Shaders / ShaderFilters and rotation

Help
#1

Im trying to write shader to handle butmap mask, where both masked object and mask can rotate.
After many hours of debbuging I come to more simple case. I just want to write a shader or filter for DisplayObject that is rotatng.
And openfl behave very wierd so I’d be happpy if somebody help me handle this.
there is simple example I did

vec4 color2 = texture2D (openfl_Texture, openfl_TextureCoordv);

 if (openfl_TextureCoordv.x > 0.5)
        {
            gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0);
        } else
        {
             gl_FragColor = vec4(0.0, 1.0, 0.0, 1.0);
        }
        if (color2.a == 0.0)
        {
          gl_FragColor = vec4(1.0, 1.0, 0.0, 1);
        }

fragment shader draws red if openfl_TextureCoordv.x > 0.5
and green if openfl_TextureCoordv.x < 0.5
It also draws yello if alpha = 0
Everything seems correct when no rotation

scr1

When rotation of filtered object or its parent changed I got another picture
scr2

It seems rotated object was drawed to another texture and i got this new texture in tha shader.
so openfl_Texture is new bigger texture. But why then red / green areas is not half of this new texture?
Well I need handle pixels of original object in shader / filter. Becouse its how filters work.
Is there any transforamtion parameters so I can get original pixels coordinate of the object in the shader?
If I want to draw a dot at (10, 10) corner of original object how can I do it with a shader?

#2

Use displayObject.shader to replace the standard DisplayObjectShader and perform a one-pass custom render of the object – this gives you visibility of the incoming texture and an ability to use this information before creating the final (transformed) fragments.

You’re using displayObject.filters = [ new ShaderFilter(...) ] which (by design) is a post-process shader to take all the cumulative rendering of the display object, flatten into a new texture and allow multiple ping-pong rendering passes on this output.

Does object.mask work for you?

ShaderFilter question
#4

You’re using displayObject.filters = [ new ShaderFilter(...) ] which (by design) is a post-process shader to take all the cumulative rendering of the display object, flatten into a new texture and allow multiple ping-pong rendering passes on this output
displayObject.shader work similar if I turn on cache as bitmap

which (by design) is a post-process shader to take all the cumulative rendering of the display object, flatten into a new texture and allow multiple ping-pong rendering passes on this outpu

I expected to have cummulative rendering but with accesing of object coordinates.
Well if its a filter than I can filter horisonatally. For example by x axis of the object. As you see on screenshot there is coordinate system of new generated texture.

I tried displayObject.shader
without cacheAs bitmap it applies on ecah child individually.
chsdr

	if(gl_FragColor.a == 0.0)
		{
		  gl_FragColor = vec4(0.0, openfl_TextureCoordv.x, 0.0, 1.0);
		} else
		{
		  gl_FragColor = vec4(openfl_TextureCoordv.y, 0.0, 0.0, 1.0);
		}

May be its correct but does not make sense for me. As I need count coordinaes relative to mask.
I did somesing like

mtrMask = _maskSprite.transform.matrix;
mtrSpr = _maskedObject.transform.matrix;
mtrMask.invert();
mtrSpr.concat(mtrMask);
So I can count matrix to transform mask coordinate space to shader space and it will work for individual objects, but this matrix must be counted for each shader individually.
I guess its possible to use some openfl variables from vertex shader but its not doucumented so dont know what is
uniform mat4 openfl_Matrix;
attribute vec4 openfl_Position;
Its position where? Matrix for what?

With cacheAsBitmap approach I got merged texture of child objects and it I could use this .
But also work not obvious.
if I change sprite.parent rotation I would get same effect as in previous message. So it seems everything merged into one global texture and it includes transformations of parents of object.
So if I rotate parent I get in shader texture with rotated object and size of shader texture is equal global bounds of rotated object spr.GetBounds(stage).

Does object.mask work for you?

mask does not work for bitmap masks. Should It work with bitmaps? I must use bitmap / Tilemap as a mask

So main problem is to transform one coordinate system to another.
What can I use for this? I dont want multipliy spr.transform.matrix, spr.parent.transform.matrix e.t.c

#5

openfl_Position represents the position of an incoming vertex to be rendered

openfl_Matrix represents the world transform to be applied to this vertex for the final fragment when rendering

Correct, using filters will flatten it first so the openfl_Position values will only be the corners of the new flattened texture and will not be very useful to you, nor the value of openfl_Matrix at that point

Flash has a system where you can use two objects with cacheAsBitmap = true and when you mask it will respect the alpha of the mask, so instead of being an off/on mask it will blend and should respect the pixels inside a bitmap. I’d like to see this supported. I have some support written at the moment here:

before the batcher that code was uncommented – you can comment the code above and uncomment this code below and it should apply the alpha mask in a basic case of two bitmaps, but this could use some more work to get it up and running for any kind of object