If you are looking for more robust masking support, the latest release of OpenFL improves on the custom shader system. Among other features, this allows sampling of additional images.
You can only apply this to a flat object (such as a Bitmap, Shape or Tilemap) but until we add framebuffer support internally to flatten more complex objects, this at least provides an option on a GL target.
Old versions of OpenFL did support masks (such as circles) using hardware, but it had crude, rough edges. In the future, I would like to see core support for masks using a system similar to the following, but handled internally:
package;
import openfl.display.BitmapData;
import openfl.display.Bitmap;
import openfl.display.DisplayObject;
import openfl.display.Shader;
import openfl.display.Sprite;
import openfl.filters.ShaderFilter;
class Main extends Sprite {
public function new () {
super ();
var bitmapData = new BitmapData (200, 200, false, 0xFFFF0000);
var bitmap = new Bitmap (bitmapData);
addChild (bitmap);
var mask = new Sprite ();
mask.graphics.beginFill (0xFF0000);
mask.graphics.drawCircle (100, 100, 100);
applyMask (bitmap, mask);
}
private function applyMask (bitmap:Bitmap, mask:DisplayObject):Void {
#if flash
bitmap.mask = mask;
#else
var bitmapDataMask = new BitmapData (bitmap.bitmapData.width, bitmap.bitmapData.height, true, 0);
bitmapDataMask.draw (mask);
var shader = new Shader ();
shader.glFragmentSource =
"varying float vAlpha;
varying vec2 vTexCoord;
uniform sampler2D uImage0;
uniform sampler2D uImage1;
void main(void) {
vec4 color = texture2D (uImage0, vTexCoord);
float mask = texture2D (uImage1, vTexCoord).a;
if (color.a == 0.0 || mask == 0.0) {
gl_FragColor = vec4 (0.0, 0.0, 0.0, 0.0);
} else {
gl_FragColor = vec4 (color.rgb / color.a, mask * color.a * vAlpha);
}
}";
shader.data.uImage1.input = bitmapDataMask;
bitmap.filters = [ new ShaderFilter (shader) ];
#end
}
}
Quick question: You don’t pass shader.data.uImage0 - is it passed implicitly? I only have my experience with shaders from ShaderToy, so no experience with passing stuff to GLSL yet.
shader.data is generated automatically based on the GL vertex and fragment sources, and when it is time to render with that shader, it gets populated with the world transform and alpha values. Values (such as our uImage1 texture) are bound automatically
The above code was written for before OpenFL had GL mask support internally, and before we revamped the shader code for the OpenFL 8 release.
If you need only on/off masks, use object.mask like in Flash. If you need to patch in support for alpha-based masking (before we get it off the TODO and implemented), take a look at this sample code, I think it should work: