Flattening Tilemap for Composite Alpha

I’m busy working on a small game with OpenFL and we’ve hit a problem. I have a number of Tiles in a Tilemap, and I want to flatten them so that I can apply a single alpha value to the composite image, instead of to the individual components separately. I want to do this to get a shadow effect, so the background isn’t a single solid color (I can’t pre-compute the background color with shadow). I’ve tried various things, including cacheAsBitmap, and BitmapData.draw(), but they either don’t work, or are way slower than I would like. We have to update the shadows every frame, because the clouds that are casting the shadows can move every frame. Here’s a small image to illustrate what I mean by applying the alpha to the composite image instead of the individual components:

Any idea of what I can try to solve this? I’m guessing the most efficient will be some direct GL calls to change the render target to a texture, and then applying alpha to that, but I’m not very familiar with that and the OpenFL render pipeline, so I’m looking for some pointers. Note that I’m open to other techniques for getting the same effect.

I’ve also written a shared google doc with some more specifics if that helps: https://docs.google.com/document/d/1TJ5xm_IwBjAeyu_5l7vrnGxSFYXYG2Hbbd0rVIhu1U8/edit

Based on https://gamedev.stackexchange.com/questions/57090/non-additive-alpha-blending-in-opengl-es-2-0-android-andengine, I do not think we could do this using a blending mode (even though you can in software with Flash Player)

This could be accomplished using a solid color instead of alpha, but if you need transparency, drawing to an intermediate texture is probably the way to go.

I’m working on cacheAsBitmap support, which is simplified bitmapData.draw support handled inside of our renderer. This won’t solve the performance (though it will automatically detect if it has changed), but I want framebuffer support to work as well. That would improve performance, but I have seen some problems on mobile when attempting to swap large framebuffers

Thanks for the reply!

Unfortunately I do need the transparency in this case. FYI, this is what the game looks like, if that helps:

The shadows in that GIF are mostly over the background blue, but that’s not generally the case (eg: see the top right), and we can’t rely on that.

That cacheAsbitmap support sounds useful, but if it’s relying on BitmapData.draw() then unfortunately it won’t solve the problems we’re having (I tried doing that already and performance wasn’t great).

How would I go about looking into framebuffers or rendering to an intermediate texture? What sort of issues are there on mobile? Would a framebuffer the size of the screen be considered large? Is this viable on WebGL, where we’re seeing the worst performance?

If you’re interested, take a look at this branch of OpenFL:

https://github.com/openfl/openfl/commits/feature/cacheAsBitmap

It will use bitmapData.draw in software right now, but uncomment this line to use framebuffers:

https://github.com/openfl/openfl/blob/feature/cacheAsBitmap/openfl/display/DisplayObject.hx#L949

There are some problems with the framebuffer approach at the moment (sometimes the dimensions seem a little wrong) but it may be worth looking at

1 Like

@singmajesty thanks! I’ll check out that branch and see if it works for me. Those ObjectPools should be great for limiting GC!

I also ran across this thread of yours, and was planning to try drawing to a hardware-only BitmapData with draw(), or if that didn’t work, poke a bit deeper and try force rendering to a texture:

That said, if cacheAsBitmap is fast and renders with hardware, that’ll be really awesome and open up the door for a lot of cool things. What still needs to happen on that branch? I saw this issue, but it’s light on details:

There is a “fast-webgl-bindings” branch on Lime that has been waiting, since it introduces backward-incompatible API changes.

I want to limit the frequency of major version releases, but I am thinking that I should update Lime with these changes (at least) in order to help improve the performance.

If you are interested in the development branch of Lime, I have just merged it in, and depending on what you are targeting, it makes a large difference on WebGL performance.

Mostly, the “cacheAsBitmap” branch in OpenFL is brand-new, and deserves additional testing. For framebuffers, if you comment out that line, you’ll see it isn’t quite right. bitmapData.draw (in the framebuffer case) needs a little help to get it be accurate with an offset matrix

1 Like

Quick update before I forget any longer: I have a “workaround” in place that seems to suffice for now. Here’s an edited snippet:

  // initialization

    shadows = new BitmapData(width, height, true, 0x000000);

    var shadowBitmap = new Bitmap(shadows);
    shadowBitmap.filters = [
      new ColorMatrixFilter([
        0, 0, 0, 0, 0,
        0, 0, 0, 0, 0,
        0, 0, 0, 0, 0,
        0, 0, 0, 1, 0
      ]),
    ];
    shadowBitmap.alpha = alpha;
    addChild(shadowBitmap);


  // once per frame

    shadows.fillRect(shadows.rect, 0x000000); // clear the shadows
    // there is a bug on ios so we use copyPixels() as a workaround here

    preShadowCast(); // change the visibility of some objects so they don't cast a shadow
    shadows.draw(tilemap);
    postShadowCast(); // restore visibility

The performance seems ok for now. It’s pretty decent on most platforms, just on HTML5 in Firefox on Linux it seems to be so-so. Once cacheAsBitmap uses a framebuffer hardware-assisted approach we might switch to that, but this works reasonably fine for now. Still running OpenFL/Lime 5.0.0/4.1.0 (just haven’t had time/need to try upgrade yet)