Hi all,
I’ve been using @singmajesty 's openfl-webm for a while and after updates to the new openfl (big hiatus because it’s based on HaxeFlixel) I seem to have weird issues with it. Some pictures to make it clearer:
Rendered by Firefox:
Renderer by a minimal OpenFl app
I’m currently using OpenFl 8.6.4. Happens both in Mac and Windows. I’m not really sure but I think that either endianess or ARGB / RGBA shenanigans going on, but I wanted to know if someone had experience this.
Thanks in advance!
What version are you updating from? Did this work properly before, or has the color always been like this?
Thanks 
Unfortunately, I can’t give you proper answers: This project has been in hibernation for literal years and to be honest your branch is the only one I’ve managed to build and run (more or less) successfully in macOS. I just cloned the game repo with a new environment and hacked at it until it worked.
Before that, it was using the very old one from haxelib… which doesn’t compile against “newer” OpenFl and leaks memory.
I’ll probably try going backwards with Lime/OpenFl versions until I get a working setup haha.
The good news is that this may be simple to fix. Maybe there’s a way to set the image format as a different color order: https://github.com/HaxeExtension/extension-webm/blob/master/webm/VpxDecoder.hx#L39
Huh, I was using the wrong repo it seems. In any case, out of the box extension-webm give different colors compared to openfl-webm:
And if I uncomment and change bitmapData.image.format = ARGB32; I get the following:
Better colors, but very blue. And if I change it to RGBA32 I get mostly red of course.
I’ll keep at it and report back, but this is good progress 
Ok, I seem to be unable to get this right. I’m kinda confused because the decoding code is basically the same it used to be in soywiz’s extension but now suddenly the colors are wrong. Has someone actually used extension-webm?
Is there any other option for video playback on openfl native?
What version of OpenFL are you coming from?
OpenFL 2 “next” or OpenFL 3 and greater all use premultiplied BGRA surfaces by default, where previously OpenFL 1, 2 and 3 “legacy” used unmultiplied RGBA textures.
It may have something do to with alpha, though WebM should be using an opaque BitmapData, so I don’t think that matters, but the color order may be wrong
The project was migrated from legacy to 8.6.4. It’s based on HaxeFlixel so it got stuck in legacy for a while.
However, the color issues are happening on a new, basic, openfl 8.6.4 application that only renders the video. Setting the bitmap data to ARGB32 seems to give me something closer to real but I have no idea where the blue overlay is coming from.
Looking at the decoding code: What’s the difference between bitmapData.image.format and bitmapData.image.buffer.data.buffer.format?
Here are the layers:
-
BitmapData is the OpenFL representation for image information. This holds a hardware texture and/or a software store of pixel information. The software image, when available, is stored as a Lime Image. That’s where our pixels live.
-
Image is the higher-level Lime API for image data. You have APIs for reading, writing and copying image information (which BitmapData in OpenFL uses for its APIs). The actual data is stored in an ImageBuffer object, available at image.buffer or behind various getters and setters (such as as image.format)
-
ImageBuffer is a basic object which holds the actual image details. It has simple properties to hold a canvas element, an array buffer of pixel information, or what format the data is in.
That said, buffer.format is a basic property to store what format the pixels are in. image.format is a getter/setter which will attempt to modify the image data to convert between formats when requested.
So this might need a requirement of properly identifying the format of the pixels after WebM decodes them, then possibly creating a conversion. Like:
imageBuffer.data = decodedData;
imageBuffer.format = ARGB32; // this should match the decoded format
imageBuffer.premultiplied = false; // this probably will be false, could try true anyway
image.buffer = imageBuffer;
image.format = RGBA32; // this will try to perform a conversion
Ok, got it working, but I don’t know why.
According to the extension’s code here: https://github.com/HaxeExtension/extension-webm/blob/457f0564c120a979ac122de27f7248d5a527dee8/project/common/ExternalInterface.cpp#L340
The resulting buffer is allocated for 4 bytes per pixel and in RGBA order. I’m no computer genius but that sounds like RGBA32 to me.
You told me that modern OpenFL defaults to BGRA32. So basically, I expected this to work:
var buffer = bitmapData.image.buffer;
buffer.data.buffer = Bytes.ofData(info[2]);
buffer.format = RGBA32;
buffer.premultiplied = false;
bitmapData.image.format = BGRA32;
bitmapData.image.version++;
Alas, it does not.This, however, works like a charm:
var buffer = bitmapData.image.buffer;
buffer.data.buffer = Bytes.ofData(info[2]);
buffer.format = ARGB32;
buffer.premultiplied = true;
bitmapData.image.format = BGRA32;
bitmapData.image.version++;
So I guess that the macro called “YUV420toRGBA” does not, in fact, give you a RGBA buffer. To be quite honest, I don’t really care.
My last question is: is this method really that much slower than the lock() -> setPixels() -> unlock() version? It seems to be good enough for my needs at the moment, I’m just curious.
In any case, thanks a lot for the help
Thank you! Would you mind doing a pull request?
I’m in the same camp – if it works, that’s ideal for now. Later a formal native video API with GL shaders to skip the software pixel swapping would be great, but for now, we just need video
I think this might be faster than setPixels () , but I’m sure there are still ways to improve the performance more
Done: https://github.com/HaxeExtension/extension-webm/pull/2
A proper native API sounds like a good idea, even if it’s limited to WebM. An updated lib with VP8 + VP9 support should be more than enough for most use cases.
1 Like
I got the WebM library working as a “native-toolkit” library (so we could build into Lime directly), but the next step would be learning from extension-webm to create some kind of formal Lime video API, then consuming that for native video built into OpenFL.
1 Like
