What is the fastest per-pixel manipulation in HTML5?

#1

What is the best and fastest mode for per-pixel manipulation in HTML5?
The desire is about 20-30 FPS, and 10-30K pixel manipulation (depends on the actual colour of the pixel) per frame, and of course refresh the image on the screen.
Can it be achieve in OpenFL?

#2

I’m interested in this too (and not only for the js target)! I’d be happy to hear some ideas on how to get the best performance for the individual targets!

Openfl rightfully does a lot of things under the hoods and obviously follows different codepaths internally to handle the various targets/compile-options. But that also makes it not so easy to read while trying to gain a few more FPS.

I’ve ~recently been playing with an experimental lib to support common operations on BitmapData-like instances (not performance oriented, but hopefully effective), and just today added what aims to be an optimization for the openfl-js target (should work with latest git openfl/lime, haxe 3.2). It’s a WIP and I’m not sure it will fit your usecase performance-wise, but! (any comments about how can I speed it up are appreciated)

If you feel like it, please test this branch of the lib. For openfl usage of the lib skim through this (basically you just assign a BitmapData instance to a Pixels var, and then fiddle with it).

Feedback welcome!

#3

Hi!
This abstract class is similar to BitmapData to Vector/ByteArray conversion - just more universal.
The performance is good enough, but the slowest part is the applyToBitmapData() function, and I think this is because of javascript and webGL limitation.
Now I need to work, so after it I will do some testing with and without applyToBitmapData() to measure the difference.

#4

Try commenting out this line in applyToBitmapData(). Should give a boost and still work (it does on my end, obviously only for html5 canvas).

#5

Yep, that is much faster, but in flash it is faster, than html5 (about 3-4x), but now not laggy, and this is good.
Further testing is coming.

#6

You might take a look at this code:

In order to improve performance when looking at HTML5 support for Evoland Classic, I devised a custom bitmap filter that combined the two post-process steps that were in the game into one. This brought the performance back to something workable. This then uses applyFilter against the BitmapData

#7

Sorry for my early post, I was wrong.
I did a per-frame test only with pixels.setPixel32(); pixels.getPixel32(); and pixels.applyToBitmapData(); and the results is impressive (near fullHD) in HTML5 for the beginning: (red graph: how many millisec for one frame, about the same amount of pixels in flash - lower is better)

Sometimes there are a very slow performance in HTML5 , but don’t know the reason yet.

#8

It may be not related, but I am noted that for the graphics-draw operations (html5) if the actual place of drawing is located behind the screen, it can very slow down performance - like in your last gif 4-7 fps.

#9

And I suggest fillRect function into Pixels.hx:

/** Sets the pixel value (with alpha) at `rect`, with `value` expressed in ARGB format. */
inline public function fillRect(rect:flash.geom.Rectangle, value:Int)
{
    // check if it is in the image
    if (rect.x >= this.width || rect.y >= this.height || rect.right < 0 || rect.bottom < 0) return;
    
    var a = (value >> 24) & 0xFF;
    var r = (value >> 16) & 0xFF;
    var g = (value >> 8) & 0xFF;
    var b = (value) & 0xFF;

    var xx:Int = Std.int(rect.x >= 0 ? rect.x : 0), yy:Int = Std.int(rect.y >= 0 ? rect.y : 0), xe:Int = Std.int(this.width >= rect.right ? rect.right : this.width - xx), ye:Int = Std.int(this.height >= rect.bottom ? rect.bottom : this.height - yy);
    var i:Int = yy * this.width + xx, pos:Int;
    
    for (yi in yy ... ye)
    {
        pos = i << 2;
        for (xi in xx ... xe)
        {
            this.bytes.set((pos + this.format.A), a);
            this.bytes.set((pos + this.format.R), r);
            this.bytes.set((pos + this.format.G), g);
            this.bytes.set((pos + this.format.B), b);
            pos += 4;
        }
        i += this.width;
    }
}
#10

If you want to see it live, here’s an HTML5 version of Evoland Classic:

http://www.openfl.org/samples/evoland/

The effect of reducing the color quality, and simulating older screens (until you pick up the upgrades to change the color) are done real-time. In a HTML5 project, it would probably be wiser to store multiple copies of the assets (or at least process once and store them), but in keeping with the spirit of a port, I wanted to try and make it work without changing the way the game worked :slight_smile:

#11

Thanks. I really need per pixel manipulation for implicit function graphing. It is more spectacular, if random order is used to calculate and plotted the values, because it can take a while, but this way you can see earlier a “draft”.
Now this is fast enough for me, so I need to optimize my other algorithms for js target. (The original algorithm was written in flash and in flashplayer it is 3x faster.)

#12

Added fillRect() as suggested, building upon the code you posted. Thanks! :stuck_out_tongue_winking_eye:

#13

You are welcome. One suggestion for even faster setPixel(), setPixel32(), getPixel(), getPixel32(), if the image width/height not changing so often, but the performance is essential.
There is var pos = (y * this.width + x) << 2; where this multiplication is the “slowest” part. So, in the PixelsData class declare:
private var y_mul_width:Array<Int> = [0];

then fill it in the constructor:
for (i in 1 ... height) y_mul_width[i] = y_mul_width[i - 1] + width;

And now, you just need to change every y * this.width to y_mul_width[y];
In flash, it is faster, but in HTML5 target I didn’t tested yet.

And I try to port from flash into Pixels a MLAA (Morphological Antialiasing) function, but it need 3x more memory like only the image.