[HTML5 - Lime5.8.2 - Openfl 6.5.0] Possible Image.copyPixels optimization?

Hi,
I am trying to fix my “famous” problem with copyPixels on HTML5 target: with BIG spritesheets (I am using a derivation of spritesheet library), when parsing the atlas to extract all the Bitmaps by copyPixels my browser blows up and my monitors turn off, as if the graphics card crashed.

I am trying to find an explanation and a way to avoid this, and doing a few experiments. I am studying Lime/Image.copyPixels: it first of all converts both the source and target images from CANVAS to DATA (source is the atlas, target is the image to extract), and then it copies the interested rectangle from the atlas to the target.

The convertion phase has no problems, so evidently the crash is caused by the copy phase, maybe memory can’t catchup with the extraction of so many images all at once, I will try to set a delay to test this hypothesis.

In the meanwhile I have a question about the source atlas convertion: it obviously needs to be converted to DATA only once, so I added a check in the ImageCanvasUtils.convertToData function:

public static function convertToData (image:Image, clear:Bool = false):Void {
		
	if (image.type != DATA)  // <---- this one
	{
			
		[...]
			
		image.type = DATA;
	}	
}

…so the source will be converted to DATA only the first time copyPixels is called.
If I put a tracking trace inside the “if” brackets I notice that it will be executed (in my app) about 500 times, against the xthousands times without the “if” check: I can’t define how many times the trace is executed, as the browser hangs, this happens when it can’t catchup with the many trace() calls.

The result is identical, so I am wondering if this check is preventing the convertToData function from doing something useful I can’t notice.

This does not prevent my app from crashing with giant atlases, but if it can introduce a small optimization I will be happy :smiley:
I don’t understand what’s happening inside the original convertToData function (maybe it already avoids multiple convertions of the same image to DATA), so I am asking if you can please have a look at it and tell me if my modification is useful.

Thanks

Now I am doing some tests on the copy phase…

I think you shouldn’t use copyPixels but just create new canvases and do a cropped drawImage.

1 Like

@elsassph And what about performances?

I noticed that ImageCanvasUtils.copyPixels is not used anywhere: why?

Is it necessary to convert to DATA in HTML5 to do copyPixels?

We use the same appoach in our game, bitmapData atlases and copyPixel individual sprites and we don’t have such problems. Though there were performance issues which we optimized here https://github.com/openfl/lime/pull/1131. But I think this PR is not in lime 5.8.2 but from 6.0.0. Not sure if this will fix your issue, but will make copyPixels faster. You could cherry-pick it in your version and try it.

@elsassph I understand your point using canvases directly, but we need them to be wrapped in BitmapData object. I’m not sure but it might be possible, in a hacky way, to create canvases and then inject them in the Image buffer.
Edit: I just checked the ImageCanvasUtils.copyPixels and it is exactly what it does - using canvases and drawImage. But to be able to do that you need to have this PR https://github.com/openfl/lime/pull/1131 in your lime, otheriwse it will use DATA

You can re-implement it yourself:

  • directly access BitmapData.image of your source and destinations,
  • like in the PR, call ImageCanvasUtil.copyPixels (note: I think you don’t need the convertToCanvas calls).

Sure, but do we gain noticeable performance benefits?
You are right about convertToCanvas but the method itself does nothing if the canvas already exists. It is kind a “make sure you have canvas” method.

copyPixels does call convertToCanvas anyway, from what I’m seeing.

Performance should be much better - it shouldn’t too hard to give it a try.

Good, I’ll try it and let you know.

It was exactly what I tried this morning: I replaced the

ImageCanvasUtil.convertToData -> ImageCanvasUtil.copyPixel
with
ImageCanvasUtil.copyPixels directly, with the 2nd convertToCanvas commented

Is this what you meant?
But about performances… I can’t tell the difference.

so the function Image.copyPixels should be modified like this

switch (type) {

	case CANVAS:
		if (alphaImage != null || sourceImage.type != CANVAS) {
			[...] // not our case
		} else {
			//ImageCanvasUtil.convertToCanvas (this);
			//ImageCanvasUtil.convertToCanvas (sourceImage);
			ImageCanvasUtil.copyPixels (this, sourceImage, sourceRect, destPoint, alphaImage, alphaPoint, mergeAlpha);
		}

right?

So with this approach you have no performance improvements ?

I am doing some tests in app, but it looks like the execution time changes from time to time, maybe because of the memory management of the browser.


EDIT: well, it seems generally faster, and significantly important not to overload execution by parsing all spritesheets at once, if there is the giant atlas between them: I am applying a delay of i.e. 3secs between the normal spritesheets and the giant one, and the execution seems to run a lot smoother.

Yeah ok. One thing that we do is getting (copyPixel) the sprite when needed. So we are not splitting the texture atlas in sprites once it is loaded, but just load it and get(and cache) the sprites as they are needed. This could reduce the initial load if you don’t need all sprites at once.

I’m suggesting to use ImageCanvasUtil.copyPixels directly - provided the implementation is efficient in using Canvas drawImage.

If your version of OpenFl doesn’t use drawImage, my advice is to drill in the objects to get the Canvases and do the drawImage yourself.

PS: I’m not recommending to modify Image.copyPixels, though it could be a way to quickly test the changes without changing the code using it.

I have updated Openfl and Lime: the previous behaviour with CANVAS images (direct ImageCanvasUtil.copyPixels call) has been restored, I just need to comment the two convertToCanvas calls, because I am already working with canvases, theoretically.

Has that solved your problems ? Do you have better performance now and browser not crashing ?

Parsing time is a lot better, but I still have “random” crashes, related to something else… I am still trying to find out what the real problem is: sometimes it crashes, I turn off the PC, turn on it again and the browser does not crash anymore… :weary:

…I am also considering the possibility that it is an hardware problem… that would be an incredible OT

Perhaps you could try setting your CPU to a lower clock speed, if your computer is dusty (among other reasons) it could overheat when it hits a heavy amount of execution.

For example, at home I have an 8-core CPU, and my motherboard by default sets it to overclock. All is fine (normally), but if I allow it to overclock, and it begins using all of the cores, after a few minutes it pushes the heat past the safe point and it shuts down. Extra fans help, but I also set it to use a more conservative CPU setting as well, because restarting wastes more time than the extra few MS of running it faster :wink:

This can also happen with a GPU, as well