Lime 3.1.0 – Image loading / GC and GL finalizer related crash (using cpp.vm.Thread)

Lime
#1

I’m trying to load bunch of images from zip file in for for loop like this:

    if (plugin.data.exists(pathTop)){
        var textureTop = BitmapData.fromBytes(ByteArray.fromBytes(plugin.data.get(pathTop).data));
        tempTerrainTexturesTop.set(terrainType.name, textureTop);
    }

which crashes after some images are loaded, when using cpp.vm.Thread. When it’s called from main thread, it works fine. Mac native target.

Crash log:

...
Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000001280
...
Thread 19 Crashed:
0   libGL.dylib                   	0x00007fff8f05d5da glDeleteShader + 13
1   com.example.client            	0x000000010079b3f2 hx::RunFinalizers() + 66
2   com.example.client            	0x000000010079e0e0 GlobalAllocator::Collect(bool, bool, bool) + 400
3   com.example.client            	0x000000010079dd86 GlobalAllocator::AllocLarge(int, bool) + 150
4   com.example.client            	0x000000010079c812 hx::InternalNew(int, bool) + 146
5   com.example.client            	0x00000001007ce0ce hx::ArrayBase::ArrayBase(int, int, int, bool) + 62
6   com.example.client            	0x0000000100796ee2 alloc_buffer_len + 194
7   lime.ndll                     	0x0000000104034988 0x104000000 + 215432
8   lime.ndll                     	0x00000001040321be 0x104000000 + 205246
9   lime.ndll                     	0x00000001040279a2 0x104000000 + 162210
10  lime.ndll                     	0x0000000104004d2f 0x104000000 + 19759
11  com.example.client            	0x00000001004600d4 lime::graphics::Image_obj::__fromBytes(hx::ObjectPtr<haxe::io::Bytes_obj>, Dynamic) + 148
12  com.example.client            	0x0000000100463241 lime::graphics::Image_obj::fromBytes(hx::ObjectPtr<haxe::io::Bytes_obj>, Dynamic) + 353
13  com.example.client            	0x000000010016e119 openfl::display::BitmapData_obj::fromBytes(hx::ObjectPtr<openfl::utils::ByteArrayData_obj>, hx::ObjectPtr<openfl::utils::ByteArrayData_obj>, Dynamic) + 457
....

Any hints?

#2

The call that makes it crash is
0 libGL.dylib 0x00007fff8f05d5da glDeleteShader + 13
openGL functions cannot be called outside of their thread.

So you can’t use BitmapData.fromBytes from outside the main thread I’m afraid.

You could extract the bytes from the zip in a thread but create the bitmapdata in main, but that could have a slight effect on framerate.

A way to do it without touching the main thread would be to extract the zip to files with cpp.vm.Thread and then loading them with http://api.openfl.org/openfl/display/BitmapData.html#fromFile which is async.

#3

Thanks. One more question – where from my application can I use things like BitmapData.fromBytes? The only place I found yet is ENTER_FRAME event handler and similar.

#4

Should be able to do it anywhere in the main thread,
but in the end it’ll end up in a enter frame event as only possible space for any code (with the exception of the static main function of your main class).

1 Like
#5

I believe we should use threads if you use Assets.loadBitmapData, I would be interested if this works for you?

This doesn’t for ZIP contents, though (for a long time) I think support for asset packing could be integrated in the Lime tools.

If you are doing it outside of Assets, perhaps you could use a Lime ThreadPool or BackgroundWorker to load the bytes, then use BitmapData.fromBytes in the main thread?

EDIT: The error actually appears to related to HXCPP garbage collection running in the wrong thread. I guess this actually comes back to use needing to come up with a thread-safe GL finalizer (and is not triggered by GL code in the bitmap data load, but just that the allocation there triggers GC activity)

#6

Yeah, you are probably right. I was working on it today and I get crashes in almost every part of my threaded code where lot of temporary objects is created. Even creating simple objects like Point (with 2 integer variables and nothing else) crashes:

Thread 21 Crashed:
0   libGL.dylib                   	0x00007fff8f05ce5e glDeleteBuffers + 18
1   lime.ndll                     	0x0000000102455981 0x102437000 + 125313
2   com.example.client            	0x00000001007907f2 hx::RunFinalizers() + 66
3   com.example.client            	0x00000001007934e0 GlobalAllocator::Collect(bool, bool, bool) + 400
4   com.example.client            	0x00000001007963ec GlobalAllocator::GetFreeBlock(int, hx::ImmixAllocator*) + 604
5   com.example.client            	0x0000000100795dd2 LocalAllocator::CallAlloc(int, unsigned int) + 178
6   com.example.client            	0x00000001006064fc com::application::types::Point_obj::__new(int, int) + 108
7   com.example.client            	0x0000000100665e02

while I do absolutely nothing with graphics in this thread. Any suggested approach I should take? I’d really need this fixed and I don’t have any idea what to do.

#7

Now I’m creating new threads from OpenFL events (like ADDED_TO_STAGE). Would it help to create threads from somewhere else, so GC in them won’t trigger GL finalizer? If yes, where in my code can I do that?

EDIT:
I added the issue to github repo:

#8

I believe I may have this resolved, but I would appreciate your help in testing and confirming if it is working properly. Check out the issue for a link to a development build with the changes :slight_smile:

#9

Fixed in dev build of lime 3.2.0, thank you very much.