Calling Loader.load results in random crashes on the Linux Neko target


#1

I’m trying to load a bitmap from a URL using the OpenFL display.Loader class, but on the Neko target on Linux, random crashes start happening after calling Loader.load. I have tried to isolate the issue to a minimal test app (it downloads and displays the OpenFL logo). I am using OpenFL 8.1.1 with Lime 6.3.1 on Linux (Ubuntu 16.04 64-bit) and Windows (Windows 7 32-bit). I’ve tested the cpp and neko targets on Linux and the neko target on Windows, and this issue appears only on the Linux Neko target.

package;

import openfl.display.Sprite;
import openfl.net.URLRequest;
import openfl.events.Event;
import openfl.events.IOErrorEvent;

import openfl.display.LoaderInfo;
import openfl.display.Loader;
import openfl.display.Bitmap;

class Main extends Sprite 
{
	private var loader:Loader;

	public function new () 
	{
		super ();
		loadURL();
	}
	
	private function loadURL()
	{
		loader = new Loader();
		loader.contentLoaderInfo.addEventListener(Event.COMPLETE, onComplete);
		loader.contentLoaderInfo.addEventListener(IOErrorEvent.IO_ERROR, onIOError);
		loader.load(new URLRequest("http://www.openfl.org/images/logo.png"));
	}
	
	private function onComplete(event:Event):Void 
	{
		trace(onComplete, event);
		var target:LoaderInfo = cast event.target;
		var loader:Loader = cast target.loader;
		var bitmap = new Bitmap(cast(loader.content, Bitmap).bitmapData.clone());
		addChild(bitmap);
	}

	private function onIOError(event:IOErrorEvent):Void 
	{
		trace("ioErrorHandler: " + event);
	}
}

On Linux, the test app runs fine on CPP, but on Neko crashes randomly (6 times out of 20 runs in my tests). The crashes happen after Loader.load has been called, but in a semi-random fashion. Most commonly the program crashes simply with a segmentation fault, but sometimes there are stack traces. I’ve collected here examples of the errors shown by the test program:

Segmentation fault (core dumped)

Called from openfl.utils.Endian.Endian_Impl::$statics line 1
Called from ApplicationMain::main line 84
Called from ApplicationMain::create line 116
Called from lime.app.Application::exec line 269
Called from lime._backend.native.NativeApplication::exec line 160
Called from a C function
Called from lime._backend.native.NativeApplication::handleRenderEvent line 452
Called from lime.graphics.Renderer::render line 61
Uncaught exception - Invalid field access : render
Segmentation fault (core dumped)

Called from openfl.utils.Endian.Endian_Impl::$statics line 1
Called from ApplicationMain::main line 84
Called from ApplicationMain::create line 116
Called from lime.app.Application::exec line 269
Called from lime._backend.native.NativeApplication::exec line 160
Called from a C function
Called from lime._backend.native.NativeApplication::handleApplicationEvent line 201
Called from lime.app._Event_Int_Void::dispatch line 101
Called from lime.system.ThreadPool::__update line 219
Called from lime.app._Event_haxe_Function_Void::dispatch line 101
Called from lime._backend.native.NativeHTTPRequest::threadPool_onProgress line 558
Uncaught exception - Invalid call

Called from openfl.utils.Endian.Endian_Impl::$statics line 1
Called from ApplicationMain::main line 84
Called from ApplicationMain::create line 116
Called from lime.app.Application::exec line 269
Called from lime._backend.native.NativeApplication::exec line 160
Called from a C function
Called from lime._backend.native.NativeApplication::handleApplicationEvent line 201
Called from lime.app._Event_Int_Void::dispatch line 101
Called from openfl.display.Stage::update line 990
Uncaught exception - Invalid field access : __deltaTime

Called from openfl.utils.Endian.Endian_Impl::$statics line 1
Called from ApplicationMain::main line 84
Called from ApplicationMain::create line 116
Called from lime.app.Application::exec line 269
Called from lime._backend.native.NativeApplication::exec line 160
Called from a C function
Called from lime._backend.native.NativeApplication::handleWindowEvent line 653
Called from lime.app._Event_Void_Void::dispatch line 101
Called from openfl.display.Stage::addWindow line 352
Called from line 1
Uncaught exception - Reading Outside Env

It would appear to me that there is some difference in how Neko and CPP do URL requests, but this goes so deep into the target implementation that I have unfortunately no idea how to proceed from here. Any help would be appreciated!


#2

Thanks for the sample code, I’ll look at this soon :slight_smile:


#3

I ran into this issue at first when trying to update an application to the current version of OpenFL. Since the Loader class worked fine on older versions, I now tried to systematically test OpenFL and Lime versions in order to find the exact update at which this issue appears for the first time. After some testing I found out that OpenFL/Lime versions up to and including 6.1.0/5.4.0 still work fine, but versions starting from 6.1.1/5.5.0 crash randomly after calling Loader.load, with the sort of errors listed in my first message.

Looking at the version logs I found that for Lime 5.5.0 the release notes mention “added GC to cURL API”. I would assume URL requests are implemented using the cURL library, so perhaps this update has something to do with the crashes in newer versions? Since the crashes happen semi-randomly and on seemingly unrelated objects (in those cases where a stack trace is generated), I wonder if these errors could perhaps be caused by the garbage collector collecting something that is still in use?


#4

Yeah, garbage collection is a tricky beast. I thought we had it all sorted out with the most recent release, but looking at this tomorrow


#5

This looks interesting:

https://groups.google.com/forum/#!topic/haxelang/6x8TNPHlMMA

Perhaps it is an issue with the Deque object in Neko (which we use in ThreadPool, which is used under the hood for the Loader class while loading) and it is returning on the wrong thread or not working properly

Is it definitely working properly on C++?


#6

I tested the above OpenFL/Lime version combinations also with CPP (on Linux) and the program runs without crashes. I changed the test program to launch 100 loaders so that the random crash will very likely appear on every run (if it happens with that OpenFL/Lime version).

class Main extends Sprite 
{
	private var loader:Loader;
	private var count:Int = 0;

	public function new () 
	{
		super ();
		for (i in 0...100)
			loadURL();
	}

	private function loadURL()
	{
		loader = new Loader();
		loader.contentLoaderInfo.addEventListener(Event.COMPLETE, onComplete);
		loader.contentLoaderInfo.addEventListener(IOErrorEvent.IO_ERROR, onIOError);
		loader.load(new URLRequest("http://www.openfl.org/images/logo.png"));
	}
	
	private function onComplete(event:Event):Void 
	{
		trace("onComplete " + count++, event);
		var target:LoaderInfo = cast event.target;
		var loader:Loader = cast target.loader;
		var bitmap = new Bitmap(cast(loader.content, Bitmap).bitmapData.clone());
		bitmap.x = Math.random()*(Lib.current.stage.stageWidth-bitmap.width);
		bitmap.y = Math.random()*(Lib.current.stage.stageHeight-bitmap.height);
		addChild(bitmap);
	}

	private function onIOError(event:IOErrorEvent):Void 
	{
		trace("ioErrorHandler: " + event);
	}
}

I compiled and ran the CPP version with the command “openfl test cpp -debug -clean”, and on both OpenFL/Lime 6.1.0/5.4.0 and 6.1.1/5.5.0 the program ran fine. I used HXCPP version 3.4.188 for 6.1.1/5.5.0, but had to downgrade to HXCPP 3.4.64 for OpenFL/Lime 6.1.0/5.4.0 due to a compiler error message about a missing ‘std.dso’ file.

The CPP targets of both OpenFL/Lime versions ran without crashes, but with an interesting difference in performance. While for 6.1.0 the Loaders finished very quickly and at a variable rate, for 6.1.1 the loaders were very slow and appeared to finish at a constant rate of what appeared to be about one per second (and some of them also timed out with an error “Timeout was reached”), looking to me as if there was no parallelization happening. Wondering about this, I ran a third test with OpenFL/Lime 8.1.1/6.3.1 (and HXCPP 3.4.188), but for this recent version the Loaders were again faster and appeared to be working in parallel. However, I’m not sure if this behaviour has anything to do with the Neko crashes.

Regarding the Neko crashes, threading issues sound like something that could also explain the seemingly random crashes. If I understand this correctly, the cURL library is used as an NDLL, and is therefore identical between Neko and C++ (unless there is some conditional compiling happening)? I assume this would mean that the issue is somehow in something the Neko virtual machine does. On the other hand, the issue is also clearly related to Loader (or something used by Loader), as I haven’t seen it in any other code, and my larger program works fine on Neko when the Loader related functionality is disabled.


#7

A long time ago, we added threading, but with no limit. This had some trouble. Next we added a pool, but the timeout logic did not account for the (possibly) delayed start time, and the thread limit was low. I believe we raised the max limit and made fixes to timeout.

I wonder if we should disable threading in Loader on Neko. You should be able to run lime test neko -Dforce-synchronous to disable ThreadPool threading entirely


#8

I tried Neko (with OpenFL/Lime 8.1.1/6.3.1) with the command “lime test neko -Dforce-synchronous” and the program runs very slowly, finishing about one loader per second. The window remains blank, so I assume the synchronous loading is blocking drawing.

However, the program still crashes, but this time many of the errors appear to be directly related to Loader and HTTP requests, so perhaps these are closer to whatever is causing the Loader related crashes? I’m not sure what the difference is between the lime and openfl run commands, the stack traces look somewhat different but I’ve before always used the openfl command?

Here are the errors that happened:

Main.hx:57: onComplete 96,[Event type=“complete” bubbles=false cancelable=false]
Called from ? line 1
Called from ApplicationMain.hx line 84
Called from ApplicationMain.hx line 114
Called from lime/app/Preloader.hx line 207
Called from lime/app/Preloader.hx line 278
Called from lime/app/Future.hx line 80
Called from lime/app/Preloader.hx line 331
Called from lime/app/Preloader.hx line 231
Called from lime/app/Preloader.hx line 353
Called from openfl/display/Preloader.hx line 56
Called from openfl/display/DisplayObject.hx line 288
Called from openfl/display/DisplayObject.hx line 551
Called from openfl/display/DisplayObject.hx line 480
Called from a C function
Called from openfl/events/EventDispatcher.hx line 217
Called from openfl/display/Preloader.hx line 297
Called from openfl/display/Preloader.hx line 244
Called from openfl/display/DisplayObject.hx line 288
Called from openfl/display/DisplayObject.hx line 551
Called from openfl/display/DisplayObject.hx line 480
Called from a C function
Called from openfl/events/EventDispatcher.hx line 217
Called from openfl/display/Preloader.hx line 114
Called from a C function
Called from lime/app/Preloader.hx line 250
Called from lime/_macros/EventMacro.hx line 101
Called from ApplicationMain.hx line 100
Called from ApplicationMain.hx line 148
Called from openfl/display/Stage.hx line 1285
Called from ApplicationMain.hx line 199
Called from ApplicationMain.hx line 272
Called from a C function
Called from ApplicationMain.hx line 295
Called from a C function
Called from Main.hx line 24
Called from Main.hx line 52
Called from openfl/display/Loader.hx line 150
Called from openfl/net/URLLoader.hx line 71
Called from lime/app/Future.hx line 80
Called from openfl/net/URLLoader.hx line 80
Called from openfl/events/EventDispatcher.hx line 96
Called from openfl/events/EventDispatcher.hx line 217
Called from openfl/display/Loader.hx line 279
Uncaught exception - Invalid field access : contentType

Main.hx:57: onComplete 13,[Event type=“complete” bubbles=false cancelable=false]
Called from ? line 1
Called from ApplicationMain.hx line 84
Called from ApplicationMain.hx line 114
Called from lime/app/Preloader.hx line 207
Called from lime/app/Preloader.hx line 278
Called from lime/app/Future.hx line 80
Called from lime/app/Preloader.hx line 331
Called from lime/app/Preloader.hx line 231
Called from lime/app/Preloader.hx line 353
Called from openfl/display/Preloader.hx line 56
Called from openfl/display/DisplayObject.hx line 288
Called from openfl/display/DisplayObject.hx line 551
Called from openfl/display/DisplayObject.hx line 480
Called from a C function
Called from openfl/events/EventDispatcher.hx line 217
Called from openfl/display/Preloader.hx line 297
Called from openfl/display/Preloader.hx line 244
Called from openfl/display/DisplayObject.hx line 288
Called from openfl/display/DisplayObject.hx line 551
Called from openfl/display/DisplayObject.hx line 480
Called from a C function
Called from openfl/events/EventDispatcher.hx line 217
Called from openfl/display/Preloader.hx line 114
Called from a C function
Called from lime/app/Preloader.hx line 250
Called from lime/_macros/EventMacro.hx line 101
Called from ApplicationMain.hx line 100
Called from ApplicationMain.hx line 148
Called from openfl/display/Stage.hx line 1285
Called from ApplicationMain.hx line 199
Called from ApplicationMain.hx line 272
Called from a C function
Called from ApplicationMain.hx line 295
Called from a C function
Called from Main.hx line 24
Called from Main.hx line 52
Called from openfl/display/Loader.hx line 150
Called from openfl/net/URLLoader.hx line 71
Called from lime/net/HTTPRequest.hx line 131
Called from lime/_backend/native/NativeHTTPRequest.hx line 356
Called from lime/system/ThreadPool.hx line 101
Called from lime/system/ThreadPool.hx line 175
Called from lime/_macros/EventMacro.hx line 101
Called from lime/_backend/native/NativeHTTPRequest.hx line 507
Called from lime/_backend/native/NativeHTTPRequest.hx line 309
Called from lime/system/ThreadPool.hx line 125
Called from lime/_macros/EventMacro.hx line 101
Called from lime/_backend/native/NativeHTTPRequest.hx line 524
Called from haxe/Timer.hx line 290
Uncaught exception - Invalid field access : length

Main.hx:57: onComplete 16,[Event type=“complete” bubbles=false cancelable=false]
Segmentation fault (core dumped)

Main.hx:57: onComplete 14,[Event type=“complete” bubbles=false cancelable=false]
Called from ? line 1
Called from ApplicationMain.hx line 84
Called from ApplicationMain.hx line 114
Called from lime/app/Preloader.hx line 207
Called from lime/app/Preloader.hx line 278
Called from lime/app/Future.hx line 80
Called from lime/app/Preloader.hx line 331
Called from lime/app/Preloader.hx line 231
Called from lime/app/Preloader.hx line 353
Called from openfl/display/Preloader.hx line 56
Called from openfl/display/DisplayObject.hx line 288
Called from openfl/display/DisplayObject.hx line 551
Called from openfl/display/DisplayObject.hx line 480
Called from a C function
Called from openfl/events/EventDispatcher.hx line 217
Called from openfl/display/Preloader.hx line 297
Called from openfl/display/Preloader.hx line 244
Called from openfl/display/DisplayObject.hx line 288
Called from openfl/display/DisplayObject.hx line 551
Called from openfl/display/DisplayObject.hx line 480
Called from a C function
Called from openfl/events/EventDispatcher.hx line 217
Called from openfl/display/Preloader.hx line 114
Called from a C function
Called from lime/app/Preloader.hx line 250
Called from lime/_macros/EventMacro.hx line 101
Called from ApplicationMain.hx line 100
Called from ApplicationMain.hx line 148
Called from openfl/display/Stage.hx line 1285
Called from ApplicationMain.hx line 199
Called from ApplicationMain.hx line 272
Called from a C function
Called from ApplicationMain.hx line 295
Called from a C function
Called from Main.hx line 24
Called from Main.hx line 52
Called from openfl/display/Loader.hx line 150
Called from openfl/net/URLLoader.hx line 71
Called from lime/net/HTTPRequest.hx line 131
Called from lime/_backend/native/NativeHTTPRequest.hx line 356
Called from lime/system/ThreadPool.hx line 101
Called from lime/system/ThreadPool.hx line 175
Called from lime/_macros/EventMacro.hx line 101
Called from lime/_backend/native/NativeHTTPRequest.hx line 507
Called from lime/_backend/native/NativeHTTPRequest.hx line 299
Called from lime/net/curl/CURL.hx line 135
Called from a C function
Called from lime/_backend/native/NativeHTTPRequest.hx line 486
Called from haxe/io/Bytes.hx line 94
Uncaught exception - OutsideBounds

Main.hx:57: onComplete 99,[Event type=“complete” bubbles=false cancelable=false]
Called from ? line 1
Called from ApplicationMain.hx line 84
Called from ApplicationMain.hx line 114
Called from lime/app/Preloader.hx line 207
Called from lime/app/Preloader.hx line 278
Called from lime/app/Future.hx line 80
Called from lime/app/Preloader.hx line 331
Called from lime/app/Preloader.hx line 231
Called from lime/app/Preloader.hx line 353
Called from openfl/display/Preloader.hx line 56
Called from openfl/display/DisplayObject.hx line 288
Called from openfl/display/DisplayObject.hx line 551
Called from openfl/display/DisplayObject.hx line 480
Called from a C function
Called from openfl/events/EventDispatcher.hx line 217
Called from openfl/display/Preloader.hx line 297
Called from openfl/display/Preloader.hx line 244
Called from openfl/display/DisplayObject.hx line 288
Called from openfl/display/DisplayObject.hx line 551
Called from openfl/display/DisplayObject.hx line 480
Called from a C function
Called from openfl/events/EventDispatcher.hx line 210
Uncaught exception - Invalid field access : hasNext

Main.hx:57: onComplete 14,[Event type=“complete” bubbles=false cancelable=false]
Called from ? line 1
Called from ApplicationMain.hx line 84
Called from ApplicationMain.hx line 114
Called from lime/app/Preloader.hx line 207
Called from lime/app/Preloader.hx line 278
Called from lime/app/Future.hx line 80
Called from lime/app/Preloader.hx line 331
Called from lime/app/Preloader.hx line 231
Called from lime/app/Preloader.hx line 353
Called from openfl/display/Preloader.hx line 56
Called from openfl/display/DisplayObject.hx line 288
Called from openfl/display/DisplayObject.hx line 551
Called from openfl/display/DisplayObject.hx line 480
Called from a C function
Called from openfl/events/EventDispatcher.hx line 217
Called from openfl/display/Preloader.hx line 297
Called from openfl/display/Preloader.hx line 244
Called from openfl/display/DisplayObject.hx line 288
Called from openfl/display/DisplayObject.hx line 551
Called from openfl/display/DisplayObject.hx line 480
Called from a C function
Called from openfl/events/EventDispatcher.hx line 217
Called from openfl/display/Preloader.hx line 114
Called from a C function
Called from lime/app/Preloader.hx line 250
Called from lime/_macros/EventMacro.hx line 101
Called from ApplicationMain.hx line 100
Called from ApplicationMain.hx line 148
Called from openfl/display/Stage.hx line 1285
Called from ApplicationMain.hx line 199
Called from ApplicationMain.hx line 272
Called from a C function
Called from ApplicationMain.hx line 295
Called from a C function
Called from Main.hx line 24
Called from Main.hx line 52
Called from openfl/display/Loader.hx line 150
Called from openfl/net/URLLoader.hx line 71
Called from lime/net/HTTPRequest.hx line 131
Called from lime/_backend/native/NativeHTTPRequest.hx line 356
Called from lime/system/ThreadPool.hx line 101
Called from lime/system/ThreadPool.hx line 175
Called from lime/_macros/EventMacro.hx line 101
Called from lime/_backend/native/NativeHTTPRequest.hx line 507
Called from lime/_backend/native/NativeHTTPRequest.hx line 309
Called from lime/system/ThreadPool.hx line 125
Called from lime/_macros/EventMacro.hx line 101
Called from lime/_backend/native/NativeHTTPRequest.hx line 524
Called from haxe/Timer.hx line 290
Uncaught exception - Invalid field access : length