OpenFL 7 AssetLibrary

tsmithf · September 11, 2018, 12:56pm

I’m trying to implement loading an AssetLibrary of images and text files using the new pak feature.

I can load the AssetLibrary as defined in the project.xml file…


<library path="assets/assetlib" id="assetlib" type="zip" preload="true" />
<assets path="assets/assetlib" library="assetlib" />

I initially set the type to pak, but was getting out of memory messages during loading, so I have been trying the zip type, which loads considerably faster and without errors.

However I’m struggling to extract the assets from the compressed library.

The library loads and the asset is visible via Assets.list(); but the normal method of extracting the bitmap data doesn’t work as the pak file is using zip compression…

var bd:BitmapData = Assets.getBitmapData("assetlib:assets/images/imagename.jpg");

I have tried various processes using the haxe built in zip but can’t seem to figure out how to target the specific asset and unzip it.

I have used the following function to unzip, but all other elements to target the asset have left me stuck…

function unzipIt(file:Entry):ByteArray{
		return ByteArray.fromBytes(Reader.unzip(file));
}

Any ideas?

singmajesty · September 11, 2018, 3:45pm

Try

<library id="assetlib" type="zip" preload="true" />
<assets path="assets/assetlib" library="assetlib" />

Then

var bd:BitmapData = Assets.getBitmapData("assetlib:assets/images/imagename.jpg");

Then if there are errors, could you share them?

tsmithf · September 11, 2018, 4:33pm

Thanks, but that method only seems to work when the AssetLibrary is using pak, but due to the file size it causes out of memory error and take ages to load.

When I use type=“zip” the AssetLibrary is found and loads, the assets are also listed correctly, but bd:BitmapData returns null

singmajesty · September 11, 2018, 5:28pm

Hmm, does it work differently if you also include <haxelib name="format" /> in your project?

EDIT: Nevermind the format library, does it work differently if you try “gzip” or “deflate” as the type instead of “zip”?

tsmithf · September 11, 2018, 5:46pm

Ah, thanks, sod’s law it is only zip that doesn’t work, deflate and gzip are fine.

btw this is html5 output, would that be the issue?

singmajesty · September 11, 2018, 5:55pm

I found the problem.

In the tools, we set type = library.type, but then if (type == "zip") type = "deflate"

However, we later push library.type into the manifest settings, not type. Whoops.

I’ve committed changes to Lime that should help resolve the behavior for the “zip” library, in the meantime, you should be able to use either “gzip” or “deflate” for compressed libraries

tsmithf · September 11, 2018, 6:18pm

Great, I’m up and running, should have tried the other methods first, oops.

Thanks

gonzos · September 11, 2018, 6:46pm

I would like to use this feature. I have folder with images (50MB) and I can’t compile project from FD if I include this library to my project. So I copy it manually in production folder and it works fine.
Question is: can I somehow create *pak, *zip file from command line and then manually copy it in production folder and use regularly in game as explained above?

singmajesty · September 11, 2018, 7:14pm

What target are you using?

gonzos · September 11, 2018, 7:16pm

My primary target is html5.

singmajesty · September 11, 2018, 8:15pm

Hmm, I’m not sure how it would affect compiling from FlashDevelop (if it uses the Lime tools properly to build), but I believe you could upload a new library to production, there should be a manifest JSON file, plus a packed binary file. Assuming there’s no caching, these could likely be replaced without updating your project

gonzos · September 11, 2018, 9:15pm

It’s not directly connected with this feature.
I have folder named /puzzleimgs under default assets/ folder.
That folder size has 45MB of images. If I include it in project with

<assets path="assets/puzzleimgs" />

with any attributes-embed, true,false
I got following error:

Called from ? line 1
Called from CommandLineTools.hx line 2091
Called from CommandLineTools.hx line 26
Called from CommandLineTools.hx line 146
Called from CommandLineTools.hx line 1876
Called from lime/project/ProjectXMLParser.hx line 26
Called from lime/project/ProjectXMLParser.hx line 59
Called from lime/project/ProjectXMLParser.hx line 2253
Called from lime/project/ProjectXMLParser.hx line 1645
Called from lime/project/ProjectXMLParser.hx line 665
Called from lime/project/ProjectXMLParser.hx line 796
Called from lime/project/ProjectXMLParser.hx line 786
Called from /usr/share/haxe/std/neko/_std/sys/FileSystem.hx line 68
Called from /usr/share/haxe/std/neko/_std/sys/FileSystem.hx line 59
Uncaught exception - std@sys_file_type
Build halted with errors.

There is no way to include this folder in my project and compile from FlashDevelop.
And as I said, I just copy it manually in production and load images with absolute path.
It’s OK. I can live with this.
But if there is way to compress 45MB of images I’ll be more happy

singmajesty · September 11, 2018, 10:23pm

It would be interesting for us to find the root cause of this issue

github.com

openfl/lime/blob/6.4.0/lime/project/ProjectXMLParser.hx#L786


		var files = FileSystem.readDirectory (path);
		
		if (targetPath != "") {
			
			targetPath += "/";
			
		}
		
		for (file in files) {
			
			if (FileSystem.isDirectory (path + "/" + file)) {
				
				if (Path.extension (file) == "bundle") {
					
					parseAssetsElementLibrary (path + "/" + file, targetPath + file, include, exclude, type, embed, library, glyphs, true);
					
				} else if (recursive) {
					
					if (filter (file, [ "*" ], exclude.split ("|"))) {
						
						parseAssetsElementDirectory (path + "/" + file, targetPath + file, include, exclude, type, embed, library, glyphs, true);

It looks like a bad path, but the trick is knowing what we’re getting there. A -verbose build might give us more detail about what’s being processed

gonzos · September 12, 2018, 11:28am

OK, I resolved issue with embed images. Issue was

bin/html5/haxe/ManifestResources.hx:1: character 1 : Malformed file. Source files must be encoded with UTF-8.

It looks like can’t parse non UTF-8 chars. After I changed some characters in file names, everything is OK.
I can use both:

<library id="assetlib" type="gzip" preload="false" embed="false" />
<assets path="assets/puzzleimg" library="assetlib" rename="puzzleimg" />

and

<assets path="assets/puzzleimg" rename="puzzleimg" embed="false" />

But if I want to get images from *pak library I get following error:

Uncaught TypeError: Cannot read property 'length' of null
    at haxe_io_Bytes.blit (Bytes.hx:616)
    at lime_utils_PackedAssetLibrary.getImage (PackedAssetLibrary.hx:160)
    at lime_utils_PackedAssetLibrary.getAsset (AssetLibrary.hx:160)
    at Function.lime_utils_Assets.getAsset (Assets.hx:148)
    at Function.lime_utils_Assets.getImage (Assets.hx:231)
    at Function.openfl_utils_Assets.getBitmapData (Assets.hx:111)
    at com_coffeebreak_screens_MainScreen.testImage (MainScreen.hx:82)
    at Lib.hx:272
    at haxe_Timer.t.run (Timer.hx:145)
    at Timer.hx:71

I tried all types of compression “zip”,“gzip”,“deflate”.

singmajesty · September 12, 2018, 6:33pm

What happens if you remove embed="false"?

gonzos · September 12, 2018, 8:26pm

Well, I’ve tried this solution and changing embed attribute from false to true causes the same error.
But, changing preload="true" make it works. No matter is embed set as true or false.
This is solution:

<library id="assetlib" type="gzip" preload="true" embed="false" />

But, since file is too big I get this on preload:
screenshot

singmajesty · September 12, 2018, 9:31pm

If you are able, perhaps you should split the assets into multiple libraries, but not use a packed library type. This should make it possible to Assets.loadLibrary and Assets.unloadLibrary bundles of assets at once, if you do not need them immediately. In general, the browser is happier with different image files than blobbed binary files in my experience

gonzos · September 12, 2018, 9:49pm

Yes, I’ll definitely go in that direction. Thanks.