Hello everybody!
I’m trying to asses OpenFL + Starling performance on HTML5 before moving a game project towards production, and I’m getting some erratic results.
I the absence of a better benchmark tool, I’ve used the Benchmark section of the official StarlingDemo sample app (https://github.com/openfl/starling/tree/master/samples/demo). I know it’s a synthetic test and won’t reflect exactly the true performance of your typical game, but it should still give a fair clue on how the renderer is doing.
Here are my results so far https://docs.google.com/spreadsheets/d/1eNgiMpIK7pIqP_8Sehg5ts_RWWzlY_baZvu9qZcTxts/edit?usp=sharing
The numbers represent the results as given by the app i.e. maximum number of sprites rendered at 60 fps. Each results cell contains a range of number I got from running the test multiple times. I’ve described in the doc the systems (one fast and one slow PC). I’ve paired different versions of Starling (2.4.2 and 1.8.13) with different versions of OpenFL & Lime (8.3+6.4 and 8.7+7.2), and tested in both Chrome and Firefox, latest versions. Both OpenFL & Starling where taken from haxelib official versions.
I’ve also included in the doc links to the builds used in the test, feel free to run them yourself. I didn’t mess with the original code of the StarlingDemo app, except to fix some minor compile errors at cross-combinations (new starling + older OpenFL or vice-versa)
Am I missing something? Why is the performance so erratic? To give just 2 examples:
- Fast PC - Starling 2.4 + OpenFL 8.7 + Lime 7.2 (Release build): Chrome 26172 vs. Firefox 5602 -> a 467% difference between browsers (if I run https://browserbench.org/JetStream/ i get 225 for Chrome and 218 for Firefox, so like 3% difference)
- Slow PC - Starling 2.4, Chrome (Release builds): OpenFL 8.3 + Lime 6.4: 4605 vs OpenFL 8.7 + Lime 7.2: 707 -> a 651% difference between different OpenFL versions, of the same system and browser.
I don’t see how this StarlingDemo app benchmark would not be relevant, in fact I do observe some correlated resultants in my real game too.
There is this GitHub issue opened on the same subject https://github.com/openfl/openfl/issues/1695
Performance in HTML5 games is a mission critical aspect. Right now, this doesn’t look very encouraging. Can we do anything about it? Where could I start? Is this specific to Stage3D, while regular OpenFL rendering is much more consistent? Is Stage3D actually slower then regular OpenFL display list using bitmaps from atlases i.e. making Starling irrelevant in the OpenFL world?
Thanks!
