[Discussion]: Changing OpenAL in favor to miniaudio?

Lime
1

OpenAL (or better labeled, OpenAL-Soft) is an audio playback API that is LGPL licensed, (see here)

While it has been using OpenAL for quite some time with no issues, we are presented with a couple of problems:

  1. Any kind of software/game that compiles to native targets (mostly affecting Windows/Mac/Linux) cannot do a clean static link executable if it uses any kind of audio
  2. OpenAL lacks and/or does not have an intuitive implementation of certain features such as stereo audio panning[1] or audio buffer editing[2] that the original Flash could provide within events, like SampleDataEvent.SAMPLE_DATA[3]

miniaudio, on the other hand, falls back on the same principle and solves some of the main concerns regarding previously, such as:

  1. it is Mit-0 licensed or even Unlicensed, so static linking could be doable
  2. As far as documentation goes, miniaudio supports StereoPanning and Pitching and Spatialization is also supported
  3. Since it’s a low-level API, there could be a way to add audio, it can also be open to implement streaming, and therefore implement samplingData events, that people could use to create their own effects like delay, reverb, pitch shifting without having to rely on a centric EFX plugin to implement what’s closest to the developer, or be exposed to a poor or unoptimized implementation that would lead to a loss of performance that could be avoided.

While, obviously this kind of API can be very liberating to implement, this also means rewriting the entire audio API aside from OpenAL (if the intended goal is to keep backwards compatibility for older games) and possibly implementing audio stuff from groundup, like audio mixing, volume, so on and so forth.

I thought it’d be discussed somewhere but I didn’t see an instance of it here, so might aswell “open” it?

  1. ↩︎

  2. ↩︎

  3. ↩︎

1 Like
2

Let me preface by saying that I have only a high-level knowledge of how we currently use OpenAL in Lime. I have not dug very deeply into our low-level implementation.

As I recall, we use MojoAL instead of OpenAL-Soft on native platforms where static linking is required. I don’t know if we technically allow forcing MojoAL instead of OpenAL-Soft on the other native platforms that support dynamic linking, but I’d be curious if you’ve tried that or not (and if a fellow contributor knows more about Lime’s support for MojoAL, please chime in!).

I remember that we added support for SampleDataEvent.SAMPLE_DATA in OpenFL 9.3. Is there any particular limitation in our existing implementation of SampleDataEvent.SAMPLE_DATA that you think would be improved if we switched to miniaudio instead?

Ultimately, I think that yes, we would be open to considering a switching to different library for audio in Lime to replace OpenAL-Soft and MojoAL. Especially if it helps us to implement things that Flash could do that we can’t do in OpenFL yet.

We’d probably want to wait until the next major version of Lime to switch over to another audio backend, but we’d commit it into a branch before that (which would be 9.0.0-dev at the time I’m writing this). It would be absolutely required to wait for a major version if any Haxe APIs need to be modified to accommodate the new library’s differing C/C++ API surface. However, there is always the chance that unexpected issues could pop up after making such a drastic change, even if it’s limited to purely internal modifications, so best to keep Lime 8 on the existing architecture to avoid too much disruption.

Would you be able to personally implement miniaudio in Lime and create a PR for it? Ideally, that would include being able to test the cpp target everywhere, including Windows, macOS, Linux, Android, and iOS. However, others can certainly help with that. Additionally, we’d need to support the HashLink and Neko targets too.

Or is this topic more to get a conversation started? I don’t want to be discouraging, but when you suggest big changes like this, it often requires you to be motivated enough to do the hard work yourself. Not because we don’t want to do it. But because everyone contributing to Lime and OpenFL are volunteers, and nobody gets paid to work on these projects full-time (though, I’d certainly love to!). We’re not necessarily stubbornly sticking to our current set of third-party libraries. We can be convinced to switch to better ones. We just don’t have time to get to everything that we wish we could, no matter how nice an improvement it would be, you know?

2 Likes
3

As I recall, we use MojoAL instead of OpenAL-Soft on native platforms where static linking is required

My experiences with Mojo-AL were… a bit disappointing
I knew that there was something else underlying, but mostly ended up with crashing the whole thing for no specific reason.

Aside from personal discomfort with it, since it’s a repository developed aside, the repository lacks from all the extensions possible, and it also exposes the risk of imparity between the same platform exported just because of static linking.

Is there any particular limitation in our existing implementation of SampleDataEvent.SAMPLE_DATA

I was tackling a bit with the sample data, I apologize, I did not know about this considering the limitations within OpenAL and I was surprised that there was an implementation for it, considering how OpenAL works, but from what I could see, SampleData seems to be restricted to the amount of data that you give, from 2048 to 8192. my first impression is that the test (provided by the Air SDK) isn’t exactly how you’d expect, plus you cannot do complex audio stuff like extracting the audio from a Sound class and use it in this socket. Furthermore, HTML5 doesn’t seem to work, but that’s aside from that, since it’s HowlerJS and all that.

With MiniAudio, for example, the choppiness would be resolved almost immediately since a lot of use cases for this is Streaming.

Also something to consider, is that when I meant functions like SampleData, I meant moreso everything surrounding raw audio data, like Sound.extract[1] or SoundMixer.computeSpectrum[2], which I believe those are important to do things like audio effects that do not require a third party to implement those.
…While there is such a thing in OpenAL-Soft (See here), we cannot implement this in static linking, since MojoAL has a small repertory of extensions[3] and ALC_SOFT_loopback isn’t an extension supported with OpenAL, so we’re kinda screwed in that regard.

Would you be able to personally implement miniaudio in Lime and create a PR for it

I would love to implement something like that, I was wondering if miniaudio has been discussed previously somewhere or if someone had already started with it.
Ultimately, I don’t expect this to be a day to night change, since it’s obviously rewriting the whole audio setup from OpenAL in favor to miniaudio, and even if I wanna tackle that, I suppose we both know that it’ll take quite some time to do it.

Or is this topic more to get a conversation started?

It’s a bit with both paragraphs, It is best to leave this kinda thing documented for a future peer that might see this and be motivated enough to implement it themselves faster than what my hands could write. I wanna know the thoughts of it and while I’m busy to start right now, I intend to test more on about it

  1. ↩︎

  2. ↩︎

  3. ↩︎

4

The HTML5 target will require a completely different implementation of any audio-related API, so we should consider it outside the scope of this discussion.

If miniaudio makes it easier to implement these additional APIs, that’s certainly a good technical reason to consider using it.

I don’t believe any discussion has taken place about replacing or providing an alternative to the current OpenAL backend within recent years that I’ve been involved in the project. Similarly, I recall no one mentioning an interest in using miniaudio in Lime before today, let alone actually starting on an implementation using it. You’ve broken all new ground here, by my recollection.

The closest related thing that I remember any discussion about is adding support for the MP3 format on native platforms. There were MP3 patent issues in the past that we had concerns over, but they are all expired now, so that is no longer the case. Now, native MP3 support just needs someone to take the lead and make it happen. I believe that minimp3 (no relation to miniaudio, as far as I can tell) was considered a good candidate library to use, especially since HashLink already seems to be using it. However, it’s my understanding that codec support is separate from libraries like OpenAL and miniaudio, so I’m just mentioning this as a point of interest.

5

The HTML5 target will require a completely different implementation of any audio-related API, so we should consider it outside the scope of this discussion.

I understand that, but I mostly don’t wanna point an issue that seems to be unrelated to native.
Interestingly enough, miniaudio does seem to have support over HTML, but it seems to be something to be implemented alongside WASM[1]

If miniaudio makes it easier to implement these additional APIs, that’s certainly a good technical reason to consider using it.

I believe miniaudio would be easier to implement since that, considering that you’re going to mix sounds into one anyway, it’ll be more structured to control such things

Now, native MP3 support just needs someone to take the lead and make it happen

Interestingly enough, miniaudio does support MP3, at least from documentation[2]

I wanna emphasize that I, myself haven’t touched miniaudio to be confident with it, but as far as I understand, since miniaudio tackles both C and CPP, I don’t think it should be a problem?

I don’t wanna make a confident statement, and right now, since it seems that some people weren’t taking interest in miniaudio (I’m surprised), from my end I could just say that I’ll research on mini audio with more extension, but should be worth to take a look over.

Since it’s a breaking change, I don’t know how to feel about those who used OpenAL for specific things like audio stuff and all, maybe add a separate audio define?

either way, I think it would be interesting to have something like that implemented, streaming wise and all

  1. ↩︎

  2. ↩︎

6

Yeah, for folks interacting directly with OpenAL in their projects, we would probably want to continue providing it (perhaps requiring a define) for some period of time.

How exactly that should be done isn’t super important right now, in my opinion. I’d rather see a working miniaudio implementation first, and worry about details like what to do with OpenAL later.

1 Like
7

I would like to be clear: MojoAL isn’t a bespoke OpenAL implementation, its an abstract over SDL that implements the API surface of OpenAL 1.1. This is incredibly useful because this works on every platform that SDL is compatible with (including consoles like Nintendo Switch).

If you have crashes from MojoAL, please be sure to create an issue report on github with steps for us to reproduce. From my personal experience, it is very stable and well supported.