MiniAudio-Wrapper for BlitzMax enables WASAPI Playback and Recording + MP3

Started by Midimaster, April 23, 2021, 14:12:00

Previous topic - Next topic

Midimaster

I proudly present a new Audio-Interface for BlitzMax:
MiniAudio Audio-Library wrapper for BlitzMax NG

This interface will enable...

  • Audio-Playback with low latency of 30msec via WASAPI

  • Realtime Samples access of already playing samples

  • MP3-Decoding

  • Easy Audio-Recording with low latency

  • Full Duplex Audio

  • Converting TAudioSamples: Format Hertz


MiniAudio is a C-Library for various OS: Mac, Linux, Windows,... All informations and source code can be found here:

https://miniaud.io/index.html

For BlitzMax you need the mima.mod/miniaudio.mod-module, which you will find in the GitHub repository.

https://github.com/MidimasterSoft/BlitzMax-Miniaudio-Wrapper

The GitHub contains:
- MiniAudioWrapper.c
- MiniAudio.bmx
- MiniAudioX.h
- Documentation for BlitzMax help
- A lot of code examples

In the example you will also find sample audio files for testing.

This is Version 1.25 from 2021-06-08.

In this topic you will find a manual and learn how to use the library.

This are the functions which are already working:


  • Playback MONO STEREO, 3-32 tracks

  • Opening MP3-FLAC- and WAV files with audioformats bejond 16bit

  • Capture Recording MONO-STEREO

  • Full Duplex: Capture and Playback at the same time

  • Loopback: Record what is "on the speakers"

  • All backend devices on all plattforms are supported incl. WASAPI on Windows

  • Formats: 8bit, 16bit, 32bit and 32bit-float

  • Samples-Rates from 8kHz-192kHzHz

  • Low Latency of 30msec

  • Convert TAudioSamples to formats Int32, Float32 and more

  • Convert TAudioSamples  Frequency

  • runs also stabil in BlitzMax DEBUG mode now 

  • Save WAV-files in various formats  upto 32 tracks

  • access to USB hardware devices with upto 32 capture/playback channels 

  • CallBack-Access, Multithread



content of the GitHub:

MiniAudio.bmx
the BlitzMax-Part of the wrapper. Shows all avaiable functions


MiniAudioWrapper.c
the C-Part of the wrapper. Do not change anything.

MiniAudioX.h
the library. Do not change anything.

OpenClose.bmx
First demo app. Opens the device for playback and shows some informations, then close it immediately.


PlayNoise.bmx
Second demo app. Produces noise only when the mouse is pressed.


MiniAudioTestLatency.bmx
Enables you to recognize the latency of the device. Listen to the time delay between the rhythmic crackl and your mouse click


StereoSinus
Demonstrates how to create Samples in Realtime. Uses STEREO to create two different waveforms.


FreeAudioMove.bmx
(including Test.ABC.ogg)
Demonstrates how to simply playback an existing Audiofile. Uses 16bit-MONO and 12kHz and standard unsigned SHORTs.


Reverb.bmx
(including Test.ABC.ogg)
Demonstrates how to manipulate an existing Audiofile in Realtime. Uses a 16bit-MONO file with 12kHz, but internal 32bit-signed-Int-Samples


CaptureRecording.bmx
(including Test.ABC.ogg)
Demonstrates how to capture an 16bit-MONO-INPUT device with 48kHz, save the samples to a TAudioSample. Also DUPLEX is activated to listen immediately to the incoming signal.


LoadMP3.bmx
(including Test.ABC.mp3)
Demonstrates how to open a MP3-file for TAudioSample and TSound, PlaySound.


ConvertAudioSample.bmx
(including Test.ABC.ogg)
Demonstrates how to convert a 16bit-Audio to a 32bit-float Audio-Format (values from -1.0 .. + 1.0).


SaveTAudioSample.bmx
(including Test.ABC.ogg)
Demonstrates how to save a  BlitzMax AudioSample to a WAV file. Works with MONO and STEREO, 8bit and 16bit and free SampleRate.


WriteWavFile.bmx
Demonstrates how to save any TBank content as a continously Audio-Stream to a WAV file. Works with 1-32 channels, 8bit and 16bit 32bit and 32bit-float and free SampleRate.



CaptureRecording_II.bmx
Demonstrates how to capture high quality STEREO IN with 48kHz and 32bit-FLOAT, and save the result to a WAV-file.


CaptureRecording_III.bmx
Demonstrates how select Hardware-Interface and capture 1-32 IN-channels of an USB-Audio-Interface with 48kHz, modifies the samples and reduce them to STEREO for immediately DUPLEX-monitoring on computer-speakers. At the end a 8-track audio file will be saved.


Installation
Copy the content of the GitHub into the MOD-folder of BlitzMax NG
The path will be mima.mod/miniaudio.mod
See the example PlayNoise.bmx to see how to use the library
Report here at Syntaxbomb, if you find bug or have questions.


Download

https://github.com/MidimasterSoft/BlitzMax-Miniaudio-Wrapper
...on the way to Egypt

Midimaster

#1
API of MiniAudio-Wrapper


Installation
You need to extract the mima.mod to the BlitzMax Mod-directory:


Miminum Code

The start-up needs 1-3 steps:

1. create an TMiniAudio-object
2  open a device type with parameters

Code (BlitzMax) Select

Global MiniAudio:TMiniAudio=New TMiniAudio
MiniAudio.GetDevice( MiniAudio.PLAYBACK, Miniaudio.FORMAT_S16, 2, 48000, MyCallBack)


You can  open the device for  4 different jobs:

  • PLAYBACK

  • CAPTURE

  • DUPLEX

  • LOOPBACK

PLAYBACK opens the default device for Audio-Out only.

CAPTURE opens the default In-device for recording only.

DUPLEX is CAPTURE and PLAYBACK at the same time. Here you can manipulate the incoming signal and immediately forward it to the speakers. Like a PlugIn.

LOOPBACK enables you to "capture the speakers". This is for recording game's or browser's audio


In the next step you have to decide your parameters

  • SampleFormat

  • Channels

  • Hertz

  • Your Callback-Function

SampleFormat is one of the following:

  • FORMAT_U8

  • FORMAT_S16

  • FORMAT_S24

  • FORMAT_S32

  • FORMAT_F32

FORMAT_U8
is the old 8bit audio format with poor quality

FORMAT_S16
is the common 16bit audio format of nearly all audio files and the default BlitzMax-Format. But be careful! BlitzMax only knows only unsigned SHORTs as variable types.

FORMAT_S24
is a 24bit audio format, which is mainly used in prof Music-Equiment.

FORMAT_S32
is a 32bit audio format, which fits perfect to BlitzMax's INTEGER variable type.

FORMAT_F32
is the new Float-Format, which WASAPI uses intern and apps like AUDACITY too.

The possible number of Channels

... depent on your hardware. The number of channels must fit to your audio hardware. Normaly 2 is the best choice for STEREO or 1 for MONO. Higher values only if your device is able to handle.

The Hertz frequency
... is related to your audio file or audio device. But you can also choose any value between 8000 and 96000, which will have side effects. Typical values are: 44100 for CDs MP3s or 48000 for music studios.

As last parameter you tell the name of your callback -function

The Callback

You need a function where you can exchange samples with MiniAudio. This function is defined by you, but called from MiniAudio every 10msec. So keep your code inside to a minimum. Do no PRINT or DRAW... commands and do never END your app inside this callback. Tell this name of the callback as  4th parameter.


our Callback function MUST look like this:
Code (BlitzMax) Select
Function MyCallBack(a%, PlaybackBuffer:Short Ptr, RecordingBuffer:Short Ptr, Frames%)
' do something with the samples
End Function


a%
...is not used at the moment

Buffer:Short Ptr
...is the adress of the buffer where you send Samples to Audio-Out

RecordingBuffer:Short Ptr
...is the adress of the buffer where you find the capture samples.

Frames%
...tells you the number of frames that the buffer contains this time. Be careful! This may vary!!!

In the case of only PLAYBACK or CAPTURE the other buffer is not allowed to use!

You are allowed to change type of both pointers (independend) to fit to your needs and selected device typ:

Code (BlitzMax) Select
... Buffer:Byte Ptr
... Buffer:Short Ptr
... Buffer:Int Ptr
... Buffer:Float Ptr



Minimum Code: OpenClose.bmx
Code (BlitzMax) Select
SuperStrict
Import mima.miniaudio

Graphics 800,600
' Setup of the device:
Global MiniAudio:TMiniAudio=New TMiniAudio
MiniAudio.GetDevice( MiniAudio.PLAYBACK, Miniaudio.FORMAT_S16, 2, 48000, MyCallBack)

MiniAudio.StartDevice()
Repeat

Until AppTerminate()
Miniaudio.KillDevice()
End

Function MyCallBack(a:Byte Ptr, PlayBuffer:Short Ptr, RecordingBuffer:Short Ptr, Frames%)
' do something with the samples
End Function
...on the way to Egypt

Midimaster

How to send Samples to Miniaudio?

This is easy now. you only have to fill the Buffer in the Callback. MiniAudio calls your Callback-function every 10msec. This is the moment when you send the next datas to MiniAudio.


Sending nothing:

This is an allowed option. If you send nothing, but only return the function, MiniAudio will send SILENCE to the speaker:
Code (BlitzMax) Select
Function MyCallBack(a%, PlaybackBuffer:Short Ptr , RecordingBuffer:Short Ptr , Frames%)
' do nothing causes silence
End Function



Or you can send anything:

In this case random number, which will result in white noise:
Code (BlitzMax) Select
Function MyCallBack(a%, PlaybackBuffer:Short Ptr , RecordingBuffer:Short Ptr , Frames%)
' here you manipulate the sound:
For Local i%=0 To frames-1
PlaybackBuffer[i] = Rand(-32000, 32000)
Next
End Function


The variable Frames% tells you how many frames you should send now. Be careful! This means not Samples, or Bytes, but Frames. A frame means 1 sample if MONO is selected or 2 samples if STEREO is choosen. Here is a table

it shows the avaiable FORMAT types and how long one frames is.
Format    Channels   Samples  Length  best pointer
--------------------------------------------------
FORMAT_U8   MONO        1       1     1 BYTE   
   "        STEREO      2       2     2 BYTEs

FORMAT_S16  MONO        1       2     1 SHORT 
   "        STEREO      2       4     2 SHORTs   
     
FORMAT_S24  MONO        1       3      ---     
   "        STEREO      2       6      ---   

FORMAT_S32  MONO        1       4     1 INT     
   "        STEREO      2       8     2 INTs   

FORMAT_F32  MONO        1       4     1 FLOAT       
   "        STEREO      2       8     2 FLOATs


This  means that in a FORMAT_U8   MONO you have to loop...
Code (BlitzMax) Select

Function MyCallBack(a%, Buffer:Byte Ptr , RecordingBuffer:Byte Ptr , Frames%)
For Local i%=0 To frames-1
PlaybackBuffer[i]=Rand(255)
Next

...to fill the buffer

but in  a FORMAT_S16 STEREO you have to loop...
Code (BlitzMax) Select

Function MyCallBack(a%, Buffer:SHORT Ptr , RecordingBuffer:Short Ptr , Frames%)
For Local i%=0 To frames-1
PlaybackBuffer[2*i +0] = Rand(-32768, +32768)  ' LEFT
PlaybackBuffer[2*i +1] = Rand(-32768, +32768)  ' RIGHT
Next



...on the way to Egypt

Henri

Hi Midimaster,

good work based on what I see. I always thought MiniAudio was a good candidate for wrapping for Blitzmax.

-Henri
- Got 01100011 problems, but the bit ain't 00000001

Midimaster

#4
Load MP3-Files

You also can open a MP3 file in BlitzMax with the MiniAudio-Library. The wrapper returns a TSound so you can directly play it with PlaySound():

Load_MP3.bmx
Code (BlitzMax) Select
SuperStrict
Import mima.miniaudio

Graphics 400,200

' Setup of the device:
Global MiniAudio:TMiniAudio=New TMiniAudio

Local MySound:MiniAudio.LoadSound("TestABC.mp3")
PlaySound MySound

Repeat
DrawText "Open and Play MP3 file",100,100
Flip
Until AppTerminate()
End

(Code and Files in the ZIP-file at first post of this topic)


You also can open a MP3 file as a TAudioSample for manipualting the Samples. The wrapper returns a TAudioSample so you can indirectly play it with PlaySound():

LoadAudioSample_MP3.bmx
Code (BlitzMax) Select
SuperStrict

Import mima.miniaudio
Graphics 400,200

' Setup of the device:
Global MiniAudio:TMiniAudio=New TMiniAudio

Local MySample:TAudioSample = MiniAudio.LoadAudioSample("TestABC.mp3")
Local MySound:TSound=LoadSound(MySample)
PlaySound MySound

Repeat
DrawText "Open and Play MP3 file",100,100
Flip
Until AppTerminate()
End

(Code and Files in the ZIP-file at first post of this topic)
...on the way to Egypt

iWasAdam

tested on a mac and it crashes at this point:
playnoise.bmx

' now start it:
'MiniAudio.StartDevice()

Midimaster

*** DONE  ********************************
The problem iWasAdam descriped was related to the
BlitzMax Debug mode and could be solved with the version 1.18
*****************************************

Thank you for testing it on a MAC. This is really helpful, as the Windows versions looks like running perfect.

to keep the worklog free from long discussion I would like to move this to the related topic here:

"How to write a wrapper"


https://www.syntaxbomb.com/index.php/topic,8388.msg347049813.html#msg347049813
...on the way to Egypt

Midimaster

Today I show a very interesting realtime manipulation:

Real Time Music Speed Changing

The algo is nearly the same like in the Reverb.Bmx example. A given Ogg.file will be manipulated before sending it to the speakers.

Today I have a standalone EXE for you. A narrator is telling anything and you can change his speed between -50% to +50%. The audio-file is included via incbin.

...on the way to Egypt

Midimaster

This shows how to convert the old school BlitzMax audio format to modern formats, how WASAPI use it:

Convert Audio Formats

The MiniAudio-Wrapper enables to convert any audio format to any other format. For BlitzMax users is important to convert the mainly used 16bit signed SHORT format, because it is always mis-interpreted in BlitzMax. BlitzMax only knows unsigned SHORT. So the real values in audio files from -32767 to +32767 are show in Blitzmax as (wrong) values  0 upto 65535.

With the converter you can convert them to new 32bit-float which knows values from -1.00 to +1.00. and are used i all professional audio software.

Convert Frequencies

Also possible is now to convert any frequency into a new one. The algorithm selects optimized dithering for any conversion direction and any frequency values.

This example shows how to convert MONO-16bit-UNSIGNED-SHORT to MONO-32bit-FLOAT:

ConvertAudioSample.bmx
Code (BlitzMax) Select
SuperStrict
Import mima.miniaudio

Graphics 800,600

' Setup of the device:
Global MiniAudio:TMiniAudio=New TMiniAudio

Local Source:TAudioSample=LoadAudioSample("TestABC.ogg")

Global ResultBank:TBank
ResultBank= MiniAudio.ConvertAudioSample(Source, Miniaudio.FORMAT_F32, 1, 12000)

'show 500 samples in old and new format:

For Local i%=1000 To 1500
Print  i + ".Sample  Source:" +  SoundBank.PeekShort(i*2) + "-->" + Resultbank.PeekFloat(i*4)
Next
End




...on the way to Egypt

Midimaster

#9
I converted the wrapper code to a module now.

Write...
Code (BlitzMax) Select
Import mima.miniaudio
to use it in your code.

I also added docs for use in the BitzMax Help. And you will find an example folder  in the mima.mod/miniaudio.mod  directory
...on the way to Egypt

Midimaster

#10
Again an update for the MiniAudio-Wrapper:

Load FLAC files

You now can load audio files in *.FLAC format for playing it with PlaySound or load it into a TAudioSample(). This enables for the first time to load 24bit audio-files in BlitzMax. The TAudioSample will be converted automatically to SF_STEREO16LE or SF_MONO16LE.

Code (BlitzMax) Select
Import mima.miniaudio

' Setup of the device:
Global MiniAudio:TMiniAudio=New TMiniAudio

Local MySound:TSound = MiniAudio.LoadSound("TestABC.flac")
PlaySound MySound

'also possible:
Local MySample:TAudioSample = MiniAudio.LoadAudioSample("TestABC.flac")
Local MySound:TSound=LoadSound(MySample)
PlaySound MySound



Load WAV files

You now can load audio files in *.WAV format with different format bejond SF_STEREO16LE for playing it with PlaySound or load it into a TAudioSample(). This enables for the first time to load 32bit audio-files in BlitzMax.

Miniaudio enables to load Wav-files also when they are in the 32bit-Integer-Format or  32bit-Float-Format.

The TAudioSample will be converted automatically to SF_STEREO16LE or SF_MONO16LE.


Code (BlitzMax) Select
Import mima.miniaudio

' Setup of the device:
Global MiniAudio:TMiniAudio=New TMiniAudio

Local MySound:TSound = MiniAudio.LoadSound("TestABC.wav")
PlaySound MySound

'also possible:
Local MySample:TAudioSample = MiniAudio.LoadAudioSample("TestABC.Wav")
Local MySound:TSound=LoadSound(MySample)
PlaySound MySound


MiniAudio in DEBUG mode

With a modification of the miniaudio.h made by COL the Miniaudio Wrapper now runs stabil also in the BlitzMax DEBUG mode. Thanks a lot to COL for his support. The modified library is now named minaudiox.h and has 6 changes which you can find when searching for the keyword "BBThread"


Download

You will find the download of the last version of the wrapper in post#1:
https://www.syntaxbomb.com/index.php/topic,8419.msg347049742.html#msg347049742




what is next?
the next step will be to select various hardware devices, f.e. USB-24bit-Audio-Devices, USB-Studio-Equipment.

...on the way to Egypt

Midimaster

new version 1.20 is online.

USB Hardware Capture

Now you can record upto 32 audio channels at the same time or playback upto 32 channels. This makes possible individual sounds for each speaker of an 7.1 systems or drive 24track mixing console for musicians. 
Code (BlitzMax) Select
...
' Setup of the device:
Const  HERTZ:Int = 48000

Global MiniAudio:TMiniAudio=New TMiniAudio
Miniaudio.SelectDevices(0,1)
MiniAudio.GetDevice_II( MiniAudio.DUPLEX, Miniaudio.FORMAT_S32, 2, Miniaudio.FORMAT_S32, 8, HERTZ, MyCallBack)


Global Source:TAudioSample=CreateAudioSample( HERTZ*61,  HERTZ, SF_MONO16LE)  '60 seconds
Global SoundBank:TBank=CreateStaticBank(Source.Samples, Source.Length)

Global WritePointer:Int

....
Repeat

Until...


Function MyCallBack(a%, PlayBackBuffer:Int Ptr, RecordingBuffer:Int Ptr, Frames%)
For Local i%=0 To frames-1
Local valueL%=RecordingBuffer[8*i]+RecordingBuffer[8*i+1]+RecordingBuffer[8*i+2]+RecordingBuffer[8*i+3]
Local valueR%=RecordingBuffer[8*i+7]+RecordingBuffer[8*i+7]'+RecordingBuffer[8*i+6]+RecordingBuffer[8*i+7]
PlayBackBuffer[2*i]= valueR
PlayBackBuffer[2*i+1]=valueR
SoundBank.PokeShort(WritePointer+2*i , valueR/65536)
Next
    WritePointer = (WritePointer + 2*Frames) Mod (HERTZ*60)
End Function




TAudioSample WAV Saving

Now you can simply save your TAudioSample as a WAV file. These BlitzMax audio-formats supported:SF_MONO8, SF_MONO16LE, SF_STEREO8, SF_STEREO16LE

Code (BlitzMax) Select
' save_taudiosample.bmx
SuperStrict
Import mima.miniaudio
' Setup of the device:
Global MiniAudio:TMiniAudio=New TMiniAudio

Global Sample:TAudioSample=LoadAudioSample("TestABC.ogg")

MiniAudio.SaveTAudioSample("TestABC.wav", Sample)

End





Multitrack WAV Saving
but you can also  save multitrack captured samples or  save multitrack (1-32 channels) TBanks as WAV. All audio-formats supported. Also continous audio streams can be saved as WAV periodically:

Code (BlitzMax) Select
' save_wav_file.bmx
SuperStrict
Import mima.miniaudio
' Setup of the device:
Global MiniAudio:TMiniAudio=New TMiniAudio

Global Stream:MMStreamID= MiniAudio.OpenWavFile("test.wav" , Miniaudio.FORMAT_F32 , 3 , 44100 )
Print "StreamID=" + Stream.Id
Global mode%, time%, Part%
While Part<10
If time < MilliSecs()
time = MilliSecs() + 500
Part=Part+1
PartRecord
Print "recording part no. " + Part
EndIf
Delay 100
Wend
MiniAudio.CloseWavFile(Stream )
End

Function PartRecord()
mode = (mode+1) Mod 3 ' only for demonstration of 3 channels

Local Bank:TBank=CreateBank(5000*4*3+12)    ' 5000samples * 32bit * 3tracks + safetybytes
Global FrameLength% = 4*3  ' 32bit*3tracks
For Local I%=0 To 4999
Local pos:Int=i*FrameLength
Local v#=Sin(i)/2
Bank.PokeFloat(Pos+mode*4, v)
Next
MiniAudio.WriteWavFile(Stream, Bank, 5000)
End Function



...on the way to Egypt

Scaremonger

Hi,
This is an awesome piece of work.

Just tested it on Linux and it fails to load the module because of filename case sensitivity. If you rename the module filename from "MiniAudio.bmx" to "miniaudio.bmx" it works great.

Si...

iWasAdam

Please link to the latest version in your posts - not point to another post where it becomes difficult to work out where to download :(

If it's now at version xxx
put a direct download to Vxxx :)

GW

Tried the mod. The samples started playing though the speaker on one my monitors despite headphones. The lib needs to pick the default device, not the first enumerated one.