BlitzmaxNG hanging on zipstream

Started by Ashmoor, July 18, 2020, 22:38:09

Previous topic - Next topic

Ashmoor


The following code makes BlitzmaxNG hand indefinitely:

Code (blitzmax) Select

Import koriolis.zipstream

Local mmTune:TSound = LoadSound("ZIP::music/music.zip//"+"example.ogg")

If mmTune=Null Then RuntimeError "can't load music"


Am I doing something wrong or is there something wrong with loading a 1mb ogg? I noticed that incbin does not work with NG always leading to out of memory errors when building.

The code above builds and output says "executing: zip test.exe" and just hangs. Is this an NG issue?

Can you guys give me an example of loading an ogg from a zip file?



Derron

#1
It is not hanging on the zipstream but on loading the OGG from the stream.

I created a file "ogg_incbin.bmx" with the following code:
Code (Blitzmax) Select

SuperStrict

Framework Brl.StandardIO
Import Brl.Audio 'TSound
Import Brl.OggLoader 'to decode OGG
Import koriolis.zipstream 'to load/stream zips
Import Brl.TextStream

Incbin "music.zip"

Local source:String = LoadText("zip::music.zip//ogg_incbin.bmx")
print "source: " + source

Local mmTune:TSound = LoadSound("zip::music.zip//"+"example.ogg")

If mmTune=Null Then Throw "can't load music"

print "loaded sound"


Within "music.zip" I placed the source code itself and an example.ogg file - the text is printed properly but it stucks on the LoadSound() command.


Edit - loading an incbinned ogg fails too (might need another import to work or so):

Incbin "example.ogg"

Local incbinTune:TSound = LoadSound("incbin::example.ogg")
if incbinTune then print "loaded incbinned tune"
if not incbinTune then print "failed loading incbinned tune"

This is because I use the "framework" - and to make the direct incbin load work here, I need to add "Import Brl.RamStream" too. Then it loads the incbinned ogg directly.


bye
Ron

Derron

#2
Code (Blitzmax) Select

SuperStrict

Framework Brl.StandardIO
Import Brl.Audio 'TSound
Import Brl.OggLoader 'to decode OGG
Import koriolis.zipstream 'to load/stream zips
'Import Brl.TextStream 'if you load a sample text file
Import Brl.RamStream

Incbin "music.zip"
Incbin "example.ogg"

'Local source:String = LoadText("zip::music.zip//ogg_incbin.bmx")
'print "source: " + source


'FILESYSTEM FILE
'works
Local oggStream:TStream = ReadStream("example.ogg")
If oggStream Then Print "loaded ogg stream"
If Not oggStream Then Print "failed loading ogg stream"

'works
Local streamTune:TSound = LoadSound( oggStream )
If streamTune Then Print "loaded sound from ogg stream"
If Not streamTune Then Print "failed loading sound from ogg stream"


'INCBIN FILE
'fails
Local incbinOggStream:TStream = ReadStream("incbin::example.ogg")
If incbinOggStream Then Print "loaded ogg stream from incbinned ogg file"
If Not incbinOggStream Then Print "failed loading ogg stream from incbinned ogg file"

'works
Local incbinTune:TSound = LoadSound("incbin::example.ogg")
If incbinTune Then Print "loaded sound from incbinned ogg file"
If Not incbinTune Then Print "failed loading sound from incbinned ogg file"

'this fails
Local incbinStreamTune:TSound = LoadSound( incbinOggStream )
If incbinStreamTune Then Print "loaded sound by stream from incbinned ogg file"
If Not incbinStreamTune Then Print "failed loading sound by stream from incbinned ogg file"


'INCBIN ZIP CONTENT
'works
Local incbinZipOggStream:TStream = ReadStream("zip?::music.zip//"+"example.ogg")
If incbinZipOggStream Then Print "loaded ogg stream from incbinned zip file"
If Not incbinZipOggStream Then Print "failed loading ogg stream from incbinned zip file"

'fails and STUCKS
Local incbinZipOggSound:TSound = LoadSound(incbinZipOggStream)
If incbinZipOggSound Then Print "loaded sound by stream from incbinned zip file"
If Not incbinZipOggSound Then Print "failed loading sound by stream from incbinned zip file"

'fails and STUCKS
Local incbinZipTune:TSound = LoadSound("zip?::music.zip//"+"example.ogg")
If incbinZipTune Then Print "loaded sound from incbinned zip file"
If Not incbinZipTune Then Print "failed loading sound from incbinned zip file"

Print "DONE."


Some test case - will raise an issue for it.


Edit: https://github.com/bmx-ng/brl.mod/issues/186

Ashmoor

Thanks Derron.

Incbin works if you include a few small files but I have 40mb worth of game music and the compiler throws an error. Here is the code:

Code (blitzmax) Select

SuperStrict

Framework Brl.StandardIO
Import Brl.Audio 'TSound
Import Brl.OggLoader 'to decode OGG
Import koriolis.zipstream 'to load/stream zips
'Import Brl.TextStream 'if you load a sample text file
Import Brl.RamStream

Incbin "music2/dream_on.ogg"
Incbin "music2/forest_of_f.ogg"
Incbin "music2/med1.ogg"
Incbin "music2/rom_mem.ogg"
Incbin "music2/rom_mem drums.ogg"
Incbin "music2/track1.ogg"
Incbin "music2/track2.ogg"
Incbin "music2/track3.ogg"
Incbin "music2/track4.ogg"
Incbin "music2/track5.ogg"
Incbin "music2/track6.ogg"
Incbin "music2/win_loop.ogg"


Print "all incbined"


Error:

Building incbin NG
[ 86%] Processing:incbin NG.bmx
[ 90%] Compiling:incbin NG.bmx.release.win32.x86.incbin.c
[ 95%] Compiling:incbin NG.bmx.gui.release.win32.x86.c
cc1.exe: out of memory allocating 536875007 bytes
Build Error: failed to compile (1) M:/Dropbox/Programming tests/.bmx/incbin NG.bmx.release.win32.x86.incbin.c
Process complete


I have 32Gb Ram and 64 free Gb on my system ssd.

What does the question mark do in "zip?::", does it replace "incbin::"?


Derron

"zip::" - it is of course a zip
"zip?::" - it might be a zip (at least koriolis.zipstream had both in it and I just tried it out)

The question mark is not a standard thing - at least I am not aware of it.

bye
Ron

Ashmoor

""zip?::" - it might be a zip (at least koriolis.zipstream had both in it and I just tried it out)"

Ok, nice.

So for incbined zip shouldn't it be
Code (blitzmax) Select

Local incbinZipOggStream:TStream = ReadStream("incbin::zip?::music.zip//"+"example.ogg")


Tried it, still gets stuck.



Derron

It "should" not ... but "could". So if you know it is a zip, just use "zip::" without the question mark.


But as written in the issue I raised... it fails as soon as "ogg_dec" (ogg vorbis decoder) tries to read from the provided memory stream of that zip extracted file (maybe some memory thingy).


bye
Ron

Ashmoor

Ok, thanks.

I wish I knew how to fix these things myself. Blitzmax is such a nice language.

Midimaster

#8
While translating my BlitzMax 1.50 app to BlitzMaxNG I have the same problem in reading OGG-file with Koriolis-ZipStream (source: https://github.com/maxmods/koriolis.mod ) from INBIN included DATEN.ZIP.
Code (BlitzMax) Select
Function ReloadAudioSounds(SoundName$)
' path is f.e.: ""ZIP::Incbin::DATEN.ZIP//Piano0.ogg""
For Local i%=0 To 36
Print i + " " + target
SoundTyp.Audio[i]=LoadSound(DatenPfad + SoundName + i + ".ogg")
Print i + " fertig"
Next
End Function

The App hangs without any error message while reading the first OGG file. Other file types are processed as expected.

Does someboy know, whether there is already a solution?

does somebody know whether there is a replacement, where I can add a ZIP file via INCBIN and the read the datas?


Of course I have found a (not very elegant) workaround:
Code (BlitzMax) Select
Function ReloadAudioSounds(SoundName$)
If FileExists(AppPfad + "tempSounds\" + "Piano0.ogg")=False
CreateDir AppPfad + "tempSounds"
EndIf

For Local i%=0 To 36
' workaround for INCBIN OGG problem 2021-06-29

Local source$ = DatenPfad + SoundName + i + ".ogg"
Local target$ = AppPfad + "tempSounds\" + SoundName + i + ".ogg"
CopyFile source , target

Print i + " " + target
' originalzeile:
'SoundTyp.Audio[i]=LoadSound(DatenPfad + SoundName + i + ".ogg")

SoundTyp.Audio[i]=LoadSound(target)
Print i + " fertig"
Next
End Function
...on the way to Egypt

Midimaster

Another strange behavoir in the context of koriolis ZipStream:

If you build your zip-file on the mac bigsur with the default finder option "Compress" the resulting ZIP-file can not be processed by koriolis zistream. As long as you build your ZIPs on Windows, f.e. with 7zip. it runs as exepected (except the ogg files, see post #8
...on the way to Egypt

Henri

Hi,

Brucey suggested using brl.io to mount a zip file (incbin is supported) and use it as a normal folder.

-Henri
- Got 01100011 problems, but the bit ain't 00000001

Midimaster

Yes I read this too, but I do not understand what it mean and how to do...

Is this a suggestion for a normal user or a possible way to remove the bug in BlitzMaxNG?
...on the way to Egypt

Henri

I'll see if I can check later today.

-Henri
- Got 01100011 problems, but the bit ain't 00000001

Derron

I tried to explain it via the github issue already. If it works out, we could paste it to here too.


bye
Ron