[Bmx Vanilla] Problem with loading big PNG (on Win10)

Started by wookie22, January 27, 2020, 17:22:17

Previous topic - Next topic

wookie22

This is strange. I have 13000x9000 RGBA png which needs 468MB of memory (I have 16GB on my machine+6GB VRAM). When I run my game right after system startup it loads OK but after opening few applications I've got EXCEPTION_ACCESS_VIOLATION. Then it shows even when I close those other applications. After running in Debug mode I found that it crashes on


Function CopyPixels( in_buf:Byte Ptr,out_buf:Byte Ptr,format,count )
MemCopy out_buf,in_buf,count*BytesPerPixel[format]
End Function


because out_buf=$00000000

And CopyPixels was called by LoadPixmapPNG in this fragment:


If pf
pixmap=CreatePixmap( width,height,pf )
Local rows:Byte Ptr Ptr=png_get_rows( png_ptr,info_ptr )
For Local y=0 Until height
CopyPixels rows[y],pixmap.PixelPtr(0,y),pf,width
Next
EndIf


what means that out_buf is pixmap.PixelPtr(0,y)


Any thoughts what to do?

Hidden Deep - 2D Action & Exploration Sci-Fi Thriller for PC/Mac/Linux
http://www.hiddendeepgame.com

Brucey

Probably your OS can no longer create a contiguous block of the required amount of RAM - due to memory fragmentation.

Questions to ask yourself -
  *  Do you need to have the whole image in memory at once? (if your screen cannot display it)
  *  Could you use some form of tiling?

fielder

can be nice to see if a bi-dimentional array of 13000x9000 make the same error.
if this is a map, maybe this kind of array (using bytes.. if bytes are enough) require a small amout of ram.

wookie22

Quote from: Brucey on January 27, 2020, 21:28:37
Probably your OS can no longer create a contiguous block of the required amount of RAM - due to memory fragmentation.

Questions to ask yourself -
  *  Do you need to have the whole image in memory at once? (if your screen cannot display it)
  *  Could you use some form of tiling?

Yes I'll do that. In fact that's what my game was to do with it - split that source png to 128x128 tiles then load to VRAM.

It is just hardly to believe that there is no contiguous block of 500MB on 16GB system with few small apps running but who knows what microsoft did...

Anyway I will split my source png to 2048x2048 I hope it will work (and then to 128x128)
Hidden Deep - 2D Action & Exploration Sci-Fi Thriller for PC/Mac/Linux
http://www.hiddendeepgame.com

Derron

Quote from: GfK on January 28, 2020, 21:48:30
Your texture will be 16384x16384 on the GPU.  This far exceeds even the 4096x4096 limitation of DX9, and the DX7 limit may be even less than that.

He is using TPixmap, not TImage - so maybe he is using it as some kind of "stream base" or so.


Maybe a little explanation of what you (@wookie22) are doing with the pixmap and why it is so big (is it coming from you?) could help us to find ideas on how to improve?



@ contiguos memory
So wasn't there some "virtual memory" file on Windows? Which grows as needed (except it was limited by the user - or disk space) ?
Else this problem could happen with a 10 or 20 mb image too (assume you create a painting software).
So I assumed that you request "100 Mb memory" and Windows maps that 100 Mb to multiple blocks - and redirects read/write to certain address offsets.


bye
Ron

wookie22

Quote from: Derron on January 29, 2020, 07:09:28
Maybe a little explanation of what you (@wookie22) are doing with the pixmap and why it is so big (is it coming from you?) could help us to find ideas on how to improve?

I have whole level as one big png (made in my own editor). So game loads it as TPixmap then split into array of TImages 128x128px. Game has pixel-perfect terrain destruction so it has to be that way (level must come as image not sort of tileset).
I think that I'll export level image as a set of 2048x2048 png's instead of one and then split them into 128x128 Timages one by one.

Yet I still don't understand why there is no 500MB free space on 16GB RAM. So Bmx don't use virtual memory?
 
Hidden Deep - 2D Action & Exploration Sci-Fi Thriller for PC/Mac/Linux
http://www.hiddendeepgame.com

Derron

QuoteGame has pixel-perfect terrain destruction so it has to be that way (level must come as image not sort of tileset).

So... you can't destroy the left of the bomb explosion circle on pixmap/image 1 and the other half on image 2?

You could even wrap all the pixmap manipulation in a new type:

Code (Blitzmax) Select

Type TMapPixmap
  Field pixmaps:TPixmap[][] 'x,y array - or single array with "wrapping"

  Method WritePixel(x:int,y:int, color:int)
     local pixmapCol:int = GetPixmapCol(x,y)
     local pixmapRow:int = GetPixmapRow(x,y)
     local pixmap:TPixmap = pixmaps[pixmapCol, pixmapRow]

     local pixmapX:int = x - pixmapCol*pixmapWidth
     local pixmapY:int = y - pixmapRow*pixmapHeight
     
     if pixmapX >= 0 and pixmapX < pixmapWidth and pixmapY >= 0 and pixmapY < pixmapHeight
          pixmap.WritePixel(pixmapX, pixmapY, color)
     endif
   End Method
...
End Type


Code incomplete and untested but you might get what I want to express.


bye
Ron

wookie22

#7
Quote from: Derron on January 29, 2020, 14:49:57
So... you can't destroy the left of the bomb explosion circle on pixmap/image 1 and the other half on image 2?

I can and there is no problem. Every small Timage has it's Pixmap so I can modify it and my game engine do this properly even if explosion affects many tiles. Only problem I was posting here is: regardless if it's right approach of wrong why I can't load 500MB image on 16GB RAM machine (and I still don't know why). All other problems are solved already but thanks anyway :)
Hidden Deep - 2D Action & Exploration Sci-Fi Thriller for PC/Mac/Linux
http://www.hiddendeepgame.com