[bb] EFFIcient Map (etc) Saving by Streaksy [ 1 week ago ]

[BlitzBot]

Title : EFFIcient Map (etc) Saving
Author : Streaksy
Posted : 1 week ago

Description : Say you want to save  a 2D map of tiles or 3D map of blocks to a file, for a common example.  If there are less than 256 tiles/blocks then you save as bytes.  If there are more then you will use Shorts or Ints, but that is wasteful if there are, say, 300 tiles/blocks because most indices will be lower than 256 and the extra byte is/are a waste of disk space and loading time most of the time.  Know what I mean?

I came up with this method of writing values to files.  Here are the specs:

 * Values 250 and under save as one byte
 * Values 502 and under save as two bytes
 * Values 65534 and under save as three bytes
 * Values 16777215 and under save as four bytes
 * Values 2147483648 and under saves as five bytes
 * Negative values write an extra byte

So, the lower the value, the fewer bytes it takes up.  Negative values take up a whole extra byte but when saving grids of map info - which this was primarily made for - negative values won't even be used, but I thought I'd support them anyway.

I hope you find it useful.  Let me know if you do.

Code :

Code: BlitzBasic

1. 'Effi is an efficient file value storage method that works best with small values.
2.  
3. ' * Values 250 and under save as one byte
4. ' * Values 502 and under save as two bytes
5. ' * Values 65534 and under save as three bytes
6. ' * Values 16777215 and under save as four bytes
7. ' * Values 2147483648 and under saves as five bytes
8. ' * Negative values write an extra byte
9.  
10.  
11.  
12. Global maxthrib=(256*256*256)-1
13.  
14.  
15.  
16.  
17.  
18.  
19.  
20.  
21.  
22.  
23.        
24. ' FILE READING / WRITING
25.  
26.  
27.  
28.  
29.  
30. Function WriteEffi(s:TStream,val:Long)  'Writes up to the value of an unsigned integer including negative sign  (-2147483648 to 2147483648)
31.         If val<0 Then WriteByte s,255;val=Abs(val)                                                      'CHR 255 = NEGATIVE
32.         If val=<250 Then WriteByte s,val;Return                                                         'CHR<250 = BYTE
33.         If val=<251+251 Then WriteByte s,251;WriteByte s,val-251;Return         'CHR 251 = BYTE+251
34.         If val=<65535 Then WriteByte s,252;WriteShort s,val;Return                      'CHR 252 = CONTROL BYTE + UNSIGNED SHORT
35.         If val=<maxthrib Then WriteByte s,253;WriteThrib s,val;Return           'CHR 253 = CONTROL BYTE + UNSIGNED 3-BYTE
36.         WriteByte s,254;WriteUSInt s,val;Return                                                         'CHR 254 = CONTROL BYTE + UNSIGNED INT
37. End Function
38.  
39.  
40. Function ReadEffi:Long(s:TStream)       'Reads up to the value of an unsigned integer including negative sign  (-2147483648 to 2147483648)
41.         Local sign=1
42.         Local b=ReadByte(s)
43.         If b=255 Then sign=-1;b=ReadByte(s)
44.         If b=<250 Then Return b*sign
45.         If b=251 Then Return (251+ReadByte(s))*sign
46.         If b=252 Then Return ReadShort(s)*sign
47.         If b=253 Then Return ReadThrib(s)*sign  'unsigned 3-byte value
48.         If b=254 Then Return ReadUSInt(s)*sign  'unsigned integer
49. End Function
50.  
51.  
52.  
53.  
54. Function WriteThrib(s:TStream,v)                'Unsigned 3-byte value (between a short and an int)
55.         WriteByte s, (v Shr 16) & $ff
56.         WriteByte s, (v Shr 8) & $ff
57.         WriteByte s, v & $ff
58. End Function
59. Function ReadThrib(s:TStream)                   'Unsigned 3-byte value (between a short and an int)
60.         Local b1=ReadByte(s)
61.         Local b2=ReadByte(s)
62.         Local b3=ReadByte(s)
63.         Return (b1 Shl 16) | (b2 Shl 8) | b3
64. End Function
65.  
66.  
67. Function WriteUSInt(s:TStream,v:Long)                   'Unsigned Integer
68.         WriteByte s, (v Shr 24) & $ff
69.         WriteByte s, (v Shr 16) & $ff
70.         WriteByte s, (v Shr 8) & $ff
71.         WriteByte s, v & $ff
72. End Function
73. Function ReadUSInt:Long(s:TStream)                              'Unsigned Integer
74.         Local b1=ReadByte(s)
75.         Local b2=ReadByte(s)
76.         Local b3=ReadByte(s)
77.         Local b4=ReadByte(s)
78.         Return (b1 Shl 24) | (b2 Shl 16) | (b3 Shl 8) | b4
79. End Function
80.  
81.  
82.  
83.  
84.  
85.  
86. ' QUICK CONVERSION IF NEEDED (USING STRINGS)
87.  
88.  
89. Function Effi$(val:Long)
90.         Local o$
91.         If val<0 Then o=Chr(255);val=Abs(val)
92.         If val=<250 Then o:+Chr(val);Return o
93.         If val=<251+251 Then o:+Chr(251);o:+Chr(val-251);Return o
94.         If val=<65535 Then o:+Chr(252);o:+Chr((val Shr 8) & $ff);o:+Chr(val & $ff);Return o
95.         If val=<maxthrib Then o:+Chr(253);o:+Chr((val Shr 16) & $ff);o:+Chr((val Shr 8) & $ff);o:+Chr(val & $ff);Return o
96.         o:+Chr(254);o:+Chr((val Shr 24) & $ff);o:+Chr((val Shr 16) & $ff);o:+Chr((val Shr 8) & $ff);o:+Chr(val & $ff);Return o
97. End Function
98.  
99. Function UnEffi:Long(i$)
100.         Local sign=1
101.         Local b=Asc(Mid(i,1,1))
102.         If b=255 Then sign=-1;i=Right(i,Len(i)-1);b=Asc(Mid(i,1,1))
103.         If b=<250 Then Return b*sign
104.         If b=251 Then Return (251+Asc(Mid(i,2,1)))*sign
105.         If b=252 Then Return ( (Asc(Mid(i,2,1)) Shl 8) | Asc(Mid(i,3,1))                )*sign
106.         If b=253 Then Return ( (Asc(Mid(i,2,1)) Shl 16) | (Asc(Mid(i,3,1)) Shl 8) | Asc(Mid(i,4,1))             )*sign
107.         If b=254 Then Return ( (Asc(Mid(i,2,1)) Shl 24) | (Asc(Mid(i,3,1)) Shl 16) | (Asc(Mid(i,4,1)) Shl 8) | Asc(Mid(i,5,1))          )*sign
108. End Function

Comments : none...