December 03, 2020, 07:53:33 PM

Author Topic: [bb] String banks/data buffers by Yasha [ 1+ years ago ]  (Read 748 times)

Offline BlitzBot

  • Jr. Member
  • **
  • Posts: 1
[bb] String banks/data buffers by Yasha [ 1+ years ago ]
« on: June 29, 2017, 12:28:38 AM »
Title : String banks/data buffers
Author : Yasha
Posted : 1+ years ago

Description : This code requires <a href="" target="_blank">MikhailV's free FastPointer DLL[/url].
This tiny library lets you treat strings exactly as you would a Blitz bank: peek and poke bytes, shorts, ints and floats.

Strings make a great way to store binary data: they're basically the same as banks, a flat data buffer, but you don't need to worry about freeing them! Unfortunately, they have a couple of major drawbacks: it's really slow to extract data as characters using the Mid command, and it's even more annoying to set data by building up a whole new string - it's even slower, and you have to return it, pass it back to wherever the old string was, etc.

No longer!

This library provides the eight Peek and Poke functions familiar from banks, but for use with strings.

The Peek functions are nothing that can't be done in pure Blitz code using Mid, but are implemented (hopefully) faster here thanks to dark pointer magic. This makes it more convenient to store data in strings to access later, because accessing the data becomes a little bit easier.

The Poke functions, on the other hand... provide something new: the ability to modify strings in-place, exactly as you would poke to a bank! Normally, Blitz strings are immutable, and when you want to change a value in your highly-complex string data type, you need to recreate and return a whole new string with the new data. But these new Poke functions mean that the string you pass into the function will be modified by it, exactly as a bank would be! This means it's now possible to set the character data within existing strings, without the massive overhead of building new strings! This is really great for speeding up code that relies on modifying very large string structures, which would take a lot of time and memory to recreate.

Tiny example:

Code: [Select]
Include ""

Print Hex(StrPeekShort("abcde", 1)) : Print ""

Local s$ = String("-", 8) ;Simplest way to pre-create a string of given length
StrPokeFloat s, 0, 42.6
StrPokeFloat s, 4, 87.19
Print s
Print StrPeekFloat(s, 0)
Print StrPeekFloat(s, 4)

WaitKey : End

As you can see, s$ changed, even though it was never set.

While this violates the Blitz Basic notion that strings should be "value types" like hardware numbers, it's really, really useful. You can effectively use strings as garbage-collected banks with this code.

The other bank functions - CreateBank, CopyBank etc. are not provided because they're kinda trivial. (To create a new string of a dynamic length for poking values into, i.e. to CreateBank, use the 'String' function.) [/i]

Code :
Code: BlitzBasic
  1. ; Fast string data access (Blitz3D/+)
  2. ;=====================================
  4. ; Requires FastPointer DLL (free):
  7. ; Offsets of char data, data length, and buffer size within a Blitz string
  11. Type FSTRING_Wrapper_   ;Singleton wrapper type, used to get pointer only
  12.         Field s$
  13. End Type
  15. Global FSTRING_private_WSlot_.FSTRING_Wrapper_, FSTRING_private_SPtr_   ;Singleton instance
  18. ; Retrieve the byte at the given offset from a string buffer
  19. Function StrPeekByte(s$, offset)
  20.         Return Asc(Mid(s, offset + 1, 1))       ;Trivial, but here for completeness
  21. End Function
  23. ; Retrieve the short int at the given offset from a string buffer
  24. Function StrPeekShort(s$, offset)
  25.         If FSTRING_private_WSlot_ = Null
  26.                 FSTRING_private_WSlot_ = New FSTRING_Wrapper_
  27.                 FSTRING_private_SPtr_ = TypePointer(FSTRING_private_WSlot_)
  28.         EndIf
  29.         FSTRING_private_WSlot_s = s
  30.         Local strPtr = MemoryFastPeekInt(FSTRING_private_SPtr_)
  31.         If offset < 0 Or offset > (MemoryFastPeekInt(strPtr + FSTRING_LEN_OFFSET) - 2)
  32.                 RuntimeError "StrPeekShort: offset out of range"
  33.         EndIf
  34.         Return MemoryFastPeekShort(MemoryFastPeekInt(strPtr + FSTRING_CHAR_OFFSET) + offset)
  35. End Function
  37. ; Retrieve the integer at the given offset from a string buffer
  38. Function StrPeekInt(s$, offset)
  39.         If FSTRING_private_WSlot_ = Null
  40.                 FSTRING_private_WSlot_ = New FSTRING_Wrapper_
  41.                 FSTRING_private_SPtr_ = TypePointer(FSTRING_private_WSlot_)
  42.         EndIf
  43.         FSTRING_private_WSlot_s = s
  44.         Local strPtr = MemoryFastPeekInt(FSTRING_private_SPtr_)
  45.         If offset < 0 Or offset > (MemoryFastPeekInt(strPtr + FSTRING_LEN_OFFSET) - 4)
  46.                 RuntimeError "StrPeekInt: offset out of range"
  47.         EndIf
  48.         Return MemoryFastPeekInt(MemoryFastPeekInt(strPtr + FSTRING_CHAR_OFFSET) + offset)
  49. End Function
  51. ; Retrieve the float at the given offset from a string buffer
  52. Function StrPeekFloat#(s$, offset)
  53.         If FSTRING_private_WSlot_ = Null
  54.                 FSTRING_private_WSlot_ = New FSTRING_Wrapper_
  55.                 FSTRING_private_SPtr_ = TypePointer(FSTRING_private_WSlot_)
  56.         EndIf
  57.         FSTRING_private_WSlot_s = s
  58.         Local strPtr = MemoryFastPeekInt(FSTRING_private_SPtr_)
  59.         If offset < 0 Or offset > (MemoryFastPeekInt(strPtr + FSTRING_LEN_OFFSET) - 4)
  60.                 RuntimeError "StrPeekFloat: offset out of range"
  61.         EndIf
  62.         Return MemoryFastPeekFloat(MemoryFastPeekInt(strPtr + FSTRING_CHAR_OFFSET) + offset)
  63. End Function
  65. ; Destructively set the byte at the given offset within a string buffer
  66. Function StrPokeByte(s$, offset, val)
  67.         If FSTRING_private_WSlot_ = Null
  68.                 FSTRING_private_WSlot_ = New FSTRING_Wrapper_
  69.                 FSTRING_private_SPtr_ = TypePointer(FSTRING_private_WSlot_)
  70.         EndIf
  71.         FSTRING_private_WSlot_s = s
  72.         Local strPtr = MemoryFastPeekInt(FSTRING_private_SPtr_)
  73.         If offset < 0 Or offset > (MemoryFastPeekInt(strPtr + FSTRING_LEN_OFFSET) - 1)
  74.                 RuntimeError "StrPokeByte: offset out of range"
  75.         EndIf
  76.         Return MemoryFastPokeByte(MemoryFastPeekInt(strPtr + FSTRING_CHAR_OFFSET) + offset, val)
  77. End Function
  79. ; Destructively set the short int at the given offset within a string buffer
  80. Function StrPokeShort(s$, offset, val)
  81.         If FSTRING_private_WSlot_ = Null
  82.                 FSTRING_private_WSlot_ = New FSTRING_Wrapper_
  83.                 FSTRING_private_SPtr_ = TypePointer(FSTRING_private_WSlot_)
  84.         EndIf
  85.         FSTRING_private_WSlot_s = s
  86.         Local strPtr = MemoryFastPeekInt(FSTRING_private_SPtr_)
  87.         If offset < 0 Or offset > (MemoryFastPeekInt(strPtr + FSTRING_LEN_OFFSET) - 2)
  88.                 RuntimeError "StrPokeShort: offset out of range"
  89.         EndIf
  90.         Return MemoryFastPokeShort(MemoryFastPeekInt(strPtr + FSTRING_CHAR_OFFSET) + offset, val)
  91. End Function
  93. ; Destructively set the integer at the given offset within a string buffer
  94. Function StrPokeInt(s$, offset, val)
  95.         If FSTRING_private_WSlot_ = Null
  96.                 FSTRING_private_WSlot_ = New FSTRING_Wrapper_
  97.                 FSTRING_private_SPtr_ = TypePointer(FSTRING_private_WSlot_)
  98.         EndIf
  99.         FSTRING_private_WSlot_s = s
  100.         Local strPtr = MemoryFastPeekInt(FSTRING_private_SPtr_)
  101.         If offset < 0 Or offset > (MemoryFastPeekInt(strPtr + FSTRING_LEN_OFFSET) - 4)
  102.                 RuntimeError "StrPokeInt: offset out of range"
  103.         EndIf
  104.         Return MemoryFastPokeInt(MemoryFastPeekInt(strPtr + FSTRING_CHAR_OFFSET) + offset, val)
  105. End Function
  107. ; Destructively set the float at the given offset within a string buffer
  108. Function StrPokeFloat(s$, offset, val#)
  109.         If FSTRING_private_WSlot_ = Null
  110.                 FSTRING_private_WSlot_ = New FSTRING_Wrapper_
  111.                 FSTRING_private_SPtr_ = TypePointer(FSTRING_private_WSlot_)
  112.         EndIf
  113.         FSTRING_private_WSlot_s = s
  114.         Local strPtr = MemoryFastPeekInt(FSTRING_private_SPtr_)
  115.         If offset < 0 Or offset > (MemoryFastPeekInt(strPtr + FSTRING_LEN_OFFSET) - 4)
  116.                 RuntimeError "StrPokeFloat: offset out of range"
  117.         EndIf
  118.         Return MemoryFastPokeFloat(MemoryFastPeekInt(strPtr + FSTRING_CHAR_OFFSET) + offset, val)
  119. End Function
  122. ;~IDEal Editor Parameters:
  123. ;~F#17#1F#24#32#40#4E#5C#6A#78
  124. ;~C#Blitz3D

Comments :

virtlands(Posted 1+ years ago)

 This is an excellent idea of yours of altering strings with pointers and stuff  [FastPointer DLL...]{ I previously downloaded the FastPointer, and purchased the FastText option. }There are so many valuable things that can be stored in strings,such as pictures (BMP,PNG,etc), other programs, UTF8 strings, unicode strings, ....If you'd like to read about UTF8, see this link:<a href="" target="_blank">[/url]The FastText Extension allows the use of UTF8, and therefore it allows a way to represent all unicodes.<a href="" target="_blank">[/url]You can store an entire program (or DLL) in a string and then it can be 'extracted' back to a file.In the case where unusual numbers must be stored in a string(such as control characters),then first use a form like s$ = String("-", 800), and then I guess come back after the program is made, and use a hex editor to force the real data in. Have you ever noticed that if you use too many "Data ..."statements in your program, that it crashes?Well I found out the hard way that it does;Using strings to store data instead shows great promise.  By the way, using  s$=chr$(7)+chr$(8)+chr$(9)+...+chr$(that) is the really clumsy way to store  data in a string, (therefore, use hex-editor).And what if the string has a chr$(34)= (") in it?Imagine the chaos that would cause, consider the scenario:S$ = " a very very long string with something "fishy" in it".NOW WHAT?-------------------------------------Some nice things about strings though:(a) They are compact, and store data one byte at a time.(b) You can use the INSTR(s,sub,n) function with strings.I definitely plan to use the 'Yasha' code.

Yasha(Posted 1+ years ago)

 <div class="quote"> And what if the string has a chr$(34)= (") in it? </div>There's no problem with a string containing Chr(34) - the double quote is just syntax for the B3D compiler and has no bearing on how strings work at runtime.Or do you mean for generating Data statements automatically, so that everything also has to be valid B3D source? The other unprintable characters (or newline) will also cause problems - you could get around this by storing the data in the Data blocks in base-80 or something like that. Just add a conversion step before they're actually used.

virtlands(Posted 1+ years ago)

 That's clever, you said "base-80". Now you've stirred up a whole hornet's nest of stuff.Someone will have to dedicate an entire topic to that. ;)Binary to Text Encoding<a href="" target="_blank">[/url]Ascii85:<a href="" target="_blank">[/url]Base 64:<a href="" target="_blank">[/url]Base 96:<a href="" target="_blank">[/url]Base 80: ???<a href="" target="_blank">[/url]

Yasha(Posted 1+ years ago)

 ...I was thinking of Ascii85, yes. Whoops.


SimplePortal 2.3.6 © 2008-2014, SimplePortal