June 24, 2018, 10:06:17 AM

Author Topic: Processing the " within a string  (Read 252 times)

Offline Mikey

  • Jr. Member
  • **
  • Posts: 20
Processing the " within a string
« on: June 12, 2018, 01:35:44 AM »

I'm using the command inside of B3D

Instr(DMY$,Chr$(34))

which does not work.
How can I process this using the instr command?

Offline Matty

  • Sr. Member
  • ****
  • Posts: 385
    • MattiesGames
Re: Processing the " within a string
« Reply #1 on: June 12, 2018, 05:27:11 AM »
save this file as test.bb in your folder and run from in the ide...it finds it fine:
Code: [Select]
;here is a string with a " in it.
infile = ReadFile("test.bb")
While(Not(Eof(infile)))
val$ = ReadLine(infile)
If(Instr(val$,Chr$(34),1)>0) Then
RuntimeError "found it!"
EndIf
Wend
CloseFile infile

Offline TomToad

  • Sr. Member
  • ****
  • Posts: 267
Re: Processing the " within a string
« Reply #2 on: June 12, 2018, 11:25:34 AM »
You need to show us more of what you are doing.  This works fine on my computer.

Code: [Select]
l$ = "This is a "+Chr(34)+"quote"+Chr(34)



Print l$

Print Instr(l$,Chr$(34))
------------------------------------------------
8 rabbits equals 1 rabbyte.

Offline RemiD

  • Hero Member
  • *****
  • Posts: 595
Re: Processing the " within a string
« Reply #3 on: June 12, 2018, 04:10:44 PM »
@TomToad>>Thanks ! i needed to add the " symbol to a string line, in the past, but did not know how to do it... Good :)

Offline Mikey

  • Jr. Member
  • **
  • Posts: 20
Processing the " within a string
« Reply #4 on: June 12, 2018, 04:49:16 PM »

I changed the code to something this before seeing the posts here

Code: [Select]
PL$= ReadLine(FH1):Print PL$:Print Len(PL$);Print Asc(Mid$(PL$,Instr(PL$,",")-1))
RL$=PL$
While Instr(RL$,",")
TC=Instr(RL$,",")
If TC
.CK If Asc(Mid$(RL$,TC-1,1))<>34
TC=TC+Instr(Mid$(RL$,TC+1),","):Goto CK
ElseIf Asc(Mid$(RL$,TC-1,1))=34
Print Mid$(RL$,2,Len(RL$)-1)
EndIf
SD=SD+1
If SD>=2
Print TC:Print Mid$(RL$,2,TC-1):HALT(1)
EndIf
RL$=Mid$(RL$,TC+1)
EndIf
; E=E+1
Wend

It's not so much the reading of the " . The other issue is that the text I'm parsing has them in odd places.

Offline Steve Elliott

  • Hero Member
  • *****
  • Posts: 1046
Re: Processing the " within a string
« Reply #5 on: June 12, 2018, 04:51:12 PM »
omg is that a goto?!   :o
Windows 10, 64-bit, 8Gb RAM, CPU Intel i5, 3.2 GHz, Nvidia GeForce GTX 1050 (2Gb)

Offline Naughty Alien

  • Sr. Member
  • ****
  • Posts: 432
Re: Processing the " within a string
« Reply #6 on: June 12, 2018, 06:04:06 PM »
 ;D

Offline col

  • Sr. Member
  • ****
  • Posts: 323
Re: Processing the " within a string
« Reply #7 on: June 12, 2018, 08:20:39 PM »
Whats wrong with 'goto'?  ;D
Any bugs in my code are proof of its hand-coded nature.

Offline Steve Elliott

  • Hero Member
  • *****
  • Posts: 1046
Re: Processing the " within a string
« Reply #8 on: June 12, 2018, 09:09:27 PM »
lol you'll goto spaghetti hell...And nobody wants to go there  ;D
Windows 10, 64-bit, 8Gb RAM, CPU Intel i5, 3.2 GHz, Nvidia GeForce GTX 1050 (2Gb)

Offline col

  • Sr. Member
  • ****
  • Posts: 323
Re: Processing the " within a string
« Reply #9 on: June 12, 2018, 09:25:56 PM »
Hehe,

In all seriousness :) instead of us making fun we could offer some help with parsing, if its wanted of course.

If you were to look up 'lexical analyzer' ( don't worry about the name - it just looks all technical and sci-fi but its not really ) you could create a powerful parser that can parse practically anything you want.

The use of a lexical analyzer is to break a big stream of characters into more manageable pieces called 'tokens'. You can then 'parse' the tokens much more easily. Think of it as breaking a huge sentence of text into words by using the space character as the separator between the words, then all 'non-space' characters get grouped into a single word - you now only need to deal with 'words' instead of the huge stream of characters.
Any bugs in my code are proof of its hand-coded nature.

Offline Mikey

  • Jr. Member
  • **
  • Posts: 20
Processing the " within a string
« Reply #10 on: June 12, 2018, 10:00:55 PM »
Quote
If you were to look up 'lexical analyzer' ( don't worry about the name - it just looks all technical and sci-fi but its not really ) you could create a powerful parser that can parse practically anything you want.

Yes that would suffice but I'm using BB and it has its limitations.
On the old site I think someone made something that would do that but it might be too much additional code.

Yes I do know that a goto statement can make you code more unorganized but not one or two of them however, if that's all you will use for directing code then : Look Out !!

Here is the line I'm trying to parse.

Code: [Select]
"07076051.wav","Two-stroke petrol engine driving small elevator, start, run, stop.","194","Engines: Petrol","EC117D","Diesel & Petrol Engines","4"

Offline col

  • Sr. Member
  • ****
  • Posts: 323
Re: Processing the " within a string
« Reply #11 on: June 13, 2018, 12:29:45 PM »
omg I'm so rusty with B3D and BB code  :))

I managed to throw this together in my lunch break. I ran out of time to fully test it though  ;)
It may seem complicated at first but it would be the beginnings of a full-on lexer and parser.
I'm so used to OO code nowadays that I tried to take an OO approach to creating a lexical analyzer and a parser to parse the line you mentioned above.

If it's too complicated then oh well, maybe you or someone may be able to cherry pick bits and pieces for their own benefit - or just throw it in the bin :D

For the sake of keeping the code and data in one source file I change the quotes ( " ) in what would be the text file to ~q ( which is a BlitzMax style 'in text' quote ) and the code parses that format accordingly. You would need to change that part to suit handling the quote symbol if you're loading from a file.
EDIT: Coffee break time - I modified the code to handle the string data as you have it ( with " ).

Also there may be slight bugs in there as I doubt it handles 'corner' cases and will need thorough testing.

Code: [Select]

Const TOKEN_UNKNOWN = 0
Const TOKEN_STRING = 1
Const TOKEN_COMMA = 2
Const TOKEN_EOL = 3
Const TOKEN_EOF = 4

Type TToken
Field value$
Field tipe
End Type

Type TLexer
Field in$
Field inlength

Field token_start
Field token_end
End Type


Function TokenTypeToString$(TokenType)
Select TokenType
Case TOKEN_UNKNOWN Return "TOKEN_UNKNOWN"
Case TOKEN_STRING Return  "TOKEN_STRING "
Case TOKEN_COMMA Return   "TOKEN_COMMA  "
Case TOKEN_EOL Return     "TOKEN_EOL    "
Case TOKEN_EOF Return     "TOKEN_EOF    "
End Select
End Function

Function Lexer_NextToken.TToken(lex.TLexer)
; default to an unknown token
Local t.TToken = New TToken
t\tipe = TOKEN_UNKNOWN

; reset the token start position
lex\token_start = lex\token_end

; end of line?
If Lexer_IsEndOfLine(lex) ; could deal with end-of-file also if wanted
t\tipe = TOKEN_EOL
t\value = "end-of-line"

; is a '~q' - change as required
Else If Lexer_IsQuote(lex)
While lex\token_end <= lex\inlength
lex\token_end = lex\token_end + 1
If Lexer_IsQuote(lex) Exit
Wend
t\tipe = TOKEN_STRING
t\value = Mid(lex\in, lex\token_start, lex\token_end - lex\token_start)
Else
; make a known symbol
If Asc(Mid(lex\in, lex\token_end, 1)) = 44 ; ','
t\tipe = TOKEN_COMMA
t\value = ","
lex\token_end = lex\token_end + 1
EndIf
EndIf

Return t
End Function

Function Lexer_IsQuote(lex.TLexer)
; change this code to suit detecting a " character instead of ~q
If Mid(lex\in, lex\token_end, 1) = Chr(34)
lex\token_end = lex\token_end + 1
Return True
EndIf
End Function

Function Lexer_IsEndOfLine(lex.TLexer)
Return lex\token_end >= lex\inlength
End Function





; you would now create a TParser to handle the token types as per the syntax that you expect
Type TParser
Field lexer.TLexer
Field token.TToken
End Type

Function Parser_Parse(parser.TParser)
Parser_NextToken(parser)

While parser\token\tipe <> TOKEN_EOL
; Select the parser\token\tipe and do something meaningful with it, here the code prints out the token data
; You would do something more meaningful with the data
Print TokenTypeToString(parser\token\tipe) + " : " + parser\token\value
Parser_NextToken(parser)
Wend
End Function

Function Parser_NextToken(parser.TParser)
parser\token = Lexer_NextToken(parser\lexer)
End Function

Local in$ = Chr(34) + "07076051.wav" + Chr(34) + ","
in = in + Chr(34) + "Two-stroke petrol engine driving small elevator, start, run, stop." + Chr(34) + ","
in = in + Chr(34) + "194" + Chr(34) + ","
in = in + Chr(34) + "Engines: Petrol" + Chr(34) + ","
in = in + Chr(34) + "EC117D" + Chr(34) + ","
in = in + Chr(34) + "Diesel & Petrol Engines" + Chr(34) + ","
in = in + Chr(34) + "4"+Chr(34)

; set up a lexer with data
Local lexer.TLexer = New TLexer
lexer\token_start = 1
lexer\token_end = 1
lexer\in = in ; "~q07076051.wav~q,~qTwo-stroke petrol engine driving small elevator, start, run, Stop.~q,~q194~q,~qEngines: Petrol~q,~qEC117D~q,~qDiesel & Petrol Engines~q,~q4~q"
lexer\inlength = Len(lexer\in)


; setup a parser with the lexer
Local parser.TParser = New TParser
parser\lexer = lexer

; use the parser to control the lexer and parse the output of the lexer tokens
Parser_Parse(parser)





Any bugs in my code are proof of its hand-coded nature.

Offline 3DzForMe

  • Full Member
  • ***
  • Posts: 196
Re: Processing the " within a string
« Reply #12 on: June 15, 2018, 12:12:50 AM »
@col. lovely piece of code,  did a slice of data parsing myself in the past with B3D, still use my clunkier code to good effect. Only one that really benefits from it is moi. Hey Ho ;) :))
happy coding ;)