Re: Anyone who uses (has used) strTok by Kat
- Posted by "Kat" <gertie at visionsix.com> Nov 18, 2004
- 687 views
On 18 Nov 2004, at 8:19, John F Dutcher wrote: > > > posted by: John F Dutcher <John_Dutcher at urmc.rochester.edu> > > > I have the luxury of a mainframe to analyze files I build or revise > in Euphoria for correctness. I tried "strTok" to parse a file, find two > matching tokens received from an input screen....and make a change to > one "field" in the parsed input, all the time "deparsing" and writing > all records to a new output file. This is followed by uploading both > files to the mainframe to compare large files with a utility. > > Parse/deparse works so beautifully .. except where multiple commas > (representing "missing" values) occur in the input. Here the > parse/deparse does not "hold" the consecutive commas when "deparsed" > and they are lost upon output. > > Has anyone dealt with this easily ?? Actually, like the name of the lib implies, possibly too subtly, it was written for string processing primarily. As in natural language processing. One easy way around it is to put in something that signifies nothing, like: string2 = parse(replace(string1,",,"," "),",") so if string1 = {the,tall,,kat} string2 = {"the","tall"," ","kat"} Choose a replacement for "" that is below your lowest valid token start character (or higher), for best results in sorttok(), depending on if you want blanks sorted above or below non-empty fields. i use this replace code currently :
function replace(sequence st,sequence old_ch,sequence new_ch) integer k sequence newst, old_ch1, old_ch2 if match("*",old_ch) then old_ch1 = old_ch[1..match("*",old_ch)-1] old_ch2 = old_ch[match("*",old_ch)+1..length(old_ch)] k = match(upper(old_ch1),upper(st)) newst = "" while k and match(upper(old_ch2),upper(st)) do newst = newst & st[1..k-1] & new_ch st = st[k..length(st)] st = st[match(upper(old_ch2),upper(st))+length(old_ch2)..length(st)] k = match(upper(old_ch1),upper(st)) end while newst = newst & st return newst else k = match(upper(old_ch),upper(st)) newst = "" while k do newst = newst & st[1..k-1] & new_ch st = st[k+length(old_ch)..length(st)] k = match(upper(old_ch),upper(st)) end while newst = newst & st return newst end if end function
I had considered adding db-types processing to strtok, or regex code, but was waiting for the user demand. The work-around for not haveing db-specific code in strtok are easy enough, i think, but i could be convinced to add to it. The lawyer said today if i am convicted, he will appeal for free and make out my will for free. Kat