OpenEuphoria

Re: Anyone who uses (has used) strTok by Kat

• Posted by "Kat" <gertie at visionsix.com> Nov 18, 2004
• 687 views

On 18 Nov 2004, at 8:19, John F Dutcher wrote:

> 
> 
> posted by: John F Dutcher <John_Dutcher at urmc.rochester.edu>
> 
> 
> I have the luxury of a mainframe to analyze files I build or revise 
> in Euphoria for correctness. I tried "strTok" to parse a file, find two
> matching tokens received from an input screen....and make a change to
> one "field" in the parsed input, all the time "deparsing" and writing
> all records to a new output file. This is followed by uploading both
> files to the mainframe to compare large files with a utility.
> 
> Parse/deparse works so beautifully  ..  except where multiple commas
> (representing "missing" values) occur in the input. Here the 
> parse/deparse does not "hold" the consecutive commas when "deparsed"
> and they are lost upon output.
> 
> Has anyone dealt with this easily ?? 

Actually, like the name of the lib implies, possibly too subtly, it was written 
for string processing primarily. As in natural language processing. One easy 
way around it is to put in something that signifies nothing, like:

string2 = parse(replace(string1,",,"," "),",")

so if
string1 = {the,tall,,kat}
string2 = {"the","tall"," ","kat"}

Choose a replacement for "" that is below your lowest valid token start 
character (or higher), for best results in sorttok(), depending on if you want 
blanks sorted above or below non-empty fields.

i use this replace code currently :

function replace(sequence st,sequence old_ch,sequence new_ch)
    integer k
    sequence newst, old_ch1, old_ch2

if match("*",old_ch)
  then

    old_ch1 = old_ch[1..match("*",old_ch)-1]
    old_ch2 = old_ch[match("*",old_ch)+1..length(old_ch)]

    k = match(upper(old_ch1),upper(st))
    newst = ""
    while k and match(upper(old_ch2),upper(st)) do
      newst = newst & st[1..k-1] & new_ch
      st = st[k..length(st)]
      st = st[match(upper(old_ch2),upper(st))+length(old_ch2)..length(st)]
      k = match(upper(old_ch1),upper(st))
    end while
    newst = newst & st
    return newst

  else

    k = match(upper(old_ch),upper(st))
    newst = ""
    while k do
      newst = newst & st[1..k-1] & new_ch
      st = st[k+length(old_ch)..length(st)]
      k = match(upper(old_ch),upper(st))
    end while
    newst = newst & st
    return newst

  end if
end function

I had considered adding db-types processing to strtok, or regex code, but 
was waiting for the user demand. The work-around for not haveing db-specific 
code in strtok are easy enough, i think, but i could be convinced to add to it. 
The lawyer said today if i am convicted, he will appeal for free and make out 
my will for free.

Kat