1. Saving a sequence

A recent contribution to the Archive (the XML-to-sequence library) made me wonder about the ways of saving sequences to disk, for later retrieval. Is there a standard way of doing it? And is there anything like sequence-to-XML or similar?

Thanks

Green Euphorian

new topic     » topic index » view message » categorize

2. Re: Saving a sequence

GreenEuphorian said...

A recent contribution to the Archive (the XML-to-sequence library) made me wonder about the ways of saving sequences to disk, for later retrieval. Is there a standard way of doing it? And is there anything like sequence-to-XML or similar?

Euphoria has built-in serialization routines. I would say this is the "standard" way of doing so.

These routines are (basically) what EDS uses to store objects in database files, which is also a "standard" way of storing a lot of Euphoria objects on disk in an organized manner.

integer fh 
fh = open("cust.dat", "wb") 
puts(fh, serialize(FirstName)) 
puts(fh, serialize(LastName)) 
puts(fh, serialize(PhoneNumber)) 
puts(fh, serialize(Address)) 
close(fh) 
 
fh = open("cust.dat", "rb") 
FirstName = deserialize(fh) 
LastName = deserialize(fh) 
PhoneNumber = deserialize(fh) 
Address = deserialize(fh) 
close(fh) 

-Greg

new topic     » goto parent     » topic index » view message » categorize

3. Re: Saving a sequence

ghaberek said...

Euphoria has built-in serialization routines. I would say this is the "standard" way of doing so.

These routines are (basically) what EDS uses to store objects in database files, which is also a "standard" way of storing a lot of Euphoria objects on disk in an organized manner.

integer fh 
fh = open("cust.dat", "wb") 
puts(fh, serialize(FirstName)) 
puts(fh, serialize(LastName)) 
puts(fh, serialize(PhoneNumber)) 
puts(fh, serialize(Address)) 
close(fh) 
 
fh = open("cust.dat", "rb") 
FirstName = deserialize(fh) 
LastName = deserialize(fh) 
PhoneNumber = deserialize(fh) 
Address = deserialize(fh) 
close(fh) 

-Greg

The manual's explanation is very terse, almost nonexistent.

Could someone please provide some more explanation?

Thanks

new topic     » goto parent     » topic index » view message » categorize

4. Re: Saving a sequence

GreenEuphorian said...

A recent contribution to the Archive (the XML-to-sequence library) made me wonder about the ways of saving sequences to disk, for later retrieval. Is there a standard way of doing it? And is there anything like sequence-to-XML or similar?

Thanks

Green Euphorian

The standard way is to use print and get from std/get.e. These store in a kind of ASCII format. I say "kind of" because you can store any sequence this way not just ASCII strings.

output:

print(fd, obj) 
puts(fd, 10) 

input:

obj = get(fd) 

I sometimes had problems when I didn't add a newline after printing.

new topic     » goto parent     » topic index » view message » categorize

5. Re: Saving a sequence

it's worth mentioning, be very sure of where your data us coming from when you read values into a program from serialize to sequences or from map save/load. a crafty user can sometimes use this input to instrument your program to do whatever.

imagine you are using a path or an expression expecting valid data that you carefully parsed before you saved it but then neglect to reparse & validate before you use what was loaded back in the next day. this is a very common security hole in many programs and languages.

new topic     » goto parent     » topic index » view message » categorize

6. Re: Saving a sequence

ne1uno said...

it's worth mentioning, be very sure of where your data us coming from when you read values into a program from serialize to sequences or from map save/load. a crafty user can sometimes use this input to instrument your program to do whatever.

imagine you are using a path or an expression expecting valid data that you carefully parsed before you saved it but then neglect to reparse & validate before you use what was loaded back in the next day. this is a very common security hole in many programs and languages.

Good advice, ne1uno. tongue

new topic     » goto parent     » topic index » view message » categorize

7. Re: Saving a sequence

Hi Ryanj,

When storing sequences in logs for debug, some of us have written routines to display sequences in a human readable form. Here are mine:

------------------------------------------------------------------------------ 
 
public function showPrintable(sequence s) 
-- returns only printable characters of a sequence 
-- non printable characters are replace by a dot 
  sequence res 
 
  res = "" 
  for i = 1 to length(s) do 
    if integer(s[i]) then 
      if (s[i] > 31) and (s[i] < 127) then 
        res &= s[i] 
      elsif s[i] = 9 then 
        res &= "\\t" 
      elsif s[i] = 13 then 
        res &= "\\r" 
      elsif s[i] = 10 then 
        res &= "\\n" 
      else 
        res &= "." 
      end if 
    else 
      res &= "." 
    end if 
  end for 
  return res 
end function 
 
------------------------------------------------------------------------------ 
 
public function sequenceDump(sequence s) 
-- prints a sequence structure in a human readable way as one string 
  integer subSequence 
  sequence result = "" 
 
  if isString(s) then 
    result = sprintf("\"%s\"", {showPrintable(s)}) 
  else 
    subSequence = 0 
    for i=1 to length(s) do 
      if sequence(s[i]) then 
        subSequence = 1 
        exit 
      end if 
    end for 
    if subSequence then 
      result = sequenceDump(s[1]) 
      for i=2 to length(s) do 
        result &= ", " & sequenceDump(s[i]) 
      end for 
      result = "{" & result & "}" 
    else 
      result = sprint(s) & "\t'" & showPrintable(s) & "'" 
    end if 
  end if 
  return result 
end function 
 
------------------------------------------------------------------------------ 
 
public function objectDump(object x, integer detailed=0) 
-- returns an object in a human readable way 
  integer subSequence 
  sequence result = "" 
 
  if integer(x) then 
    result = sprintf("%d", {x}) 
  elsif atom(x) then 
    if (x >= 0) and (x = floor(x)) then 
      result = sprintf("%.0f", {x}) 
    elsif (x < 0) and (x = floor(x+1)) then 
      result = sprintf("%.0f", {x}) 
    else 
      result = sprintf("%f", {x}) 
    end if 
  elsif isString(x) then 
    if detailed then 
      result = sprintf("'%s'\t%s", {showPrintable(x), sprint(x)}) 
    else 
      result = sprintf("'%s'", {showPrintable(x)}) 
    end if 
  else 
    subSequence = 0 
    for i=1 to length(x) do 
      if sequence(x[i]) then 
        subSequence = 1 
        exit 
      end if 
    end for 
    if subSequence then 
      result = objectDump(x[1], 0) 
      for i=2 to length(x) do result &= ", " & objectDump(x[i], 0) end for 
      result = "{" & result & "}" 
    else 
      if detailed then 
        result = sprintf("%s\t'%s'", {sprint(x), showPrintable(x)}) 
      else 
        result = sprintf("%s", {sprint(x)}) 
      end if 
    end if 
  end if 
  return result 
end function 
 

Regards

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

8. Re: Saving a sequence

GreenEuphorian said...

The manual's explanation is very terse, almost nonexistent.

Could someone please provide some more explanation?

Thanks

I don't understand this library...

I did a quick test on a small nested sequence:

  • original 60 k
  • serialized version is 120 k (now a flat sequence)
  • dump produces a 30 k binary file

Since dump overwrites files you can save only one object in a file.

Conclusion--something is amiss in this library.

_tom

new topic     » goto parent     » topic index » view message » categorize

9. Re: Saving a sequence

jmduro said...

When storing sequences in logs for debug, some of us have written routines to display sequences in a human readable form. Here are mine:

Thanks for sharing.

What are you using for the isString function?

_tom

new topic     » goto parent     » topic index » view message » categorize

Search



Quick Links

User menu

Not signed in.

Misc Menu