OpenEuphoria

Re: internal storage

• Posted by Derek Parnell <ddparnell at bigpond.com> Sep 20, 2004
• 434 views

Jim Hendricks wrote:
> 
> The manual states:
> 
>  Performance Note:
>     Does this mean that all atoms are stored in memory as 8-byte 
>     floating-point numbers? No. The Euphoria interpreter usually 
>     stores integer-valued atoms as machine integers (4 bytes) to 
>     save space and improve execution speed. When fractional results 
>     occur or numbers get too big, conversion to floating-point happens 
>     automatically.
> 
> My question is are string sequences then stored as 4 byte atoms or as 
> 1 byte atoms?
> 
> This is quite important to the app I am porting since I am parsing about
> 4 Meg of character data and the process performs much better if I can
> parse the character data to an intermediate state in memory which causes
> having all the data in memory at once. I could always go with an
> intermediate state to temp files, but this causes the process to run 1000%
> slower( on windoze ). I can live with requiring 4 Meg of memory consumption
> for this process, but not so keen on requiring 16 Meg of memory consumption.

All 'characters' are stored as 4-byte integers and not stored as single 
bytes.

If you have large text strings in RAM, you might need to rethink your 
design, or let the operating system deal with the virtual memory.

Another consideration is that each sequence is stored in contiguous RAM
so rather than have a giant single sequence containing all your
text, it is better to break it up into a sequence of sequences.

E.G. 

sequence theFile
theFile = {
  "line one", 
  "line two", 
  "line three"
   }

This is four sequences: one for the file and another for each line. So it
takes up four independant RAM blocks. This can effect RAM paging swapping
for very large sequences.

-- 
Derek Parnell
Melbourne, Australia