OpenEuphoria

Re: internal storage

• Posted by CoJaBo <cojabo at suscom.net> Sep 20, 2004
• 422 views

Derek Parnell wrote:
> 
> Jim Hendricks wrote:
> > 
> > The manual states:
> > 
> >  Performance Note:
> >     Does this mean that all atoms are stored in memory as 8-byte 
> >     floating-point numbers? No. The Euphoria interpreter usually 
> >     stores integer-valued atoms as machine integers (4 bytes) to 
> >     save space and improve execution speed. When fractional results 
> >     occur or numbers get too big, conversion to floating-point happens 
> >     automatically.
> > 
> > My question is are string sequences then stored as 4 byte atoms or as 
> > 1 byte atoms?
> > 
> > This is quite important to the app I am porting since I am parsing about
> > 4 Meg of character data and the process performs much better if I can
> > parse the character data to an intermediate state in memory which causes
> > having all the data in memory at once. I could always go with an
> > intermediate state to temp files, but this causes the process to run 1000%
> > slower( on windoze ). I can live with requiring 4 Meg of memory consumption
> > for this process, but not so keen on requiring 16 Meg of memory consumption.
> 
> All 'characters' are stored as 4-byte integers and not stored as single 
> bytes.
This DEFINATLY should be improved in a new version of Euphoria.
There are many times where I use allocated memory to get around
this problem.
Euphoria 2.5(or 2.6 if it would take too long) should use 1-byte
instead of 4-byte whenever possible.

> 
> If you have large text strings in RAM, you might need to rethink your 
> design, or let the operating system deal with the virtual memory.
> 
> Another consideration is that each sequence is stored in contiguous RAM
> so rather than have a giant single sequence containing all your
> text, it is better to break it up into a sequence of sequences.
> 
> E.G. 
> 
> sequence theFile
> theFile = {
>   "line one", 
>   "line two", 
>   "line three"
>    }
> 
> This is four sequences: one for the file and another for each line. So it
> takes up four independant RAM blocks. This can effect RAM paging swapping
> for very large sequences.
> 
> -- 
> Derek Parnell
> Melbourne, Australia
>