OpenEuphoria

Re: too much memory use!

• Posted by Jiri Babor <jbabor at PARADISE.NET.NZ> Feb 20, 2002
• 493 views

Kat,

if you have a memory challenged system and want to avoid hard disk
thrashing, there is a number of different schemes you can use, one of
them is outlined below:

----------------------------------------------------------------------
Select a suitable word delimiter. I chose 'space' (ascii 32).

Determine size of the data file.

Allocate sufficient memory for output.

In a while loop
    isolate individual words
    poke them into the reserved memory
    separate them with your chosen delimiter
    update the memory pointer
and keep at it until the end of input file is reached.

Flush the system.
----------------------------------------------------------------------

I am not sure what you intend to do with the data, but it is basically
trivial to read it back into a sequence and write it to a file, as a
whole or even as individual tokens.

I hope this makes some sense to you.

jiri

----------------------------------------------------------------------
include machine.e               -- allocate
include file.e                  -- seek, where

constant false = 0, true = 1
constant d = 32                 -- space chosen as delimiter
sequence word
atom a,p
integer c,e,f,inword,size

f = open("kat1.eml", "rb")

-- get file size
e = seek(f, -1)                 -- go to end of file
size = where(f)
e = seek(f, 0)                  -- go to back to start of file

-- allocate memory for output
a = allocate(size + 1)
if a = 0 then
    puts(1, "Memory allocation failed...\n")
    abort(1)
end if

-- initialize
word = {}
inword = false                  -- flag
p = a                           -- current memory pointer

-- main loop
c = getc(f)
while c != -1 do
    if find(c, {32,13,10}) then
        if inword then
            poke(p, word & d)
            p += length(word) + 1
            inword = false
            word = {}
        end if
    else
        inword = true
        word &= c
    end if
    c = getc(f)
end while
close(f)

-- flush
if inword then
    poke(p, word & d)
    p += length(word) + 1
end if

-- a little test - DO NOT try it with a 12 Mb file !
puts(1, peek({a, p-a}))