OpenEuphoria

1. Why are my maps chewing on my RAM? or, Adventures in memory (de)allocation

• Posted by ghaberek (admin) Sep 08, 2019
• 978 views

I'm working on the new code base for the website, which also acts as a proving ground for Euphoria MVC.

This problem is pretty obvious when you think about it, but it caught me off guard and my hand-spun web server started chewing pretty hard on my RAM.

This is a simplified example of the code I'm working with. Can you spot the problem? Hint: models are maps, which use eumem.

namespace messages 
 
constant MESSAGE = model:define( ... ) 
 
-- 
-- return a message object matching the provided query string 
-- 
public function fetch_one( sequence query ) 
 
    object message = model:fetch_one( MESSAGE, query ) 
 
    if map( message ) then 
 
        integer message_id = map:get( message, "id" ) 
 
        -- retrieve the associated replies 
        sequence replies = model:fetch_all( MESSAGE, 
            "WHERE parent_id = %d AND is_deleted = 0", 
            {message_id} 
        ) 
 
        map:put( message, "replies", replies ) 
 
    end if 
 
    return message 
end function 

Do you see it yet?

The pseudo memory library automatically tags the objects it allocates with delete_routine() to mark the object's slot as "free" when it goes out of scope.

So when you allocate a bunch of maps, put them into a sequence, and then store that sequence in another map, what happens when the map goes out of scope?

Nothing! Cleanup routines are chained, but they're not recursive, so the maps just hang around even though the sequence they're in went out with the main map.

We have to catch the object going out of scope and clean them up manually:

    -- ... 
    return delete_routine( message, routine_id("cleanup_message") ) 
end function 
 
procedure cleanup_message( object message ) 
 
    if map( message ) then 
 
        sequence replies = map:get( message, "replies", {} ) 
 
        for i = 1 to length( replies ) do 
 
            -- force the object's cleanup routine 
            delete( replies[i] ) 
 
        end for 
 
    end if 
 
end procedure 

And I'm hesitant to call this a bug, since it's operating as designed, even though it's a bit counter-intuitive.

And I wouldn't have even noticed this if I was running this as a CGI script in Apache, since the maps would all live and die within the brief lifetime of each CGI call.

But I've got a rudimentary web server cobbled together that runs the code directly, so things like this become a lot more obvious.

This makes me wonder what other "interesting" memory management problems folks might have and not even realize.

-Greg

3. Re: Why are my maps chewing on my RAM? or, Adventures in memory (de)allocation

• Posted by ghaberek (admin) Sep 09, 2019
• 940 views

You're not wrong, and that's how previous user-contributed implementations have worked. For example...

sequence table = map_new( 256 ) -- number of buckets 
 
table = map_insert( table, "key", "value" ) -- insert data, return updated sequence 
 
object value = map_fetch( table, "key" ) -- fetch stored data from the sequence 

The goal of the eumem library, and its use in maps and stacks is allow pass-by-reference instead of requiring pass-by-value.

map table = map:new() -- 'table' is a reference to the eumem:ram_space sequence 
 
map:put( table, "key", "value" ) -- notice there is no return value here 
 
object value = map:get( table, "key" ) -- this is pretty much the same 

I think there's an advantage to be had in not having to pass-by-value everything. This method, IMHO, makes for cleaner and easier to understand code.

But, like with any other language that deals with passing references, keeping track of when things can be freed becomes an additional burden.

-Greg