1. LL1 Memory Pattern

• Posted by xecronix 1 week ago
• 292 views

LL1 Memory Pattern. Safe and Scalable Object-like Structures Using eumem

Author: Ronald Weidner

This post summarizes a design pattern discussion I had while building an LL(1) token stream structure in OpenEuphoria. The goal was to build a reusable, memory-safe, pointer-driven object without needing full OOP. The result was a fully testable, efficient, and idiomatic structure using ''eumem''.

Why This Pattern?

Euphoria passes everything by value. This means that without explicit handling, all state is copied, mutated in isolation, and discarded unless reassigned. To simulate pass-by-reference structures, I built a reusable LL(1) stream backed by ''eumem''.

This design allows:

• Pointer-based memory
• Shared state between procedures
• Explicit free and lifecycle control
• Type-safe tagging
• Struct-like layouts

The Layout

enum 
    __TYPE__,       -- must be first 
    LL1_INDEX,      -- current stream position 
    LL1_DATA,       -- sequence of tokens 
    __MYSIZE__      -- must be last 

constant LL1_ID = "LL1$T54yhwe%^%$^$@3yjhw@$%^" 

Each LL1 instance is created using:

public function new(sequence tokens) 
    return eumem:malloc({LL1_ID, 1, tokens, SIZEOF_LL1}) 
end function 

And type validation is performed with:

public type LL1(atom ptr) 
    if eumem:valid(ptr, __MYSIZE__) then 
        if equal(eumem:ram_space[ptr][__TYPE__], LL1_ID) then 
            return 1 
        end if 
    end if 
    return 0 
end type 

Safe for Multiple Instances

You do not need a dynamic ID. Every LL1 object shares the same ''LL1_ID''. That tag is not for uniqueness. It's for type validation. Each call to ''new()'' creates a new memory block with its own pointer, index, and token list. They're separate instances, but all valid for ''type LL1''.

What Happens on Free?

When calling:

LL1:free(ptr) 

All internal fields, including ''LL1_INDEX'', are erased. ''eumem:free()'' deletes the entire struct. After that:

• ''eumem:valid(ptr, MYSIZE)'' will return 0
• ''type LL1(ptr)'' will fail
• Accessing the freed pointer will result in garbage or error

The caller is responsible for setting the pointer to NULL if needed.

Why This Matters

This pattern has proven valuable in:

• FreeBASIC
• C
• Now Euphoria

It's simple, scales well, and allows for object-like design without embracing full OOP or class inheritance.

You can use this for:

• ASTs
• Tokenizers
• Stacks
• Symbol tables
• Virtual machines
• Dynamic data structures

Final Thought

Euphoria gives you enough control to simulate objects, but not enough to hide from responsibility. This pattern bridges the gap. It's clean, performant, and honest.

Thanks for reading. I would love to hear how others have solved pass-by-reference or struct emulation in their own projects.

Ronald Weidner

3. Re: LL1 Memory Pattern

• Posted by xecronix 1 week ago
• 286 views

Here are the tests. So you know this isn't vaporware. LOL

Ronald Weidner

-- File: tests/ll1_stream_test.ex 
include std/unittest.e 
include std/sequence.e 
include std/eumem.e 
include ../lib/engine/ll1_stream.e 
 
-- 🧪 Setup & Test Data 
sequence test_tokens = {"A", "B", "C"} 
atom stream = LL1:new(test_tokens) 
 
-- 🧪 Test peek() does not move the pointer 
test_equal("1. current returns first token A", "A", LL1:current(stream)) 
test_equal("2. peek returns second token B", "B", LL1:peek(stream)) 
 
-- 🧪 Test next() advances the pointer 
test_equal("3. next returns second token", "B", LL1:next(stream)) 
test_equal("4. current returns second token", "B", LL1:current(stream)) 
 
-- 🧪 Test back() reverses the pointer 
test_equal("5. back then next returns 'A' again", "A", LL1:back(stream)) 
test_equal("6. current returns second token", "A", LL1:current(stream)) 
 
-- 🧪 Test recall() gives the previous token 
LL1:next(stream) -- move to B 
test_equal("7. recall returns 'A'", "A", LL1:recall(stream)) 
test_equal("8. current returns 'B'", "B", LL1:current(stream)) 
test_equal("9. peek returns 'C'", "C", LL1:peek(stream)) 
 
-- 🧪 Test has_more() and has_less()  -- We should be on B 
test_true("10. has_less is true", LL1:has_less(stream)) 
test_true("11. has_more is true", LL1:has_more(stream)) 
 
-- 🧪 Test has_less() from the start  -- We should be on B 
test_equal("12. back returns 'A' The start of stream", "A", LL1:back(stream)) 
test_false("13. has less should be false", LL1:has_less(stream)) 
 
-- 🧪 Test has_more() from the end  -- We should be on A 
test_equal("14. Test moving forward", "B", LL1:next(stream)) 
test_equal("15. Test moving forward", "C", LL1:next(stream)) 
test_false("16. has more should be false", LL1:has_more(stream)) 
LL1:free(stream) 
 
-- 🧪 Edge Case: Empty stream 
atom empty_stream = LL1:new({}) 
test_false("empty stream has more", LL1:has_more(empty_stream)) 
test_false("empty stream has less", LL1:has_less(empty_stream)) 
test_equal("next on empty stream returns 0", 0, LL1:next(empty_stream)) 
test_equal("peek on empty stream returns 0", 0, LL1:peek(empty_stream)) 
LL1:free(empty_stream) 
 
-- Done 
test_report()