OpenEuphoria: Forum: Architectural Discussion: Implementing ECS and Data-Oriented Programming (DOP)

1. Architectural Discussion: Implementing ECS and Data-Oriented Programming (DOP)

Posted by euphoric (admin) 3 weeks ago
295 views

Hey everyone,

Over the last few years, programming patterns in game engines and distributed backend systems have swung heavily away from classic Object-Oriented inheritance toward Data-Oriented Programming (DOP) and Entity-Component-System (ECS) architectures.

The core tenets of DOP (championed by books like Yehonathan Sharvit's Data-Oriented Programming, or the Clojure ecosystem) are:

Separate code from data: Pure functions/procedures operate on separate, passive data structures.

Represent data with generic structures: Instead of rigid classes or fixed structs, pass generic associative maps/dictionaries between functions.

Treat data as immutable: State transitions return new or modified data rather than mutating in place.

Separate schema validation from data representation: Data remains flexible maps; validation happens optionally at boundaries (like Euphoria's custom type system).

In modern ECS, this lines up nicely:

Entities: Pure integer IDs.

Components: Plain data records/maps (e.g., Position, Velocity, Renderable).

Systems: Pure logic/functions that query entities containing specific component signatures and process them.

Lately, I've been transitioning my own architectural style toward passing maps between everything-treating data as flexible dictionaries rather than tightly coupled records. Since OpenEuphoria has native map support via std/map.e, I started exploring how clean an ECS/DOP implementation would actually look in modern Euphoria.

Quick Implementation Spike

In OpenEuphoria, std/map.e implements hash tables where map variables are handles referencing dynamic internal storage. Combining std/map.e with sequence indices gives us a clean way to model entities and systems:

OpenEuphoria: Euphoria v4.0

include std/map.e as map 
 
-- 1. Component Stores: Map component keys to entity data 
constant COMPONENT_POSITION = "pos" 
constant COMPONENT_VELOCITY = "vel" 
 
-- World state can just be a map of component stores 
map:map world = map:new() 
map:put(world, COMPONENT_POSITION, map:new()) 
map:put(world, COMPONENT_VELOCITY, map:new()) 
 
-- 2. Entity Creation: Entities are just unique IDs 
integer next_entity_id = 1 
 
function create_entity() 
    integer id = next_entity_id 
    next_entity_id += 1 
    return id 
end function 
 
-- Helper: attach component map to entity 
procedure add_component(integer entity, sequence comp_name, map:map comp_data) 
    map:map store = map:get(world, comp_name) 
    map:put(store, entity, comp_data) 
end procedure 
 
-- 3. Spawning an entity using generic data maps (pure DOP style) 
integer player = create_entity() 
 
map:map pos = map:new() 
map:put(pos, "x", 10.0) 
map:put(pos, "y", 25.0) 
add_component(player, COMPONENT_POSITION, pos) 
 
map:map vel = map:new() 
vel = map:put(vel, "dx", 1.5) 
vel = map:put(vel, "dy", 0.0) 
add_component(player, COMPONENT_VELOCITY, vel) 
 
-- 4. System: MovementSystem (Pure procedure operating over data) 
procedure movement_system(atom dt) 
    map:map positions  = map:get(world, COMPONENT_POSITION) 
    map:map velocities = map:get(world, COMPONENT_VELOCITY) 
     
    -- Query: entities with velocities 
    sequence moving_entities = map:keys(velocities) 
     
    for i = 1 to length(moving_entities) do 
        integer id = moving_entities[i] 
         
        if map:has(positions, id) then 
            map:map p = map:get(positions, id) 
            map:map v = map:get(velocities, id) 
             
            atom new_x = map:get(p, "x") + (map:get(v, "dx") * dt) 
            atom new_y = map:get(p, "y") + (map:get(v, "dy") * dt) 
             
            map:put(p, "x", new_x) 
            map:put(p, "y", new_y) 
        end if 
    end for 
end procedure

Questions for the Forum:

DOP & Passing Maps Everywhere:

For those who have leaned into passing std/map.e instances around as the primary payload between routines: how do you manage lifecycle and memory? Since maps in Euphoria use reference handles created via map:new(), are you manually calling map:delete() to reclaim memory, or do you stick to standard Euphoria nested sequences ({"key", value}) to let the garbage collector/reference counter manage disposal automatically?

Map Overhead vs. Flat Sequences: In C++ or Rust, ECS thrives on cache lines and contiguous memory arrays. In Euphoria, std/map.e is convenient for sparse components (e.g., only 10 out of 5,000 entities have a Flying or ParticleEmitter component). But in tight game/processing loops (60 FPS or large batches), does hashing via map:get() and map:put() become too slow compared to indexing flat sequences like positions_x[entity_id]?

Copy-on-Write vs. In-Place Map Mutation: One of the core DOP rules is state immutability, but in Euphoria, passing massive nested sequences and modifying deeply nested nodes can trigger copy-on-write branching. Does using std/map.e mitigate this by acting as a mutable handle, or does it end up being a performance bottleneck of its own?

Schema Validation via type:

DOP advocates checking schemas at boundaries rather than baking them into classes. Has anyone written custom Euphoria type definitions that validate std/map.e structures (e.g., ensuring a map passed to a system contains specific keys like "x" and "y") without burning too many cycles on every function call?

Prior Art: Has anyone built a reusable entity-manager, blackboard, or DOP-style pipeline using std/map.e in 4.1?

Curious to hear how other Euphoria devs approach data-first architecture when building larger apps.

new topic » topic index » view message » categorize

2. Re: Architectural Discussion: Implementing ECS and Data-Oriented Programming (DOP)

Posted by euphoric (admin) 3 weeks ago
280 views

Interesting thought experiment, but having gone down similar rabbit holes with Euphoria game loops and state engines back in the 4.0 beta days, I'll give you the ground truth: std/map.e will hit a wall fast if you use it like this in a high-churn loop, but you can do DOP if you play to Euphoria's actual strengths.

Here is the breakdown on your questions:

1. The Memory & Handle Trap (std/map.e)

You hit the nail on the head in question #1. std/map.e is not a first-class language type; it's a wrapper around an internal lookup table that assigns an integer handle.

If you create nested maps for every component on every entity:

Euphoria's native garbage collector will not clean them up.

If an entity dies, you can't just set world[id] = 0 and walk away. You have to write an explicit cascade: delete the entity map, delete every component map inside it, and reclaim the ID.

Forget a single map:delete(), and you have a classic C-style memory leak in a high-level interpreted language.

If you want automatic memory management and DOP, plain old nested sequences are infinitely safer because reference counting handles cleanup the millisecond an entity drops out of scope.

2 & 3. Performance: The Hash Penalty vs. Copy-on-Write

In C++, ECS is about CPU cache lines. In Euphoria, there are no cache lines-everything is an object pointer under the hood.

std/map.e uses hashing and bucket lookups. In a tight loop running at 60 FPS (or chewing through 10,000 records in a batch):

-- Doing 4 hash lookups per entity per frame: 
atom new_x = map:get(p, "x") + (map:get(v, "dx") * dt) 
The overhead of hashing string keys ("x", "dx") inside the loop is brutal.

The Euphoria Alternative: Enum/Constant Offsets on Flat Sequences If you want DOP speed without the OO nonsense, the idiomatic Euphoria way has always been flat parallel sequences or 1D arrays with enum offsets, not dynamic string maps:

-- Component indices (Zero hash overhead) 
enum POS_X, POS_Y 
enum VEL_DX, VEL_DY 
 
-- Dense flat stores (indexed directly by Entity ID) 
sequence pos = repeat({0.0, 0.0}, MAX_ENTITIES) 
sequence vel = repeat({0.0, 0.0}, MAX_ENTITIES) 
 
-- Update loop (Direct index, blistering fast): 
for id = 1 to active_count do 
    pos[id][POS_X] += vel[id][VEL_DX] * dt 
    pos[id][POS_Y] += vel[id][VEL_DY] * dt 
end for

Because pos[id] is mutated in-place and Euphoria's copy-on-write optimizer recognizes single-reference sequences, it will mutate directly in memory without cloning the entire tree, provided you don't assign temporary variables to slices right before the write.

4. Schema Validation via type

Custom types are awesome in Euphoria, but putting map:has() checks inside a type definition will destroy your frame rate if without type_check isn't enabled in release mode:

type PositionMap(map:map m) 
    return map:has(m, "x") and map:has(m, "y") 
end type

If you pass that into a procedure, Euphoria validates it on every call. Fine for debugging, but in production, you definitely want without type_check at the top of your system files, or just validate once at the factory boundary when the entity is spawned.

The Verdict

If you're writing a business app, database processor, or UI state manager where updates happen on user events (milliseconds don't matter), passing std/map.e handles around is fine and decoupled.

If you are writing an actual game or real-time simulation, ditch std/map.e for your core loop. Use integer IDs to index flat, pre-allocated sequence pools with enum constants. You get 90% of the DOP architectural cleanliness, full native garbage collection, and it will run roughly 50x faster.

Good to see someone still pushing architectural patterns here, though! What are you building that needs this?

new topic » goto parent » topic index » view message » categorize

3. Re: Architectural Discussion: Implementing ECS and Data-Oriented Programming (DOP)

Posted by jimcbrown (admin) 3 weeks ago
220 views

Interesting (and seemingly AI generated or at least AI enhanced).

Couple of good points though - could we come up with atom key = map:precompute_key(m, "x") and then use map:has_pkey(m, key) and map:get_via_pkey(m, key) ? Do we have to compute the key every time or can we just do it once, to speed up the loop?

Likewise, OE 4.2 has references - so when the top-level world[id] = 0 happens, the object can call a custom delete_routine() - perhaps we could implement a custom GC-like pass here for maps, that searches for child maps that might be unused, and deletes them. (It could be a shallow one, since if the child map has its own nested maps, then the child's own delete_routine should take care of the grandchildren, etc.)

new topic » goto parent » topic index » view message » categorize

4. Re: Architectural Discussion: Implementing ECS and Data-Oriented Programming (DOP)

Posted by jmduro 3 weeks ago
197 views

Five years ago, I proposed a way to do OOP-like programming using OEU's preprocessor.

Here is its test example, test_classes.oex:

include builtins\timedate.e   
 
function get_years(atom secs) 
  atom days = floor(secs/86400) 
  return floor(days/365.25) 
end function 
 
class person 
  public timedate birth_date = date() 
  public sequence birth_place = "" 
  public sequence mother_tongue = "" 
  procedure describe() 
    atom secs = timedate_diff(this.birth_date, date()) 
    printf(1, "\nHello! I speak %s.\n", {this.mother_tongue}) 
    printf(1, "I am %d years old. I was born in %s.\n", {get_years(secs), this.birth_place}) 
  end procedure 
 
end class 
 
person englishman = new() 
englishman.birth_date = parse_date_string("May 14 2010",{"Mmmm d yyyy"}) 
englishman.birth_place = "Liverpool" 
englishman.mother_tongue = "english" 
 
person frenchman = new() 
frenchman.birth_date = parse_date_string("August 10 1960",{"Mmmm d yyyy"}) 
frenchman.birth_place = "Thionville" 
frenchman.mother_tongue = "french" 
 
person german = new() 
german.birth_date = parse_date_string("February 11 1993",{"Mmmm d yyyy"}) 
german.birth_place = "Munich" 
german.mother_tongue = "german" 
 
englishman.describe() 
frenchman.describe() 
german.describe() 
 
maybe_any_key()

It converted this code to OEU code, test_classes.ex:

include std/datetime.e 
include std/console.e 
include classes.e 
 
------------------------------------------------------------------------------ 
 
function get_years(atom secs) 
  atom days = floor(secs/86400) 
  return floor(days/365.25) 
end function 
 
------------------------------------------------------------------------------ 
 
function describe(sequence entity, sequence params) 
  atom secs = diff(getProperty(entity, "birth_date"), now()) 
  printf(1, "\nHello! I speak %s.\n", {getProperty(entity, "mother_tongue")}) 
  if length(params) then 
    printf(1, "I am %d years old. I was born in %s (%s).\n", {get_years(secs), getProperty(entity, "birth_place"), params[1]}) 
  else 
    printf(1, "I am %d years old. I was born in %s.\n", {get_years(secs), getProperty(entity, "birth_place")}) 
  end if 
  return 0 
end function 
 
------------------------------------------------------------------------------ 
 
procedure main(sequence cmd) 
  object void 
 
  sequence TPerson = class("TPerson") 
  TPerson = addProperty(TPerson, "birth_date", now()) 
  TPerson = addProperty(TPerson, "birth_place", "") 
  TPerson = addProperty(TPerson, "mother_tongue", "") 
  TPerson = addMethod(TPerson, "describe", routine_id("describe")) 
 
  sequence englishman = classes:new(TPerson) 
  englishman = setProperty(englishman, "birth_date", datetime:new(2010, 5, 14)) 
  englishman = setProperty(englishman, "birth_place", "Liverpool") 
  englishman = setProperty(englishman, "mother_tongue", "english") 
 
  sequence frenchman = classes:new(TPerson) 
  frenchman = setProperty(frenchman, "birth_date", datetime:new(1960, 8, 10)) 
  frenchman = setProperty(frenchman, "birth_place", "Thionville") 
  frenchman = setProperty(frenchman, "mother_tongue", "french") 
 
  sequence german = classes:new(TPerson) 
  german = setProperty(german, "birth_date", datetime:new(1993, 2, 23)) 
  german = setProperty(german, "birth_place", "Munich") 
  german = setProperty(german, "mother_tongue", "german") 
 
  void = callMethod(englishman, "describe", {"Merseyside"}) 
  void = callMethod(frenchman, "describe", {"Lorraine"}) 
  void = callMethod(german, "describe", {"Bavaria"}) 
end procedure  -- main 
 
------------------------------------------------------------------------------ 
 
-- Main 
main(command_line()) 
maybe_any_key()

I don't remember all the details. I will rework this to make it run again.

Jean-Marc

new topic » goto parent » topic index » view message » categorize

5. Re: Architectural Discussion: Implementing ECS and Data-Oriented Programming (DOP)

Posted by jmduro 3 weeks ago
195 views

Here is the class library:

namespace classes 
-- classes.e 
-- do not modify this file 
 
------------------------------------------------------------------------------ 
-- INTERNAL CONSTANTS 
------------------------------------------------------------------------------ 
 
constant CLASS_NAME   =1, CLASS_PROPERTY=2,   CLASS_METHOD=3 
constant PROPERTY_NAME=1, PROPERTY_VALUE=2 
constant METHOD_NAME  =1, METHOD_ROUTINE_ID=2 
constant ENTITY_CLASS =1, ENTITY_VALUES=2 
 
------------------------------------------------------------------------------ 
-- GLOBAL CLASS FUNCTIONS 
------------------------------------------------------------------------------ 
 
-- define a Class 
public function class(sequence className) 
  return {className, {}, {}} 
end function 
 
------------------------------------------------------------------------------ 
 
-- define a Class property 
public function addProperty(sequence class, sequence propertyName, object defaultValue) 
  class[CLASS_PROPERTY] = append(class[CLASS_PROPERTY], {propertyName, defaultValue}) 
  return class 
end function 
 
------------------------------------------------------------------------------ 
 
-- define a Class method 
public function addMethod(sequence class, sequence methodName, integer routineId) 
  class[CLASS_METHOD] = append(class[CLASS_METHOD], {methodName, routineId}) 
  return class 
end function 
 
------------------------------------------------------------------------------ 
 
-- find property 
public function findProperty(sequence class, sequence property) 
  for i = 1 to length(class[CLASS_PROPERTY]) do 
    if equal(property, class[CLASS_PROPERTY][i][PROPERTY_NAME]) then 
      return i 
    end if 
  end for 
  return 0 
end function 
 
------------------------------------------------------------------------------ 
 
-- find method routine_id 
public function findMethod(sequence class, sequence method) 
  for i = 1 to length(class[CLASS_METHOD]) do 
    if equal(method, class[CLASS_METHOD][i][METHOD_NAME]) then 
      return class[CLASS_METHOD][i][METHOD_ROUTINE_ID] 
    end if 
  end for 
  return 0 
end function 
 
------------------------------------------------------------------------------ 
-- GLOBAL ENTITY FUNCTIONS 
------------------------------------------------------------------------------ 
 
-- create an Entity of the class 
public function new(sequence class) 
  sequence values =  {} 
  for i = 1 to length(class[CLASS_PROPERTY]) do 
    values = append(values, class[CLASS_PROPERTY][i][PROPERTY_VALUE]) 
  end for 
  return {class, values} 
end function 
 
------------------------------------------------------------------------------ 
 
-- get property value 
public function getProperty(sequence entity, sequence property, object defaultValue=0) 
  integer propertyID = findProperty(entity[ENTITY_CLASS], property) 
  if propertyID then 
    return entity[ENTITY_VALUES][propertyID] 
  end if 
  return defaultValue 
end function 
 
------------------------------------------------------------------------------ 
 
-- set property value 
public function setProperty(sequence entity, sequence property, object val) 
  integer propertyID = findProperty(entity[ENTITY_CLASS], property) 
  if propertyID then 
    entity[ENTITY_VALUES][propertyID] = val 
  end if 
  return entity 
end function 
 
------------------------------------------------------------------------------ 
 
-- call a method (function) within an Entity 
public function callMethod(sequence entity, sequence method, object methodParams= {}) 
  integer methodID = findMethod(entity[ENTITY_CLASS], method) 
  if methodID then 
    return call_func(methodID, {entity, methodParams}) 
  end if 
  return -1 
end function

And an outdated version of the preprocessor that I began to update but I did not finish the job because noone was interested in.

include std/io.e 
include std/text.e 
include std/regex.e 
include std/search.e 
include std/sequence.e 
include std/datetime.e 
include std/console.e 
include common.e 
 
constant ATTR=1, PROC=2, FUNC=3 
 
------------------------------------------------------------------------------ 
 
function get_next_non_white(sequence code, integer from) 
  -- skip white 
  while (code[from] = ' ') or (code[from] = '\t') do 
    from += 1 
    if from > length(code) then return 0 end if 
  end while 
  return from 
end function 
 
------------------------------------------------------------------------------ 
 
function manage_routine(sequence code, sequence lines, integer start) 
  integer from, upto 
   
  for i = start+1 to length(lines) do 
    if begins("end ", trim(lines[i])) then 
      code &= lines[i] & "\n" 
      start = i 
      exit 
    end if 
    sequence rule = "self\\.([_A-Za-z][_A-Za-z0-9]*)" 
    -- puts(f_out, rule & "\n") 
    regex r = regex:new(rule) 
    object found = regex:find_all(r, lines[i]) 
    if sequence(found) then 
      for j = 1 to length(found) do 
        {from, upto} = found[j][2] 
        sequence name = lines[i][from..upto] 
        {from, upto} = found[j][1] 
        lines[i] = replace(lines[i], sprintf("entity[%s]", {upper(name)}), from, upto) 
      end for 
      code &= lines[i] & "\n" 
      puts(f_debug, lines[i] & "\n") 
    end if 
  end for 
  return {code, start} 
end function 
 
------------------------------------------------------------------------------ 
 
procedure main(sequence cmd) 
  f_debug = open(InitialDir & "debug.log", "w") 
  debug_level = DEBUG 
  debug_time_stamp = 0 
 
  sequence inFileName, outFileName 
   
  for i = 3 to length(cmd) do 
    switch cmd[i] do 
      case "-i" then 
        inFileName = cmd[i+1] 
      case "-o" then 
        outFileName = cmd[i+1] 
    end switch 
  end for 
 
  integer f_out = open(outFileName, "w") 
  sequence lines = read_lines(inFileName) 
  integer from, upto 
  object found, found2 
  sequence class_name = "", name, params, elements = {} 
  integer start = 0 
   
  sequence code = "" 
   
  -- find class definition 
  regex r = regex:new("class (.*)$") 
  for i = 1 to length(lines) do 
    puts(f_debug, lines[i] & "\n") 
    found = regex:find(r, lines[i]) 
    if sequence(found) then 
      {from, upto} = found[2] 
      class_name = lines[i][from..upto] 
      start = i 
      exit 
    else 
      code &= lines[i] & "\n" 
    end if 
  end for 
   
  -- add target sequence attributes 
  code &= "enum type %ENUM_TYPE% %ENUM_ITEMS% end type\n\n" 
  code &= "sequence %CLASS_NAME% = {%CLASS_CONTENT%}\n\n" 
   
  -- analyze class definition 
  sequence enums = {} 
  sequence class_definition = {} 
  integer i = start+1 
  while i <= length(lines) do 
    sequence trimmed = trim(lines[i]) 
    puts(f_debug, lines[i] & "\n") 
    if match("end class", lines[i]) then 
      start = i 
      exit 
    end if 
    if begins("integer", trimmed) then 
      r = regex:new("integer (.*)$") 
      class_definition= append(class_definition, "0") 
      found = regex:find(r, lines[i]) 
      {from, upto} = found[2] 
      name = lines[i][from..upto] 
      enums = append(enums, upper(name)) 
      elements = append(elements, {name, ATTR}) 
    elsif begins("atom", trimmed) then 
      r = regex:new("atom (.*)$") 
      class_definition= append(class_definition, "0.0") 
      found = regex:find(r, lines[i]) 
      {from, upto} = found[2] 
      name = lines[i][from..upto] 
      enums = append(enums, upper(name)) 
      elements = append(elements, {name, ATTR}) 
    elsif begins("sequence", trimmed) then 
      r = regex:new("sequence (.*)$") 
      class_definition= append(class_definition, "\"\"") 
      found = regex:find(r, lines[i]) 
      {from, upto} = found[2] 
      name = lines[i][from..upto] 
      enums = append(enums, upper(name)) 
      elements = append(elements, {name, ATTR}) 
    elsif begins("object", trimmed) then 
      r = regex:new("object (.*)$") 
      class_definition= append(class_definition, "0") 
      found = regex:find(r, lines[i]) 
      {from, upto} = found[2] 
      name = lines[i][from..upto] 
      enums = append(enums, upper(name)) 
      elements = append(elements, {name, ATTR}) 
    elsif begins("datetime", trimmed) then 
      r = regex:new("datetime (.*)$") 
      class_definition= append(class_definition, "now()") 
      found = regex:find(r, lines[i]) 
      {from, upto} = found[2] 
      name = lines[i][from..upto] 
      enums = append(enums, upper(name)) 
      elements = append(elements, {name, ATTR}) 
    elsif begins("procedure", trimmed) then 
      r = regex:new("procedure (.*)\\((.*)\\)$") 
      found = regex:find(r, lines[i]) 
      {from, upto} = found[2] 
      name = lines[i][from..upto] 
      enums = append(enums, upper(name)) 
      {from, upto} = found[3] 
      params = lines[i][from..upto] 
      code &= sprintf("procedure %s(sequence entity, sequence params)\n", {name}) 
      {code, i} = manage_routine(code, lines, i) 
      elements = append(elements, {name, PROC}) 
    elsif begins("function", trimmed) then 
      r = regex:new("function (.*)\\((.*)\\)$") 
      found = regex:find(r, lines[i]) 
      {from, upto} = found[2] 
      name = lines[i][from..upto] 
      enums = append(enums, upper(name)) 
      {from, upto} = found[3] 
      params = lines[i][from..upto] 
      code &= sprintf("function %s(sequence entity, sequence params)\n", {name}) 
      {code, i} = manage_routine(code, lines, i) 
      elements = append(elements, {name, FUNC}) 
    elsif begins("end procedure", trimmed) then 
      class_definition= append(class_definition, "0") 
      code &= lines[i] & "\n" 
      code &= sprintf("%s[%s] = routine_id(\"%s\")\n", {class_name, upper(name), name}) 
    elsif begins("end function", trimmed) then 
      class_definition= append(class_definition, "0") 
      code &= lines[i] & "\n" 
      code &= sprintf("%s[%s] = routine_id(\"%s\")\n", {class_name, upper(name), name}) 
    else 
      common:error_message("Unexpected variable type!", 1) 
    end if 
    i += 1 
  end while 
  analyze_object(class_definition, class_name, DEBUG) 
  analyze_object(enums, "enums", DEBUG) 
   
  -- replace target sequence attributes 
  code = replace_all(code, "%ENUM_TYPE%", upper(class_name)) 
  code = replace_all(code, "%ENUM_ITEMS%", join(enums, ',')) 
  code = replace_all(code, "%CLASS_NAME%", class_name) 
  code = replace_all(code, "%CLASS_CONTENT%", join(class_definition, ',')) 
   
  -- add converted code 
  puts(f_out, code) 
   
  -- add internal functions 
  puts(f_out, "\n") 
  puts(f_out, "procedure class_proc(sequence entity, integer method, sequence params = {})\n") 
  puts(f_out, "  call_proc(entity[method], {entity, params})\n") 
  puts(f_out, "end procedure\n") 
  puts(f_out, "\n") 
   
  puts(f_out, "function class_func(sequence entity, integer method, sequence params = {})\n") 
  puts(f_out, "  return call_func(entity[method], {entity, params})\n") 
  puts(f_out, "end function\n") 
  puts(f_out, "\n") 
   
  -- manage class entities 
  sequence entity_name = "" 
  sequence entities = "" 
  for l = start+1 to length(lines) do 
    -- lines[l] = trim(lines[l]) 
    puts(f_debug, lines[l] & "\n") 
    sequence rule = sprintf("sequence (.*) = new\\(%s\\)$", {class_name}) 
    -- puts(f_out, rule & "\n") 
    r = regex:new(rule) 
    found = regex:find(r, lines[l]) 
    if sequence(found) then 
      {from, upto} = found[2] 
      entity_name = lines[l][from..upto] 
      entities = append(entities, entity_name) 
      printf(f_out, "sequence %s = %s\n", {entity_name, class_name}) 
    else 
      r = regex:new("([_A-Za-z0-9]*)\\.([_A-Za-z0-9]*)") 
      found = regex:find(r, lines[l]) 
      if sequence(found) then 
        {from, upto} = found[2] 
        entity_name = lines[l][from..upto] 
        if find(entity_name, entities) then  -- class referenced 
          -- check if it refers to a routine 
          {from, upto} = found[3] 
          name = lines[l][from..upto] 
          {from, upto} = found[1] 
          integer p = vlookup(name, elements, 1, 2, 0) 
          printf(f_debug, "p = %d\n", {p}) 
          switch p do 
            case ATTR then 
              lines[l] = replace(lines[l], sprintf("%s[%s]", {entity_name, upper(name)}), from, upto) 
            case PROC then 
              from = find('(', lines[l], upto+1) 
              upto = find(')', lines[l], upto+1) 
              if upto > from+1 then 
                params = lines[l][from+1..upto-1] 
              else 
                params = {} 
              end if 
              lines[l] = sprintf("class_proc(%s, %s, %s)\n", {entity_name, upper(name), params}) 
            case FUNC then 
              from = find('(', lines[l], upto+1) 
              upto = find(')', lines[l], upto+1) 
              if upto > from+1 then 
                params = lines[l][from+1..upto-1] 
              else 
                params = {} 
              end if 
              lines[l] = sprintf("class_proc(%s, %s, %s)\n", {entity_name, upper(name), params}) 
            case else 
          end switch 
        end if 
      end if 
      puts(f_out, lines[l] & "\n") 
    end if 
  end for 
  analyze_object(entities, "entities", DEBUG) 
   
  close(f_out) 
  close(f_debug) 
end procedure  -- main 
 
------------------------------------------------------------------------------ 
 
-- Main 
main(command_line()) 
maybe_any_key()

As it was a work in progress, I'm not sure it runs again without modifications.

Jean-Marc

new topic » goto parent » topic index » view message » categorize

6. Re: Architectural Discussion: Implementing ECS and Data-Oriented Programming (DOP)

Posted by jmduro 3 weeks ago
192 views

I found the last running version. Here it is, complete with no additional library.

The pre-processor, classes.ex:

include std/io.e 
include std/text.e 
include std/regex.e 
include std/search.e 
include std/sequence.e 
 
constant ATTR=1, PROC=2, FUNC=3 
 
------------------------------------------------------------------------------ 
 
function replace_all(sequence s, sequence old, sequence new) 
  integer lg = length(old) 
  integer index = match(old, s, 1) 
  while index != 0 do 
    s = replace(s, new, index, index+lg-1) 
    index = match(old, s, index+lg) 
  end while 
  return s 
end function 
 
------------------------------------------------------------------------------ 
 
function get_next_non_white(sequence code, integer from) 
  -- skip white 
  while (code[from] = ' ') or (code[from] = '\t') do 
    from += 1 
    if from > length(code) then return 0 end if 
  end while 
  return from 
end function 
 
------------------------------------------------------------------------------ 
 
function manage_routine(sequence code, sequence lines, integer start) 
  integer from, upto 
   
  for i = start+1 to length(lines) do 
    if begins("end ", trim(lines[i])) then 
      code &= lines[i] & "\n" 
      start = i 
      exit 
    end if 
    sequence rule = "self\\.([_A-Za-z][_A-Za-z0-9]*)" 
    regex r = regex:new(rule) 
    object found = regex:find_all(r, lines[i]) 
    if sequence(found) then 
      for j = 1 to length(found) do 
        {from, upto} = found[j][2] 
        sequence name = lines[i][from..upto] 
        {from, upto} = found[j][1] 
        lines[i] = replace(lines[i], sprintf("entity[%s]", {upper(name)}), from, upto) 
      end for 
      code &= lines[i] & "\n" 
    end if 
  end for 
  return {code, start} 
end function 
 
------------------------------------------------------------------------------ 
 
sequence cmds = command_line() 
sequence inFileName, outFileName 
 
for i = 3 to length(cmds) do 
  switch cmds[i] do 
    case "-i" then 
      inFileName = cmds[i+1] 
    case "-o" then 
      outFileName = cmds[i+1] 
  end switch 
end for 
 
integer f_out = open(outFileName, "w") 
sequence lines = read_lines(inFileName) 
integer from, upto 
object found, found2 
sequence class_name = "", name, params, elements = {} 
integer start = 0 
 
sequence code = "" 
 
-- find class definition 
regex r = regex:new("class (.*)$") 
for i = 1 to length(lines) do 
  found = regex:find(r, lines[i]) 
  if sequence(found) then 
    {from, upto} = found[2] 
    class_name = lines[i][from..upto] 
    start = i 
    exit 
  else 
    code &= lines[i] & "\n" 
  end if 
end for 
 
-- add target sequence attributes 
code &= "enum type %ENUM_TYPE% %ENUM_ITEMS% end type\n\n" 
code &= "sequence %CLASS_NAME% = {%CLASS_CONTENT%}\n\n" 
 
-- analyze class definition 
sequence enums = {} 
sequence class_definition = {} 
integer i = start+1 
while i <= length(lines) do 
  sequence trimmed = trim(lines[i]) 
  if match("end class", lines[i]) then 
    start = i 
    exit 
  end if 
  if begins("integer", trimmed) then 
    r = regex:new("integer (.*)$") 
    class_definition= append(class_definition, "0") 
    found = regex:find(r, lines[i]) 
    {from, upto} = found[2] 
    name = lines[i][from..upto] 
    enums = append(enums, upper(name)) 
    elements = append(elements, {name, ATTR}) 
  elsif begins("atom", trimmed) then 
    r = regex:new("atom (.*)$") 
    class_definition= append(class_definition, "0.0") 
    found = regex:find(r, lines[i]) 
    {from, upto} = found[2] 
    name = lines[i][from..upto] 
    enums = append(enums, upper(name)) 
    elements = append(elements, {name, ATTR}) 
  elsif begins("sequence", trimmed) then 
    r = regex:new("sequence (.*)$") 
    class_definition= append(class_definition, "\"\"") 
    found = regex:find(r, lines[i]) 
    {from, upto} = found[2] 
    name = lines[i][from..upto] 
    enums = append(enums, upper(name)) 
    elements = append(elements, {name, ATTR}) 
  elsif begins("object", trimmed) then 
    r = regex:new("object (.*)$") 
    class_definition= append(class_definition, "0") 
    found = regex:find(r, lines[i]) 
    {from, upto} = found[2] 
    name = lines[i][from..upto] 
    enums = append(enums, upper(name)) 
    elements = append(elements, {name, ATTR}) 
  elsif begins("datetime", trimmed) then 
    r = regex:new("datetime (.*)$") 
    class_definition= append(class_definition, "now()") 
    found = regex:find(r, lines[i]) 
    {from, upto} = found[2] 
    name = lines[i][from..upto] 
    enums = append(enums, upper(name)) 
    elements = append(elements, {name, ATTR}) 
  elsif begins("procedure", trimmed) then 
    r = regex:new("procedure (.*)\\((.*)\\)$") 
    found = regex:find(r, lines[i]) 
    {from, upto} = found[2] 
    name = lines[i][from..upto] 
    enums = append(enums, upper(name)) 
    {from, upto} = found[3] 
    params = lines[i][from..upto] 
    code &= sprintf("procedure %s(sequence entity, sequence params)\n", {name}) 
    {code, i} = manage_routine(code, lines, i) 
    elements = append(elements, {name, PROC}) 
  elsif begins("function", trimmed) then 
    r = regex:new("function (.*)\\((.*)\\)$") 
    found = regex:find(r, lines[i]) 
    {from, upto} = found[2] 
    name = lines[i][from..upto] 
    enums = append(enums, upper(name)) 
    {from, upto} = found[3] 
    params = lines[i][from..upto] 
    code &= sprintf("function %s(sequence entity, sequence params)\n", {name}) 
    {code, i} = manage_routine(code, lines, i) 
    elements = append(elements, {name, FUNC}) 
  elsif begins("end procedure", trimmed) then 
    class_definition= append(class_definition, "0") 
    code &= lines[i] & "\n" 
    code &= sprintf("%s[%s] = routine_id(\"%s\")\n", {class_name, upper(name), name}) 
  elsif begins("end function", trimmed) then 
    class_definition= append(class_definition, "0") 
    code &= lines[i] & "\n" 
    code &= sprintf("%s[%s] = routine_id(\"%s\")\n", {class_name, upper(name), name}) 
  else 
    printf(2, "Unexpected variable type at line %d!: \n", {i, lines[i]}) 
    abort(1) 
  end if 
  i += 1 
end while 
 
-- replace target sequence attributes 
code = replace_all(code, "%ENUM_TYPE%", upper(class_name)) 
code = replace_all(code, "%ENUM_ITEMS%", join(enums, ',')) 
code = replace_all(code, "%CLASS_NAME%", class_name) 
code = replace_all(code, "%CLASS_CONTENT%", join(class_definition, ',')) 
 
-- add converted code 
puts(f_out, code) 
 
-- add internal functions 
puts(f_out, "\n") 
puts(f_out, "procedure class_proc(sequence entity, integer method, sequence params = {})\n") 
puts(f_out, "  call_proc(entity[method], {entity, params})\n") 
puts(f_out, "end procedure\n") 
puts(f_out, "\n") 
 
puts(f_out, "function class_func(sequence entity, integer method, sequence params = {})\n") 
puts(f_out, "  return call_func(entity[method], {entity, params})\n") 
puts(f_out, "end function\n") 
puts(f_out, "\n") 
 
-- manage class entities 
sequence entity_name = "" 
sequence entities = "" 
for l = start+1 to length(lines) do 
  sequence rule = sprintf("sequence (.*) = new\\(%s\\)$", {class_name}) 
  r = regex:new(rule) 
  found = regex:find(r, lines[l]) 
  if sequence(found) then 
    {from, upto} = found[2] 
    entity_name = lines[l][from..upto] 
    entities = append(entities, entity_name) 
    printf(f_out, "sequence %s = %s\n", {entity_name, class_name}) 
  else 
    r = regex:new("([_A-Za-z0-9]*)\\.([_A-Za-z0-9]*)") 
    found = regex:find(r, lines[l]) 
    if sequence(found) then 
      {from, upto} = found[2] 
      entity_name = lines[l][from..upto] 
      if find(entity_name, entities) then  -- class referenced 
        -- check if it refers to a routine 
        {from, upto} = found[3] 
        name = lines[l][from..upto] 
        {from, upto} = found[1] 
        integer p = vlookup(name, elements, 1, 2, 0) 
        switch p do 
          case ATTR then 
            lines[l] = replace(lines[l], sprintf("%s[%s]", {entity_name, upper(name)}), from, upto) 
          case PROC then 
            from = find('(', lines[l], upto+1) 
            upto = find(')', lines[l], upto+1) 
            if upto > from+1 then 
              params = lines[l][from+1..upto-1] 
            else 
              params = {} 
            end if 
            lines[l] = sprintf("class_proc(%s, %s, %s)\n", {entity_name, upper(name), params}) 
          case FUNC then 
            from = find('(', lines[l], upto+1) 
            upto = find(')', lines[l], upto+1) 
            if upto > from+1 then 
              params = lines[l][from+1..upto-1] 
            else 
              params = {} 
            end if 
            lines[l] = sprintf("class_proc(%s, %s, %s)\n", {entity_name, upper(name), params}) 
          case else 
        end switch 
      end if 
    end if 
    puts(f_out, lines[l] & "\n") 
  end if 
end for 
 
close(f_out)

The OOP-style test program, test_classes.oex:

include std/datetime.e 
include std/console.e 
 
function get_years(atom secs) 
  atom days = floor(secs/86400) 
  return floor(days/365.25) 
end function 
 
class person 
  datetime birth_date 
  sequence birth_place 
  sequence mother_tongue 
  procedure describe(sequence params) 
    atom secs = diff(self.birth_date, now()) 
    printf(1, "\nHello! I speak %s.\n", {self.mother_tongue}) 
    if length(params) then 
      printf(1, "I am %d years old. I was born in %s (%s).\n", {get_years(secs), self.birth_place, params[1]}) 
    else 
      printf(1, "I am %d years old. I was born in %s.\n", {get_years(secs), self.birth_place}) 
    end if 
  end procedure 
end class 
 
sequence englishman = new(person) 
englishman.birth_date = datetime:new(2010, 5, 14) 
englishman.birth_place = "Liverpool" 
englishman.mother_tongue = "english" 
 
sequence frenchman = new(person) 
frenchman.birth_date = datetime:new(1960, 8, 10) 
frenchman.birth_place = "Thionville" 
frenchman.mother_tongue = "french" 
 
sequence german = new(person) 
german.birth_date = datetime:new(1993, 2, 23) 
german.birth_place = "Munich" 
german.mother_tongue = "german" 
 
englishman.describe({"Merseyside"}) 
frenchman.describe({"Lorraine"}) 
german.describe({"Bavaria"}) 
 
maybe_any_key()

The shell to run the program with the pre-processor, runme.bat (Linux):

eui -p oex:classes.ex test_classes.oex

The result:

duro@hp-zbook15g2:~/Data/Euphoria/v4/myOOP$ ./runme.bat test_classes.oex  
 
Hello! I speak english. 
I am 16 years old. I was born in Liverpool (Merseyside). 
 
Hello! I speak french. 
I am 66 years old. I was born in Thionville (Lorraine). 
 
Hello! I speak german. 
I am 33 years old. I was born in Munich (Bavaria).

The resulting converted program, test_classes.pp.oex, used in the background:

include std/datetime.e 
include std/console.e 
 
function get_years(atom secs) 
  atom days = floor(secs/86400) 
  return floor(days/365.25) 
end function 
 
enum type PERSON BIRTH_DATE,BIRTH_PLACE,MOTHER_TONGUE,DESCRIBE end type 
 
sequence person = {now(),"","",0} 
 
procedure describe(sequence entity, sequence params) 
    atom secs = diff(entity[BIRTH_DATE], now()) 
    printf(1, "\nHello! I speak %s.\n", {entity[MOTHER_TONGUE]}) 
    if length(params) then 
      printf(1, "I am %d years old. I was born in %s (%s).\n", {get_years(secs), entity[BIRTH_PLACE], params[1]}) 
    else 
      printf(1, "I am %d years old. I was born in %s.\n", {get_years(secs), entity[BIRTH_PLACE]}) 
    end if 
  end procedure 
person[DESCRIBE] = routine_id("describe") 
 
procedure class_proc(sequence entity, integer method, sequence params = {}) 
  call_proc(entity[method], {entity, params}) 
end procedure 
 
function class_func(sequence entity, integer method, sequence params = {}) 
  return call_func(entity[method], {entity, params}) 
end function 
 
 
sequence englishman = person 
englishman[BIRTH_DATE] = datetime:new(2010, 5, 14) 
englishman[BIRTH_PLACE] = "Liverpool" 
englishman[MOTHER_TONGUE] = "english" 
 
sequence frenchman = person 
frenchman[BIRTH_DATE] = datetime:new(1960, 8, 10) 
frenchman[BIRTH_PLACE] = "Thionville" 
frenchman[MOTHER_TONGUE] = "french" 
 
sequence german = person 
german[BIRTH_DATE] = datetime:new(1993, 2, 23) 
german[BIRTH_PLACE] = "Munich" 
german[MOTHER_TONGUE] = "german" 
 
class_proc(englishman, DESCRIBE, {"Merseyside"}) 
 
class_proc(frenchman, DESCRIBE, {"Lorraine"}) 
 
class_proc(german, DESCRIBE, {"Bavaria"}) 
 
 
maybe_any_key()

Jean-Marc

new topic » goto parent » topic index » view message » categorize

7. Re: Architectural Discussion: Implementing ECS and Data-Oriented Programming (DOP)

Posted by euphoric (admin) 3 weeks ago
188 views

jmduro said...

Here is the class library:

Jean-Marc, this thread is about DATA-oriented programming, not OBJECT-oriented programming. grin

Do your algorithms apply to DOP as well?

new topic » goto parent » topic index » view message » categorize

8. Re: Architectural Discussion: Implementing ECS and Data-Oriented Programming (DOP)

Posted by jmduro 3 weeks ago
191 views

Oops sorry, I don't even know what DOP is about. I just know that the same technique can be applied to data structures. I did some experiments with that in the past.

Jean-Marc

new topic » goto parent » topic index » view message » categorize

Search

Quick Links

User menu

Not signed in.

OpenEuphoria

1. Architectural Discussion: Implementing ECS and Data-Oriented Programming (DOP)

2. Re: Architectural Discussion: Implementing ECS and Data-Oriented Programming (DOP)

3. Re: Architectural Discussion: Implementing ECS and Data-Oriented Programming (DOP)

4. Re: Architectural Discussion: Implementing ECS and Data-Oriented Programming (DOP)

5. Re: Architectural Discussion: Implementing ECS and Data-Oriented Programming (DOP)

6. Re: Architectural Discussion: Implementing ECS and Data-Oriented Programming (DOP)

7. Re: Architectural Discussion: Implementing ECS and Data-Oriented Programming (DOP)

8. Re: Architectural Discussion: Implementing ECS and Data-Oriented Programming (DOP)

Search

Include:

Quick Links

User menu

Misc Menu