OpenEuphoria

1. C Struct Support

• Posted by mattlewis (admin) Dec 29, 2010
• 2094 views

Now that 4.0 is out, it's time to start thinking about...4.1! In fact, there is a wiki page with a high level overview of what's being planned:

EuphoriaRoadmap

One of the items on that list is for support for dealing with C-style structures in memory (not to be confused with data structures of native euphoria objects). Anyone who has dealt with an external API has probably needed to deal with these, managing offsets and data formats, peeking and poking...The goal is to make all of this easier for the euphoria programmer.

I've started playing around with some ideas, and put the code up into a mercurial repository here:

http://scm.openeuphoria.org/hg/mem_struct/

(You can also get it zipped: http://scm.openeuphoria.org/hg/mem_struct/archive/tip.zip )

That code has a couple of preprocessors for declaring and using structures, as well as a support library that implements the structures. This is really just something for testing the syntax, etc. A final implementation would probably be built into euphoria. There's a README with some documentation explaining some of the ideas more fully than this post, along with some examples.

In any case, it allows the coder to build structures, such as:

mem_struct numbers 
	integer(8) byte 
	integer(16) word 
	integer(32) dword 
	integer(64) longlong 
	atom(32) float 
	atom(64) double 
end mem_struct 
 
mem_union int_or_float 
	integer(32) int 
	atom(32)    float 
end mem_union 
 
mem_struct wc_str 
	pointer( integer( 16, array, null_terminated ) ) wchars 
end mem_struct 

...and then to be used like:

atom example = allocate( sizeof("basic:numbers") ) 
example.basic:numbers.byte = 1 
example.basic:numbers.word = 2 
example.basic:numbers.dword = 3 
example.basic:numbers.longlong = 4 
example.basic:numbers.float = 5.5 
example.basic:numbers.double = 6.6 
 
printf( 1, "byte: %g\n", example.basic:numbers.byte ) 
printf( 1, "word: %g\n", example.basic:numbers.word ) 
printf( 1, "dword: %g\n", example.basic:numbers.dword ) 
printf( 1, "longlong: %g\n", example.basic:numbers.longlong ) 
printf( 1, "float: %g\n", example.basic:numbers.float ) 
printf( 1, "double: %g\n", example.basic:numbers.double ) 

The namespacing used here is a bit of a hack to make the preprocessors easier to build (please don't get hung up on that aspect).

I'd like to get some discussion going about this topic (but please feel free to ask questions or make other suggestions about other 4.1 topics).

Matt

Forked into: EUPHORIA struct support

3. Re: C Struct Support

• Posted by mattlewis (admin) Dec 29, 2010
• 1883 views

Yes. There are preprocessor/proof of concept examples that do this. I've also got some examples of using unions.

Yes.

I'm not sure what you mean by this. This won't affect the direct interface between euphoria code and DLLs. It should make it easier to communicate with those DLLs that use memory structures. Basically, you'll use the same pointer you would now, except that instead of using peek() and poke(), you can use a more descriptive syntax. Also, you shouldn't have to manually calculate offsets.

Ideally, we'll be able to define and use any sort of structure that you'd encounter in C code. At this time, I'm not planning to support bitfields, since they're non-standard, and compilers appear to implement them differently.

The main benefit will be for people who write code that directly interfaces with external libraries or the back end of the interpreter. I recommend taking a look at the code I posted earlier. I have several examples, including the Euphoria Symbol Table as it exists in the interpreter, plus a few Win32 structures I grabbed from Win32Lib.

So anyone who writes code that does this sort of thing should chime in. If I don't get any feedback, you'll probably be stuck with whatever I come up with.

Matt

5. Re: C Struct Support

• Posted by mattlewis (admin) Dec 29, 2010
• 1943 views

This feature won't deal with name mangling, if that's what you mean. That's another thing that's compiler dependent. It's just a much nicer way of dealing with data structures in RAM than peek and poke.

The easiest way I've found to wrap C code is to write a DLL that wraps the C API and exposes an "extern C" interface. That's what I've done with wxEuphoria.

Matt

7. Re: C Struct Support

• Posted by coconut Dec 29, 2010
• 1872 views

Are those structure definitions like types so that one can declare variables of those types and to transfert data from struct to sequence I expect a simple syntax like:

mem_struct numbers  
	integer(8) byte  
	integer(16) word  
	integer(32) dword  
	integer(64) longlong  
	atom(32) float  
	atom(64) double  
end mem_struct  
 
--numbers is a data type 
numbers a 
--  is not a data type 
atom a 
a=alloc(sizeof(numbers)) 
--or better 
atom a=alloc(numbers) 
-- and for transfert of data between sequence and structure 
a={32,65535,-128000,1386836683, 3.1416, 3.1e100} 
-- if length(sequence) < struct_member_count what append? zero trailing members? 
-- the inverse should be as simple as 
sequence  b=a 

jacques

9. Re: C Struct Support

• Posted by eumario Dec 29, 2010
• 1898 views

mem_struct numbers  
	integer(8) byte  
	integer(16) word  
	integer(32) dword  
	integer(64) longlong  
	atom(32) float  
	atom(64) double  
end mem_struct  
 
--numbers is a data type 
numbers a 
--  is not a data type 
atom a 
a=alloc(sizeof(numbers)) 
--or better 
atom a=alloc(numbers) 
-- and for transfert of data between sequence and structure 
a={32,65535,-128000,1386836683, 3.1416, 3.1e100} 
-- if length(sequence) < struct_member_count what append? zero trailing members? 
-- the inverse should be as simple as 
sequence  b=a 

This is the kind of issue that I am running in with myself with doing my OOP Setup. I'm going to look into your pre-processor, and may be merging in your mem struct with my OOP setup.

11. Re: C Struct Support

• Posted by jimcbrown (admin) Dec 30, 2010
• 1804 views

cstruct:structure("APPBARDATA",""" 

    DWORD  cbSize;           // sizeof(APPBARDATA)  
    HWND   hWnd;             // handle of appbar  
    UINT   uCallbackMessage; // see below  
    UINT   uEdge;            // see below  
    RECT   rc;               // see below  
    LPARAM lParam; 
""") 

 
ABD = cstruct:new("APPBARDATA") 
 
cstruct:set(ABD,"uEdge",ABE_LEFT) 
? cstruct:get(ABD,"uEdge") 

Oh, I approve!

One practical note on the "copy & paste from C code" bit though:

But, LPARAM, RECT, etc are not defined on e.g. Linux/GNU. Aside from the basic C types (char, short, long, etc) how is the library/preprocessor to know what a given type means? If we hardcode it to know about DWORD, etc should we also hardcode it to know about uint8_t, off_t, etc?

13. Re: C Struct Support

• Posted by raseunew Dec 30, 2010
• 1836 views

here's my first take on C structure support

 
cstruct.e 
 
namespace cstruct 
 
/* 

   reserved structure primitive names 
   (need associated byte sizes) 
*/ 
 
   int8 
   int16 
   int32 
   int64 
   uint8 
   uint16 
   uint32 
   uint64 
   float 
   double 
   byte 
   char 
   ubyte 
   uchar 
   word 
   dword 
   longlong 
   pointer 
 
struct   --// public type = atom 
 
--// create new memstruct 
 
public function new(sequence ns_struct) 
   return allocate(sizeof(ns_struct)) 
end function 
 
/* 

   string 
*/ 
mem_struct c_str 
   pointer( integer( 8, array, null_terminated ) ) chars 
end mem_struct 
 
/* 

   wide string 
*/ 
mem_struct wc_str 
   pointer( integer( 16, array, null_terminated ) ) wchars 
end mem_struct 
 
/* 

   allocate array of [n] items 
 
   expands to TYPE(TYPESIZE, array, N) n 
*/ 
mem_struct vector[N, TYPE=int32] 
   TYPE[N] v 
end mem_struct 
 
usage : 
 
basic.e 
 
namespace basic 
 
include cstruct.e as cs 
 
/* 

   person structure 
*/ 
 
mem_struct person 
   cs:uchar[32] name     null_terminated 
   cs:int32     age 
   cs:c_str     company 
   cs:wc_str    address1 
end mem_struct 
 
usage 2: 
 
test.e 
 
include cstruct.e as cs 
include basic.e   as basic 
 
/* 

   array of int32 
 
   expands to integer(32, array, 10) 
*/ 
 
cs:struct A = cs:new(basic:vector[10]) 
 
/* 

   array of int64 
 
   expands to integer(64, array, 10) 
*/ 
 
cs:struct B = cs:new(basic:vector[10, cs:int64]) 
 
--// single person 
 
cs:struct P = cs:new(basic:person()) 
 
--// assignment 
 
P.basic.person.name     = cs:string("My Name") 
P.basic:person.age      = 44 
P.basic.person.company  = cs:string("My Company") 
P.basic.person.address1 = cs:w_string("My Address") 
 
--// pointer dereferencing 
 
printf(1, "name     = %s\n", { P.basic:person.name.* }) 
printf(1, "age      = %d\n", { P.basic:person.age }) 
printf(1, "company  = %s\n", { P.basic:person.company.* }) 
printf(1, "address1 = %s\n", { P.basic:person.address1.wchars }) 
 
--// pointer address @ 
 
printf(1, "pointer address = %d\n", { @P.basic:person.name }) 
 
--// array of persons 
 
cs:struct D = cs:new(basic:vector[10, basic:person()]) 
 
/* 

   vector/array indexing 
 
   instance . [idx] 
*/ 
 
cs:struct E = cs:new(basic:vector[10]) 
 
for i = 0 to 9 do 
   --// implicit v member access 
   E.[i] = i 
   --// explicit v member access 
   E.v[i] = i 
end for 
 

15. Re: C Struct Support

• Posted by raseunew Dec 30, 2010
• 1807 views

also i like coconut's idea of assigning to a structure

--or better  
atom a=alloc(numbers)  
-- and for transfert of data between sequence and structure  
a={32,65535,-128000,1386836683, 3.1416, 3.1e100}  

but with the structure name as a prefix

mem_struct person 
   cs:c_str name 
   cs:int32 age 
   cs:c_str addr1 
   cs:c_str addr2 
end mem_struct 
 
cs:struct P = cs:new(person()) 
P           = person:{ "my name", 44, "my address1", "my address2" }