1. Preliminary libffi progress

I've been playing around with libffi and here's what I've come up with so far.

  • Structures: You can create new "types" (using ffi_type) that can describe direct values like int, float, etc. or you can describe structures which are basically an array of types. Then you can get back the offsets of these types using ffi_get_struct_offsets(). Combined with libffi's built-in knowledge of native C types, this allows me to describe, allocate, and then peek and poke structures with very little effort.

  • C functions: You can pass any of these new "types" you've created (including structures!) to a Call InterFace or "CIF", and then allocate the required memory for arguments and return value, then perform the call with ffi_call(). libffi will take care of figuring out how to pass and receive values with its built-in knowledge of ABIs and type sizes and alignments. And it can even pass and receive structures by value!

  • Call-backs: I haven't explored this much yet, but libffi also supports creating call-backs that can receive a variable number of arguments, as well as receive non-integer (floating point) values and structures by value. These are things that Euphoria currently does not support. I need to do more digging on this but so far it looks promising.

Internally, I've created std/ffi.e that completely replaces, but remains code-compatible with, std/dll.e. What I mean by "code compatible" is that the API doesn't change, but the ABI - implementation and underlying values (like constants) - are completely different. From the end-programmer's perspective, replacing std/dll.e with std/ffi.e should be 100% compatible as long as you weren't doing anything funny with the actual values of things like C_POINTER, etc.

The biggest change is the addition of what I'm calling define_c_type() (to parallel define_c_var/func/proc) which allows you to build a new type from existing types. I also created special types based on C_POINTER for C_STRING and C_WSTRING that will automatically allocate/poke/free string values during a function call.

Here's an example of what I have working so far (at least the basic Raylib demo works). I'm looking to solicit feedback at this stage. I feel this is more worthwhile than the existing memstruct approach. So if there's support for this, I'd like to get it worked into 4.2 now, instead of memstruct. Then we can revisit building classes and structures into the language syntax later.

include std/ffi.e 
 
-- Vector2, 2 components 
--typedef struct Vector2 { 
--    float x;                // Vector x component 
--    float y;                // Vector y component 
--} Vector2; 
public constant RL_VECTOR2 = define_c_type({ 
    C_FLOAT, -- x // Vector x component 
    C_FLOAT  -- y // Vector y component 
}) 
 
-- Color, 4 components, R8G8B8A8 (32bit) 
--typedef struct Color { 
--    unsigned char r;        // Color red value 
--    unsigned char g;        // Color green value 
--    unsigned char b;        // Color blue value 
--    unsigned char a;        // Color alpha value 
--} Color; 
public constant RL_COLOR = define_c_type({ 
    C_UCHAR, -- r // Color red value 
    C_UCHAR, -- g // Color green value 
    C_UCHAR, -- b // Color blue value 
    C_UCHAR  -- a // Color alpha value 
}) 
 
atom raylib = open_dll( "raylib.dll" ) 
 
constant xInitWindow = define_c_proc( raylib, "+InitWindow", {C_INT,C_INT,C_STRING} ) 
constant xGetWindowPosition = define_c_func( raylib, "+GetWindowPosition", {}, RL_VECTOR2 ) 
constant xClearBackground = define_c_proc( raylib, "+ClearBackground", {RL_COLOR} ) 
 
public procedure InitWindow( integer width, integer height, sequence title ) 
    c_proc( xInitWindow, {width,height,title} ) -- string is allocated/freed automatically 
end procedure 
 
public function GetWindowPosition() 
    return c_func( xGetWindowPosition, {} ) -- returns {x,y} 
end function 
 
public procedure ClearBackground( sequence color ) 
    c_proc( xClearBackground, {color} ) -- color is {r,g,b,a} 
end procedure 

-Greg

new topic     » topic index » view message » categorize

2. Re: Preliminary libffi progress

This looks great Greg. I bet this library will come in handy wrapping other libraries too. Also, looks like it makes it a lot easier to wrap libraries too.

EDIT: Is there a link where we can see the code to libffi.e so far?

new topic     » goto parent     » topic index » view message » categorize

3. Re: Preliminary libffi progress

Icy_Viking said...

Is there a link where we can see the code to libffi.e so far?

I need to clean it up a bit and then I'll push it to GitHub and share the link. Keep in mind that this is a temporary approach and ultimately libffi would be compiled into the backend and then exposed using machine_func().

Edit: putting together some rough documentation, I came up with this feature comparison table:

Feature std/dll.e std/ffi.e
Define C variables YES YES
Define C functions YES YES
Define machine functions YES YES
STDCALL convention YES YES
CDECL convention YES YES
FASTCALL convention NO YES
PASCAL convention NO YES
THISCALL convention NO YES
Pass structures by value NO YES
Receive structures by value NO YES
Variable-length call-backs NO YES
Floating-point call-backs NO YES

-Greg

new topic     » goto parent     » topic index » view message » categorize

4. Re: Preliminary libffi progress

ghaberek said...
Icy_Viking said...

Is there a link where we can see the code to libffi.e so far?

I need to clean it up a bit and then I'll push it to GitHub and share the link. Keep in mind that this is a temporary approach and ultimately libffi would be compiled into the backend and then exposed using machine_func().

Edit: putting together some rough documentation, I came up with this feature comparison table:

Feature std/dll.e std/ffi.e
Define C variables YES YES
Define C functions YES YES
Define machine functions YES YES
STDCALL convention YES YES
CDECL convention YES YES
FASTCALL convention NO YES
PASCAL convention NO YES
THISCALL convention NO YES
Pass structures by value NO YES
Receive structures by value NO YES
Variable-length call-backs NO YES
Floating-point call-backs NO YES

-Greg

This is looking great so far Greg, even if its in the early stages. I think this will really help in wrapping libraries. Making it much easier to wrap libraries and not have to make shim libraries or declaring structs as separate entities.

new topic     » goto parent     » topic index » view message » categorize

5. Re: Preliminary libffi progress

As promised, here's the current working code: https://github.com/ghaberek/libffi-euphoria. I've included Win32 and Win64 binaries for raylib and libffi and a couple of working raylib demos.

You should be able to run the raylib demos on Windows with the following steps. More work is need to get going on Linux but AFAICT libffi is in the Debian repos so "apt install libffi" should work.

git clone https://github.com/ghaberek/libffi-euphoria 
cd libffi-euphoria 
copy bin\win64\*.dll C:\Euphoria\bin 
eui examples\core_basic_window.ex 

-Greg

new topic     » goto parent     » topic index » view message » categorize

6. Re: Preliminary libffi progress

ghaberek said...

As promised, here's the current working code: https://github.com/ghaberek/libffi-euphoria. I've included Win32 and Win64 binaries for raylib and libffi and a couple of working raylib demos.

You should be able to run the raylib demos on Windows with the following steps. More work is need to get going on Linux but AFAICT libffi is in the Debian repos so "apt install libffi" should work.

git clone https://github.com/ghaberek/libffi-euphoria 
cd libffi-euphoria 
copy bin\win64\*.dll C:\Euphoria\bin 
eui examples\core_basic_window.ex 

-Greg

Thanks Greg. If I can understand the code base well enough I wouldn't mind helping you out with this. This might work better than having a struct function. Though I do think Euphoria would benefit from a struct type of some kind.

new topic     » goto parent     » topic index » view message » categorize

7. Re: Preliminary libffi progress

Icy_Viking said...

This might work better than having a struct function. Though I do think Euphoria would benefit from a struct type of some kind.

If you wish to compare, here is a version without libffi that can run the core_basic_windows.ex demo:

include std/dll.e  
include std/machine.e  
include structs.e 
 
public constant LIGHTGRAY  = { 200, 200, 200, 255 } -- Light Gray 
 
------------------------------------------------------------------------------------ 
-- Structures Definition 
------------------------------------------------------------------------------------ 
 
-- Vector2, 2 components  
public sequence RL_VECTOR2 = 
"typedef struct Vector2 {" & 
"  float x;" &                -- Vector x component  
"  float y;" &                -- Vector y component  
"} Vector2;" 
  
-- Vector3, 3 components 
public sequence RL_VECTOR3 = 
"typedef struct Vector3 {" & 
"    float x;" &                -- Vector x component 
"    float y;" &                -- Vector y component 
"    float z;" &                -- Vector z component 
"} Vector3;" 
 
-- Color, 4 components, R8G8B8A8 (32bit) 
public sequence RL_COLOR = 
"typedef struct Color {" & 
"    unsigned_char r;" &        -- Color red value  
"    unsigned_char g;" &        -- Color green value  
"    unsigned_char b;" &        -- Color blue value  
"    unsigned_char a;" &        -- Color alpha value  
"} Color;"  
  
sequence Vector2 = allocateStructure(RL_VECTOR2) 
sequence Vector3 = allocateStructure(RL_VECTOR3) 
sequence Color = allocateStructure(RL_COLOR) 
 
constant raylib = open_dll( "raylib.dll" ), 
  xInitWindow        = define_c_proc( raylib, "+InitWindow", {C_INT,C_INT,C_POINTER} ), 
  xWindowShouldClose = define_c_func( raylib, "+WindowShouldClose", {}, C_BOOL ), 
  xCloseWindow       = define_c_proc( raylib, "+CloseWindow", {} ), 
  xGetWindowPosition = define_c_func( raylib, "+GetWindowPosition", {}, C_POINTER ), 
  xSetTargetFPS      = define_c_proc( raylib, "+SetTargetFPS", {C_INT} ), 
  xClearBackground   = define_c_proc( raylib, "+ClearBackground", {C_POINTER} ), 
  xBeginDrawing      = define_c_proc( raylib, "+BeginDrawing", {} ), 
  xEndDrawing        = define_c_proc( raylib, "+EndDrawing", {} ), 
  xDrawText          = define_c_proc( raylib, "+DrawText", {C_POINTER,C_INT,C_INT,C_INT,C_POINTER} ), 
$ 
 
public procedure InitWindow( integer width, integer height, sequence title ) 
	c_proc( xInitWindow, {width,height,allocate_string(title)} ) -- title string is allocated/freed automatically 
end procedure 
 
public function WindowShouldClose() 
	return c_func( xWindowShouldClose, {} ) 
end function 
 
public procedure CloseWindow() 
	c_proc( xCloseWindow, {} ) 
end procedure 
 
public function GetWindowPosition() 
	free(Vector2[SU_ADDRESS]) 
	Vector2[SU_ADDRESS] = c_func( xGetWindowPosition, {} ) -- returns {x,y} 
	return readStructure(Vector2) 
end function 
 
public procedure SetTargetFPS( integer fps ) 
	c_proc( xSetTargetFPS, {fps} ) 
end procedure 
 
public procedure ClearBackground( sequence color ) 
	writeStructure(Color, color ) 
	c_proc( xClearBackground, Color[SU_ADDRESS] ) -- color is {r,g,b,a} 
end procedure 
 
public procedure BeginDrawing() 
	c_proc( xBeginDrawing, {} ) 
end procedure 
 
public procedure EndDrawing() 
	c_proc( xEndDrawing, {} ) 
end procedure 
 
public procedure DrawText( sequence text, integer posX, integer posY, integer fontSize, sequence color ) 
	writeStructure(Color, color ) 
	c_proc( xDrawText, {allocate_string(text),posX,posY,fontSize,Color[SU_ADDRESS]} ) 
end procedure 
 
new topic     » goto parent     » topic index » view message » categorize

8. Re: Preliminary libffi progress

Looks like there is multiple ways to go about this. However the FFI option looks much easier.

When running the 32-bit version, I get the following error:

ffi.e:775 in function define_c_func() ffi_prep_cif() returned FFI_BAD_ABI (2)

The 64-bit version doesn't show any errors, but doesn't run, I assume due to not having a 64-bit installtion of Euphoria, which I need to do.

new topic     » goto parent     » topic index » view message » categorize

9. Re: Preliminary libffi progress

jmduro said...

If you wish to compare, here is a version without libffi that can run the core_basic_windows.ex demo:

It looks like you're allocating the structures to memory and then passing the pointer around. Raylib is passing structures by value. Does this work? Have you tried it with Raylib? Are you using 32-bit or 64-bit Euphoria?

Icy_Viking said...

When running the 32-bit version, I get the following error:

ffi.e:775 in function define_c_func() ffi_prep_cif() returned FFI_BAD_ABI (2)

The 64-bit version doesn't show any errors, but doesn't run, I assume due to not having a 64-bit installtion of Euphoria, which I need to do.

I'll be honest: I didn't test the 32-bit version at all. I try not to use 32-bit anymore.

My guess is that FFI_MS_CDECL isn't the correct calling convention for what I'm trying to do, or there's something else I'm missing here.

Could you provide the full back trace from ex.err? Just the "called from" lines so I know which define_c_func() call triggered this.

-Greg

new topic     » goto parent     » topic index » view message » categorize

10. Re: Preliminary libffi progress

ghaberek said...
jmduro said...

If you wish to compare, here is a version without libffi that can run the core_basic_windows.ex demo:

It looks like you're allocating the structures to memory and then passing the pointer around. Raylib is passing structures by value. Does this work? Have you tried it with Raylib? Are you using 32-bit or 64-bit Euphoria?

Icy_Viking said...

When running the 32-bit version, I get the following error:

ffi.e:775 in function define_c_func() ffi_prep_cif() returned FFI_BAD_ABI (2)

The 64-bit version doesn't show any errors, but doesn't run, I assume due to not having a 64-bit installtion of Euphoria, which I need to do.

I'll be honest: I didn't test the 32-bit version at all. I try not to use 32-bit anymore.

My guess is that FFI_MS_CDECL isn't the correct calling convention for what I'm trying to do, or there's something else I'm missing here.

Could you provide the full back trace from ex.err? Just the "called from" lines so I know which define_c_func() call triggered this.

-Greg

Hey Greg. Here is the called from lines from ex.err

ffi.e:775 in function define_c_func()  
ffi_prep_cif() returned FFI_BAD_ABI (2) 
  
    lib = 1710096384 
    name = {73'I',110'n',105'i',116't',87'W',105'i',110'n',100'd',111'o', 
119'w'} 
    arg_types = {1795457132,1795457132,12319936} 
    rtype = 1795457204 
    abi = 8 
    fn = 1710416016 
    pname = 44806000 
    parg_types = 44806080 
    nargs = 3 
    cif = 44806000 
    status = 2 
 
\ffi.e:784 in function define_c_proc()   
    lib = 1710096384 
    name = {43'+',73'I',110'n',105'i',116't',87'W',105'i',110'n',100'd',111'o', 
119'w'} 
    arg_types = {1795457132,1795457132,12319936} 
 
called from:..... raylib.e:311  

I can send the full ex.err if needed.

new topic     » goto parent     » topic index » view message » categorize

11. Re: Preliminary libffi progress

Icy_Viking said...

Hey Greg. Here is the called from lines from ex.err

ffi.e:775 in function define_c_func()  
ffi_prep_cif() returned FFI_BAD_ABI (2) 
  
    lib = 1710096384 
    name = {73'I',110'n',105'i',116't',87'W',105'i',110'n',100'd',111'o', 
119'w'} 
    arg_types = {1795457132,1795457132,12319936} 
    rtype = 1795457204 
    abi = 8 
    fn = 1710416016 
    pname = 44806000 
    parg_types = 44806080 
    nargs = 3 
    cif = 44806000 
    status = 2 
 
\ffi.e:784 in function define_c_proc()   
    lib = 1710096384 
    name = {43'+',73'I',110'n',105'i',116't',87'W',105'i',110'n',100'd',111'o', 
119'w'} 
    arg_types = {1795457132,1795457132,12319936} 
 
called from:..... raylib.e:311  

I can send the full ex.err if needed.

Nope, that's enough for me to trace out the problem. Pretty sure the issue is here: std/ffi.e:59

That X86_32 should just be X86. Please make that change locally and try it again and let me know the result.

57    ifdef X86_64 then 
58        with define X86_WIN64 
59    elsifdef X86 then -- <-- change this from X86_32 to just X86 
60        with define X86_WIN32 
61    end ifdef 

-Greg

new topic     » goto parent     » topic index » view message » categorize

12. Re: Preliminary libffi progress

ghaberek said...
Icy_Viking said...

Hey Greg. Here is the called from lines from ex.err

ffi.e:775 in function define_c_func()  
ffi_prep_cif() returned FFI_BAD_ABI (2) 
  
    lib = 1710096384 
    name = {73'I',110'n',105'i',116't',87'W',105'i',110'n',100'd',111'o', 
119'w'} 
    arg_types = {1795457132,1795457132,12319936} 
    rtype = 1795457204 
    abi = 8 
    fn = 1710416016 
    pname = 44806000 
    parg_types = 44806080 
    nargs = 3 
    cif = 44806000 
    status = 2 
 
\ffi.e:784 in function define_c_proc()   
    lib = 1710096384 
    name = {43'+',73'I',110'n',105'i',116't',87'W',105'i',110'n',100'd',111'o', 
119'w'} 
    arg_types = {1795457132,1795457132,12319936} 
 
called from:..... raylib.e:311  

I can send the full ex.err if needed.

Nope, that's enough for me to trace out the problem. Pretty sure the issue is here: std/ffi.e:59

That X86_32 should just be X86. Please make that change locally and try it again and let me know the result.

57    ifdef X86_64 then 
58        with define X86_WIN64 
59    elsifdef X86 then -- <-- change this from X86_32 to just X86 
60        with define X86_WIN32 
61    end ifdef 

-Greg

That fixed it, sorta. The window flashes for a brief second then disappears. I'll look through the example code and see if I can fix it. All the Raylib info does show up in the command prompt.

EDIT: I ran it through both EUI and EUIW and both times the windows appears for just a brief second before closing. However the Raylib info in the command prompt does say the window closed sucessfully. I looked at both the wrapper code and example code. I can't find what I'd need to change or its just my system being weird. The problem is to figure out why the window closes right away instead of staying open.

new topic     » goto parent     » topic index » view message » categorize

13. Re: Preliminary libffi progress

ghaberek said...

It looks like you're allocating the structures to memory and then passing the pointer around. Raylib is passing structures by value. Does this work? Have you tried it with Raylib? Are you using 32-bit or 64-bit Euphoria?

I tried it with Raylib on OEU 4.1 Windows 32-bit. I get a full list of messages with no error but it seems to open a window and close it immediatly after instead of waiting for the user to close the window.

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

14. Re: Preliminary libffi progress

jmduro said...
ghaberek said...

It looks like you're allocating the structures to memory and then passing the pointer around. Raylib is passing structures by value. Does this work? Have you tried it with Raylib? Are you using 32-bit or 64-bit Euphoria?

I tried it with Raylib on OEU 4.1 Windows 32-bit. I get a full list of messages with no error but it seems to open a window and close it immediatly after instead of waiting for the user to close the window.

Jean-Marc

I have the same issue.

new topic     » goto parent     » topic index » view message » categorize

15. Re: Preliminary libffi progress

Icy_Viking said...
jmduro said...
ghaberek said...

It looks like you're allocating the structures to memory and then passing the pointer around. Raylib is passing structures by value. Does this work? Have you tried it with Raylib? Are you using 32-bit or 64-bit Euphoria?

I tried it with Raylib on OEU 4.1 Windows 32-bit. I get a full list of messages with no error but it seems to open a window and close it immediatly after instead of waiting for the user to close the window.

I have the same issue.

Alright, I did some testing using 32-bit Euphoria 4.1 and Raylib. I think the problem might be that libffi isn't setting the return value memory to NULL before making the call, and since I told it C_BOOL was an unsigned int (4-bytes) it was returning the garbage in the upper three bytes of the value, so I'd see something like 0x43542100 come back for "false" which Euphoria rightfully treated as non-zero "true" so the the while loop exited right away. I changed C_BOOL to char value (one byte) instead and that seems to have helped. I also fixed the incorrect ifdef for x86 detection and added a couple more examples that utilize various structure-by-value functions.

-Greg

new topic     » goto parent     » topic index » view message » categorize

16. Re: Preliminary libffi progress

ghaberek said...
Icy_Viking said...
jmduro said...
ghaberek said...

It looks like you're allocating the structures to memory and then passing the pointer around. Raylib is passing structures by value. Does this work? Have you tried it with Raylib? Are you using 32-bit or 64-bit Euphoria?

I tried it with Raylib on OEU 4.1 Windows 32-bit. I get a full list of messages with no error but it seems to open a window and close it immediatly after instead of waiting for the user to close the window.

I have the same issue.

Alright, I did some testing using 32-bit Euphoria 4.1 and Raylib. I think the problem might be that libffi isn't setting the return value memory to NULL before making the call, and since I told it C_BOOL was an unsigned int (4-bytes) it was returning the garbage in the upper three bytes of the value, so I'd see something like 0x43542100 come back for "false" which Euphoria rightfully treated as non-zero "true" so the the while loop exited right away. I changed C_BOOL to char value (one byte) instead and that seems to have helped. I also fixed the incorrect ifdef for x86 detection and added a couple more examples that utilize various structure-by-value functions.

-Greg

Excellent work Greg! It works! I can help to finish the Raylib wrapper if you'd like. I'd of course push updates to my local repository then you can merge them later if you wanna. I'll be playing around with the examples and seeing what FFI can do.

EDIT: Here's my fork of Greg's repoistory, I'll be using it as a playground for testing/coding for now: https://github.com/gAndy50/libffi-euphoria

Hopefully this will help Greg as well.

--Raylib's Keyboard Input example in Eu code  
 
include raylib.e 
 
procedure main() 
 
 integer screenWidth = 800 
 integer screenHeight = 600 
  
 InitWindow(screenWidth,screenHeight,"Keyboard Input Example") 
  
 sequence ballPos = {screenWidth / 2, screenHeight / 2} 
  
 SetTargetFPS(60) 
  
 while not WindowShouldClose() do 
  
    --BallPos [1] -is X position, [2] -is Y position 
    if IsKeyDown(KEY_RIGHT) then 
    	ballPos[1] += 2.0 
    	elsif IsKeyDown(KEY_LEFT) then 
    		ballPos[1] -= 2.0 
    end if 
     
	if IsKeyDown(KEY_UP) then 
		ballPos[2] -= 2.0 
		elsif IsKeyDown(KEY_DOWN) then 
			ballPos[2] += 2.0 
	end if 
	 
	--Keep the ball from going off the screen 
	if ballPos[1] <= 0 then 
		ballPos[1] += 2.0 
		elsif ballPos[1] + 50 >= screenWidth then 
			ballPos[1] -= 2.0 
	end if 
	 
	if ballPos[2] <= 0 then 
		ballPos[2] += 2.0 
		elsif ballPos[2] + 50 >= screenHeight then 
			ballPos[2] -= 2.0 
	end if 
  
    BeginDrawing() 
     
	ClearBackground(RAYWHITE) 
	 
    DrawText("Move the ball with arrow keys", 10,10,20, DARKGRAY) 
     
    DrawCircleV(ballPos,50,MAROON) 
     
    EndDrawing() 
 	 
 end while 
  
 CloseWindow() 
	 
end procedure 
 
main() 
 
new topic     » goto parent     » topic index » view message » categorize

17. Re: Preliminary libffi progress

Icy_Viking said...

Excellent work Greg! It works! I can help to finish the Raylib wrapper if you'd like.

Thanks! Keep in mind that this was just a proof-of-concept. I don't plan to use libffi externally moving forward. I'd like it to replace the existing C interfacing code in the Euphoria backend. And I'm still unsure of the design for define_c_type(). I'm open to suggestions on that.

I also need to do more work to get callbacks working. Right now we've no way of declaring the incoming type for callback parameters and everything is assumed to be a pointer-sized integer. I want to provide a means of specifying and pre-handling the parameter types. This will be especially helpful when dealing with some of the event handlers in IUP.

include iup.e 
 
function action_cb( atom id, atom posX, atom posY ) 
 
    return IUP_DEFAULT 
end function 
constant ACTION_CB = call_back( routine_id("action_cb"), {C_POINTER,C_FLOAT,C_FLOAT} ) 
 
function dropfiles_cb( atom id, sequence filename, integer num, integer x, integer y ) 
 
    return IUP_DEFAULT 
end function 
constant DROPFILES_CB = call_back( routine_id("dropfiles_cb"), {C_POINTER,C_STRING,C_INT,C_INT,C_INT} ) 
 
procedure main() 
 
    IupOpen() 
 
    atom canvas = IupCanvas( NULL ) 
    IupSetCallback( canvas, "ACTION", ACTION_CB ) 
    IupSetCallback( canvas, "DROPFILES", DROPFILES_CB ) 
 
    -- etc. 
 
    IupClose() 
 
end procedure 
 
main() 

-Greg

new topic     » goto parent     » topic index » view message » categorize

18. Re: Preliminary libffi progress

ghaberek said...
Icy_Viking said...

Excellent work Greg! It works! I can help to finish the Raylib wrapper if you'd like.

Thanks! Keep in mind that this was just a proof-of-concept. I don't plan to use libffi externally moving forward. I'd like it to replace the existing C interfacing code in the Euphoria backend. And I'm still unsure of the design for define_c_type(). I'm open to suggestions on that.

I also need to do more work to get callbacks working. Right now we've no way of declaring the incoming type for callback parameters and everything is assumed to be a pointer-sized integer. I want to provide a means of specifying and pre-handling the parameter types. This will be especially helpful when dealing with some of the event handlers in IUP.

include iup.e 
 
function action_cb( atom id, atom posX, atom posY ) 
 
    return IUP_DEFAULT 
end function 
constant ACTION_CB = call_back( routine_id("action_cb"), {C_POINTER,C_FLOAT,C_FLOAT} ) 
 
function dropfiles_cb( atom id, sequence filename, integer num, integer x, integer y ) 
 
    return IUP_DEFAULT 
end function 
constant DROPFILES_CB = call_back( routine_id("dropfiles_cb"), {C_POINTER,C_STRING,C_INT,C_INT,C_INT} ) 
 
procedure main() 
 
    IupOpen() 
 
    atom canvas = IupCanvas( NULL ) 
    IupSetCallback( canvas, "ACTION", ACTION_CB ) 
    IupSetCallback( canvas, "DROPFILES", DROPFILES_CB ) 
 
    -- etc. 
 
    IupClose() 
 
end procedure 
 
main() 

-Greg

So you want to re-write the C interfacing backend entirely? Is that what I'm understanding? I mean I think the FFI is a good idea. It works and it has made it easier to wrap libraries and especially when dealing with structs. As for the define_c_type, it works for now. I don't mind it. I also like that C_STRING takes care of the memory allocation and such. Oh wait you want to implement FFI into the Euphoria backend, I think I get it now. As it stands using FFI externally works for now. Maybe implementning FFI into the backend would be good for 4.2.0 for now and after that we can work on adding structs/classes to Eu.

new topic     » goto parent     » topic index » view message » categorize

19. Re: Preliminary libffi progress

ghaberek said...

Alright, I did some testing using 32-bit Euphoria 4.1 and Raylib. I think the problem might be that libffi isn't setting the return value memory to NULL before making the call, and since I told it C_BOOL was an unsigned int (4-bytes) it was returning the garbage in the upper three bytes of the value, so I'd see something like 0x43542100 come back for "false" which Euphoria rightfully treated as non-zero "true" so the the while loop exited right away. I changed C_BOOL to char value (one byte) instead and that seems to have helped. I also fixed the incorrect ifdef for x86 detection and added a couple more examples that utilize various structure-by-value functions.

In the Windows SDK, BOOL is one byte long, I don't know why, while in Euphoria C_BOOL is four bytes long.

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

20. Re: Preliminary libffi progress

ghaberek said...

I also need to do more work to get callbacks working. Right now we've no way of declaring the incoming type for callback parameters and everything is assumed to be a pointer-sized integer. I want to provide a means of specifying and pre-handling the parameter types. This will be especially helpful when dealing with some of the event handlers in IUP.

Another one to consider is libcurl's CURLOPT_XFERINFOFUNCTION callback, in the end I had to define completely separate ones for 32 and 64 bit code - libcurl always puts 4 int64s on the stack, but since there is simply no way to tell the legacy call_back() any different, it pulls 32 bits per arg, so the only thing that can currently be done is to stitch together 8 int32s back into 4 int64s when runnning on 32 bit.

new topic     » goto parent     » topic index » view message » categorize

21. Re: Preliminary libffi progress

petelomax said...

Another one to consider is libcurl's CURLOPT_XFERINFOFUNCTION callback, in the end I had to define completely separate ones for 32 and 64 bit code - libcurl always puts 4 int64s on the stack, but since there is simply no way to tell the legacy call_back() any different, it pulls 32 bits per arg, so the only thing that can currently be done is to stitch together 8 int32s back into 4 int64s when runnning on 32 bit.

Yep. Being able to specify the incoming types for callbacks should help this significantly. Even 32-bit systems support 64-bit types so we can't assume everything is pointer-sized.

-Greg

new topic     » goto parent     » topic index » view message » categorize

22. Re: Preliminary libffi progress

I've added support for callbacks using libffi "closures" API. To test this I added a couple IUP examples along with a modified version of the IUP wrapper I've been working on. You can indicate the argument and return types of a callback in basically the same way you do for a C function, as shown in iup_canvas.ex and iup_editor.ex. (Hypothetically this could also allow for callback procedures if you specify the return type as C_VOID, but I haven't implemented that yet.)

I'm still looking for feedback on the structures aspect of the implementation, because I'm still not in love with define_c_type() as it's written. Eventually I'd like to use libffi as the underlying logic to get memstruct implemented, but I still need a way to tell define_c_func() about memstruct types and vice-versa. What does that even look like?

Edit: Link to repo: https://github.com/ghaberek/libffi-euphoria

-Greg

new topic     » goto parent     » topic index » view message » categorize

23. Re: Preliminary libffi progress

I've finished the wrapper part for raylib using Greg's euphoria's version of FFI. Greg this FFI library has been great. I know its still early, but the results are promising! I'll probably add a couple more examples in the coming days.

You can get it from my fork: https://github.com/gAndy50/libffi-euphoria

new topic     » goto parent     » topic index » view message » categorize

24. Re: Preliminary libffi progress

Hi Gregg and Pete

If ffi replaces dll.e, and Phix doesn't need to call dll.e, are there going to be more compatability issues when using Andy's (copious) output of libraries, eg SDL2, raylib and so on? If ffi is going to make things easier for Euphoria, is it going to make things more difficult for Phix? Or will Phix users just be able to call ffi, and have the same access to libraries using it for Euphoria? Are we looking at a separation fork event?

Cheers

SlightlyWorriedChris

new topic     » goto parent     » topic index » view message » categorize

25. Re: Preliminary libffi progress

ChrisB said...

If ffi replaces dll.e, and Phix doesn't need to call dll.e, are there going to be more compatability issues when using Andy's (copious) output of libraries, eg SDL2, raylib and so on? If ffi is going to make things easier for Euphoria, is it going to make things more difficult for Phix? Or will Phix users just be able to call ffi, and have the same access to libraries using it for Euphoria?

I don't think so? I plan to keep the existing std/dll.e library around for a while and just mark its routines with deprecate until it's finally removed. I cannot speak to Phix as I've honestly no idea how DLL interfacing works there.

ChrisB said...

Are we looking at a separation fork event?

I've always maintained that this project (the "OpenEuphoria" group) maintains the reference implementation for Euphoria and that Phix is a dialect of that. If Pete wants to maintain Phix's compatibility with Euphoria that's great, but I'm not concerned about the reverse.

-Greg

new topic     » goto parent     » topic index » view message » categorize

26. Re: Preliminary libffi progress

ghaberek said...
ChrisB said...

If ffi replaces dll.e, and Phix doesn't need to call dll.e, are there going to be more compatability issues when using Andy's (copious) output of libraries, eg SDL2, raylib and so on? If ffi is going to make things easier for Euphoria, is it going to make things more difficult for Phix? Or will Phix users just be able to call ffi, and have the same access to libraries using it for Euphoria?

I don't think so? I plan to keep the existing std/dll.e library around for a while and just mark its routines with deprecate until it's finally removed. I cannot speak to Phix as I've honestly no idea how DLL interfacing works there.

The Phix equivalent of dll.e is actually builtins\VM\pcfunc.e and that's a) likely to stay around a while (see below) and b) quite probably capable of having a new define_c_type() and whatever added to it.
After all, everything in that file and much more besides was all reverse-engineered from scratch to match Euphoria as closely as possible, in the first place.
Generally speaking, as long as things clearly crash on the right source code line, and force something like the following (only with lower-level code), that'll do, in my view.

--/*   
        puts(1, "Eu prints this, Phix does not\n")   
--*/   
--/* */ puts(1, "Phix prints this, Eu does not\n")    
ghaberek said...
ChrisB said...

Are we looking at a separation fork event?

I've always maintained that this project (the "OpenEuphoria" group) maintains the reference implementation for Euphoria and that Phix is a dialect of that. If Pete wants to maintain Phix's compatibility with Euphoria that's great, but I'm not concerned about the reverse.

-Greg

Agreed. I'm not planning a separation fork event just yet, but I won't rule one out. If you really want to be worried, Phix supports the following right now (and has done for quite some time):

    elsif platform()=LINUX then 
        #ilASM{ 
                mov eax,[var] 
                shl eax,2 
                push eax 
                call "libc.so.6","getenv" 
                add esp,4 
                lea edi,[pRes] 
                call :%pStoreMint 
              } 

Which is wholly incompatible with Euphoria, and may one day become the preferred manner of Phix interfacing with so/dlls.
As I mentioned, should it go that way it might force a whole layer of extra work on top of Andy and anyone else's output,
but as long as Phix can output the right, clear, meaningful, and genuinely helpful error messages, I'm not worried at all.

One thing I'm not planning is incorporating ffi.c or whatever it is into the phix executable like I believe Greg is planning,
maybe there could be (and I wouldn't deliberately stop) an ffi.dll knocking about for the few cases that actually need it.

new topic     » goto parent     » topic index » view message » categorize

Search



Quick Links

User menu

Not signed in.

Misc Menu