1. Preliminary libffi progress
- Posted by ghaberek (admin) Sep 25, 2022
- 1622 views
I've been playing around with libffi and here's what I've come up with so far.
- Structures: You can create new "types" (using ffi_type) that can describe direct values like int, float, etc. or you can describe structures which are basically an array of types. Then you can get back the offsets of these types using ffi_get_struct_offsets(). Combined with libffi's built-in knowledge of native C types, this allows me to describe, allocate, and then peek and poke structures with very little effort.
- C functions: You can pass any of these new "types" you've created (including structures!) to a Call InterFace or "CIF", and then allocate the required memory for arguments and return value, then perform the call with ffi_call(). libffi will take care of figuring out how to pass and receive values with its built-in knowledge of ABIs and type sizes and alignments. And it can even pass and receive structures by value!
- Call-backs: I haven't explored this much yet, but libffi also supports creating call-backs that can receive a variable number of arguments, as well as receive non-integer (floating point) values and structures by value. These are things that Euphoria currently does not support. I need to do more digging on this but so far it looks promising.
Internally, I've created std/ffi.e that completely replaces, but remains code-compatible with, std/dll.e. What I mean by "code compatible" is that the API doesn't change, but the ABI - implementation and underlying values (like constants) - are completely different. From the end-programmer's perspective, replacing std/dll.e with std/ffi.e should be 100% compatible as long as you weren't doing anything funny with the actual values of things like C_POINTER, etc.
The biggest change is the addition of what I'm calling define_c_type() (to parallel define_c_var/func/proc) which allows you to build a new type from existing types. I also created special types based on C_POINTER for C_STRING and C_WSTRING that will automatically allocate/poke/free string values during a function call.
Here's an example of what I have working so far (at least the basic Raylib demo works). I'm looking to solicit feedback at this stage. I feel this is more worthwhile than the existing memstruct approach. So if there's support for this, I'd like to get it worked into 4.2 now, instead of memstruct. Then we can revisit building classes and structures into the language syntax later.
include std/ffi.e -- Vector2, 2 components --typedef struct Vector2 { -- float x; // Vector x component -- float y; // Vector y component --} Vector2; public constant RL_VECTOR2 = define_c_type({ C_FLOAT, -- x // Vector x component C_FLOAT -- y // Vector y component }) -- Color, 4 components, R8G8B8A8 (32bit) --typedef struct Color { -- unsigned char r; // Color red value -- unsigned char g; // Color green value -- unsigned char b; // Color blue value -- unsigned char a; // Color alpha value --} Color; public constant RL_COLOR = define_c_type({ C_UCHAR, -- r // Color red value C_UCHAR, -- g // Color green value C_UCHAR, -- b // Color blue value C_UCHAR -- a // Color alpha value }) atom raylib = open_dll( "raylib.dll" ) constant xInitWindow = define_c_proc( raylib, "+InitWindow", {C_INT,C_INT,C_STRING} ) constant xGetWindowPosition = define_c_func( raylib, "+GetWindowPosition", {}, RL_VECTOR2 ) constant xClearBackground = define_c_proc( raylib, "+ClearBackground", {RL_COLOR} ) public procedure InitWindow( integer width, integer height, sequence title ) c_proc( xInitWindow, {width,height,title} ) -- string is allocated/freed automatically end procedure public function GetWindowPosition() return c_func( xGetWindowPosition, {} ) -- returns {x,y} end function public procedure ClearBackground( sequence color ) c_proc( xClearBackground, {color} ) -- color is {r,g,b,a} end procedure
-Greg
2. Re: Preliminary libffi progress
- Posted by Icy_Viking Sep 25, 2022
- 1596 views
This looks great Greg. I bet this library will come in handy wrapping other libraries too. Also, looks like it makes it a lot easier to wrap libraries too.
EDIT: Is there a link where we can see the code to libffi.e so far?
3. Re: Preliminary libffi progress
- Posted by ghaberek (admin) Sep 25, 2022
- 1586 views
Is there a link where we can see the code to libffi.e so far?
I need to clean it up a bit and then I'll push it to GitHub and share the link. Keep in mind that this is a temporary approach and ultimately libffi would be compiled into the backend and then exposed using machine_func().
Edit: putting together some rough documentation, I came up with this feature comparison table:
Feature | std/dll.e | std/ffi.e |
---|---|---|
Define C variables | YES | YES |
Define C functions | YES | YES |
Define machine functions | YES | YES |
STDCALL convention | YES | YES |
CDECL convention | YES | YES |
FASTCALL convention | NO | YES |
PASCAL convention | NO | YES |
THISCALL convention | NO | YES |
Pass structures by value | NO | YES |
Receive structures by value | NO | YES |
Variable-length call-backs | NO | YES |
Floating-point call-backs | NO | YES |
-Greg
4. Re: Preliminary libffi progress
- Posted by Icy_Viking Sep 25, 2022
- 1561 views
Is there a link where we can see the code to libffi.e so far?
I need to clean it up a bit and then I'll push it to GitHub and share the link. Keep in mind that this is a temporary approach and ultimately libffi would be compiled into the backend and then exposed using machine_func().
Edit: putting together some rough documentation, I came up with this feature comparison table:
Feature | std/dll.e | std/ffi.e |
---|---|---|
Define C variables | YES | YES |
Define C functions | YES | YES |
Define machine functions | YES | YES |
STDCALL convention | YES | YES |
CDECL convention | YES | YES |
FASTCALL convention | NO | YES |
PASCAL convention | NO | YES |
THISCALL convention | NO | YES |
Pass structures by value | NO | YES |
Receive structures by value | NO | YES |
Variable-length call-backs | NO | YES |
Floating-point call-backs | NO | YES |
-Greg
This is looking great so far Greg, even if its in the early stages. I think this will really help in wrapping libraries. Making it much easier to wrap libraries and not have to make shim libraries or declaring structs as separate entities.
5. Re: Preliminary libffi progress
- Posted by ghaberek (admin) Sep 26, 2022
- 1549 views
As promised, here's the current working code: https://github.com/ghaberek/libffi-euphoria. I've included Win32 and Win64 binaries for raylib and libffi and a couple of working raylib demos.
You should be able to run the raylib demos on Windows with the following steps. More work is need to get going on Linux but AFAICT libffi is in the Debian repos so "apt install libffi" should work.
git clone https://github.com/ghaberek/libffi-euphoria cd libffi-euphoria copy bin\win64\*.dll C:\Euphoria\bin eui examples\core_basic_window.ex
-Greg
6. Re: Preliminary libffi progress
- Posted by Icy_Viking Sep 26, 2022
- 1563 views
As promised, here's the current working code: https://github.com/ghaberek/libffi-euphoria. I've included Win32 and Win64 binaries for raylib and libffi and a couple of working raylib demos.
You should be able to run the raylib demos on Windows with the following steps. More work is need to get going on Linux but AFAICT libffi is in the Debian repos so "apt install libffi" should work.
git clone https://github.com/ghaberek/libffi-euphoria cd libffi-euphoria copy bin\win64\*.dll C:\Euphoria\bin eui examples\core_basic_window.ex
-Greg
Thanks Greg. If I can understand the code base well enough I wouldn't mind helping you out with this. This might work better than having a struct function. Though I do think Euphoria would benefit from a struct type of some kind.
7. Re: Preliminary libffi progress
- Posted by jmduro Sep 26, 2022
- 1542 views
This might work better than having a struct function. Though I do think Euphoria would benefit from a struct type of some kind.
If you wish to compare, here is a version without libffi that can run the core_basic_windows.ex demo:
include std/dll.e include std/machine.e include structs.e public constant LIGHTGRAY = { 200, 200, 200, 255 } -- Light Gray ------------------------------------------------------------------------------------ -- Structures Definition ------------------------------------------------------------------------------------ -- Vector2, 2 components public sequence RL_VECTOR2 = "typedef struct Vector2 {" & " float x;" & -- Vector x component " float y;" & -- Vector y component "} Vector2;" -- Vector3, 3 components public sequence RL_VECTOR3 = "typedef struct Vector3 {" & " float x;" & -- Vector x component " float y;" & -- Vector y component " float z;" & -- Vector z component "} Vector3;" -- Color, 4 components, R8G8B8A8 (32bit) public sequence RL_COLOR = "typedef struct Color {" & " unsigned_char r;" & -- Color red value " unsigned_char g;" & -- Color green value " unsigned_char b;" & -- Color blue value " unsigned_char a;" & -- Color alpha value "} Color;" sequence Vector2 = allocateStructure(RL_VECTOR2) sequence Vector3 = allocateStructure(RL_VECTOR3) sequence Color = allocateStructure(RL_COLOR) constant raylib = open_dll( "raylib.dll" ), xInitWindow = define_c_proc( raylib, "+InitWindow", {C_INT,C_INT,C_POINTER} ), xWindowShouldClose = define_c_func( raylib, "+WindowShouldClose", {}, C_BOOL ), xCloseWindow = define_c_proc( raylib, "+CloseWindow", {} ), xGetWindowPosition = define_c_func( raylib, "+GetWindowPosition", {}, C_POINTER ), xSetTargetFPS = define_c_proc( raylib, "+SetTargetFPS", {C_INT} ), xClearBackground = define_c_proc( raylib, "+ClearBackground", {C_POINTER} ), xBeginDrawing = define_c_proc( raylib, "+BeginDrawing", {} ), xEndDrawing = define_c_proc( raylib, "+EndDrawing", {} ), xDrawText = define_c_proc( raylib, "+DrawText", {C_POINTER,C_INT,C_INT,C_INT,C_POINTER} ), $ public procedure InitWindow( integer width, integer height, sequence title ) c_proc( xInitWindow, {width,height,allocate_string(title)} ) -- title string is allocated/freed automatically end procedure public function WindowShouldClose() return c_func( xWindowShouldClose, {} ) end function public procedure CloseWindow() c_proc( xCloseWindow, {} ) end procedure public function GetWindowPosition() free(Vector2[SU_ADDRESS]) Vector2[SU_ADDRESS] = c_func( xGetWindowPosition, {} ) -- returns {x,y} return readStructure(Vector2) end function public procedure SetTargetFPS( integer fps ) c_proc( xSetTargetFPS, {fps} ) end procedure public procedure ClearBackground( sequence color ) writeStructure(Color, color ) c_proc( xClearBackground, Color[SU_ADDRESS] ) -- color is {r,g,b,a} end procedure public procedure BeginDrawing() c_proc( xBeginDrawing, {} ) end procedure public procedure EndDrawing() c_proc( xEndDrawing, {} ) end procedure public procedure DrawText( sequence text, integer posX, integer posY, integer fontSize, sequence color ) writeStructure(Color, color ) c_proc( xDrawText, {allocate_string(text),posX,posY,fontSize,Color[SU_ADDRESS]} ) end procedure
8. Re: Preliminary libffi progress
- Posted by Icy_Viking Sep 26, 2022
- 1583 views
Looks like there is multiple ways to go about this. However the FFI option looks much easier.
When running the 32-bit version, I get the following error:
ffi.e:775 in function define_c_func() ffi_prep_cif() returned FFI_BAD_ABI (2)
The 64-bit version doesn't show any errors, but doesn't run, I assume due to not having a 64-bit installtion of Euphoria, which I need to do.
9. Re: Preliminary libffi progress
- Posted by ghaberek (admin) Sep 27, 2022
- 1453 views
If you wish to compare, here is a version without libffi that can run the core_basic_windows.ex demo:
It looks like you're allocating the structures to memory and then passing the pointer around. Raylib is passing structures by value. Does this work? Have you tried it with Raylib? Are you using 32-bit or 64-bit Euphoria?
When running the 32-bit version, I get the following error:
ffi.e:775 in function define_c_func() ffi_prep_cif() returned FFI_BAD_ABI (2)
The 64-bit version doesn't show any errors, but doesn't run, I assume due to not having a 64-bit installtion of Euphoria, which I need to do.
I'll be honest: I didn't test the 32-bit version at all. I try not to use 32-bit anymore.
My guess is that FFI_MS_CDECL isn't the correct calling convention for what I'm trying to do, or there's something else I'm missing here.
Could you provide the full back trace from ex.err? Just the "called from" lines so I know which define_c_func() call triggered this.
-Greg
10. Re: Preliminary libffi progress
- Posted by Icy_Viking Sep 27, 2022
- 1421 views
If you wish to compare, here is a version without libffi that can run the core_basic_windows.ex demo:
It looks like you're allocating the structures to memory and then passing the pointer around. Raylib is passing structures by value. Does this work? Have you tried it with Raylib? Are you using 32-bit or 64-bit Euphoria?
When running the 32-bit version, I get the following error:
ffi.e:775 in function define_c_func() ffi_prep_cif() returned FFI_BAD_ABI (2)
The 64-bit version doesn't show any errors, but doesn't run, I assume due to not having a 64-bit installtion of Euphoria, which I need to do.
I'll be honest: I didn't test the 32-bit version at all. I try not to use 32-bit anymore.
My guess is that FFI_MS_CDECL isn't the correct calling convention for what I'm trying to do, or there's something else I'm missing here.
Could you provide the full back trace from ex.err? Just the "called from" lines so I know which define_c_func() call triggered this.
-Greg
Hey Greg. Here is the called from lines from ex.err
ffi.e:775 in function define_c_func() ffi_prep_cif() returned FFI_BAD_ABI (2) lib = 1710096384 name = {73'I',110'n',105'i',116't',87'W',105'i',110'n',100'd',111'o', 119'w'} arg_types = {1795457132,1795457132,12319936} rtype = 1795457204 abi = 8 fn = 1710416016 pname = 44806000 parg_types = 44806080 nargs = 3 cif = 44806000 status = 2 \ffi.e:784 in function define_c_proc() lib = 1710096384 name = {43'+',73'I',110'n',105'i',116't',87'W',105'i',110'n',100'd',111'o', 119'w'} arg_types = {1795457132,1795457132,12319936} called from:..... raylib.e:311
I can send the full ex.err if needed.
11. Re: Preliminary libffi progress
- Posted by ghaberek (admin) Sep 27, 2022
- 1436 views
Hey Greg. Here is the called from lines from ex.err
ffi.e:775 in function define_c_func() ffi_prep_cif() returned FFI_BAD_ABI (2) lib = 1710096384 name = {73'I',110'n',105'i',116't',87'W',105'i',110'n',100'd',111'o', 119'w'} arg_types = {1795457132,1795457132,12319936} rtype = 1795457204 abi = 8 fn = 1710416016 pname = 44806000 parg_types = 44806080 nargs = 3 cif = 44806000 status = 2 \ffi.e:784 in function define_c_proc() lib = 1710096384 name = {43'+',73'I',110'n',105'i',116't',87'W',105'i',110'n',100'd',111'o', 119'w'} arg_types = {1795457132,1795457132,12319936} called from:..... raylib.e:311
I can send the full ex.err if needed.
Nope, that's enough for me to trace out the problem. Pretty sure the issue is here: std/ffi.e:59
That X86_32 should just be X86. Please make that change locally and try it again and let me know the result.
57 ifdef X86_64 then 58 with define X86_WIN64 59 elsifdef X86 then -- <-- change this from X86_32 to just X86 60 with define X86_WIN32 61 end ifdef
-Greg
12. Re: Preliminary libffi progress
- Posted by Icy_Viking Sep 27, 2022
- 1426 views
Hey Greg. Here is the called from lines from ex.err
ffi.e:775 in function define_c_func() ffi_prep_cif() returned FFI_BAD_ABI (2) lib = 1710096384 name = {73'I',110'n',105'i',116't',87'W',105'i',110'n',100'd',111'o', 119'w'} arg_types = {1795457132,1795457132,12319936} rtype = 1795457204 abi = 8 fn = 1710416016 pname = 44806000 parg_types = 44806080 nargs = 3 cif = 44806000 status = 2 \ffi.e:784 in function define_c_proc() lib = 1710096384 name = {43'+',73'I',110'n',105'i',116't',87'W',105'i',110'n',100'd',111'o', 119'w'} arg_types = {1795457132,1795457132,12319936} called from:..... raylib.e:311
I can send the full ex.err if needed.
Nope, that's enough for me to trace out the problem. Pretty sure the issue is here: std/ffi.e:59
That X86_32 should just be X86. Please make that change locally and try it again and let me know the result.
57 ifdef X86_64 then 58 with define X86_WIN64 59 elsifdef X86 then -- <-- change this from X86_32 to just X86 60 with define X86_WIN32 61 end ifdef
-Greg
That fixed it, sorta. The window flashes for a brief second then disappears. I'll look through the example code and see if I can fix it. All the Raylib info does show up in the command prompt.
EDIT: I ran it through both EUI and EUIW and both times the windows appears for just a brief second before closing. However the Raylib info in the command prompt does say the window closed sucessfully. I looked at both the wrapper code and example code. I can't find what I'd need to change or its just my system being weird. The problem is to figure out why the window closes right away instead of staying open.
13. Re: Preliminary libffi progress
- Posted by jmduro Sep 28, 2022
- 1380 views
It looks like you're allocating the structures to memory and then passing the pointer around. Raylib is passing structures by value. Does this work? Have you tried it with Raylib? Are you using 32-bit or 64-bit Euphoria?
I tried it with Raylib on OEU 4.1 Windows 32-bit. I get a full list of messages with no error but it seems to open a window and close it immediatly after instead of waiting for the user to close the window.
Jean-Marc
14. Re: Preliminary libffi progress
- Posted by Icy_Viking Sep 28, 2022
- 1374 views
It looks like you're allocating the structures to memory and then passing the pointer around. Raylib is passing structures by value. Does this work? Have you tried it with Raylib? Are you using 32-bit or 64-bit Euphoria?
I tried it with Raylib on OEU 4.1 Windows 32-bit. I get a full list of messages with no error but it seems to open a window and close it immediatly after instead of waiting for the user to close the window.
Jean-Marc
I have the same issue.
15. Re: Preliminary libffi progress
- Posted by ghaberek (admin) Sep 28, 2022
- 1351 views
It looks like you're allocating the structures to memory and then passing the pointer around. Raylib is passing structures by value. Does this work? Have you tried it with Raylib? Are you using 32-bit or 64-bit Euphoria?
I tried it with Raylib on OEU 4.1 Windows 32-bit. I get a full list of messages with no error but it seems to open a window and close it immediatly after instead of waiting for the user to close the window.
I have the same issue.
Alright, I did some testing using 32-bit Euphoria 4.1 and Raylib. I think the problem might be that libffi isn't setting the return value memory to NULL before making the call, and since I told it C_BOOL was an unsigned int (4-bytes) it was returning the garbage in the upper three bytes of the value, so I'd see something like 0x43542100 come back for "false" which Euphoria rightfully treated as non-zero "true" so the the while loop exited right away. I changed C_BOOL to char value (one byte) instead and that seems to have helped. I also fixed the incorrect ifdef for x86 detection and added a couple more examples that utilize various structure-by-value functions.
-Greg
16. Re: Preliminary libffi progress
- Posted by Icy_Viking Sep 28, 2022
- 1376 views
It looks like you're allocating the structures to memory and then passing the pointer around. Raylib is passing structures by value. Does this work? Have you tried it with Raylib? Are you using 32-bit or 64-bit Euphoria?
I tried it with Raylib on OEU 4.1 Windows 32-bit. I get a full list of messages with no error but it seems to open a window and close it immediatly after instead of waiting for the user to close the window.
I have the same issue.
Alright, I did some testing using 32-bit Euphoria 4.1 and Raylib. I think the problem might be that libffi isn't setting the return value memory to NULL before making the call, and since I told it C_BOOL was an unsigned int (4-bytes) it was returning the garbage in the upper three bytes of the value, so I'd see something like 0x43542100 come back for "false" which Euphoria rightfully treated as non-zero "true" so the the while loop exited right away. I changed C_BOOL to char value (one byte) instead and that seems to have helped. I also fixed the incorrect ifdef for x86 detection and added a couple more examples that utilize various structure-by-value functions.
-Greg
Excellent work Greg! It works! I can help to finish the Raylib wrapper if you'd like. I'd of course push updates to my local repository then you can merge them later if you wanna. I'll be playing around with the examples and seeing what FFI can do.
EDIT: Here's my fork of Greg's repoistory, I'll be using it as a playground for testing/coding for now: https://github.com/gAndy50/libffi-euphoria
Hopefully this will help Greg as well.
--Raylib's Keyboard Input example in Eu code include raylib.e procedure main() integer screenWidth = 800 integer screenHeight = 600 InitWindow(screenWidth,screenHeight,"Keyboard Input Example") sequence ballPos = {screenWidth / 2, screenHeight / 2} SetTargetFPS(60) while not WindowShouldClose() do --BallPos [1] -is X position, [2] -is Y position if IsKeyDown(KEY_RIGHT) then ballPos[1] += 2.0 elsif IsKeyDown(KEY_LEFT) then ballPos[1] -= 2.0 end if if IsKeyDown(KEY_UP) then ballPos[2] -= 2.0 elsif IsKeyDown(KEY_DOWN) then ballPos[2] += 2.0 end if --Keep the ball from going off the screen if ballPos[1] <= 0 then ballPos[1] += 2.0 elsif ballPos[1] + 50 >= screenWidth then ballPos[1] -= 2.0 end if if ballPos[2] <= 0 then ballPos[2] += 2.0 elsif ballPos[2] + 50 >= screenHeight then ballPos[2] -= 2.0 end if BeginDrawing() ClearBackground(RAYWHITE) DrawText("Move the ball with arrow keys", 10,10,20, DARKGRAY) DrawCircleV(ballPos,50,MAROON) EndDrawing() end while CloseWindow() end procedure main()
17. Re: Preliminary libffi progress
- Posted by ghaberek (admin) Sep 29, 2022
- 1303 views
Excellent work Greg! It works! I can help to finish the Raylib wrapper if you'd like.
Thanks! Keep in mind that this was just a proof-of-concept. I don't plan to use libffi externally moving forward. I'd like it to replace the existing C interfacing code in the Euphoria backend. And I'm still unsure of the design for define_c_type(). I'm open to suggestions on that.
I also need to do more work to get callbacks working. Right now we've no way of declaring the incoming type for callback parameters and everything is assumed to be a pointer-sized integer. I want to provide a means of specifying and pre-handling the parameter types. This will be especially helpful when dealing with some of the event handlers in IUP.
include iup.e function action_cb( atom id, atom posX, atom posY ) return IUP_DEFAULT end function constant ACTION_CB = call_back( routine_id("action_cb"), {C_POINTER,C_FLOAT,C_FLOAT} ) function dropfiles_cb( atom id, sequence filename, integer num, integer x, integer y ) return IUP_DEFAULT end function constant DROPFILES_CB = call_back( routine_id("dropfiles_cb"), {C_POINTER,C_STRING,C_INT,C_INT,C_INT} ) procedure main() IupOpen() atom canvas = IupCanvas( NULL ) IupSetCallback( canvas, "ACTION", ACTION_CB ) IupSetCallback( canvas, "DROPFILES", DROPFILES_CB ) -- etc. IupClose() end procedure main()
-Greg
18. Re: Preliminary libffi progress
- Posted by Icy_Viking Sep 29, 2022
- 1297 views
Excellent work Greg! It works! I can help to finish the Raylib wrapper if you'd like.
Thanks! Keep in mind that this was just a proof-of-concept. I don't plan to use libffi externally moving forward. I'd like it to replace the existing C interfacing code in the Euphoria backend. And I'm still unsure of the design for define_c_type(). I'm open to suggestions on that.
I also need to do more work to get callbacks working. Right now we've no way of declaring the incoming type for callback parameters and everything is assumed to be a pointer-sized integer. I want to provide a means of specifying and pre-handling the parameter types. This will be especially helpful when dealing with some of the event handlers in IUP.
include iup.e function action_cb( atom id, atom posX, atom posY ) return IUP_DEFAULT end function constant ACTION_CB = call_back( routine_id("action_cb"), {C_POINTER,C_FLOAT,C_FLOAT} ) function dropfiles_cb( atom id, sequence filename, integer num, integer x, integer y ) return IUP_DEFAULT end function constant DROPFILES_CB = call_back( routine_id("dropfiles_cb"), {C_POINTER,C_STRING,C_INT,C_INT,C_INT} ) procedure main() IupOpen() atom canvas = IupCanvas( NULL ) IupSetCallback( canvas, "ACTION", ACTION_CB ) IupSetCallback( canvas, "DROPFILES", DROPFILES_CB ) -- etc. IupClose() end procedure main()
-Greg
So you want to re-write the C interfacing backend entirely? Is that what I'm understanding? I mean I think the FFI is a good idea. It works and it has made it easier to wrap libraries and especially when dealing with structs. As for the define_c_type, it works for now. I don't mind it. I also like that C_STRING takes care of the memory allocation and such. Oh wait you want to implement FFI into the Euphoria backend, I think I get it now. As it stands using FFI externally works for now. Maybe implementning FFI into the backend would be good for 4.2.0 for now and after that we can work on adding structs/classes to Eu.
19. Re: Preliminary libffi progress
- Posted by jmduro Sep 30, 2022
- 1254 views
Alright, I did some testing using 32-bit Euphoria 4.1 and Raylib. I think the problem might be that libffi isn't setting the return value memory to NULL before making the call, and since I told it C_BOOL was an unsigned int (4-bytes) it was returning the garbage in the upper three bytes of the value, so I'd see something like 0x43542100 come back for "false" which Euphoria rightfully treated as non-zero "true" so the the while loop exited right away. I changed C_BOOL to char value (one byte) instead and that seems to have helped. I also fixed the incorrect ifdef for x86 detection and added a couple more examples that utilize various structure-by-value functions.
In the Windows SDK, BOOL is one byte long, I don't know why, while in Euphoria C_BOOL is four bytes long.
Jean-Marc
20. Re: Preliminary libffi progress
- Posted by petelomax Oct 01, 2022
- 1237 views
I also need to do more work to get callbacks working. Right now we've no way of declaring the incoming type for callback parameters and everything is assumed to be a pointer-sized integer. I want to provide a means of specifying and pre-handling the parameter types. This will be especially helpful when dealing with some of the event handlers in IUP.
Another one to consider is libcurl's CURLOPT_XFERINFOFUNCTION callback, in the end I had to define completely separate ones for 32 and 64 bit code - libcurl always puts 4 int64s on the stack, but since there is simply no way to tell the legacy call_back() any different, it pulls 32 bits per arg, so the only thing that can currently be done is to stitch together 8 int32s back into 4 int64s when runnning on 32 bit.
21. Re: Preliminary libffi progress
- Posted by ghaberek (admin) Oct 01, 2022
- 1182 views
Another one to consider is libcurl's CURLOPT_XFERINFOFUNCTION callback, in the end I had to define completely separate ones for 32 and 64 bit code - libcurl always puts 4 int64s on the stack, but since there is simply no way to tell the legacy call_back() any different, it pulls 32 bits per arg, so the only thing that can currently be done is to stitch together 8 int32s back into 4 int64s when runnning on 32 bit.
Yep. Being able to specify the incoming types for callbacks should help this significantly. Even 32-bit systems support 64-bit types so we can't assume everything is pointer-sized.
-Greg
22. Re: Preliminary libffi progress
- Posted by ghaberek (admin) Oct 01, 2022
- 1184 views
I've added support for callbacks using libffi "closures" API. To test this I added a couple IUP examples along with a modified version of the IUP wrapper I've been working on. You can indicate the argument and return types of a callback in basically the same way you do for a C function, as shown in iup_canvas.ex and iup_editor.ex. (Hypothetically this could also allow for callback procedures if you specify the return type as C_VOID, but I haven't implemented that yet.)
I'm still looking for feedback on the structures aspect of the implementation, because I'm still not in love with define_c_type() as it's written. Eventually I'd like to use libffi as the underlying logic to get memstruct implemented, but I still need a way to tell define_c_func() about memstruct types and vice-versa. What does that even look like?
Edit: Link to repo: https://github.com/ghaberek/libffi-euphoria
-Greg
23. Re: Preliminary libffi progress
- Posted by Icy_Viking Oct 03, 2022
- 1125 views
I've finished the wrapper part for raylib using Greg's euphoria's version of FFI. Greg this FFI library has been great. I know its still early, but the results are promising! I'll probably add a couple more examples in the coming days.
You can get it from my fork: https://github.com/gAndy50/libffi-euphoria
24. Re: Preliminary libffi progress
- Posted by ChrisB (moderator) Oct 21, 2022
- 930 views
Hi Gregg and Pete
If ffi replaces dll.e, and Phix doesn't need to call dll.e, are there going to be more compatability issues when using Andy's (copious) output of libraries, eg SDL2, raylib and so on? If ffi is going to make things easier for Euphoria, is it going to make things more difficult for Phix? Or will Phix users just be able to call ffi, and have the same access to libraries using it for Euphoria? Are we looking at a separation fork event?
Cheers
25. Re: Preliminary libffi progress
- Posted by ghaberek (admin) Oct 21, 2022
- 930 views
If ffi replaces dll.e, and Phix doesn't need to call dll.e, are there going to be more compatability issues when using Andy's (copious) output of libraries, eg SDL2, raylib and so on? If ffi is going to make things easier for Euphoria, is it going to make things more difficult for Phix? Or will Phix users just be able to call ffi, and have the same access to libraries using it for Euphoria?
I don't think so? I plan to keep the existing std/dll.e library around for a while and just mark its routines with deprecate until it's finally removed. I cannot speak to Phix as I've honestly no idea how DLL interfacing works there.
Are we looking at a separation fork event?
I've always maintained that this project (the "OpenEuphoria" group) maintains the reference implementation for Euphoria and that Phix is a dialect of that. If Pete wants to maintain Phix's compatibility with Euphoria that's great, but I'm not concerned about the reverse.
-Greg
26. Re: Preliminary libffi progress
- Posted by petelomax Oct 22, 2022
- 901 views
If ffi replaces dll.e, and Phix doesn't need to call dll.e, are there going to be more compatability issues when using Andy's (copious) output of libraries, eg SDL2, raylib and so on? If ffi is going to make things easier for Euphoria, is it going to make things more difficult for Phix? Or will Phix users just be able to call ffi, and have the same access to libraries using it for Euphoria?
I don't think so? I plan to keep the existing std/dll.e library around for a while and just mark its routines with deprecate until it's finally removed. I cannot speak to Phix as I've honestly no idea how DLL interfacing works there.
The Phix equivalent of dll.e is actually builtins\VM\pcfunc.e and that's a) likely to stay around a while (see below) and b) quite probably capable of having a new define_c_type() and whatever added to it.
After all, everything in that file and much more besides was all reverse-engineered from scratch to match Euphoria as closely as possible, in the first place.
Generally speaking, as long as things clearly crash on the right source code line, and force something like the following (only with lower-level code), that'll do, in my view.
--/* puts(1, "Eu prints this, Phix does not\n") --*/ --/* */ puts(1, "Phix prints this, Eu does not\n")
Are we looking at a separation fork event?
I've always maintained that this project (the "OpenEuphoria" group) maintains the reference implementation for Euphoria and that Phix is a dialect of that. If Pete wants to maintain Phix's compatibility with Euphoria that's great, but I'm not concerned about the reverse.
-Greg
Agreed. I'm not planning a separation fork event just yet, but I won't rule one out. If you really want to be worried, Phix supports the following right now (and has done for quite some time):
elsif platform()=LINUX then #ilASM{ mov eax,[var] shl eax,2 push eax call "libc.so.6","getenv" add esp,4 lea edi,[pRes] call :%pStoreMint }
Which is wholly incompatible with Euphoria, and may one day become the preferred manner of Phix interfacing with so/dlls.
As I mentioned, should it go that way it might force a whole layer of extra work on top of Andy and anyone else's output,
but as long as Phix can output the right, clear, meaningful, and genuinely helpful error messages, I'm not worried at all.
One thing I'm not planning is incorporating ffi.c or whatever it is into the phix executable like I believe Greg is planning,
maybe there could be (and I wouldn't deliberately stop) an ffi.dll knocking about for the few cases that actually need it.