1. define_c_func/proc and 64-bit return types
- Posted by ghaberek (admin) Jul 29, 2012
- 1382 views
I'm trying to wrap a function that returns a int64_t value.
I know I can pass int64_t as a long long via two C_LONG values:
procedure some_routine( atom param ) atom hi_dword = and_bits( param / #100000000, #FFFFFFFF ) atom lo_dword = and_bits( param, #FFFFFFFF ) c_proc( xsome_routine, {lo_dword,hi_dword} ) end procedure
But how (or can) I do the reverse?
It would be nice if I could declare the function like this:
constant xsome_routine = define_c_func( somelib, "some_routine", {}, {C_LONG,C_LONG} )
And then get back {lo_dword,hi_dword} when I call the function.
Although I suppose the best declaration would simply be:
constant xsome_routine = define_c_func( somelib, "some_routine", {}, C_LONGLONG )
-Greg
2. Re: define_c_func/proc and 64-bit return types
- Posted by jimcbrown (admin) Jul 30, 2012
- 1328 views
I'm trying to wrap a function that returns a int64_t value.
I know I can pass int64_t as a long long via two C_LONG values:
procedure some_routine( atom param ) atom hi_dword = and_bits( param / #100000000, #FFFFFFFF ) atom lo_dword = and_bits( param, #FFFFFFFF ) c_proc( xsome_routine, {lo_dword,hi_dword} ) end procedure
But how (or can) I do the reverse?
It would be nice if I could declare the function like this:
constant xsome_routine = define_c_func( somelib, "some_routine", {}, {C_LONG,C_LONG} )
And then get back {lo_dword,hi_dword} when I call the function.
Although I suppose the best declaration would simply be:
constant xsome_routine = define_c_func( somelib, "some_routine", {}, C_LONGLONG )
-Greg
I'm not sure if you can with 4.0 .. but 4.1 actually has a C_LONGLONG type and has support for doing this builtin already.
3. Re: define_c_func/proc and 64-bit return types
- Posted by ghaberek (admin) Jul 30, 2012
- 1293 views
I'm not sure if you can with 4.0 .. but 4.1 actually has a C_LONGLONG type and has support for doing this builtin already.
Interesting... There is a C_DWORDLONG in std/dll.e, but it's set the same as C_DOUBLE which AFAIK returns a 64-bit floating point value. I just want a 64-bit int.
--** double 64-bits C_DOUBLE = #03000008, --** dwordlong 64-bits C_DWORDLONG = #03000008,
Another problem I've run into is a series of functions that take and/or return the following structure by value!
struct x {
float a;
float b;
float c;
float d;
}
Now, I can pass four floats to the function quite easily, but I'm thinking this 128-bit return value may be a problem for Euphoria. 
So at this point I may have two options:
- Modify the source of the library to return int64_t and this quad-float structure by reference, then provide my custom-built library instead of the original.
- Write a shim library in C that links to the original library, reads the troublesome values and returns a pointer to them.
-Greg
4. Re: define_c_func/proc and 64-bit return types
- Posted by jimcbrown (admin) Jul 30, 2012
- 1309 views
I'm not sure if you can with 4.0 .. but 4.1 actually has a C_LONGLONG type and has support for doing this builtin already.
Interesting... There is a C_DWORDLONG in std/dll.e, but it's set the same as C_DOUBLE which AFAIK returns a 64-bit floating point value. I just want a 64-bit int.
--** double 64-bits C_DOUBLE = #03000008, --** dwordlong 64-bits C_DWORDLONG = #03000008,
That's for 4.0, not 4.1
C_DWORDLONG is the wrong constant... but strangely, C_DWORDLONG and C_LONGLONG both point to the same value in my copy of 4.1 ! I'll have to ask Matt about it sometime, when he's around...
In 4.1, C_LONGLONG is set to #03000002 - but that hex constant won't work in 4.0, since support for it is lacking in the backend.
Another problem I've run into is a series of functions that take and/or return the following structure by value!
struct x {
float a;
float b;
float c;
float d;
}
Now, I can pass four floats to the function quite easily, but I'm thinking this 128-bit return value may be a problem for Euphoria.
So at this point I may have two options:
- Modify the source of the library to return int64_t and this quad-float structure by reference, then provide my custom-built library instead of the original.
- Write a shim library in C that links to the original library, reads the troublesome values and returns a pointer to them.
-Greg
The shim library is better, as it allows users to use your code in places where they can't update or install your modified library and have to use the original library. Also, no need to rebuild the shim library everytime a new version of the original comes out.
5. Re: define_c_func/proc and 64-bit return types
- Posted by ghaberek (admin) Jul 30, 2012
- 1288 views
That's for 4.0, not 4.1
C_DWORDLONG is the wrong constant... but strangely, C_DWORDLONG and C_LONGLONG both point to the same value in my copy of 4.1 ! I'll have to ask Matt about it sometime, when he's around...
In 4.1, C_LONGLONG is set to #03000002 - but that hex constant won't work in 4.0, since support for it is lacking in the backend.
I was wondering if I could have just hard-coded a value in myself. I guess not! I'll keep this in mind as I move forward.
The shim library is better, as it allows users to use your code in places where they can't update or install your modified library and have to use the original library. Also, no need to rebuild the shim library everytime a new version of the original comes out.
Agreed. A shim it shall be. This is also known as "the wxEuphoria solution" 
(or is it "the wxWidgets problem" ?)
Thanks!
-Greg
6. Re: define_c_func/proc and 64-bit return types
- Posted by mattlewis (admin) Jul 30, 2012
- 1313 views
C_DWORDLONG is the wrong constant... but strangely, C_DWORDLONG and C_LONGLONG both point to the same value in my copy of 4.1 ! I'll have to ask Matt about it sometime, when he's around...
In 4.1, C_LONGLONG is set to #03000002 - but that hex constant won't work in 4.0, since support for it is lacking in the backend.
Several things needed to be changed. In particular, things like C_INT and C_LONG were treated the same way in 4.0, which is a valid assumption for a 32-bit OS, but not for 64-bits.
I'm not sure what the intent of C_DWORDLONG was originally, but I must have suspected it was meant to be a 64-bit integer, and updated it accordingly. Note that the 64-bit types should work on 32-bit 4.1.
Another problem I've run into is a series of functions that take and/or return the following structure by value!
struct x {
float a;
float b;
float c;
float d;
}
Now, I can pass four floats to the function quite easily, but I'm thinking this 128-bit return value may be a problem for Euphoria.
Hmmm....how does this actually work with the compiler? What's the convention?
So at this point I may have two options:
- Modify the source of the library to return int64_t and this quad-float structure by reference, then provide my custom-built library instead of the original.
- Write a shim library in C that links to the original library, reads the troublesome values and returns a pointer to them.
I agree with Jim. The shim is probably easier to deal with, though still a PITA. It might be easier, actually, to build the shim, then see if you can't rip out the machine code, poke it into memory and call it that way. That would alleviate the requirement of having to ship an extra, platform dependent binary.
Note, however, that Windows has defined its own 64-bit calling convention that is subtly different than what's specified by the AMD64 spec (or whatever it's properly called).
Matt
7. Re: define_c_func/proc and 64-bit return types
- Posted by ghaberek (admin) Jul 30, 2012
- 1339 views
If you must know (which you must because it will help LOL), I'm wrapping Allegro 5.
Here's some relevant information:
// allegro5/color.h
struct ALLEGRO_COLOR
{
float r, g, b, a;
};
// allegro5/bitmap.h
ALLEGRO_COLOR al_map_rgba(unsigned char r, unsigned char g, unsigned char b, unsigned char a);
Hmmm....how does this actually work with the compiler? What's the convention?
I made a quick program to call one of the offending functions and here is what I got back from objdump. I have no idea what I'm looking at since I haven't done Assembly coding in well over a decade. I've also been programming all weekend so my eyes are literally tied of reading all this code.
ALLEGRO_COLOR color = al_map_rgba( 255, 0, 255, 0 ); 401b59: 8d 5d d8 lea -0x28(%ebp),%ebx 401b5c: c7 44 24 10 00 00 00 movl $0x0,0x10(%esp) 401b63: 00 401b64: c7 44 24 0c ff 00 00 movl $0xff,0xc(%esp) 401b6b: 00 401b6c: c7 44 24 08 00 00 00 movl $0x0,0x8(%esp) 401b73: 00 401b74: c7 44 24 04 ff 00 00 movl $0xff,0x4(%esp) 401b7b: 00 401b7c: 89 1c 24 mov %ebx,(%esp) 401b7f: e8 94 f7 ff ff call 401318 <_al_map_rgba> 401b84: 50 push %eax 401b85: d9 45 dc flds -0x24(%ebp) 401b88: d9 45 e0 flds -0x20(%ebp) 401b8b: d9 45 e4 flds -0x1c(%ebp)
I agree with Jim. The shim is probably easier to deal with, though still a PITA. It might be easier, actually, to build the shim, then see if you can't rip out the machine code, poke it into memory and call it that way. That would alleviate the requirement of having to ship an extra, platform dependent binary.
Note, however, that Windows has defined its own 64-bit calling convention that is subtly different than what's specified by the AMD64 spec (or whatever it's properly called).
I really like the idea of writing a machine-code shim. It's fast, it's dangerous, it lives on the edge. Let's do it!
-Greg
8. Re: define_c_func/proc and 64-bit return types
- Posted by ghaberek (admin) Jul 30, 2012
- 1291 views
I think I'm on the right track. How does this look? (Functions al_malloc/al_free are just Allegro's wrappers around malloc/free.)
al_shim.h
#include <stdio.h> #include <allegro5/allegro.h> ALLEGRO_COLOR *eu_map_rgba( unsigned int r, unsigned int g, unsigned int b, unsigned int a );
al_shim.c
#include <stdio.h>
#include <allegro5/allegro.h>
#include "al_shim.h"
ALLEGRO_COLOR *eu_map_rgba( unsigned int r, unsigned int g, unsigned int b, unsigned int a )
{
ALLEGRO_COLOR *color = al_malloc( sizeof(ALLEGRO_COLOR) );
*color = al_map_rgba( r, g, b, a );
return color;
}
shimtest.c
#include <stdio.h>
#include <allegro5/allegro.h>
#include "al_shim.h"
int main( int argc, char **argv )
{
if( !al_init() ) {
fprintf( stderr, "failed to initialize allegro!\n" );
return -1;
}
ALLEGRO_COLOR *color = eu_map_rgba( 255, 0, 255, 0 );
printf( "color = (%0.3f,%0.3f,%0.3f,%0.3f)\n", color->r, color->g, color->b, color->a );
al_free( color );
return 0;
}
Output...
color = (1.000,0.000,1.000,0.000)
Hey look, I'm a C programmer! (I don't do this very often so it's always exciting and/or stressful.)
-Greg
9. Re: define_c_func/proc and 64-bit return types
- Posted by petelomax Aug 01, 2012
- 1256 views
In 4.1, C_LONGLONG is set to #03000002
That is a 16-bit float, if you remember such things.
[Assuming you are being totally explicit with your types], a signed 64 bit int should be #01000008 and an unsigned 64 bit int should be #02000008
Several things needed to be changed. In particular, things like C_INT and C_LONG were treated the same way in 4.0, which is a valid assumption for a 32-bit OS, but not for 64-bits.
Just to nitpick, it is the 64-bit application, not the 64-bit OS it must run on, that makes such assumptions invalid.
As far as Ansi/Unicode versions of dll routines go, don't we just leave that up to the programmer? Should 64-bit really be any different?
As you say, 32-bit apps need to be able to throw 64-bit ints around, so I would add explicit C_I32, C_I64, C_UI32, C_UI64, etc as well as changing C_INT from #01000004 on 32-bit to #01000008 on 64-bit. I would also say a 64-bit compiler should generate 64-bit apps; if you want to generate 32-bit apps on a 64-bit box then you should (also) install the 32-bit compiler. If the source has to differ, so be it.
Pete
10. Re: define_c_func/proc and 64-bit return types
- Posted by mattlewis (admin) Aug 01, 2012
- 1227 views
Several things needed to be changed. In particular, things like C_INT and C_LONG were treated the same way in 4.0, which is a valid assumption for a 32-bit OS, but not for 64-bits.
Just to nitpick, it is the 64-bit application, not the 64-bit OS it must run on, that makes such assumptions invalid.
No, it actually is the operating system, at least with respect to how C data types are defined there.
As far as Ansi/Unicode versions of dll routines go, don't we just leave that up to the programmer? Should 64-bit really be any different?
I'm...not sure what you're getting at here. This is a windows specific thing (which in C/C++ is handled with compiler macros). How is it not being left to the programmer?
As you say, 32-bit apps need to be able to throw 64-bit ints around, so I would add explicit C_I32, C_I64, C_UI32, C_UI64, etc as well as changing C_INT from #01000004 on 32-bit to #01000008 on 64-bit. I would also say a 64-bit compiler should generate 64-bit apps; if you want to generate 32-bit apps on a 64-bit box then you should (also) install the 32-bit compiler. If the source has to differ, so be it.
I'm not seeing the utility here. When you're interfacing with external code, it's typically defined in normal C data types (int, long, pointers, etc). Forcing the euphoria programmer to figure this out is just going to make writing portable code more difficult.
As far as the C interface code is concerned, I think it's best to leave as much to the compiler as possible. So if you need to pass a 64-bit int in a 32-bit program, you would use the C_LONGLONG, which is probably how the published interface reads. The compiler (that's used to build euphoria or a translated euphoria program) will always know the correct size of an int or a long or a pointer on the specific platform. There's no reason to make the euphoria programmer (who often has very limited knowledge of C) try to figure this all out on his own. And even if he knows what he's doing, he'll still make mistakes. I think we should help him avoid mistakes, rather than encourage them.
Matt
11. Re: define_c_func/proc and 64-bit return types
- Posted by ghaberek (admin) Aug 01, 2012
- 1242 views
As far as Ansi/Unicode versions of dll routines go, don't we just leave that up to the programmer? Should 64-bit really be any different?
I'm pretty sure, especially on *NIX, that a 64-bit application will attempt to load 64-bit shared libraries, i.e. open_dll("libwhatever.so") will result in a call to e.g. /usr/lib/libwhatever.so on from a 32-bit app and /usr/lib64/libwhatever.so from a 64-bit app. So it isn't as "manual" of a process as calling FunctionA vs FunctionW. (I could be wrong but this is my basic understanding.)
As you say, 32-bit apps need to be able to throw 64-bit ints around, so I would add explicit C_I32, C_I64, C_UI32, C_UI64, etc as well as changing C_INT from #01000004 on 32-bit to #01000008 on 64-bit.
I think this is a great idea. Perhaps we could use some ifdef's to set the generic constants based on bitness:
public constant C_INT32 = #01000004, C_INT64 = #01000008 ifdef EU64 then public constant C_INT = C_INT64 elsedef public constant C_INT = C_INT32 end ifdef
I would also say a 64-bit compiler should generate 64-bit apps; if you want to generate 32-bit apps on a 64-bit box then you should (also) install the 32-bit compiler. If the source has to differ, so be it.
I'm sure the translator could output the code differently based on a flag, and I know with GCC it's just -m32 versus -m64. Perhaps the translator could simply have the same options, and revert to its native bitness by default.
-Greg
12. Re: define_c_func/proc and 64-bit return types
- Posted by jimcbrown (admin) Aug 01, 2012
- 1221 views
In 4.1, C_LONGLONG is set to #03000002
That is a 16-bit float, if you remember such things.
[Assuming you are being totally explicit with your types], a signed 64 bit int should be #01000008 and an unsigned 64 bit int should be #02000008
I didn't create those constants, someone else did. I didn't realize that there was a standard convention though... where are you getting this from?
Several things needed to be changed. In particular, things like C_INT and C_LONG were treated the same way in 4.0, which is a valid assumption for a 32-bit OS, but not for 64-bits.
Just to nitpick, it is the 64-bit application, not the 64-bit OS it must run on, that makes such assumptions invalid.
It's actually up to the compiler, not the application. The OS does have influence here however, as most compilers follow the convention of the OS that they are compiling to (or perhaps it's that the OS follows the convention of the compiler that is used to build the OS?).
As far as Ansi/Unicode versions of dll routines go, don't we just leave that up to the programmer? Should 64-bit really be any different?
I don't understand what you mean. Leave what up to the programmer?
As you say, 32-bit apps need to be able to throw 64-bit ints around, so I would add explicit C_I32, C_I64, C_UI32, C_UI64, etc
This works for me. Even C typically has a set of types/typedefs that define int8_t and uint8_t, etc.
as well as changing C_INT from #01000004 on 32-bit to #01000008 on 64-bit.
Why? C's int is still 4 bytes on gcc/linux/gnu .. What would be the point? What platform has 8 byte sized ints such that the C compiler has sizeof(int) == 8?
I would also say a 64-bit compiler should generate 64-bit apps; if you want to generate 32-bit apps on a 64-bit box then you should (also) install the 32-bit compiler. If the source has to differ, so be it.
Pete
Euphoria gets away with it because it is not a compiler per se. The 64bit vs 32bit headaches can (mostly) be left to the underlying C compiler, but we do have ifdefs to allow the source to be flexible in those cases where the programmer has to (or simply wants to) deal with those issues upfront.
13. Re: define_c_func/proc and 64-bit return types
- Posted by mattlewis (admin) Aug 01, 2012
- 1215 views
As far as Ansi/Unicode versions of dll routines go, don't we just leave that up to the programmer? Should 64-bit really be any different?
I'm pretty sure, especially on *NIX, that a 64-bit application will attempt to load 64-bit shared libraries, i.e. open_dll("libwhatever.so") will result in a call to e.g. /usr/lib/libwhatever.so on from a 32-bit app and /usr/lib64/libwhatever.so from a 64-bit app. So it isn't as "manual" of a process as calling FunctionA vs FunctionW. (I could be wrong but this is my basic understanding.)
Ah. Yes. It's generally impossible to link to a shared/dynamic library that isn't the same architecture (e.g., x86 vs x86-64). The operating system handles this for you. Windows has WOW64 and Linux has its own way of separating these things.
As you say, 32-bit apps need to be able to throw 64-bit ints around, so I would add explicit C_I32, C_I64, C_UI32, C_UI64, etc as well as changing C_INT from #01000004 on 32-bit to #01000008 on 64-bit.
I think this is a great idea. Perhaps we could use some ifdef's to set the generic constants based on bitness:
Noooooooooooooooooooooooo...
public constant C_INT32 = #01000004, C_INT64 = #01000008 ifdef EU64 then public constant C_INT = C_INT64 elsedef public constant C_INT = C_INT32 end ifdef
What is gained here? I could possibly see having explicit C_INTN type of constants. But I really see only downside to linking these to the more generic C_INT type of constants.
Constructing the ifdefs for C_LONG is an exercise left to the reader.
I would also say a 64-bit compiler should generate 64-bit apps; if you want to generate 32-bit apps on a 64-bit box then you should (also) install the 32-bit compiler. If the source has to differ, so be it.
I'm sure the translator could output the code differently based on a flag, and I know with GCC it's just -m32 versus -m64. Perhaps the translator could simply have the same options, and revert to its native bitness by default.
Yes, the translator handles this:
$ euc -? euc options: ... [-arch architecture] <0356>:: Specify the target architecture (X86, X86_64, ARM) ...
Obviously, this is a 4.1 feature, since 4.0 is only available on a single architecture.
Matt
14. Re: define_c_func/proc and 64-bit return types
- Posted by mattlewis (admin) Aug 01, 2012
- 1203 views
In 4.1, C_LONGLONG is set to #03000002
That is a 16-bit float, if you remember such things.
[Assuming you are being totally explicit with your types], a signed 64 bit int should be #01000008 and an unsigned 64 bit int should be #02000008
I didn't create those constants, someone else did. I didn't realize that there was a standard convention though... where are you getting this from?
The values follow a pattern. Unfortunately, the original pattern can't be kept and also be portable, which is why some were changed for 4.1.
IIRC, the 4.0 C code for calling uses some bitwise flags for optimizing the way C arguments and/or return values are handled. We lost some of these optimizations, because the assumptions are not always true, but I suspect they were not significant in any meaningful sense (at least on modern hardware).
Matt
15. Re: define_c_func/proc and 64-bit return types
- Posted by petelomax Aug 04, 2012
- 1262 views
where are you getting this from?
There's a pattern, as matt said, but if it has to break I can cope.
As far as Ansi/Unicode versions of dll routines go, don't we just leave that up to the programmer? Should 64-bit really be any different?
I don't understand what you mean. Leave what up to the programmer?
When you run things on 64-bit Windows, 32-bit programs load dlls from C:\Windows\SYSWOW64 whereas 64-bit applications load them from C:\Windows\system32 (just to maximise confusion). True, that should make no difference, but I suspect it probably will.
as well as changing C_INT from #01000004 on 32-bit to #01000008 on 64-bit.
Why? C's int is still 4 bytes on gcc/linux/gnu .. What would be the point? What platform has 8 byte sized ints such that the C compiler has sizeof(int) == 8?
Oh. size_t is however 8. My only experience with 64-bit was writing a kernel mode driver, utterly blind, no debugging facilities whatsoever (until very late in the day when I figured out how to squirt things down a named pipe that another program could then display). So that code chucks a few 8 byte ints at routines that probably (I will never know) only read 4 bytes (thank god for little endian). One thing I can tell you is that the 64-bit version of that code is utterly and completely different to the 32-bit code that preceded it. Now that someone has said it, I will concede that OpenEu has a translator in the middle (which I did not have in C++) to do some and possibly all of the mangling.
Pete
Edit: I just found this, see table N2 in http://www.viva64.com/en/a/0004/#ID0ERG
16. Re: define_c_func/proc and 64-bit return types
- Posted by jimcbrown (admin) Aug 04, 2012
- 1300 views
where are you getting this from?
There's a pattern, as matt said, but if it has to break I can cope.
I see it now. Doesn't seem to have been documented, but in retrospect it's obvious.
As far as Ansi/Unicode versions of dll routines go, don't we just leave that up to the programmer? Should 64-bit really be any different?
I don't understand what you mean. Leave what up to the programmer?
When you run things on 64-bit Windows, 32-bit programs load dlls from C:\Windows\SYSWOW64 whereas 64-bit applications load them from C:\Windows\system32 (just to maximise confusion). True, that should make no difference, but I suspect it probably will.
I have no doubt that new kinds of DLL Hell will be discovered. But yes, creating and maintaining 32bit vs 64bit versions of a dll should be left up to the programmer ... even so, why not make life as easy and simple as possible for that programmer?
as well as changing C_INT from #01000004 on 32-bit to #01000008 on 64-bit.
Why? C's int is still 4 bytes on gcc/linux/gnu .. What would be the point? What platform has 8 byte sized ints such that the C compiler has sizeof(int) == 8?
Oh. size_t is however 8.
I have no problems with an additional include that mimics types.h that adds a C_SIZE_T and C_INTPTR_T and so on.
My only experience with 64-bit was writing a kernel mode driver, utterly blind, no debugging facilities whatsoever (until very late in the day when I figured out how to squirt things down a named pipe that another program could then display). So that code chucks a few 8 byte ints at routines that probably (I will never know) only read 4 bytes (thank god for little endian). One thing I can tell you is that the 64-bit version of that code is utterly and completely different to the 32-bit code that preceded it. Now that someone has said it, I will concede that OpenEu has a translator in the middle (which I did not have in C++) to do some and possibly all of the mangling.
Pete
Edit: I just found this, see table N2 in http://www.viva64.com/en/a/0004/#ID0ERG
That's an interesting read. Read world ILP64 systems seem to be rare.
17. Re: define_c_func/proc and 64-bit return types
- Posted by mattlewis (admin) Aug 04, 2012
- 1213 views
I just found this, see table N2 in http://www.viva64.com/en/a/0004/#ID0ERG
Yes, I dealt with most of those issues when making euphoria 64-bit. Fortunately, C99 is a lot friendlier than C++ for writing portable code.
Matt
