1. Nokia n900 and Euphoria
- Posted by Jerome May 13, 2011
- 5872 views
Hunting through the forums I found some posts about porting euphoria to arm processors: http://openeuphoria.org/forum/112156.wc#112156, http://openeuphoria.org/forum/110757.wc#110757. I recently got a nokia n900 cell phone that runs the Linux Maemo operating system with an ARM Cortex processor. So following the suggestions in the posts and making some tweaks I was able to get the EU 4.0.2 interpreter working on my phone! But now I've run into the following issue which I don't know anything about. Any code that requires the call_back() function in std/dll.e fails with the following error:
Internal error: CallBack routine id patch failed: missing magic
This error is related to be_machine.c at line 2372 in the source code, but that's as much as I know. Some googling shows info about magic numbers, but I'm not sure how to proceed. Any thoughts or suggestions?
Thanks in advance!
- Ira
P.S. I've done this with Scratchbox, EU 4.0.2, and the Nokia n900 SDK on Arch Linux
2. Re: Nokia n900 and Euphoria
- Posted by SDPringle May 13, 2011
- 5851 views
I think we have been here before. Someone says they want to port to ARM and probably and thanks to it being at least nearly all ANSI C, I'll bet they do, but the changes if any never find there way back to the repository. Unfortunate. As this problem probably has already been solved.
How long did it take you to get it working on your phone? The magic numbers are where we have to put the identifiers for the EUPHORIA routine. You'll have to change the CALLBACK_SIZE macro. Beyond that, most people will need either an emulator or such a phone to try and test code.
Shawn
3. Re: Nokia n900 and Euphoria
- Posted by mattlewis (admin) May 13, 2011
- 5851 views
Internal error: CallBack routine id patch failed: missing magic
P.S. I've done this with Scratchbox, EU 4.0.2, and the Nokia n900 SDK on Arch Linux
Very nice! It might be a little more complicated than just what Shawn said about changing the size of the callback code, since ARM machine code looks different than x86. I had to do some investigating to figure out how to make x86_64 callbacks work (and OSX callbacks weren't much fun, either).
The first step is to get the output from something like objdump. Then we need to figure out how to change the euphoria backend.
You'll need to run it on the be_runtime.o file, and look for the output for cdecl_call_back. If you need help, please put the output into a Pastey.
Also, it would be great to get this into the official code!
Matt
4. Re: Nokia n900 and Euphoria
- Posted by Jerome May 14, 2011
- 5822 views
I think we have been here before. Someone says they want to port to ARM and probably and thanks to it being at least nearly all ANSI C, I'll bet they do, but the changes if any never find there way back to the repository. Unfortunate. As this problem probably has already been solved.
How long did it take you to get it working on your phone?
It took me a week of working on and off to get a working interpreter on my phone. Some of that time was devoted to Scratchbox and phone specefic issues though as well. The biggest challenge I ran into was that some Euphoria programs would run fine in my Scratchbox environment but crash on my phone. For example, directly or indirectly including std/text.e in a program on my phone would crash with "Internal Error". There didn't seem to be any other error information, so debugging became a guessing game for me. I'm not sure if this is ARM specific or specific to my operating system on the phone. In any case, I would be happy to submit my changes.
The first step is to get the output from something like objdump. Then we need to figure out how to change the euphoria backend.
I changed the CALLBACK_SIZE macro to 126 (my guess based on objdump) per Shawn's suggestion and I no longer get the missing magic error. It does seem though that callback routines don't actually run though, so editing the backend seems necessary. I'm happy to give it a try if you can point me in the right direction. I've posted the objdump results for be_runtime.o in a Pastey here: http://openeuphoria.org/pastey/67.wc
Thanks,
Ira
5. Re: Nokia n900 and Euphoria
- Posted by mattlewis (admin) May 16, 2011
- 5711 views
I changed the CALLBACK_SIZE macro to 126 (my guess based on objdump) per Shawn's suggestion and I no longer get the missing magic error. It does seem though that callback routines don't actually run though, so editing the backend seems necessary. I'm happy to give it a try if you can point me in the right direction. I've posted the objdump results for be_runtime.o in a Pastey here: http://openeuphoria.org/pastey/67.wc
Sorry, forgot to mention the flags. Use the -d flag to get the disassembly.
Matt
6. Re: Nokia n900 and Euphoria
- Posted by Jerome May 16, 2011
- 5680 views
Sorry, forgot to mention the flags. Use the -d flag to get the disassembly.
Matt
No problem; pastey with dissasembly is located here: http://openeuphoria.org/pastey/68.wc
Thanks,
Ira
7. Re: Nokia n900 and Euphoria
- Posted by mattlewis (admin) May 16, 2011
- 5708 views
Sorry, forgot to mention the flags. Use the -d flag to get the disassembly.
Matt
No problem; pastey with dissasembly is located here: http://openeuphoria.org/pastey/68.wc
Hmmm....didn't get any symbols. Also, from your previous pastey, I see:
0000cd10 g F .text 0000007c cdecl_call_back
Unfortunately, I think the output may have been too big for the pastey, since the objdump output ends at 1960. So, third try...look for the block that starts at 0000cd10 (which should be cdecl_call_back) and paste that in. It might also be good to try building in debug mode, which might give us the symbols, to make it easier to spot.
Matt
8. Re: Nokia n900 and Euphoria
- Posted by Jerome May 17, 2011
- 5619 views
Unfortunately, I think the output may have been too big for the pastey, since the objdump output ends at 1960. So, third try...look for the block that starts at 0000cd10 (which should be cdecl_call_back) and paste that in. It might also be good to try building in debug mode, which might give us the symbols, to make it easier to spot.
Sorry, you are right! I didn't double check to make sure all the text went into the Pastey... Ok, so I have two new pasteys which should hvae everything: http://openeuphoria.org/pastey/69.wc and http://openeuphoria.org/pastey/70.wc. I re-built in debug mode, and then copied out the symbols table and the blocks of dissasembly that refer to cdecl_call_back, which is now at 0000cfd4.
Thanks again,
Ira
9. Re: Nokia n900 and Euphoria
- Posted by mattlewis (admin) May 17, 2011
- 5644 views
Sorry, you are right! I didn't double check to make sure all the text went into the Pastey... Ok, so I have two new pasteys which should hvae everything: http://openeuphoria.org/pastey/69.wc and http://openeuphoria.org/pastey/70.wc. I re-built in debug mode, and then copied out the symbols table and the blocks of dissasembly that refer to cdecl_call_back, which is now at 0000cfd4.
Heh...well, we got some callbacks in pastey 70. Based on that, 124 looks like the right answer, and it seems to be finding it. I think that the blx instruction is relative, not absolute, however. I don't know much about ARM assembly, but this looks like the relevant part:
d500: e59f0020 ldr r0, [pc, #32] ; d528 <call_back9+0x7c> d504: e51b1008 ldr r1, [fp, #-8] d508: e51b200c ldr r2, [fp, #-12] d50c: e51b3010 ldr r3, [fp, #-16] d510: e12fff3c blx ip d514: e1a03000 mov r3, r0 d518: e1a00003 mov r0, r3 d51c: e24bd004 sub sp, fp, #4 ; 0x4 d520: e8bd8800 pop {fp, pc} d524: 00000000 .word 0x00000000 d528: 12345678 .word 0x12345678So, rather than putting in the absolute address, I think we need to have a relative address. We were using a similar thing with 64-bit code, though for various reasons, made this use an absolute address. We might want to do something similar here.
Matt
10. Re: Nokia n900 and Euphoria
- Posted by SDPringle May 21, 2011
- 5458 views
Jerome,
Could you paste a patch of the changes you have made so far?
Shawn
11. Re: Nokia n900 and Euphoria
- Posted by Jerome May 21, 2011
- 5627 views
So, rather than putting in the absolute address, I think we need to have a relative address. We were using a similar thing with 64-bit code, though for various reasons, made this use an absolute address. We might want to do something similar here.
Ok, so it looks like I made a few mistakes early on by making too many changes at once... So I retraced my steps, double checked my changes to the eu source code (there are only a few), and cross-compiled the interpreter again. Now, most things seem to be working well, including call_back functions (I tested a simple case). So pushing forward, I cross-compiled wxWidgets according to this post: http://maemogeek.blogspot.com/2007/12/wxgtk-working-on-maemo.html. Finally, I cross-compiled wxEuphoria 0.15 and was able to run some of the demos! Here is a screenshot of the about.exw demo running on my phone: http://dl.dropbox.com/u/5805068/About.png.
There seem to be a few issues with wxEu, for example mouse events work but a call to mouse_event_position() crashes. In any case, it looks like I can write GUI programs with wxEu on my phone, which was my goal.
Thanks for your help and patience!
- Ira
12. Re: Nokia n900 and Euphoria
- Posted by SDPringle May 21, 2011
- 5423 views
Typical, thanks for your help, but you'll get no code from me...
13. Re: Nokia n900 and Euphoria
- Posted by Jerome May 21, 2011
- 5408 views
Typical, thanks for your help, but you'll get no code from me...
Sorry, I did not mean to apply I was done and wouldn't share code! What is the best way to share changes to the source code? I made changes in be_call.c, be_machine.c, and some of the std includes. Also, I'm sure of my changes are not the most appropriate, but they seem to work on my ARM system.
Thanks,
Ira
14. Re: Nokia n900 and Euphoria
- Posted by SDPringle May 21, 2011
- 5342 views
I would be adamant about you getting full credit to porting to ARM and I am thinking about buying a Nokia phone. :) From the modified files we can create patches, integrate them into current development so that following releases won't need to be recoded. If we are careful about alignment issues, the next release should cross compile out of the box.
Shawn
15. Re: Nokia n900 and Euphoria
- Posted by mattlewis (admin) May 22, 2011
- 5372 views
What is the best way to share changes to the source code? I made changes in be_call.c, be_machine.c, and some of the std includes. Also, I'm sure of my changes are not the most appropriate, but they seem to work on my ARM system.
I think you were also working with 4.0. For a change like this, we'll probably want to add the port into 4.1 (but I guess that's open for discussion).
Mercurial can generate patches for you with the "export" command (i.e., "hg export"). Locally, you can just use the sequential rev number (those numbers don't necessarily mean anything outside of your repo), so something like:
$ hg export -o arm_patches 1234:1240...replacing 1234:1240 with whatever your revision numbers were, of course. Then you can email the patch file. Before we integrate it into the official repo, we'll need to decide on where to put it (i.e., 4.0 or 4.1). It will probably start out in its own branch, just to isolate it for testing, etc, before we integrate it.
Try "hg help export" for more details on using the export command.
Matt
16. Re: Nokia n900 and Euphoria
- Posted by Jerome May 24, 2011
- 5437 views
Mercurial can generate patches for you with the "export" command (i.e., "hg export"). Locally, you can just use the sequential rev number (those numbers don't necessarily mean anything outside of your repo), so something like:
$ hg export -o arm_patches 1234:1240
Ok, so I think I'm close to generating a patch. So here is the last issue I'm running into on my ARM phone. I used gdb (thanks for the tip Matt!) to debug why I sometimes get an 'Illegal Instruction' error on my phone for some of the demos:
Program received signal SIGILL, Illegal Instruction. 0x002ff9b4 in decompress (c=251) at be_decompress.c:75 75 d = (double)*(float *)string_ptr;
So this has to do with 4-byte floating point numbers but I'm not sure why there would be an error. Thoughts?
Thanks,
Ira
17. Re: Nokia n900 and Euphoria
- Posted by mattlewis (admin) May 24, 2011
- 5302 views
Program received signal SIGILL, Illegal Instruction. 0x002ff9b4 in decompress (c=251) at be_decompress.c:75 75 d = (double)*(float *)string_ptr;
So this has to do with 4-byte floating point numbers but I'm not sure why there would be an error. Thoughts?
Yeah, that seems like it should work. Try adding a float variable to the function:
float f; .... f = *(float*)string_ptr; d = (double) f;
Also, does this operation always fail? Try something like the following (stand alone):
// bug.c // gcc bug.c -o bug #include <stdio.h> int main(){ char string_ptr[10]; float f; double d; // 3.5 in 32-bit floating point: string_ptr[0] = 0; string_ptr[1] = 0; string_ptr[2] = 96; string_ptr[3] = 64; f = *(float*)string_ptr; d = (double) f; d = (double)*(float*)string_ptr; printf("float: %g double: %g\n", f, d ); return 0; }
Matt
18. Re: Nokia n900 and Euphoria
- Posted by SDPringle May 25, 2011
- 5228 views
I'd say it is an alignment issue. Is string_ptr a multiple of 4? If it comes from a file, probably not. You can't just say to the (ARM) procesor there is a floating point number at string_ptr unless string_ptr is a 4-byte aligned address. You can use memcpy to put it into a float declared in the function and then convert it to double.
Don't do this:
d = (double)*(float*)string_ptr; // attempt to load a float from an unaligned address.
nor this:
float f; f = *(float*)string_ptr; // attempt to load a float from an unaligned address. d = (double)f;
but you can do this:
float f; d = (double)*(float*)memcpy((void*)&f, (void*)string_ptr, 4);
So, where is your patch so far?
19. Re: Nokia n900 and Euphoria
- Posted by mattlewis (admin) May 25, 2011
- 5206 views
I'd say it is an alignment issue. Is string_ptr a multiple of 4? If it comes from a file, probably not. You can't just say to the (ARM) procesor there is a floating point number at string_ptr unless string_ptr is a 4-byte aligned address. You can use memcpy to put it into a float declared in the function and then convert it to double.
Ah, yes, I'd forgotten about the alignment issue. That makes sense.
Matt
20. Re: Nokia n900 and Euphoria
- Posted by Jerome May 25, 2011
- 5172 views
I'd say it is an alignment issue. Is string_ptr a multiple of 4? If it comes from a file, probably not. You can't just say to the (ARM) procesor there is a floating point number at string_ptr unless string_ptr is a 4-byte aligned address. You can use memcpy to put it into a float declared in the function and then convert it to double.
Ah, yes, I'd forgotten about the alignment issue. That makes sense.
Thanks to you both for helping. I believe I have a working patch that I created based on revision 4925 of the trunk. I wasn't sure if the patch should have been based on a particular branch... In any case, I just tested my patch by doing the following:
- cloned the eu repository
- hg import ARM_patch
- copy the repository into Scratchbox
- on my Scratchbox_x86 target (This way it can use a pre-built eui binary),
./configure --plat ARM
make source
- switch to my Scratchbox_ARMEL target,
make
Everything goes smoothly, except for making eu_coverage. I then tested the new binaries on my phone and the demos work well. Also, there is no longer the "Internal Error" with the memcpy fix. I mostly tested eui and eubind with various sample programs.
Is there an official euphoria email to send the patch?
Thanks,
Ira
21. Re: Nokia n900 and Euphoria
- Posted by SDPringle May 25, 2011
- 5175 views
From Jerome's code listing
* on my Scratchbox_x86 target (This way it can use a pre-built eui binary), \\ ./configure ##--##plat ARM \\ make source \\ * switch to my Scratchbox_ARMEL target, \\ make
Here, I am a little surprised Jerome could supply ARM to '--plat' when that was only for the OS. If this goes to the translator, you should get an error. Whether you use Windows or Linux, the translator should answer the same way the following command line:
D:\Documents and Settings\Pringle>euc -con -plat ARM hello.ex Unknown platform: ARM. Supported platforms are: WIN, LINUX, FREEBSD, OSX, OPENB SD, NETBSD
It seems you changed the functionality of --plat argument in 'configure' to set the ARCH variable in Makefile.eu. Instead of testing for--plat ARM, you should make configure test an argument like --arch ARM for that purpose.
22. Re: Nokia n900 and Euphoria
- Posted by Jerome May 25, 2011
- 5220 views
It seems you changed the functionality of --plat argument in 'configure' to set the ARCH variable in Makefile.eu. Instead of testing for--plat ARM, you should make configure test an argument like --arch ARM for that purpose.
Ok, I can do that. For testing on my system, I had hardcoded ARM related changes and was trying to avoid that in the patch. I used the --plat option because I noticed it when running ./configure --help. I actually didn't notice the --arch option in the configure script. Essentially, I have a couple places in the source files that look for an #ifdef ARM and make the neccessary changes, like the CALLBACK size macro for example. Is this the right way to go about it?
Thanks,
Ira
23. Re: Nokia n900 and Euphoria
- Posted by mattlewis (admin) May 25, 2011
- 5203 views
It seems you changed the functionality of --plat argument in 'configure' to set the ARCH variable in Makefile.eu. Instead of testing for--plat ARM, you should make configure test an argument like --arch ARM for that purpose.
Ok, I can do that. For testing on my system, I had hardcoded ARM related changes and was trying to avoid that in the patch. I used the --plat option because I noticed it when running ./configure --help. I actually didn't notice the --arch option in the configure script. Essentially, I have a couple places in the source files that look for an #ifdef ARM and make the neccessary changes, like the CALLBACK size macro for example. Is this the right way to go about it?
The arch configurability is still pretty new. But I'd agree that the ARM belongs there. The platform is still Linux. I don't know enough about ARM to know if one ARM is basically the same as any other. It's possible we'll need to get more specific at some point. Currently, the arch option is used to distinguish between x86 and x86-64.
Matt
24. Re: Nokia n900 and Euphoria
- Posted by SDPringle May 25, 2011
- 5200 views
Inside the 4.0.2 distributed version of EUPHORIA's configure script there are these lines:
if echo "$UNAME_MACHINE" | grep "i[1-7]86" > /dev/null; then echo ARCH=ix86 >> "$PREFIX"${CONFIG_FILE} elif echo "$UNAME_MACHINE" | grep "x86_64" > /dev/null; then echo ARCH=ix86 >> "$PREFIX"${CONFIG_FILE} elif echo "$UNAME_MACHINE" | grep ARM > /dev/null; then echo ARCH=ARM >> "$PREFIX"${CONFIG_FILE} fi
Now, this is from a call to uname. The intention was that you call configure and it detects what kind of CPU you have. Now, it does make sense to override, ARCH. So, you can add a case in the switch statement simliar to the one for plat. Copying the case for plat and changing --plat for --arch, and PLAT for ARCH you'll get:
--arch*) VAL=`echo $1 | cut -d = -f 2` if [ "$VAL" = "$1" ]; then shift ; VAL=$1 fi echo "ARCH=$VAL" >> "$PREFIX"${CONFIG_FILE} ;;
Now you'll need to use --arch ARM or run configure with out arguments on an ARM processor (under emulation). The --arch argument and the value of ARCH should take either ix86 or ARM.
Looking in be_callc.c we see an example of ARCH use.
#if ARCH == ix86 #define push() asm("movl %0,%%ecx; pushl (%%ecx);" : /* no out */ : "r"(last_offset) : "%ecx" ) #define pop() asm( "movl %0,%%ecx; addl (%%ecx),%%esp;" : /* no out */ : "r"(as_offset) : "%ecx" ) #endif
So the idea is to use
#if ARCH == ARM // ARM code #endif #if ARCH == ix86 // Intel x86 code #endif
Shawn
25. Re: Nokia n900 and Euphoria
- Posted by Jerome May 25, 2011
- 5195 views
So the idea is to use
#if ARCH == ARM // ARM code #endif #if ARCH == ix86 // Intel x86 code #endif
I might be off base, but I think c preprocessor comparasions can only work with integers. Take the following the example:
#include <stdio.h> int main() { #if ARCH == ARM printf("Arch is ARM \n"); #else printf("Arch is something else \n") #endif return 0; }
Compiling and running with:
1) gcc -DARCH=x86 test.c -o test outputs "Arch is ARM"
2) gcc -DARCH=ARM test.c -o test outputs "Arch is ARM"
The code works as expected with the following:
#include <stdio.h> int main() { #if ARCH == 1 printf("Arch is ARM \n"); #else printf("Arch is something else \n") #endif return 0; }
1) gcc -DARCH=1 test.c -o test outputs "Arch is ARM"
2) gcc -DARCH=0 test.c -o test outputs "Arch is something else"
Thanks,
Ira
26. Re: Nokia n900 and Euphoria
- Posted by SDPringle May 25, 2011
- 5147 views
Ah, the illumination that occurs from testing. Did you change any EUPHORIA code in the standard library?
27. Re: Nokia n900 and Euphoria
- Posted by SDPringle May 25, 2011
- 5161 views
We just need a couple of #define lines.
#include <stdio.h> #define ARM 0xA #define ix86 86 int main() { #if ARCH == 1 printf("Arch is ARM \n"); #else printf("Arch is something else \n") #endif return 0; }
wcc386 -DARCH=ARM arch.c wlink name arch.exe file arch.obj .\arch Arch is ARM wcc386 ARCH=ix86 arch.c wlink name arch.exe file arch.obj .\arch Arch is not ARM
So we just need to add these lines to execute.h in source:
#define ARM 0xA #define ix86 86
28. Re: Nokia n900 and Euphoria
- Posted by Jerome May 26, 2011
- 5112 views
So we just need to add these lines to execute.h in source:
#define ARM 0xA #define ix86 86
Got it! I removed all changes for ARM to be an option for --plat and now use the --arch ARM in the configure script. I also updated my #ifdef ARM to now use #if ARCH == ARM and then created a new patch. Lastly, I did not need to make any changes to the standard library in the source. What's next?
Thanks,
Ira