1. Nokia n900 and Euphoria

Hunting through the forums I found some posts about porting euphoria to arm processors: http://openeuphoria.org/forum/112156.wc#112156, http://openeuphoria.org/forum/110757.wc#110757. I recently got a nokia n900 cell phone that runs the Linux Maemo operating system with an ARM Cortex processor. So following the suggestions in the posts and making some tweaks I was able to get the EU 4.0.2 interpreter working on my phone! But now I've run into the following issue which I don't know anything about. Any code that requires the call_back() function in std/dll.e fails with the following error:

Internal error: CallBack routine id patch failed: missing magic 

This error is related to be_machine.c at line 2372 in the source code, but that's as much as I know. Some googling shows info about magic numbers, but I'm not sure how to proceed. Any thoughts or suggestions?

Thanks in advance!
- Ira

P.S. I've done this with Scratchbox, EU 4.0.2, and the Nokia n900 SDK on Arch Linux

new topic     » topic index » view message » categorize

2. Re: Nokia n900 and Euphoria

I think we have been here before. Someone says they want to port to ARM and probably and thanks to it being at least nearly all ANSI C, I'll bet they do, but the changes if any never find there way back to the repository. Unfortunate. As this problem probably has already been solved.

How long did it take you to get it working on your phone? The magic numbers are where we have to put the identifiers for the EUPHORIA routine. You'll have to change the CALLBACK_SIZE macro. Beyond that, most people will need either an emulator or such a phone to try and test code.

Shawn

new topic     » goto parent     » topic index » view message » categorize

3. Re: Nokia n900 and Euphoria

Jerome said...
Internal error: CallBack routine id patch failed: missing magic 

P.S. I've done this with Scratchbox, EU 4.0.2, and the Nokia n900 SDK on Arch Linux

Very nice! It might be a little more complicated than just what Shawn said about changing the size of the callback code, since ARM machine code looks different than x86. I had to do some investigating to figure out how to make x86_64 callbacks work (and OSX callbacks weren't much fun, either).

The first step is to get the output from something like objdump. Then we need to figure out how to change the euphoria backend.

You'll need to run it on the be_runtime.o file, and look for the output for cdecl_call_back. If you need help, please put the output into a Pastey.

Also, it would be great to get this into the official code!

Matt

new topic     » goto parent     » topic index » view message » categorize

4. Re: Nokia n900 and Euphoria

SDPringle said...

I think we have been here before. Someone says they want to port to ARM and probably and thanks to it being at least nearly all ANSI C, I'll bet they do, but the changes if any never find there way back to the repository. Unfortunate. As this problem probably has already been solved.

How long did it take you to get it working on your phone?

It took me a week of working on and off to get a working interpreter on my phone. Some of that time was devoted to Scratchbox and phone specefic issues though as well. The biggest challenge I ran into was that some Euphoria programs would run fine in my Scratchbox environment but crash on my phone. For example, directly or indirectly including std/text.e in a program on my phone would crash with "Internal Error". There didn't seem to be any other error information, so debugging became a guessing game for me. I'm not sure if this is ARM specific or specific to my operating system on the phone. In any case, I would be happy to submit my changes.

mattlewis said...

The first step is to get the output from something like objdump. Then we need to figure out how to change the euphoria backend.

I changed the CALLBACK_SIZE macro to 126 (my guess based on objdump) per Shawn's suggestion and I no longer get the missing magic error. It does seem though that callback routines don't actually run though, so editing the backend seems necessary. I'm happy to give it a try if you can point me in the right direction. I've posted the objdump results for be_runtime.o in a Pastey here: http://openeuphoria.org/pastey/67.wc

Thanks,
Ira

new topic     » goto parent     » topic index » view message » categorize

5. Re: Nokia n900 and Euphoria

Jerome said...

I changed the CALLBACK_SIZE macro to 126 (my guess based on objdump) per Shawn's suggestion and I no longer get the missing magic error. It does seem though that callback routines don't actually run though, so editing the backend seems necessary. I'm happy to give it a try if you can point me in the right direction. I've posted the objdump results for be_runtime.o in a Pastey here: http://openeuphoria.org/pastey/67.wc

Sorry, forgot to mention the flags. Use the -d flag to get the disassembly.

Matt

new topic     » goto parent     » topic index » view message » categorize

6. Re: Nokia n900 and Euphoria

mattlewis said...

Sorry, forgot to mention the flags. Use the -d flag to get the disassembly.

Matt

No problem; pastey with dissasembly is located here: http://openeuphoria.org/pastey/68.wc

Thanks,
Ira

new topic     » goto parent     » topic index » view message » categorize

7. Re: Nokia n900 and Euphoria

Jerome said...
mattlewis said...

Sorry, forgot to mention the flags. Use the -d flag to get the disassembly.

Matt

No problem; pastey with dissasembly is located here: http://openeuphoria.org/pastey/68.wc

Hmmm....didn't get any symbols. Also, from your previous pastey, I see:

0000cd10 g     F .text	0000007c cdecl_call_back 

Unfortunately, I think the output may have been too big for the pastey, since the objdump output ends at 1960. So, third try...look for the block that starts at 0000cd10 (which should be cdecl_call_back) and paste that in. It might also be good to try building in debug mode, which might give us the symbols, to make it easier to spot.

Matt

new topic     » goto parent     » topic index » view message » categorize

8. Re: Nokia n900 and Euphoria

mattlewis said...

Unfortunately, I think the output may have been too big for the pastey, since the objdump output ends at 1960. So, third try...look for the block that starts at 0000cd10 (which should be cdecl_call_back) and paste that in. It might also be good to try building in debug mode, which might give us the symbols, to make it easier to spot.

Sorry, you are right! I didn't double check to make sure all the text went into the Pastey... Ok, so I have two new pasteys which should hvae everything: http://openeuphoria.org/pastey/69.wc and http://openeuphoria.org/pastey/70.wc. I re-built in debug mode, and then copied out the symbols table and the blocks of dissasembly that refer to cdecl_call_back, which is now at 0000cfd4.

Thanks again,
Ira

new topic     » goto parent     » topic index » view message » categorize

9. Re: Nokia n900 and Euphoria

Jerome said...

Sorry, you are right! I didn't double check to make sure all the text went into the Pastey... Ok, so I have two new pasteys which should hvae everything: http://openeuphoria.org/pastey/69.wc and http://openeuphoria.org/pastey/70.wc. I re-built in debug mode, and then copied out the symbols table and the blocks of dissasembly that refer to cdecl_call_back, which is now at 0000cfd4.

Heh...well, we got some callbacks in pastey 70. Based on that, 124 looks like the right answer, and it seems to be finding it. I think that the blx instruction is relative, not absolute, however. I don't know much about ARM assembly, but this looks like the relevant part:

    d500:	e59f0020 	ldr	r0, [pc, #32]	; d528 <call_back9+0x7c>  
    d504:	e51b1008 	ldr	r1, [fp, #-8]  
    d508:	e51b200c 	ldr	r2, [fp, #-12]  
    d50c:	e51b3010 	ldr	r3, [fp, #-16]  
    d510:	e12fff3c 	blx	ip  
    d514:	e1a03000 	mov	r3, r0  
    d518:	e1a00003 	mov	r0, r3  
    d51c:	e24bd004 	sub	sp, fp, #4	; 0x4  
    d520:	e8bd8800 	pop	{fp, pc}  
    d524:	00000000 	.word	0x00000000  
    d528:	12345678 	.word	0x12345678 
So, rather than putting in the absolute address, I think we need to have a relative address. We were using a similar thing with 64-bit code, though for various reasons, made this use an absolute address. We might want to do something similar here.

Matt

new topic     » goto parent     » topic index » view message » categorize

10. Re: Nokia n900 and Euphoria

Jerome,

Could you paste a patch of the changes you have made so far?

Shawn

new topic     » goto parent     » topic index » view message » categorize

11. Re: Nokia n900 and Euphoria

mattlewis said...

So, rather than putting in the absolute address, I think we need to have a relative address. We were using a similar thing with 64-bit code, though for various reasons, made this use an absolute address. We might want to do something similar here.

Ok, so it looks like I made a few mistakes early on by making too many changes at once... So I retraced my steps, double checked my changes to the eu source code (there are only a few), and cross-compiled the interpreter again. Now, most things seem to be working well, including call_back functions (I tested a simple case). So pushing forward, I cross-compiled wxWidgets according to this post: http://maemogeek.blogspot.com/2007/12/wxgtk-working-on-maemo.html. Finally, I cross-compiled wxEuphoria 0.15 and was able to run some of the demos! Here is a screenshot of the about.exw demo running on my phone: http://dl.dropbox.com/u/5805068/About.png.

There seem to be a few issues with wxEu, for example mouse events work but a call to mouse_event_position() crashes. In any case, it looks like I can write GUI programs with wxEu on my phone, which was my goal.

Thanks for your help and patience!
- Ira

new topic     » goto parent     » topic index » view message » categorize

12. Re: Nokia n900 and Euphoria

Typical, thanks for your help, but you'll get no code from me...

new topic     » goto parent     » topic index » view message » categorize

13. Re: Nokia n900 and Euphoria

SDPringle said...

Typical, thanks for your help, but you'll get no code from me...

Sorry, I did not mean to apply I was done and wouldn't share code! What is the best way to share changes to the source code? I made changes in be_call.c, be_machine.c, and some of the std includes. Also, I'm sure of my changes are not the most appropriate, but they seem to work on my ARM system.

Thanks,
Ira

new topic     » goto parent     » topic index » view message » categorize

14. Re: Nokia n900 and Euphoria

I would be adamant about you getting full credit to porting to ARM and I am thinking about buying a Nokia phone. :) From the modified files we can create patches, integrate them into current development so that following releases won't need to be recoded. If we are careful about alignment issues, the next release should cross compile out of the box.

Shawn

new topic     » goto parent     » topic index » view message » categorize

15. Re: Nokia n900 and Euphoria

Jerome said...

What is the best way to share changes to the source code? I made changes in be_call.c, be_machine.c, and some of the std includes. Also, I'm sure of my changes are not the most appropriate, but they seem to work on my ARM system.

I think you were also working with 4.0. For a change like this, we'll probably want to add the port into 4.1 (but I guess that's open for discussion).

Mercurial can generate patches for you with the "export" command (i.e., "hg export"). Locally, you can just use the sequential rev number (those numbers don't necessarily mean anything outside of your repo), so something like:

$ hg export -o arm_patches 1234:1240 
...replacing 1234:1240 with whatever your revision numbers were, of course. Then you can email the patch file. Before we integrate it into the official repo, we'll need to decide on where to put it (i.e., 4.0 or 4.1). It will probably start out in its own branch, just to isolate it for testing, etc, before we integrate it.

Try "hg help export" for more details on using the export command.

Matt

new topic     » goto parent     » topic index » view message » categorize

16. Re: Nokia n900 and Euphoria

mattlewis said...

Mercurial can generate patches for you with the "export" command (i.e., "hg export"). Locally, you can just use the sequential rev number (those numbers don't necessarily mean anything outside of your repo), so something like:

$ hg export -o arm_patches 1234:1240 

Ok, so I think I'm close to generating a patch. So here is the last issue I'm running into on my ARM phone. I used gdb (thanks for the tip Matt!) to debug why I sometimes get an 'Illegal Instruction' error on my phone for some of the demos:

Program received signal SIGILL, Illegal Instruction.  
0x002ff9b4 in decompress (c=251) at be_decompress.c:75 
75        d = (double)*(float *)string_ptr; 
 

So this has to do with 4-byte floating point numbers but I'm not sure why there would be an error. Thoughts?

Thanks,
Ira

new topic     » goto parent     » topic index » view message » categorize

17. Re: Nokia n900 and Euphoria

Jerome said...
Program received signal SIGILL, Illegal Instruction.  
0x002ff9b4 in decompress (c=251) at be_decompress.c:75 
75        d = (double)*(float *)string_ptr; 
 

So this has to do with 4-byte floating point numbers but I'm not sure why there would be an error. Thoughts?

Yeah, that seems like it should work. Try adding a float variable to the function:

    float f; 
    .... 
    f = *(float*)string_ptr; 
    d = (double) f; 

Also, does this operation always fail? Try something like the following (stand alone):

// bug.c 
// gcc bug.c -o bug 
#include <stdio.h> 
 
int main(){ 
	char string_ptr[10]; 
	float f; 
	double d; 
	 
	// 3.5 in 32-bit floating point: 
	string_ptr[0] = 0; 
	string_ptr[1] = 0; 
	string_ptr[2] = 96; 
	string_ptr[3] = 64; 
	 
	f = *(float*)string_ptr; 
	d = (double) f; 
	d = (double)*(float*)string_ptr; 
	printf("float: %g  double: %g\n", f, d ); 
	 
	return 0; 
} 

Matt

new topic     » goto parent     » topic index » view message » categorize

18. Re: Nokia n900 and Euphoria

I'd say it is an alignment issue. Is string_ptr a multiple of 4? If it comes from a file, probably not. You can't just say to the (ARM) procesor there is a floating point number at string_ptr unless string_ptr is a 4-byte aligned address. You can use memcpy to put it into a float declared in the function and then convert it to double.

Don't do this:

d = (double)*(float*)string_ptr; // attempt to load a float from an unaligned address. 

nor this:

float f; 
f = *(float*)string_ptr; // attempt to load a float from an unaligned address. 
d = (double)f; 

but you can do this:

float f; 
d = (double)*(float*)memcpy((void*)&f, (void*)string_ptr, 4); 

So, where is your patch so far?

new topic     » goto parent     » topic index » view message » categorize

19. Re: Nokia n900 and Euphoria

SDPringle said...

I'd say it is an alignment issue. Is string_ptr a multiple of 4? If it comes from a file, probably not. You can't just say to the (ARM) procesor there is a floating point number at string_ptr unless string_ptr is a 4-byte aligned address. You can use memcpy to put it into a float declared in the function and then convert it to double.

Ah, yes, I'd forgotten about the alignment issue. That makes sense.

Matt

new topic     » goto parent     » topic index » view message » categorize

20. Re: Nokia n900 and Euphoria

mattlewis said...
SDPringle said...

I'd say it is an alignment issue. Is string_ptr a multiple of 4? If it comes from a file, probably not. You can't just say to the (ARM) procesor there is a floating point number at string_ptr unless string_ptr is a 4-byte aligned address. You can use memcpy to put it into a float declared in the function and then convert it to double.

Ah, yes, I'd forgotten about the alignment issue. That makes sense.

Thanks to you both for helping. I believe I have a working patch that I created based on revision 4925 of the trunk. I wasn't sure if the patch should have been based on a particular branch... In any case, I just tested my patch by doing the following:

  • cloned the eu repository
  • hg import ARM_patch
  • copy the repository into Scratchbox
  • on my Scratchbox_x86 target (This way it can use a pre-built eui binary),
    ./configure --plat ARM
    make source
  • switch to my Scratchbox_ARMEL target,
    make

Everything goes smoothly, except for making eu_coverage. I then tested the new binaries on my phone and the demos work well. Also, there is no longer the "Internal Error" with the memcpy fix. I mostly tested eui and eubind with various sample programs.

Is there an official euphoria email to send the patch?

Thanks,
Ira

new topic     » goto parent     » topic index » view message » categorize

21. Re: Nokia n900 and Euphoria

From Jerome's code listing

* on my Scratchbox_x86 target (This way it can use a pre-built eui binary), \\ 
       ./configure ##--##plat ARM \\ 
        make source  \\ 
* switch to my Scratchbox_ARMEL target, \\  
        make 

Here, I am a little surprised Jerome could supply ARM to '--plat' when that was only for the OS. If this goes to the translator, you should get an error. Whether you use Windows or Linux, the translator should answer the same way the following command line:

 
D:\Documents and Settings\Pringle>euc -con -plat ARM hello.ex 
Unknown platform: ARM.  Supported platforms are: WIN, LINUX, FREEBSD, OSX, OPENB 
SD, NETBSD 
 

It seems you changed the functionality of --plat argument in 'configure' to set the ARCH variable in Makefile.eu. Instead of testing for--plat ARM, you should make configure test an argument like --arch ARM for that purpose.

new topic     » goto parent     » topic index » view message » categorize

22. Re: Nokia n900 and Euphoria

SDPringle said...

It seems you changed the functionality of --plat argument in 'configure' to set the ARCH variable in Makefile.eu. Instead of testing for--plat ARM, you should make configure test an argument like --arch ARM for that purpose.

Ok, I can do that. For testing on my system, I had hardcoded ARM related changes and was trying to avoid that in the patch. I used the --plat option because I noticed it when running ./configure --help. I actually didn't notice the --arch option in the configure script. Essentially, I have a couple places in the source files that look for an #ifdef ARM and make the neccessary changes, like the CALLBACK size macro for example. Is this the right way to go about it?

Thanks,
Ira

new topic     » goto parent     » topic index » view message » categorize

23. Re: Nokia n900 and Euphoria

Jerome said...
SDPringle said...

It seems you changed the functionality of --plat argument in 'configure' to set the ARCH variable in Makefile.eu. Instead of testing for--plat ARM, you should make configure test an argument like --arch ARM for that purpose.

Ok, I can do that. For testing on my system, I had hardcoded ARM related changes and was trying to avoid that in the patch. I used the --plat option because I noticed it when running ./configure --help. I actually didn't notice the --arch option in the configure script. Essentially, I have a couple places in the source files that look for an #ifdef ARM and make the neccessary changes, like the CALLBACK size macro for example. Is this the right way to go about it?

The arch configurability is still pretty new. But I'd agree that the ARM belongs there. The platform is still Linux. I don't know enough about ARM to know if one ARM is basically the same as any other. It's possible we'll need to get more specific at some point. Currently, the arch option is used to distinguish between x86 and x86-64.

Matt

new topic     » goto parent     » topic index » view message » categorize

24. Re: Nokia n900 and Euphoria

Inside the 4.0.2 distributed version of EUPHORIA's configure script there are these lines:

 
 
if echo "$UNAME_MACHINE" | grep "i[1-7]86" > /dev/null; then 
	echo ARCH=ix86 >> "$PREFIX"${CONFIG_FILE} 
elif echo "$UNAME_MACHINE" | grep "x86_64" > /dev/null; then 
	echo ARCH=ix86 >> "$PREFIX"${CONFIG_FILE} 
elif echo "$UNAME_MACHINE" | grep ARM > /dev/null; then 
	echo ARCH=ARM >> "$PREFIX"${CONFIG_FILE} 
fi 

Now, this is from a call to uname. The intention was that you call configure and it detects what kind of CPU you have. Now, it does make sense to override, ARCH. So, you can add a case in the switch statement simliar to the one for plat. Copying the case for plat and changing --plat for --arch, and PLAT for ARCH you'll get:

	--arch*) 
		VAL=`echo $1 | cut -d = -f 2` 
		if [ "$VAL" = "$1" ]; then 
			shift ; VAL=$1 
		fi 
 
		echo "ARCH=$VAL" >> "$PREFIX"${CONFIG_FILE} 
		;; 
 

Now you'll need to use --arch ARM or run configure with out arguments on an ARM processor (under emulation). The --arch argument and the value of ARCH should take either ix86 or ARM.

Looking in be_callc.c we see an example of ARCH use.

 
#if ARCH == ix86  
#define push() asm("movl %0,%%ecx; pushl (%%ecx);" : /* no out */ : "r"(last_offset) : "%ecx" ) 
#define  pop() asm( "movl %0,%%ecx; addl (%%ecx),%%esp;" : /* no out */ : "r"(as_offset) : "%ecx" ) 
#endif 
 

So the idea is to use

#if ARCH == ARM  
// ARM code 
#endif 
 
#if ARCH == ix86 
// Intel x86 code 
#endif 

Shawn

new topic     » goto parent     » topic index » view message » categorize

25. Re: Nokia n900 and Euphoria

SDPringle said...

So the idea is to use

#if ARCH == ARM  
// ARM code 
#endif 
 
#if ARCH == ix86 
// Intel x86 code 
#endif 

I might be off base, but I think c preprocessor comparasions can only work with integers. Take the following the example:

#include <stdio.h> 
int main() { 
 
#if ARCH == ARM 
  printf("Arch is ARM \n"); 
#else 
  printf("Arch is something else \n") 
#endif 
return 0; 
} 

Compiling and running with:
1) gcc -DARCH=x86 test.c -o test outputs "Arch is ARM"
2) gcc -DARCH=ARM test.c -o test outputs "Arch is ARM"

The code works as expected with the following:

#include <stdio.h> 
int main() { 
 
#if ARCH == 1 
  printf("Arch is ARM \n"); 
#else 
  printf("Arch is something else \n") 
#endif 
return 0; 
} 

1) gcc -DARCH=1 test.c -o test outputs "Arch is ARM"
2) gcc -DARCH=0 test.c -o test outputs "Arch is something else"

Thanks,
Ira

new topic     » goto parent     » topic index » view message » categorize

26. Re: Nokia n900 and Euphoria

Ah, the illumination that occurs from testing. Did you change any EUPHORIA code in the standard library?

new topic     » goto parent     » topic index » view message » categorize

27. Re: Nokia n900 and Euphoria

We just need a couple of #define lines.

 
#include <stdio.h> 
#define ARM 0xA 
#define ix86 86 
  
int main() {  
  
#if ARCH == 1  
  printf("Arch is ARM \n");  
#else  
  printf("Arch is something else \n")  
#endif  
return 0;  
}  

wcc386 -DARCH=ARM arch.c 
wlink name arch.exe file arch.obj 
.\arch 
 
Arch is ARM 
 
wcc386 ARCH=ix86 arch.c 
wlink name arch.exe file arch.obj 
.\arch 
 
Arch is not ARM 
 

So we just need to add these lines to execute.h in source:

#define ARM 0xA 
#define ix86 86 

new topic     » goto parent     » topic index » view message » categorize

28. Re: Nokia n900 and Euphoria

SDPringle said...

So we just need to add these lines to execute.h in source:

#define ARM 0xA 
#define ix86 86 

Got it! I removed all changes for ARM to be an option for --plat and now use the --arch ARM in the configure script. I also updated my #ifdef ARM to now use #if ARCH == ARM and then created a new patch. Lastly, I did not need to make any changes to the standard library in the source. What's next?

Thanks,
Ira

new topic     » goto parent     » topic index » view message » categorize

Search



Quick Links

User menu

Not signed in.

Misc Menu