OpenEuphoria

2. Re: Trying to use gdb on OS X

• Posted by mattlewis (admin) Jun 06, 2013
• 3560 views

I often do things like:

$ print StdPrint( 1, _some_eu_object_12345, 1 ) 

That's equivalent to "? some_eu_object", so you get something reasonable. For strings, you can do things like "print EPuts(1, _seq_to_print )" (from memory...may not be exactly right).

Matt

4. Re: Trying to use gdb on OS X

• Posted by jaygade Jun 09, 2013
• 3547 views

So here's where I'm at. For some reason, the computer isn't passing along (-1) or (#FFFFFFFF) correctly in the function call for and_bits() in be_runtime.c. It only seems to fail when combined with a larger integer integer that is promoted to an atom, like #DEADBEE5.

Here's my test program:

include std/convert.e 
include std/math.e 
 
printf(1, "1. and_bits(#3EADBEE5, -1): Exp: %x | Res: %x\n", {#3EADBEE5, and_bits(#3EADBEE5, -1)}) 
printf(1, "2. and_bits(#DEADBEE5, -1): Exp: %x | Res: %x\n", {#DEADBEE5, and_bits(#DEADBEE5, -1)}) 
printf(1, "3. and_bits(-1, #DEADBEE5): Exp: %x | Res: %x\n", {#DEADBEE5, and_bits(-1, #DEADBEE5)}) 
printf(1, "4. and_bits(-1, -1): Exp: %x | Res: %x\n", {-1, and_bits(-1, -1)}) 
printf(1, "5. and_bits(-1, #FFFFFFFF): Exp: %x | Res: %x\n", {-1, and_bits(-1, #FFFFFFFF)}) 
printf(1, "6. and_bits(#FFFFFFFF, #FFFFFFFF): Exp: %x | Res: %x\n", {#FFFFFFFF, and_bits(#FFFFFFFF, #FFFFFFFF)}) 
printf(1, "7. and_bits(3735928549, -1): Exp: %x | Res: %x\n", {3735928549, and_bits(3735928549, -1)}) 
printf(1, "8. and_bits(3735928549, #DEADBEE5): Exp: %x | Res: %x\n", {3735928549, and_bits(3735928549, #DEADBEE5)}) 
 
-- Fails tests 2, 3, 5, 7 

1. and_bits(#3EADBEE5, -1): Exp: 3EADBEE5 | Res: 3EADBEE5 
2. and_bits(#DEADBEE5, -1): Exp: DEADBEE5 | Res: 0 
3. and_bits(-1, #DEADBEE5): Exp: DEADBEE5 | Res: 0 
4. and_bits(-1, -1): Exp: FFFFFFFF | Res: FFFFFFFF 
5. and_bits(-1, #FFFFFFFF): Exp: FFFFFFFF | Res: 0 
6. and_bits(#FFFFFFFF, #FFFFFFFF): Exp: FFFFFFFF | Res: FFFFFFFF 
7. and_bits(3735928549, -1): Exp: DEADBEE5 | Res: 0 
8. and_bits(3735928549, #DEADBEE5): Exp: DEADBEE5 | Res: DEADBEE5 

Using gdb, I've traced the program to be_runtime.c line 1761.

Dand_bits (a=0x804e20, b=0xbfffe220) at be_runtime.c:1761 
1761		return and_bits( (unsigned long)(a->dbl), (unsigned long)(b->dbl)); 

At this point, a->dbl and b->dbl have the correct values (but in double format). The cast should take care of that and put both values on the stack for the and_bits() call.

(gdb) p/x (unsigned long)(a->dbl)  
$58 = 0xdeadbee5 
(gdb) p/x (unsigned long)(b->dbl) 
$59 = 0xffffffff 

I can tell gdb to return at that point and it works correctly, so the asm code must be being generated incorrectly. Now, my understanding of x86 assembler is even worse than my understanding of C, and my understanding of floating point and vector instructions is non-existent.

Here's the disassembly for line 1761:

0x002c1498 <Dand_bits+13>:	mov    0xc(%ebp),%eax 
0x002c149b <Dand_bits+16>:	movsd  (%eax),%xmm0 
0x002c149f <Dand_bits+20>:	movapd %xmm0,%xmm1 
0x002c14a3 <Dand_bits+24>:	lea    0x26de9(%ebx),%eax 
0x002c14a9 <Dand_bits+30>:	movapd (%eax),%xmm0 
0x002c14ad <Dand_bits+34>:	movapd %xmm0,%xmm2 
0x002c14b1 <Dand_bits+38>:	cmplesd %xmm1,%xmm2 
0x002c14b6 <Dand_bits+43>:	lea    0x26df9(%ebx),%eax 
0x002c14bc <Dand_bits+49>:	movapd (%eax),%xmm0 
0x002c14c0 <Dand_bits+53>:	minsd  %xmm0,%xmm1 
0x002c14c4 <Dand_bits+57>:	xorpd  %xmm0,%xmm0 
0x002c14c8 <Dand_bits+61>:	maxsd  %xmm0,%xmm1 
0x002c14cc <Dand_bits+65>:	lea    0x26de9(%ebx),%eax 
0x002c14d2 <Dand_bits+71>:	movapd (%eax),%xmm0 
0x002c14d6 <Dand_bits+75>:	andpd  %xmm2,%xmm0 
0x002c14da <Dand_bits+79>:	subpd  %xmm0,%xmm1 
0x002c14de <Dand_bits+83>:	cvttpd2dq %xmm1,%xmm1 
0x002c14e2 <Dand_bits+87>:	movdqa %xmm2,%xmm0 
0x002c14e6 <Dand_bits+91>:	psllq  $0x1f,%xmm0 
0x002c14eb <Dand_bits+96>:	pxor   %xmm0,%xmm1 
0x002c14ef <Dand_bits+100>:	movd   %xmm1,%edx 
0x002c14f3 <Dand_bits+104>:	mov    0x8(%ebp),%eax 
0x002c14f6 <Dand_bits+107>:	movsd  (%eax),%xmm0 
0x002c14fa <Dand_bits+111>:	movapd %xmm0,%xmm1 
0x002c14fe <Dand_bits+115>:	lea    0x26de9(%ebx),%eax 
0x002c1504 <Dand_bits+121>:	movapd (%eax),%xmm0 
0x002c1508 <Dand_bits+125>:	movapd %xmm0,%xmm2 
0x002c150c <Dand_bits+129>:	cmplesd %xmm1,%xmm2 
0x002c1511 <Dand_bits+134>:	lea    0x26df9(%ebx),%eax 
0x002c1517 <Dand_bits+140>:	movapd (%eax),%xmm0 
0x002c151b <Dand_bits+144>:	minsd  %xmm0,%xmm1 
0x002c151f <Dand_bits+148>:	xorpd  %xmm0,%xmm0 
0x002c1523 <Dand_bits+152>:	maxsd  %xmm0,%xmm1 
0x002c1527 <Dand_bits+156>:	lea    0x26de9(%ebx),%eax 
0x002c152d <Dand_bits+162>:	movapd (%eax),%xmm0 
0x002c1531 <Dand_bits+166>:	andpd  %xmm2,%xmm0 
0x002c1535 <Dand_bits+170>:	subpd  %xmm0,%xmm1 
0x002c1539 <Dand_bits+174>:	cvttpd2dq %xmm1,%xmm1 
0x002c153d <Dand_bits+178>:	movdqa %xmm2,%xmm0 
0x002c1541 <Dand_bits+182>:	psllq  $0x1f,%xmm0 
0x002c1546 <Dand_bits+187>:	pxor   %xmm0,%xmm1 
0x002c154a <Dand_bits+191>:	movd   %xmm1,%eax 
0x002c154e <Dand_bits+195>:	mov    %edx,0x4(%esp) 
0x002c1552 <Dand_bits+199>:	mov    %eax,(%esp) 
0x002c1555 <Dand_bits+202>:	call   0x2c1423 <and_bits> 

By the time that and_bits() gets called,

Breakpoint 4, and_bits (a=3735928549, b=0) at be_runtime.c:1754 
1754		a = a & b; 

b is equal to 0 when it should be 0xFFFFFFFF.

gcc --version 
i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5666) (dot 3) 

I may try to remove the --ffast-math option and see what happens. I should also take a closer look at test 4 and figure out why it it passing.

6. Re: Trying to use gdb on OS X

• Posted by jimcbrown (admin) Jun 09, 2013
• 3391 views

This seems similiar to the and_bits() issue that was seen in ARM a few months back.

http://openeuphoria.org/forum/m/120902.wc

http://openeuphoria.org/forum/m/120910.wc

Considering that, unlike ARM, OSX runs on identical hardware to the x86/x64 version (and using the same compiler suite to boot!), even if the results are suppose to be formally undefined/implementation dependent, I'm not sure why we'd get different results. But that appears to be what is happening here.

8. Re: Trying to use gdb on OS X

• Posted by jimcbrown (admin) Jun 09, 2013
• 3317 views

That is odd.

Well, we know that your code works. Maybe the right thing to do is to add your code around an #ifdef EOSX.

Perhaps, but so is any bitwise operation involving more than 32bits according to the other thread.

10. Re: Trying to use gdb on OS X

• Posted by jimcbrown (admin) Jun 09, 2013
• 3329 views

Well, speed is an issue here. Doing these casts unnecessarily on x86 might slow it down (but then gain it might not).

I can't see a downside to that.

Ouch.

I'd say we should use int64_t, as that's standardized, while 'long long' is really a GNU extension.

Thankfully we no longer have to worry about Watcom. (IIRC it doesn't support either, we'd have to use __int64 or something there if we needed a 64bit integer type for some reason.)

12. Re: Trying to use gdb on OS X

• Posted by jimcbrown (admin) Jun 09, 2013
• 3324 views

Hmm, that's a good point. I guess one day in the far future, there might be a platform that gcc supports with 256 sized ints where long long isn't 64bits ... but that's probably not worrying about right now.

I felt that we should do this anyways, considering that only int64_t et al guarantee a particular size in bits. This is a huge job though.

Well, #ifdefs take no runtime performance hit at all. Just saying.

Thinking it over, you're probably right. It's not like hardware is getting any slower, anyways.

A new branch is probably best... I don't think it's a big deal if you commit straight to 4.0 though. Someone needs to test on ARM and the other platforms to make sure nothing breaks anyways...

Hmm. Maybe I was thinking of MSVC...

14. Re: Trying to use gdb on OS X

• Posted by mattlewis (admin) Jun 10, 2013
• 3245 views

Yes, we do this on 4.1, but he's trying to get 4.0 working under OSX. This is very disappointing. Could you try the following program?

#include <stdio.h> 
 
void convert_double( double d ){ 
	long l = (long) d; 
	unsigned long u = (unsigned long) d; 
	printf("%g -> %lx %lx\n", d, l, u ); 
} 
 
int main(){ 
	convert_double( (double) -1 ); 
	convert_double( (double) 0xdeadbeef ); 
	return 0; 
} 
 

I tried this two different ways:

$ gcc -m32 test.c -o test && ./test 
-1 -> ffffffff ffffffff 
3.73593e+09 -> 80000000 deadbeef 
 
gcc -m64 test.c -o test && ./test 
-1 -> ffffffffffffffff ffffffffffffffff 
3.73593e+09 -> deadbeef deadbeef 

Matt

16. Re: Trying to use gdb on OS X

• Posted by jaygade Jun 10, 2013
• 3215 views

I can't try the test until tonight (after 4:30pm PDT), although I'm sure I'll get similar results since that was what I was seeing in gdb from the compiled code. However when typing the casts directly into gdb they worked correctly.

I don't know how/why it broke. My OS X has an older GCC (I think I posted the version number up thread).

It now seems odd to me that the casting has always worked and was never broken in other versions; you would expect to see some odd errors whenever a pointer got stored as a double and then cast back into an int.

Whoever wrote the t_math.e bit tests and used a number that was sure to be cast as a double did a good job of catching this.

18. Re: Trying to use gdb on OS X

• Posted by mattlewis (admin) Jun 10, 2013
• 3190 views

The basic problem is that the behavior is undefined from the perspective of the C specs, so it's very possible that the version of gcc on OSX started doing something different, pre-empting the normal x86 behavior.

You might want to look into the function we added for these casts for ARM in the default branch. I don't know if that's a better solution than using a 64-bit integer or not.

Matt

20. Re: Trying to use gdb on OS X

• Posted by jaygade Jun 10, 2013
• 3235 views

#include <stdio.h> 
 
void convert_double( double d ){ 
	long l = (long) d; 
	unsigned long u = (unsigned long) d; 
	printf("%g -> %lx %lx\n", d, l, u ); 
} 
 
int main(){ 
	convert_double( (double) -1 ); 
	convert_double( (double) 0xdeadbeef ); 
	return 0; 
} 
 

$ gcc -m32 test.c -o test && ./test 
-1 -> ffffffff ffffffff 
3.73593e+09 -> 80000000 deadbeef 
 
gcc -m64 test.c -o test && ./test 
-1 -> ffffffffffffffff ffffffffffffffff 
3.73593e+09 -> deadbeef deadbeef 

OS X results:

jason$ ./test-dbl32 
-1 -> ffffffff 0 
3.73593e+09 -> 80000000 deadbeef 
 
jason$ ./test-dbl64 
-1 -> ffffffffffffffff ffffffffffffffff 
3.73593e+09 -> deadbeef deadbeef 

22. Re: Trying to use gdb on OS X

• Posted by mattlewis (admin) Jun 11, 2013
• 3173 views

In cases where that's big enough, yes it will work. The problem comes when you have a value that's too large for the target type. I hadn't even considered stuff like SSE instructions.

Matt

24. Re: Trying to use gdb on OS X

• Posted by jimcbrown (admin) Jun 11, 2013
• 3062 views

Yes. The prealphas already have for some time.

It's already a problem on 64bit, since we have long doubles (or extended floating points) that could in theory hold larger integer values than a 64bit int. We need a way to do bitwise ops on these superlarge values...

Nothing modern that I can think of. Win32s maybe? Or really old 32bit DOS systems?

26. Re: Trying to use gdb on OS X

• Posted by jaygade Jun 11, 2013
• 3087 views

Does Euphoria use long doubles or extended floats internally now?

Right now, a normal double can hold an integer with 52 bits of precision, and those 52 bits will fit into a 64-bit type. So what I'm doing is taking that double with 52 bits, converting it to a 64 bit integer, and keeping lowest 32 bits. Then we can perform bitwise operations on the number. Remember, this is all internal to the C backend routines -- no double in these cases should be outside the range of a 32-bit integer because that is how the backend is written. (Although I don't think there are any explicit checks for this.)

Going forward maybe we need explicit routines to handle real<-->integer conversions for users regardless of the underlying type or size. I see that the library has to_integer() which returns either 0 or a default value if the number is out of bounds of a Euphoria 29-bit integer.

I think it's really a compiler function; the compiler knows enough about the underlying machine to know whether it can be done in the processor or with a routine (like the old pre-387 days). I was looking into it and I saw that djgpp supports long long. I guess it depends on how old of machines we want to support with new code, or else suggest users use existing older software with older machines.

If and when we work with more embedded architectures, we'll have to solve whatever other problems crop up then.

How does Euphoria handle converting doubles too large for a Euphoria integer now? (Tested on Windows with 4.05...) It fails with a type check error. Should this behavior change?

Remember, this is all for internal conversions between an integral atom stored as a C double and one stored as a C integer used either as an integer (for bitwise ops) or as a pointer. Not for user type conversions between arbitrary doubles and integers (at this point).

I created the change because I compiled 4.0 on OS X, found that I was failing the *bits() routines in t_math.e, and tried to figure out why. I found that certain doubles were not being converted to unsigned longs correctly. Although I didn't realize it at the time, the same problem seems to be related to the ARM port failing the same tests, only I came up with a different solution.

Converting between (larger) doubles and (larger) integers and performing bitwise operations on them is certainly a worthwhile goal, but it wasn't the main thrust of my bugfixing.

I realize that this only applies to 32-bit versions of Euphoria, not necessarily to 64-bit versions, but it will have to be accounted for if there is one codebase which can be compiled to both.

28. Re: Trying to use gdb on OS X

• Posted by jaygade Jun 11, 2013
• 3126 views

Should I create a ticket? I don't want this issue to get lost either.

Yes, I know this, and I thought that I had addressed it a couple of times unless I'm missing some other point.

Currently 32-bit Euphoria has 29-bit integers, and it can perform bitwise operations on integers up to 32 bits in size. Bitwise operations on integers larger than 32 bits produce incorrect results.

Integers which are between 30 and 32 bits in size are stored as doubles. In order to preserve all 32 bits, the C backend has to cast the double to a C integer (long or unsigned long), perform the operation, and then store it back to a double.

The C standard says that casting a double to a long that is too big to fit into the long is undefined, and that casting double that is too big or is negative to an unsigned long is undefined.

There was a bug in the Euphoria C backend which relied upon this undefined behavior working a certain way. The fact that it worked until now is why it wasn't caught until we started moving into ARM and OS X platforms. The bug manifested itself as incorrect results from bitwise operations when the arguments were negative or were larger than 29 bits.

At this point I am not trying to address other behavior or larger values, although some very good points have been raised. Changing the D*bits() functions to work on the full range of integers that can be accurately represented is an interesting, but separate, problem.

30. Re: Trying to use gdb on OS X

• Posted by jaygade Jun 11, 2013
• 3136 views

Okay -- I should know that. smh.