OpenEuphoria

Re: math.e and misc.e

• Posted by Matt Lewis <matthewwalkerlewis at gm?il?com> Aug 02, 2007
• 438 views

Juergen Luethje wrote:
> 
> Thanks for the explanation, Matt!
> Just to be sure: When the 17th digit is 5 or higher, should then the
> 18th digit be rounded up, or is it better to keep the original 16th digit?

I'm not sure there's an easy rule like that.  Taking a further look,
it's possible that the seventeenth digit could affect the rounding.  
If you look at how I figure out the fractional part, I look bit by 
bit to see if it should be one or zero.

Basically, I start by putting all of the digits into a sequence, so

0.1234 => {1,2,3,4} = decimals 
[decimals is the var name in the code]

Then, I make a similar representation for 1 / 2:

0.5 => {5} = sub 

This is the value of (binary) 0.1.  If the decimals sequence compares bigger
than the sub sequence, then I set the bit to 1, and subtract it from 
decimals.  Then I divide sub by 2 to get {2,5}, and keep going until
I've got enough to fill up the entire precision, or decimals has dropped 
to all zeros.

A double has 52 bits of precision, plus one more, if it's a normalized value
(meaning there's an implicit 1 at the beginning), so really a total of
53.  But I also consider rounding based on the 54th bit.  Watching my
'sub' sequence, it takes 3 or four bits (i.e., dividing sub by 2) before
the first non-zero digit in sub moves farther down (which makes sense
when you condsider that log10(2) ~ 3.xx).  In other words:

bit  1st non-0 sub
---  --------- ---------
1    1         {5}
2    1         {2,5}
3    1         {1,2,5}
4    2         {0,6,2,5}
...
53   16        {0,.....}
54   17        {0,.....}
55   17        {0,.....}
56   17        {0,.....}
57   18        {0,.....}

So best practice is probably to go ahead and use 17 digits for maximum
precision.  But whether the 17th digit gives any added precision
depends on a lot more than it's own value, because we're converting the
base-10 fraction to a base-2 fraction.  But the 18th digit will have 
absolutely no affect on the value.

Of course, the above all considers that you're using scientific notation,
so something with a lot of leading fractional zeroes:

0.000000000000000000123456789

makes a difference.  But then you have to consider the exponent.  For 
numbers with big or small magnitudes, you have other issues, because the
representations get sparse in those areas.

Matt