1. Re: math.e and misc.e
- Posted by Matt Lewis <matthewwalkerlewis at gm?il?com> Aug 02, 2007
- 439 views
Juergen Luethje wrote: > > Thanks for the explanation, Matt! > Just to be sure: When the 17th digit is 5 or higher, should then the > 18th digit be rounded up, or is it better to keep the original 16th digit? I'm not sure there's an easy rule like that. Taking a further look, it's possible that the seventeenth digit could affect the rounding. If you look at how I figure out the fractional part, I look bit by bit to see if it should be one or zero. Basically, I start by putting all of the digits into a sequence, so 0.1234 => {1,2,3,4} = decimals [decimals is the var name in the code] Then, I make a similar representation for 1 / 2: 0.5 => {5} = sub This is the value of (binary) 0.1. If the decimals sequence compares bigger than the sub sequence, then I set the bit to 1, and subtract it from decimals. Then I divide sub by 2 to get {2,5}, and keep going until I've got enough to fill up the entire precision, or decimals has dropped to all zeros. A double has 52 bits of precision, plus one more, if it's a normalized value (meaning there's an implicit 1 at the beginning), so really a total of 53. But I also consider rounding based on the 54th bit. Watching my 'sub' sequence, it takes 3 or four bits (i.e., dividing sub by 2) before the first non-zero digit in sub moves farther down (which makes sense when you condsider that log10(2) ~ 3.xx). In other words: bit 1st non-0 sub --- --------- --------- 1 1 {5} 2 1 {2,5} 3 1 {1,2,5} 4 2 {0,6,2,5} ... 53 16 {0,.....} 54 17 {0,.....} 55 17 {0,.....} 56 17 {0,.....} 57 18 {0,.....} So best practice is probably to go ahead and use 17 digits for maximum precision. But whether the 17th digit gives any added precision depends on a lot more than it's own value, because we're converting the base-10 fraction to a base-2 fraction. But the 18th digit will have absolutely no affect on the value. Of course, the above all considers that you're using scientific notation, so something with a lot of leading fractional zeroes: 0.000000000000000000123456789 makes a difference. But then you have to consider the exponent. For numbers with big or small magnitudes, you have other issues, because the representations get sparse in those areas. Matt