OpenEuphoria

Re: Phix, numbers, printf, log10

• Posted by RobertS 2 months ago
• 758 views

Thank you for all your answers!

This is weird. I swear,

? floor(log10(1000))
? floor(log10(1000 - 1e-16))
? floor(log10(1000 + 1e-16))

gives me
3
2
2
(Win 10, Phix 1.0.4 64-bit.) I suppose log10(1000) is calculated more precisely than log10(not exactly 1000).

Also found this:

99800000000000000 is integer
9.98 * power(10, 16) is integer
9.98e16 is not integer

But I must admit that I'm picking really very tiny nits here.

What got me started now was the
printf(1, "%1.2e", 2.48)
2.49e+0
thing, when I began experimenting with 64-bit Phix for the next Hypatia version.

For two related purposes: to round numbers to n significant digits, and to print numbers with n significant digits.

To round a number to n digits I have to get the magnitude, divide the number by power(10, magnitude) so that there is one digit before the decimal point, then I can use round() to round it to n-1 digits after the decimal point, then multiply the rounded result with the original magnitude. If there is a simpler or better way of doing it, I'm not aware of it.

(If round() gets the last digit wrong by +/-1, so be it, but what I want to avoid is the risk of rounding the number to a wrong number of digits, which might give a much larger error.)

For printing numbers with n significant digits (up to 15 with 64-bit, which sprint() doesn't do) I use a similar method — divide the number by power(10, magnitude+1), convert it to a string with sprintf() so that the string starts with "0.", remove the "0.", then insert the decimal point where it belongs (or add leading zeros) according to the original magnitude.

Previously I used sprintf() with "%e", and read the magnitude from what's after "e" in the string, which meant that whichever way sprintf() rounded the number, mantissa and exponent were consistent. Since finding that "%e" often gets the last digit wrong (see above, 2.48), I use "%F" which doesn't, but this requires me to get the exponent from somewhere else, and this opens up the possibility of getting it wrong (999... getting rounded up to 1000... by sprintf(), but log10 says magnitude is still that of 999..., and the result is printed as 100...). I think I got it under control now, but I'm still wary.

I know that some inaccuracies are unavoidable, due to binary vs. decimal — I only want to understand where they lurk, which ones I could avoid, which ones I can work around, which ones need special attention because they may possibly make something wrong by a factor of 10. And, what to tell the hypothetical user of the upcoming 64-bit version of Hypatia about how far they can, or can not, trust the results. I suppose it will have to be, "Internal accuracy is 18-19 digits, I show you only 15, but the last one may still be wrong by +/-1" — I just want to be sure it's never more than that.

So, this is a fact finding mission, not a complaint!
Thanks again,
Robert