Re: Strange problem
Robert Craig wrote:
>> My main confusion was that when I print()ed the results of
>> the expressions it said 1 instead of .9999999999. Is that
>> because of the 10 digit cutoff?
>
>Yes. It *rounds* to 10 digits, but doesn't print trailing 0's.
That is why most systems choose to truncate rather than round.
Truncation produces a printout much closer to the truth than
rounding. Rounding, as in this case, can conceal the fact that
a calculation will not work because of the actual underlying number.
I would suggest that the rounding be removed and truncation be
used in the next release...at all levels. It should be faster, and it
would solve the other noted problems. Same should apply for
prints of float32 or float64. They should trunc(to zeros if necessary)
at the last possible significant digit, even if more digits are requested
than are significant.
Everett L.(Rett) Williams
rett at gvtc.com
|
Not Categorized, Please Help
|
|
