Re: Strange problem
- Posted by Everett Williams <rett at GVTC.COM> Mar 20, 2000
- 472 views
Robert Craig wrote: >> My main confusion was that when I print()ed the results of >> the expressions it said 1 instead of .9999999999. Is that >> because of the 10 digit cutoff? > >Yes. It *rounds* to 10 digits, but doesn't print trailing 0's. That is why most systems choose to truncate rather than round. Truncation produces a printout much closer to the truth than rounding. Rounding, as in this case, can conceal the fact that a calculation will not work because of the actual underlying number. I would suggest that the rounding be removed and truncation be used in the next release...at all levels. It should be faster, and it would solve the other noted problems. Same should apply for prints of float32 or float64. They should trunc(to zeros if necessary) at the last possible significant digit, even if more digits are requested than are significant. Everett L.(Rett) Williams rett at gvtc.com