OpenEuphoria

Re: atom_to_float32 precision

• Posted by Robert Craig <rds at RapidEup?or?a.com> Oct 25, 2007
• 641 views

Jonas Temple wrote:
> Is there any way to control the precision of the output of atom_to_float32?
> I want to take a number that might have the value:
> 
> 123.450000
> 
> And ensure it comes out as:
> 
> 123.45

I'm not sure exactly what you are trying to achieve,
but it sounds like you are encountering an age-old problem
with floating-point numbers that has been discussed here 
many times before. A number like 123.45 can't be 
represented *exactly* in binary floating-point form, 
no matter how many bits you use. There will always be a 
tiny error. This problem can drive you crazy when you are trying
to get money calculations involving pennies to work out
exactly. If you choose 2 decimal places for display (%.2f),
printf() or sprintf() should round it nicely for you, for display purposes.
However it will never be *precisely* 123.45 that is stored
in memory. More likely 123.44999999999999999... or 
123.4500000000...000001 or something along those lines.
See:
http://www.openeuphoria.org/cgi-bin/esearch.exu?fromMonth=6&fromYear=1&toMonth=A&toYear=C&postedBy=&keywords=round+decimal+binary+floating-point

Regards,
   Rob Craig
   Rapid Deployment Software
   http://www.RapidEuphoria.com