OpenEuphoria

Re: rounding problem

• Posted by Chris Bensler <bensler at nt.net> Oct 24, 2006
• 511 views

Ricardo M. Forno wrote:
> 
> Ricardo M. Forno wrote:
> > 
> > George Walters wrote:
> > > 
> > > 
> > > Using EU 2.4 and Win32
> > > 
> > > IF I trace and display the value of temp (a variable) the display shows
> > > temp = .7149999738
> > > 
> > > sprintf("%10.4f, temp)  displays .7150 which is not correct if its value
> > > is
> > > .7149999738
> > > 
> > > On the data base (mysql) the value displayed is .7150.
> > > 
> > > I need to solve this riddle since I'm having invoice totals incorrect. So
> > > how does sprintf know that .7150 is the value but yet .7149999738 is 
> > > displayed on the trace?? I"m confused.
> > 
> > Actually, 0.7150 is correct for 0.7149999738 since you specified it to be
> > 4 digits (i.e., rounded to 4 digits).
> > If you are using floating numbers for invoicing, you'll found nearly
> > impossible to get exact results.
> > This is a property of floating numbers, the hardware, and the binary
> > and decimal arithmetics; Euphoria isn't guilty.
> > Solution: if you want results with 4 decimals, for example, use integers
> > that are 10,000 times greater than the original numbers. Round or truncate
> > them by means of the float() function [truncate: z = floor(x * 10000);
> > round:
> > floor(x * 10000 + 0.5)]. When printing them, divide them into 10,000,
> > only for the task of showing them, not to further compute results.
> > In other words: if you want exact results using cents, then operate
> > with cents.
> > Regards.
> 
> Oops! I meant the floor() function, not float().

This is called fixed point math.

Chris Bensler
~ The difference between ordinary and extraordinary is that little extra ~
http://empire.iwireweb.com - Empire for Euphoria