1. rounding problem
- Posted by George Walters <gwalters at sc.rr.com>
Oct 23, 2006
-
Last edited Oct 24, 2006
Using EU 2.4 and Win32
IF I trace and display the value of temp (a variable) the display shows
temp = .7149999738
sprintf("%10.4f, temp) displays .7150 which is not correct if its value is
.7149999738
On the data base (mysql) the value displayed is .7150.
I need to solve this riddle since I'm having invoice totals incorrect. So
how does sprintf know that .7150 is the value but yet .7149999738 is
displayed on the trace?? I"m confused.
3. Re: rounding problem
- Posted by Ricardo M. Forno <rmforno at tutopia.com>
Oct 23, 2006
-
Last edited Oct 24, 2006
George Walters wrote:
>
>
> Using EU 2.4 and Win32
>
> IF I trace and display the value of temp (a variable) the display shows
> temp = .7149999738
>
> sprintf("%10.4f, temp) displays .7150 which is not correct if its value is
> .7149999738
>
> On the data base (mysql) the value displayed is .7150.
>
> I need to solve this riddle since I'm having invoice totals incorrect. So
> how does sprintf know that .7150 is the value but yet .7149999738 is
> displayed on the trace?? I"m confused.
Actually, 0.7150 is correct for 0.7149999738 since you specified it to be
4 digits (i.e., rounded to 4 digits).
If you are using floating numbers for invoicing, you'll found nearly
impossible to get exact results.
This is a property of floating numbers, the hardware, and the binary
and decimal arithmetics; Euphoria isn't guilty.
Solution: if you want results with 4 decimals, for example, use integers
that are 10,000 times greater than the original numbers. Round or truncate
them by means of the float() function [truncate: z = floor(x * 10000); round:
floor(x * 10000 + 0.5)]. When printing them, divide them into 10,000,
only for the task of showing them, not to further compute results.
In other words: if you want exact results using cents, then operate
with cents.
Regards.
4. Re: rounding problem
- Posted by Chris Burch <chriscrylex at aol.com>
Oct 23, 2006
-
Last edited Oct 24, 2006
Hi
Quick fix
* 10000
add 1
floor
/10000
Note, you'll never get more accurate than that, unless you use specialised
libraries, looks like mysql is doing that for you.
Chris
http://euallegro.wikispaces.com
http://members.aol.com/chriscrylex/euphoria.htm
http://uboard.proboards32.com/
http://members.aol.com/chriscrylex/EUSQLite/eusql.html
5. Re: rounding problem
- Posted by Ricardo M. Forno <rmforno at tutopia.com>
Oct 23, 2006
-
Last edited Oct 24, 2006
Ricardo M. Forno wrote:
>
> George Walters wrote:
> >
> >
> > Using EU 2.4 and Win32
> >
> > IF I trace and display the value of temp (a variable) the display shows
> > temp = .7149999738
> >
> > sprintf("%10.4f, temp) displays .7150 which is not correct if its value is
> > .7149999738
> >
> > On the data base (mysql) the value displayed is .7150.
> >
> > I need to solve this riddle since I'm having invoice totals incorrect. So
> > how does sprintf know that .7150 is the value but yet .7149999738 is
> > displayed on the trace?? I"m confused.
>
> Actually, 0.7150 is correct for 0.7149999738 since you specified it to be
> 4 digits (i.e., rounded to 4 digits).
> If you are using floating numbers for invoicing, you'll found nearly
> impossible to get exact results.
> This is a property of floating numbers, the hardware, and the binary
> and decimal arithmetics; Euphoria isn't guilty.
> Solution: if you want results with 4 decimals, for example, use integers
> that are 10,000 times greater than the original numbers. Round or truncate
> them by means of the float() function [truncate: z = floor(x * 10000); round:
> floor(x * 10000 + 0.5)]. When printing them, divide them into 10,000,
> only for the task of showing them, not to further compute results.
> In other words: if you want exact results using cents, then operate
> with cents.
> Regards.
Oops! I meant the floor() function, not float().
6. Re: rounding problem
Ricardo M. Forno wrote:
>
> Ricardo M. Forno wrote:
> >
> > George Walters wrote:
> > >
> > >
> > > Using EU 2.4 and Win32
> > >
> > > IF I trace and display the value of temp (a variable) the display shows
> > > temp = .7149999738
> > >
> > > sprintf("%10.4f, temp) displays .7150 which is not correct if its value
> > > is
> > > .7149999738
> > >
> > > On the data base (mysql) the value displayed is .7150.
> > >
> > > I need to solve this riddle since I'm having invoice totals incorrect. So
> > > how does sprintf know that .7150 is the value but yet .7149999738 is
> > > displayed on the trace?? I"m confused.
> >
> > Actually, 0.7150 is correct for 0.7149999738 since you specified it to be
> > 4 digits (i.e., rounded to 4 digits).
> > If you are using floating numbers for invoicing, you'll found nearly
> > impossible to get exact results.
> > This is a property of floating numbers, the hardware, and the binary
> > and decimal arithmetics; Euphoria isn't guilty.
> > Solution: if you want results with 4 decimals, for example, use integers
> > that are 10,000 times greater than the original numbers. Round or truncate
> > them by means of the float() function [truncate: z = floor(x * 10000);
> > round:
> > floor(x * 10000 + 0.5)]. When printing them, divide them into 10,000,
> > only for the task of showing them, not to further compute results.
> > In other words: if you want exact results using cents, then operate
> > with cents.
> > Regards.
>
> Oops! I meant the floor() function, not float().
This is called fixed point math.
Chris Bensler
~ The difference between ordinary and extraordinary is that little extra ~
http://empire.iwireweb.com - Empire for Euphoria
7. Re: rounding problem
Am I right in thinking that working in pence saves memory as well?
Thanks,
Alex
8. Re: rounding problem
Ricardo M. Forno wrote:
> them by means of the float() function [truncate: z = floor(x * 10000); round:
> floor(x * 10000 + 0.5)]. When printing them, divide them into 10,000,
This is exactly what my round functin does, but produces a different result than
sprintf. I would like to know how sprintf does it's rounding. Anyone know?
global function round(atom a, integer b)
atom f
if b < 0 then
return a -- invalid argument, do nothing
elsif b = 0 then
return floor(a+.5)
else
f = power(10,b)
return floor(a * f +.5 / f)
end if
end function
9. Re: rounding problem
These are the rounding functions that I use:
global type nonnegative_int (object x)
if integer(x) then
return x >= 0
end if
return 0 -- FALSE
end type
global function round_half_even (object x, nonnegative_int digits)
-- rounding according to the IEEE standard
-- in: x : (sequence of) number(s) to round
-- digits: desired number of decimal places
atom p, ret
if atom(x) then
p = power(10, digits)
ret = floor(x*p + 0.5)
if remainder(ret, 2) then
ret -= 1
end if
return ret/p
end if
for i = 1 to length(x) do
x[i] = round_half_even(x[i], digits)
end for
return x
end function
global function round_half_up (object x, nonnegative_int digits)
-- "commercial rounding"
-- in: x : (sequence of) number(s) to round
-- digits: desired number of decimal places
atom p
if atom(x) then
p = power(10, digits)
if x >= 0 then
return floor( x*p + 0.5)/p
else
return -floor(-x*p + 0.5)/p
end if
end if
for i = 1 to length(x) do
x[i] = round_half_up(x[i], digits)
end for
return x
end function
-- Demo
? round_half_even( 2.65, 1)
? round_half_up ( 2.65, 1)
? round_half_even( 2.75, 1)
? round_half_up ( 2.75, 1)
puts(1, "\n")
? round_half_even(-2.65, 1)
? round_half_up (-2.65, 1)
? round_half_even(-2.75, 1)
? round_half_up (-2.75, 1)
Regards,
Juergen
10. Re: rounding problem
George Walters wrote:
>
> Ricardo M. Forno wrote:
> > them by means of the float() function [truncate: z = floor(x * 10000);
> > round:
> > floor(x * 10000 + 0.5)]. When printing them, divide them into 10,000,
>
> This is exactly what my round functin does, but produces a different result
> than
> sprintf. I would like to know how sprintf does it's rounding. Anyone know?
>
> }}}
<eucode>
> global function round(atom a, integer b)
> atom f
>
> if b < 0 then
> return a -- invalid argument, do nothing
> elsif b = 0 then
> return floor(a+.5)
> else
> f = power(10,b)
> return floor(a * f +.5 / f)
> end if
>
> end function
> </eucode>
{{{
Please check the sligth change I made to your round() routine.
Division by 10,000 should be made *after* rounding.
It is named round1(). I also included my RoundCents routine from my
General Functions package.
Regards.
include genfunc.e -- General Functions
global function round(atom a, integer b)
atom f
if b < 0 then
return a -- invalid argument, do nothing
elsif b = 0 then
return floor(a+.5)
else
f = power(10,b)
return floor(a * f +.5 / f)
end if
end function
global function round1(atom a, integer b)
atom f
if b < 0 then
return a -- invalid argument, do nothing
elsif b = 0 then
return floor(a+.5)
else
f = power(10,b)
return floor(a * f +.5) / f
end if
end function
atom x
for i = 1 to 10 do
x = rand(100) / 6.56745321
printf(1, "%10.10f %10.10f %10.4f %10.4f %10.4f\n", {x, round(x, 4),
round(x, 4), round1(x, 4), RoundCents(x, 10000)})
end for
11. Re: rounding problem
- Posted by George Walters <gwalters at sc.rr.com>
Oct 24, 2006
-
Last edited Oct 25, 2006
Hmm, thanks I'll play with that some...
12. Re: rounding problem
- Posted by George Walters <gwalters at sc.rr.com>
Oct 24, 2006
-
Last edited Oct 25, 2006
Thanks Juergen, I'll spin them around some.
13. Re: rounding problem
I have discovered, I believe, the root of my rounding problem. It lies in the
fact that
I am using MySql to store the data. I had specified the numeric values like
"double(12,4)" or "double(12,2)" whereas MySql will do rounding on what you
store. GEEZ!! If I stored 1.250 I would get back from Mysql 1.249999 plus some
less
significant digits at the end. Changing my data base specification to "double"
with no size specification now gives me back exactly what I have stored. See
this reference if you're interested.
http://dev.mysql.com/doc/refman/5.0/en/numeric-types.html
Thanks for all the help. I did have an error in my round function which was
pointed out by Ricardo and Juergen.
I'm trudging on with more testing now.
>
> This is exactly what my round functin does, but produces a different result
> than
> sprintf. I would like to know how sprintf does it's rounding. Anyone know?
>
> }}}
<eucode>
> global function round(atom a, integer b)
> atom f
>
> if b < 0 then
> return a -- invalid argument, do nothing
> elsif b = 0 then
> return floor(a+.5)
> else
> f = power(10,b)
> return floor(a * f +.5) / f
> end if
>
> end function
> </eucode>
{{{
