Re: round off problem
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Matt, you're right about the 32 vs 64 conversion. Forget the last
message on the convertData blow up....my typo caused it. I changed the
mysql specifications to double(xx,yy) as needed and the numeric
significance is now considerably more. I'm having no round off issue
reading at least 2000 financial records. I'll need to do some more
checking however on larger datasets.<br>
<br>
Thanks for everybody's help and suggestions... very enlightening on how
each one solves the same problem.<br>
<br>
Matt Lewis wrote:<br>
<blockquote cite="mid1493394626-1463792382-1088439042 at boing.topica.com"
type="cite">
George Walters wrote:
</pre>
<blockquote type="cite">
<pre wrap="">I have imported general ledger data from a working system in
another OS
into mysql using Matt's ODBC tools. Trying to finish the EU general
ledger software, I've run into serious round off issues. Here's a
couple of examples to show the problem.
A typical account balance:
Mysql shows -1143.46 which is correct
EU trace shows -1143.459961 which is not accurate to 15 places as the EU
manual states.
Totalling the credits and debits over the entire file of transactions
gives the following (printing with a mask of %12.2f.)
Debits = 4642782.63
Credits= 4642782.56
Off by 4 cents. The input data from the other OS is balanced. It looks
like there's about 8 places of accuracy not 15.
The data records are always read into a sequence, then arithmetic is
done with atoms.
Any Ideas?
</pre>
</blockquote>
<pre wrap=""><!---->
I think you may be having trouble with a 32-bit floating point number vs
a 64-bit number. Try:
? float64_to_atom( atom_to_float64( 1143.46 ) )
? float32_to_atom( atom_to_float32( 1143.46 ) )
I get:
1143.46
1143.459961
What are the actual datatypes in the MySQL database? And how are you
checking the value from MySQL? You can see how odbc.ew converts data
by looking at the convertData() function.
Matt Lewis
</pre>
</blockquote>
</body>
</html>
|
Not Categorized, Please Help
|
|
