Re: sprintf usage with %d gives a rounding problem
- Posted by Kat <gertie at PELL.NET> Feb 15, 2001
- 401 views
On 15 Feb 2001, at 10:00, Jan Willem Teunisse wrote: > For a Euro conversion of a financial bank-statements interface file, I > have to put an integer value into strings sequence in order to output > this to an Ascci flatfile record. > > The length of the string value is max. 18 characters, therefore I use an > atom variable instead of an integer declaration. > > For the integer_to_string conversion I use the sprintf(%d,a) function > This works OK for most of the times, but today we ran into a problem > that the following value (displayed by the exw interpreter debugger)gets > the wrong result. By the way: the value of atom a is the result of the > sum of several bank-statements. > > Value a = 9200616 (according the exw debugger screen) > the result of the > sa = sprintf("%d",a) > gives the value in the sequence of "9200615" and this should be > "9200616" > > Looking further into the situation, I noticed that real value of a is > 9200615.9999999963 This will always be a problem in doing math on binary puters, sorry to say. Human deal in decimal math, puters deal in binary math, and the two do not translate well. I suggest, for accuracy and reliability, you use string math. Basically speaking, all the operands are strings (so they can be any size and accuracy) which you parse, and perform math on with a lookup table of results. If you have multiply("5","5"), you look up multiply[5][5] and get "25" which you apply appropriately. I don't know if there are any Eu routines for this yet, but i have seen some Perl routines. Warning: string math is incredibly slow, but you cannot beat the accuracy (and number of significant digits), it's been used to calculate pi out to thousands of decimal points. Possibly you can use string math only in multiplication and division, and use regular integer math for addition and multiplication. Or use repeated addition for multiplication, and a subtract loop for division. Kat