1. display/text:format floating point errors

  • Posted by Senator Oct 15, 2020
  • 1150 views

I still use atoms/decimal to store monetary data, but this is the only error/rounding error
I have noticed after a subtraction operation

std/math.e round(bal,100) seems to have solved the problem.
Raw data value from database = 95.4

 display_bal=text:format("$ [:8,,.2]",bal) 
 display("\n[]",bal)      95.3999999999999 
 display("\n[,2]",bal)    95.39 
 display("\n[g]",bal)     95.3999999999999 
 printf(1,"\n%5.2f",bal)  95.4<=====================correct 
                                   debit    balance 
98 10-28 autolog household       $ 250.00  $ 404.60 
99 10-28 autolog FOOD groceries  $ 500.00  $ -95.39<=display_bal=text:format("$ [:8,,.2]",bal)  


display_bal=text:format("$ [:8,,.2]",round(bal,100)) 


                                 debit    balance 
98 10-28 autolog household       $ 250.00  $ 404.60 
99 10-28 autolog FOOD groceries  $ 500.00  $ -95.40<=display_bal=text:format("$ [:8,,.2]", round(bal,100)) 
? bal = 95.4

Regards, Ken

new topic     » topic index » view message » categorize

Search



Quick Links

User menu

Not signed in.

Misc Menu