1. display/text:format floating point errors
- Posted by Senator Oct 15, 2020
- 703 views
I still use atoms/decimal to store monetary data, but this is the only error/rounding error
I have noticed after a subtraction operation
std/math.e round(bal,100) seems to have solved the problem.
Raw data value from database = 95.4
display_bal=text:format("$ [:8,,.2]",bal) display("\n[]",bal) 95.3999999999999 display("\n[,2]",bal) 95.39 display("\n[g]",bal) 95.3999999999999 printf(1,"\n%5.2f",bal) 95.4<=====================correct debit balance 98 10-28 autolog household $ 250.00 $ 404.60 99 10-28 autolog FOOD groceries $ 500.00 $ -95.39<=display_bal=text:format("$ [:8,,.2]",bal)
display_bal=text:format("$ [:8,,.2]",round(bal,100))
debit balance 98 10-28 autolog household $ 250.00 $ 404.60 99 10-28 autolog FOOD groceries $ 500.00 $ -95.40<=display_bal=text:format("$ [:8,,.2]", round(bal,100)) ? bal = 95.4
Regards, Ken