Re: Bug in math.e?
- Posted by DerekParnell (admin) Jun 13, 2010
- 1552 views
Thomas said...
When running som test with round in std/math.e i got some weird result.
This was what i got round(22/7,3) = 3
I may be wrong in expecting 3.143?
It's not well documented but to get the result you were expecting you need to code ...
round(22/7, 1000)
The precision parameter is in terms of powers-of-ten.
| Value | Precision | Result |
|---|---|---|
| 1.2345 | 1 | 1 |
| 1.2345 | 10 | 1.2 |
| 1.2345 | 100 | 1.23 |
| 1.2345 | 1000 | 1.234 |
