Re: sprintf() precision arbitrarily at six decimal places?

new topic     » goto parent     » topic index » view thread      » older message » newer message
petelomax said...
euphoric said...

Pete, feel free to jump in here. This behavior also occurs with Phix.

You have not actually explained what your problem is:
... Why exactly do you need variable precision outputs?

I have an atom of value with 8 decimal places. When I want to use it in an API call, I have to make it text; I generally use sprintf("%f",{v}) because I cannot have exponential notation. However, I just recently discovered that sprintf() and sprint() do not return my complete number! I was shocked to discover this.

petelomax said...
euphoric said...

What's the best way to insure all decimal places are returned without going scientific notation for very-precise values?

IEEE 754 floats are not exact and hence when using them "all decimal places" is simply meaningless (see link).

What's the maximum precision I can get away with and still be confident that the number I want is the number I'll get?

I think you answer below... 16 digits. I can run with that.

petelomax said...
atom w = 0.123456789123456789123456789  
--                    ...^^^ ==> ...68(ish) [on 32-bit] 

Phix limits the precision to 16 on 32-bit and 20 on 64-bit, as that is the underlying hardware limit, and which is why sprintf("%0.25f",{w}) yields "0.1234567891234568" on Phix whereas OE yields "0.1234567891234568100000000".

new topic     » goto parent     » topic index » view thread      » older message » newer message

Search



Quick Links

User menu

Not signed in.

Misc Menu