Re: sprintf() precision arbitrarily at six decimal places?

new topic     » goto parent     » topic index » view thread      » older message » newer message
euphoric said...

I have an atom of value with 8 decimal places. When I want to use it in an API call, I have to make it text; I generally use sprintf("%f",{v}) because I cannot have exponential notation. However, I just recently discovered that sprintf() and sprint() do not return my complete number! I was shocked to discover this.

I suspected as much. You really should hold the precise number it needs and specify that as eg %.8f - I remember having much trouble trying to figure out how many dp each of the various API args needed, especially when the requirements for the same field often changed from one call to the next.

euphoric said...

What's the maximum precision I can get away with and still be confident that the number I want is the number I'll get?

I think you answer below... 16 digits. I can run with that.

Be warned: some of those API calls kicked me for passing too many trailing zeroes.
Independently of that, using the max 16dp can trigger the odd 0.1234000000000001 whereas using say 12 dp would round that glitch away.

new topic     » goto parent     » topic index » view thread      » older message » newer message

Search



Quick Links

User menu

Not signed in.

Misc Menu