How EU rounds atoms

new topic     » topic index » view thread      » older message » newer message

After rereading the thread BUG! Erratic.....
I tried to program around the problem by firing a laser-gun
at floats (atomize them, floor() them) and look at the real
nasty results.

atom var
var=9999999949
? var
var=9999999995
? var
var=99999999949
? var
var=99999999950
? var
var=999999999949
? var
var=999999999950
? var
var=9999999999499
? var
var=9999999999500
? var
? 1/0

so much for rounding, where an 8-bytes value should be able
to hold   16 ExaBytes-1  [16*power(1024,6)-1 if unsigned].

antoine

new topic     » topic index » view thread      » older message » newer message

Search



Quick Links

User menu

Not signed in.

Misc Menu