RE: BUG! Erratic for loop behavior

new topic     » goto parent     » topic index » view thread      » older message » newer message

I added rounding to the magnify function, and it seems to work all the 
time now. Right up to 16 decimal places.

<CODE>
function round(atom a, integer place)
 atom b
   b = floor(a)
   return b + ((a-b) >= (0.5*power(10,-place)))
end function

function magnify(atom a)
 sequence str
 integer f
   str = sprintf("%.16g",a)
   f = find('.',str)
   if f then
      a *= power(10,length(str)-f)
      return round(a,0)
   else
      return a
   end if
end function
<END CODE>


Chris


Chris Bensler wrote:
> I just did some further testing, and it seems that magnifying a decimal 
> value doesn't help either.
> 
> ? magnify(0.12345678901) = 12345678901 -- 0
> ? magnify(0.12345678901) = 123456789   -- 0
> ? magnify(0.12345678901) = 12345679    -- 0
> 
> This seems like a devastating problem. We use fractional numbers all the 
> 
> time. How is it possible that it doesn't show up more often?
> 
> Do we need to resort to binary math? Would that even help?
> 
> What can be done to ensure that my program is calculating the math 
> properly?
> 
> 
> Chris
> 
> 
> Chris Bensler wrote:
> > How does atom_to_float help?
> > 
> > 
> > sequence x     x= atom_to_float32(0.1)
> > 
> > ? x
> > ? float32_to_atom(x+x+x+x+x+x+x+x+x+x)
> > 
> > 
> > How do other languages deal with the general use of decimals?
> > 
> > Chris
> > 
> > Andy Serpa wrote:
> > > > .1 + .1 + .1 + .1 + .1 + .1 + .1 + .1 + .1 does not equal 1 -- just try 
> > > > it.  It will display as one if you print it, but if you do this:
> > > > 
> > > > x = .1 + .1 + .1 + .1 + .1 + .1 + .1 + .1 + .1 
> > > 
> > > Oops.  One ".1" short, but you get the point...
> > > 
> > > 
> 
> 
>

new topic     » goto parent     » topic index » view thread      » older message » newer message

Search



Quick Links

User menu

Not signed in.

Misc Menu