Welcome to the Second Life Forums Archive

These forums are CLOSED. Please visit the new forums HERE

Floating point precision is laughably innacurate.

Sleeper Guillaume
Explorer Achiever
Join date: 8 Jan 2003
Posts: 120
02-06-2003 10:59
There is code out there for rotating a texture on an object. It looked something like:

CODE

float rot = 0.0;

while (rot = rot + 0.1) {
if (rot = 10.1) {
rot = 0.0;
llDoTheRotationEtcEtcEtc(rot, etc);
}
}


This is pretty much an unbounded loop, for if
you print the values of rot you'll see stray numbers
appear after the 1st decimal point rather quickly.

But perhaps that is a bad example.

Consider this other example, one where we try to pretty-print (aka trimming off most of the garbage decimal places) the coords of a vector by multiplying, llRound()'ing and dividing: (Note that I haven't tested this exact code, but it illustrates something similar to what I had been trying to do in-world.

CODE

float x = 57.234323; // we pretend this is some x coord
// as returned by say llGetPos()

x = llRound(x * 100) / 100.0 ;
llSay(0, "x is " + (string)x);


Do you have a guess about what this printed out? Did you perhaps guess it would print out 57.23? That would be my guess. Unfortunately, in that one division by 100.0 we accrue .000007 in error so our pretty-printing fails and we end up with 57.229993 (or was it 57.230007?)

Can the environment's floating precision be improved?
Sleeper Guillaume
Explorer Achiever
Join date: 8 Jan 2003
Posts: 120
02-06-2003 11:07
Well darn I left the / out of the first [/code]... and I can't see the edit button to change the main post....

Edit: ah found the main post's edit button ... was shot way out to the right because of the formatting :)
Wednesday Grimm
Ex Libris
Join date: 9 Jan 2003
Posts: 934
02-06-2003 11:37
Always always always use a delta (or as scientists call it, "fudge factor";) when comparing floating point numbers, never never ever depend on the value of a floating point being exact.

For example, this is bad
CODE

for (x = 0.0; x != 10.0; x += 0.1)

and this is good (well, "ok", "good" would not use a float in a loop counter)
CODE

for (x = 0.0; x < 10.0; x+=0.1)


This is more a property of how floating point numbers are stored rather than any inaccuracy of the simulator. We just have to take this in to account when writing our scripts.

As for pretty-printing, you would probably do better with string manipulation ie:
CODE

llWhisper(0, llGetSubString((string)27.7777777, 0, 4));