Welcome to the Second Life Forums Archive

These forums are CLOSED. Please visit the new forums HERE

LSL math errors?

Fenix Eldritch
Mostly harmless
Join date: 30 Jan 2005
Posts: 201
05-24-2008 07:40
In my continuing effort to eliminate prim seams from my builds, I may have come across a possible source... Below is a snippet from a script I put together to help me get more precision when positioning a prim. With it I can theoretically access six decimals as opposed to the viewer build tool's three.

CODE

position(string axis, float value)
{
vector pos_current = llGetPos();

//depending on what axis is, modify the corresponding attribute x,y, or z
if(axis == "x")
pos_current.x = value;
if(axis == "y")
pos_current.y = value;
if(axis == "z")
pos_current.z = value;

llSetPos(pos_current);
llOwnerSay("Repositioning Complete! "+(string)llGetPos());

vector pos_new = llGetPos();
llSay(0, "Coordinates are: "+(string)pos_new.x+" "
+(string)pos_new.y+" "+(string)pos_new.z);
}


Now, my problem is this: the "Repositioning Complete!" line reports the correct value, the one I entered. However, the second report which breaks up the vector, reports an erroneous value! Mind you, these values are minuscule...

From: someone

[7:22] Object: Coordinates are: 110.000000 103.370003 400.000000

[7:22] Object: Repositioning Complete! <110.00000, 103.37000, 400.00000>

[7:22] Object: Coordinates are: 110.000000 103.370003 400.000000


... But that still shouldn't happen! Right?
Dora Gustafson
Registered User
Join date: 13 Mar 2007
Posts: 779
05-24-2008 08:23
Floating point mantissa is 24 bits long including sign. So you can have 7 meaning full decimal digits at the most. What you see can not be avoided, it is a rounding error coming from converting between binary and decimal numbers
103.370003 truncated to 7 decimal digits is 103.3700
digits beyond that are senseless
_____________________
From Studio Dora
Kidd Krasner
Registered User
Join date: 1 Jan 2007
Posts: 1,938
05-24-2008 13:25
It may or may not involve a rounding error. It could just be an inconsistency in the conversion to string.

The only thing we can be sure from this information is that (string) applied to a vector prints out one fewer digits than applying (string) to the individual floats. For example, on the z coordinate, (string)llGetPos prints 400.00000, i.e., 5 decimal places, but (string)pos_new.z prints 400.000000, i.e. 6 decimal places. There shouldn't be any rounding error converting 400.0 to a float and back, and the two results are mathematically identical. The difference is entirely in the number of decimal places produced by the float to string conversion.
Strife Onizuka
Moonchild
Join date: 3 Mar 2004
Posts: 5,887
05-24-2008 14:25
To accurately represent every float, you need at least 9 digits.

Here is what you really need to know, floats can't perfectly express numbers that aren't some combination of powers of two. There is no way to express 0.1 as a float.

Everything you don't need to know:

Floating Point arithmetic was first standardized in the 1980's and hasn't changed much since then. Just about all computers have good hardware support for floating point arithmetic, supporting three different floating point types: single, double, long double

single: 23 bits for the mantissa, 8 bits for the exponent, 1 bit for the sign
double: 52 bits for the mantissa, 11 bits for the exponent, 1 bit for the sign
long double: varies depending upon the cpu, in x86, it's the same as double but with 68 bits of mantissa, on PPC it's 128 bits in total.

How the storage works (for the normalized range):
div = (2 ^ number_of_bits_in_mantissa);
offset = (2 ^ (number_of_bits_in_exponent - 1)) - 1;

value = sign * (1 + (mantissa / div)) * 2 ^ (exponent - offset);

-----------

In the last ten years the type "decimal" has been standardized and has increasing hardware support, though mostly only in mainframes. The decimal type was created in response to the problems of doing monetary calculation with floats (double and single).

decimal, unlike floats is base 10 and not base 2 for it's exponent. This has the advantage of removing the 0.1 problem.

Why LSL doesn't use decimal:
1) limited hardware support
2) much slower then floats
3) no physics engines targeting the decimal type.

-----------

If you want to transmit floats without incurring any loss or requiring any fancy parsing functions, try my Float2Hex function:

http://wiki.secondlife.com/wiki/Float2Hex
_____________________
Truth is a river that is always splitting up into arms that reunite. Islanded between the arms, the inhabitants argue for a lifetime as to which is the main river.
- Cyril Connolly

Without the political will to find common ground, the continual friction of tactic and counter tactic, only creates suspicion and hatred and vengeance, and perpetuates the cycle of violence.
- James Nachtwey
Ollj Oh
Registered User
Join date: 28 Aug 2007
Posts: 522
05-24-2008 14:35
The havok engine only calculates down to a finite element of 0.010 distance, same goes for smallest prim/bounding box scale.
the z buffer resolution is also very limited and full of minor errors like gaps between 2 planar planes.
A rounding error of 0.000003 within the resolution of a 16 bit float is NOT your problem and not worth caring for because the engine wont be that accurate, ever. It chooses speed and bandwidth saving over accuracy.

if you still want accuracy, use float=llFloor(integer/1000.) to round it on a meaningfull grid, any smaller grid wont be noticable for buildings.
if you want accuracy, make sure NO prim is accidently rotated by 1° too much, thats a lot more grief in building than anything else.
Fenix Eldritch
Mostly harmless
Join date: 30 Jan 2005
Posts: 201
05-26-2008 08:43
Thanks everyone for the explanations!