integer math error

default
{
state_entry()
{
}

touch_start(integer total_number)
{
llSay(0, (string)(200 * .7));
llSay(0, (string)((integer)(200 * .7)));
}
}


Casting 140.000000 to an integer = 139? I have found this round down error with other numbers also and it is causing a serious problem with some scripts.

Welcome to floating point math. The problem is that the 200 *.7 doesn't result in an exact number due to the limited precision of floats. It will be something like 139.9999999999. When you cast it to an integer, it's not a round, it's a truncate that drops the fractional part so the result is 139.

The first llSay doesn't show all the precision so it does a round to display the closest number, but that's almost always not what's internally stored.

There is an llRound function that might help, but in general don't rely on floats for exact values.

One workaround is to do it like this which returns 140:

default {
	state_entry() {
		llSay(0, (string) (200 * 7 / 10));
	}
}

Note that the rounding problem in this case starts not with the multiplication but with the constant 0.7, which has no exact binary representation. In binary it would have the representation 0.10110_0110_ (meaning that the sequence 0110 is repeated). The closest 32-bit floating point number to that is the value rounded down to binary 0.101100110011001100110011 which, when converted back to decimal, is 0.699999988079. That will produce a result which is slightly less than 140, and since all fractional parts are truncated when converting to an integer....

This is not quite the usual rounding thing, it is actually an LSL Mono bug. You will get 140 for both if compiled with old LSL, but 140 and 139 with Mono.

The bug should be fixed when the 1.25 server finally rolls out.

http://jira.secondlife.com/browse/SVC-3328

Just confirming the fix , Ravanne's sample script prints 140 under both LSO and Mono under 1.25.1 on the beta grid.

Glad to hear they fixed it, just hope the 1.25 version doesn't introduce too many new bugs.

Bah. This should NOT be "fixed." It is correct behavior according to the IEEE standard and thus it is the old LSL that was actually broken. See the comment I put in SVC-3328.

LSL behavior seems to be following a float to int assignment as seen in gcc. The string representations follow %d and %f from the printf family.

The original point still remains however. Relying on exact values for floats or chained operations behaving as if they had infinite precision is very bad programming practice.

Yup, yup

As a general note, it is better not to cast if you can properly use a function to say what you want, e.g. round, floor, ceiling.

The discussion about how the casts work is valid, but better coding practice is to make sure your script is not as susceptible to compiler differences. In short, just use llRound.

I only cast from float to int if I do not care so much, like with llFrand to get a channel.