Float literal precision

When typing float literals, how many significant digits (and/or decimal places) should I include to extract the maximum precision from LSL? The 32 digits of the Windows calculator seem a tad excessive and do little for a script's legibility.

(I'm tightening up some code that involves lots of PI and DEG_TO_RAD, etc.)

A float in LSL has a 24 bits mantissa; 2^24=16777216, so 8 decimal digits are all you ever need.
If you set aside one bit for the sign you get 2^23=8388608, indicating that 7 decimal digits are sufficient.

Thanks Dora, eight will do nicely.

you actually get a fraction over 7 significant decimal digits, so you might want to include up to 9 for absolute surety of the closest possible rounding if it really matters to you... although in practice, much over 5 places is just being thorough rather than necessary unless you're dealing with really small and really large multipliers

It's a fine line between thorough and anally retentive - just one or two decimal places

Agreed.. depending on what you're doing, in most cases that level of precision isn't really that that important.

Precisely, you have log10(2^23) or ~6.924 significant decimal digits of precision.

That means you can most definitely store 6 significant digits without loss of accuracy, and ~92% of the time you can store 7 without a least-significant-digit error of one place. You probably won't get 8 significant decimal digits.

shouldn't that be 2^24 (first significand bit is assumed) ? or ~7.225 significant digits TBH I can't remember what rounding model is used... round towards even I think (so the 25th bit would round up if the 24th was 0, down otherwise)

Yeah, I forgot about the implied "1", since 0, NaN, and Inf all have special "coded" representations.

Since the first bit is implied: 1, it doesn't add to the number of possible mantissas which will be 2^23=
8388608 or (count them) 7 decimal digits.
For those who want more of the background see:
http://steve.hollasch.net/cgindex/coding/ieeefloat.html
As you can see I dispute that: If the leading bit wasn't implied, it would have to be set always, thus giving us a number of 2^22 different mantissas. So the quote should have said: "As a result, the mantissa has effectively 23 bits of resolution".

Please correct me if I am wrong

Note: in the above I don't count the special values: zero, denormalized, infinity and Not a Number.

a simple test should suffice...

llOwnerSay( (string)16777216.0 ); //-- 2^24 as a float, will be printed as such, 8 significant digits
llOwnerSay( (string)16777217.0 ); //--2^24+1, will print 16777216.0
llOwnerSay( (string)16777.216 ); //-- 2^24 / 100, will print as such

the reason it adds to the possible significands, is because it inherently represents the first possible one. put another way, the sinificand only represents a whole number, and the floating point is placed separately by the exponent

ETA:
think of ANY number other than zero, and give it a binary representation.... there will always be one bit set highest. if you always discard that bit, and only count the ones after it until the last 1 bit, you'll always be able to represent that number with one less bits used if you know that when you read it back out, you automatically add a bit to the front.

the exponent controls whether you add that imaginary first bit or not so that you don't end up not being able to represent zero. it uses a special value of -127 (or -128 I forget which offhand) it also controls NaN and infinities

I just don't understand what you are saying! Is it 23 or 24 bits effective mantissa?
The following example shows that it is NOT 24:

default
{
    touch_end(integer num)
    {
        float x= llPow( 2.0, 23.0 );
        llOwnerSay("(2^23)-1= "+(string)(x-1.0));
        llOwnerSay("2^23= "+(string)x);
        llOwnerSay("(2^23)+1= "+(string)(x+1.0));
        x= llPow( 2.0, 24.0 );
        llOwnerSay("(2^24)-1= "+(string)(x-1.0));
        llOwnerSay("2^24= "+(string)x);
        llOwnerSay("(2^24)+1= "+(string)(x+1.0));
    }
}

Result:
Object: (2^23)-1= 8388607.000000
Object: 2^23= 8388608.000000
Object: (2^23)+1= 8388609.000000
Object: (2^24)-1= 16777220.000000
Object: 2^24= 16777220.000000
Object: (2^24)+1= 16777220.000000

ah, now I see where the error lies... and I did screw up my example.
2^2 (two bits?) == 4 == 100b(three bits to represent)
ie
2^24 yields 25 bit places (1 followed by 24 zeros)
so it's actually (numerically)
2^23 - 1 + 2^23 yeilds 24 bit places filled. (or 2^24 - 1, which lsl will round before subtracting) (24 ones)

or 16777215

llPow is showing early rounding errors though, as 2^24 should actually represent (because it's only 1 set bit moved 24 places in binary)

I can only refer to the IEEE specifications. Nothing else in LSL is documented, that includes the way literals are converted to float and how they are rounded.
According to http://steve.hollasch.net/cgindex/coding/ieeefloat.html the fraction has 23 bits and I state that the mantissa has 23 bits resolution. The fact that the mantissa has one more bit doesn't change that because it is always 1 (except in a few special cases).
So this quote is not right:it should have been: "As a result, the mantissa has effectively 23 bits of resolution, by way of 23 fraction bits."

the binary number is never reduced to a fraction below 1 (LSL uses the IEEE754 spec... 1985 version I believe)

the action is take any binary number, get the sign (goes in first bit), and make the number possitive if it wasn't. get the binary power to bring the number to 1.xxx, place that in the exponent, then remove the floating point, and anny leading or trailing zeros. next remove the first remaining digit on the left (always a 1 because you removed all leading zeros), then place the next 23 bits in the significand.

when it's read back out, you tack a 1 on the front, offset it by the exponent, and apply the sign.

that extra binary 1 isn't stored, but it's assumed (because when all leading zeros are gone, the next digit must always be 1, allowing it to be assumed in storage), allowing you an extra binary bit place beyond the 23 stored bits. which is why tables that list the bits for the significand often list them as 23 + 1

to put it another way, imagine your number is 5, in binary that'd be 101, when you convert that to a float, you get zero sign, exponent of 2 (for our purpose we'll ignore the bias) which is the binary fraction 1.01 and the significand stores only the bits after the . or 01 with zeroes padding the end (which are unimportant to the example)... you just stored a 3 bit number in 2 bits, free bit courtesy of the knowledge that it'll always be 1 on the front

I've written a number of functions that convert floats to strings. There are a few floats that need 9 digits to represent them without loosing a single bit. I recommend using 9 digits but 8 bits will work in the majority of numbers.

As a structure floats are split into three sections:
You have a sign bit.
8 bits for the exponent.
23 bits for the mantissa.

If you treat each as an unsigned integer then this is how the value is calculated:

(1 - (sign << 1)) * llPow(2, (exponent | !exponent) - 150) * (((!!exponent) << 23) + mantissa);
(I'm pretty sure I've gotten it correct, I've written this from memory).

I wrote a couple functions a good while ago that converts integers to floats in this manner (and back again).
https://wiki.secondlife.com/wiki/User:Strife_Onizuka#Float_.3C-Union-.3E_Integer

Well, I think the question is what is the max number of decimal digits, as a range, which can store all possible values within that range, without precision loss. While you can store *some* values with more than 7 significant digits, not all values in the range of 0.0-99,999,999.0 can be stored without loss of significance.

The problem is that the significance of decimal digits does not map directly to the significance of binary digits, so some numbers will actually fit, while others will not.

Thank you all for a very informative discussion of what seemed to me, at a first glance, a very simple and straightforward question.

This, along with the other float thread, also ties in very nicely with another script I've been working on, a calculator, which has exposed the problems associated with LSL math functions and 32 bit floats to me rather startlingly.

Incidentally, for the calculator, a HUD, for fun, I've incorporated an animated circular slide rule and cursor. The cursor is a sharp edge of a prism that only just pokes through the face of the two slide rule prims (by 0.01mm at its working scale), and I found that in a few positions on my screen it disappeared from view (e.g. at y = -1.077), but would reappear when the whole thing was moved just one pixel to either side. I'll now put that effect down to the server code using 32 bit floats and some position coordinates falling into the gaps between binary fractions.

actually display is up to your video card, although I think only 32bit floats are transmitted

Bleh. It's still all the fault of those 32 bit floats - and the people who triggered the idea in the first place