Looking for benchmarks.

Babbage Linden put up the first internal collaborative MONO simulator today and a few of us logged in to test out some scripts. We quickly found some issues that made it clear that it isn't ready for an public preview. I didn't even get to finish the benchmark script I was trying to write.

Anyways, we'll be writing some benchmarks internally, but if any of you have a benchmark script, one you've already used to test how fast the LSL scripting engine runs for specific operations, that you would like us to run on MONO before we open it up for preview then feel free to send me or Babbage a copy and we'll try to test it out.

Initially the MONO scripting engine is expected to be much faster for numerical intensive loops and computations, and only a little bit faster for calls to llFoo(). So we're interested in benchmarks that test one or the other.

(!) Wow! Everyone's been saying this was vaporware. Glad to see it's being tested.

What about string / list manipulation? Do you think it will be any faster for that?

Babbage Linden could probably answer that question. I would expect string manipulation to be in the much faster category but am not certain. I don't know how lists were implemented in the MONO port.

Give me a list string/manipulalion benchmark and I'll return preliminary results

i'll have some math stress tests for you that will check results (currently writing them).

Babbage just emailed a preliminary result on a simple loop counter that ran 23.096/0.134 seconds on legacy/MONO, which put MONO at about 170x the legacy system. That is probably near the highest gain we'll see. Real scripts that move objects around or scan their environment won't speed up that much.

It occurs to me that the MONO speed boost could cause some complex script systems to break. If a script is explicitly (or accidentally) relying on an event on one script completing before a second near simultaneously triggered event on another script then the change from MONO could move the completion times around to cause some things to miss their cues. Dunno. In any case, we plan on supporting both MONO and legacy for a period to ease the transition.

Back to the subject... how would you like to have your benchmark's preliminary results posted here? If so send me your benchmark. There is no rush -- I won't be able to test them until Tuesday next week at the earliest. Also, I'll bet you'll be able to run your own tests sometime in January.

I have to say the only thing I need to have faster are list/string manipulation inside loops.

I do a lot of number crunching but do not have much in the way of performance problems.

It would be nice if some ll commands were faster, but they have built in delays so I do not see that happening anytime soon.

I would be curious to hear what other people see as bottlenecks in their code and I will write some benchmarks around those issues.

When will you have enough up that we can log in to a Mono-infected* sim and see how physics-related scripts react to Mono? I'd love to see what impact it has on Cyberflight, since a lot of the code in there is to tweak the script to keep it running smoothly and providing the same boost in more and less laggy sims.

*

Okay, just because I'm a little dense, let me see if I got this straight:

1. you linked the fully buzzword compliant but rather immature Mono VM into a sim (porting would be an exaggeration as Mono is native to Linux)
   
2. You didn't map in the LSL API
   
3. You find that empty loops run faster than in LSL
   
4. You expect API time to dominate script run-time, though
   
5. You fear that as Mono is faster it will break script function
   
6. Replacing LSL with Mono will break all extant scripts (unless you have a cross-compiler up your sleeve)
   

Did I understand that correctly?

For comparison I ran some quick tests on the venerable Tcl scripting language which:

1. was designed from the start to be an embeddable scripting language
   
2. doesn't have a VM
   
3. does not need garbage collection
   
4. has a syntax that you don't need to know .NET for
   
5. is not based around a window model
   
6. has asynchronous event handling
   
7. has error handing for events
   
8. has intrinsic arrays, lists, iterators, associative arrays, sockets
   
9. has XML, HTML, HTTP, SOAP, and DOM support if needed
   

Now, of course, this test was run on a machine that was not busy running a sim, but it isn't exactly slow:
Do you have hopes of Mono actually being an improvement?

hmmm it depends...

is self replication and crashing the sim a good benchmark?

seriously though....

I'd love to see benchmarks run on the most notoriously slow, non sleep penalized parts of LSL such as long nexted if thens.

1. Posting arbitrary stats is useless, we don't know your computer specs; for all we know you ran that on a PD11 or DeepBlue.
2. From what i can tell of TCL doesn't compile. If it doesn't have a VM what do you call the enviroment?
3. If you have been following Babbage's blog you would know the progress.

The LSL API is SLOOOW.
They have writen an LSL compiler that saves the scripts state.
They have writen an LSL compiler that handles LSL text.
Of course empty loops run faster, LSL was an interpreted language, Mono can be run on the cpu directly.
The speed increase will break some scripts that depend on timing quirks of LSL. IMHO it's a non issue.

The speed gains will be in the exectution of the bytecode. In the past the bytecode had to be fed though a giant switch statement to be executed. Now it gets compiled into machine code. The overhead bytecode for calling functions has been (obviously) reduced which will result in those functions executing majonaly faster. You don't know what your talking about if you think an interpreted langauge can compete in speed to one that is compiled to machine code.

http://shootout.alioth.debian.org/benchmark.php?test=all&lang=csharp&lang2=tcl
while mono didn't score as well overall (it eats memory), it was over all faster.

here is an example of why tcl sucks:
http://wiki.tcl.tk/1173
why do i say it sucks? because an optimizing compiler should be able to do this for you.

I'd like this script to be tested.

It runs a 1000 iterations of each block, i expect the first and second text to be slow in legacy mode (10->20 min), i expect a significant gain. The other 2 loops i expect somegain but nothing as drastic.

Any and all output would be very nice to have. If the output is very long/large just post in txt file as attachment.

integer fui(float a)
{//union float to integer
    integer b = (a<0) << 31;
    if((a = llFabs(a)) < 1.1754943508222875079687365372222e-38)
        b = b | (integer)((a * 8.5070591730234615865843651857942e+37) * 8388608.0);
    else
    {
        integer c = llFloor(llLog(a) / 0.69314718055994530941723212145818);
        b = b | ((c + 127) << 23 ) | (integer)(((a / llPow(2.0, c)) - 1) * 8388608.0);
    }
    return b;
}

float iuf(integer a)
{//union integer to float
    integer c= (a >> 23) & 0xff;
    float b = (a & 0x7fffff) / 8388608.0;
    if(c == 0xff)
    {
        llOwnerSay((string)c);//rem out for production.
        b = 1.0 / 0.0; //NaN or Infinity (since getting them crashes the script, we do that).
    }
    else if(c)
        b = (b + 1) * llPow(2, c - 127);
    else
        b = b / 8.5070591730234615865843651857942e+37;
    return b * ((a >> 31) | 1);
}

string FloatToSci(float Feed)
{//just give me gcvt, and save us all effort. My way is by far not the best way, but it works.
    //if we had doubles _none_ of this would be needed.
    //with doubles there would be enough percion as to not require remainder tracking
    //not to mention i could just use logs instead.
    //for safely translating floats into strings.
    //with doubles i could just use 
    //FloatToSci(float Feed)
    //{
    //    double a = llLog10(llFabs(Feed));
    //    integer b = llCeil(a) - 1; 
    //    if((a -= b) < 1.204119982655924780854955578898)
    //    {//ensures the output is greater then 16
    //        --b;
    //        ++a;
    //    }
    //    string c = (string)llPow(10,a);
    //    if(b)
    //    {
    //        c+="E";
    //        if(b>0)
    //            c+="+";
    //        c+=(string)b;
    //    }
    //    if(Feed < 0)
    //        c = "-" + c;
    //    return c;
    //}
    //Alot less pain and fewer iterations.?
    integer c;
    integer d;
    integer e = 24;

    float AFeed = llFabs(Feed);
    integer Exp;
    string Mant = (string)AFeed;

    if(AFeed<16.0) //unsure about optical
    {
        if(AFeed!=(float)Mant) //optical fails
        {
            //d is the actual exponant
            //e is the target exponant
            //c is the integer portion
            //m is the remainder portion
            //as the loop runs the bitshift is moved towards the target, and the target is adjusted.
            //the goal is to have the result mostly in c
            //then that integer is converted to a string.
            //with the use of c & m we the math is done with 62 bits.
            //if we had the long long datatype this wouldn't be needed.
            integer m;
            c=(integer)((AFeed / llPow(2,d = llFloor(llLog(AFeed) / 0.69314718055994530941723212145818))) * 16777216.0);

            //time to apply the rest of the bit shift AFeed<1
            while(e>d)
            {
                //overflow test
                if(c >= 0x19999999) //(0x80000000 / 5) // it's >= because the float could overflow the int.
                {//overflow predicted, devide by 2
                    m = (m << 1) + (c & 1);
                    c = c >> 1;
                    ++d;
                }
                else
                {//won't overflow, multiply in 5
                    c = c * 5 + m%5;
                    m /= 5;
                    --e;
                }
            }
            //time to apply the rest of the bit shift AFeed>1
            //this might overflow m, though that could only happen if AFeed<16.0 is bypassed.
            while(e<d)
            {
                if(c & 0xC0000000) //c >= 0x40000000 //(0x80000000 / 2)
                {//overflow predicted, devide by 5
                    m = m * 5 + c%5;
                    c /= 5;
                    ++e;
                }
                else
                {//won't overflow, multiply in 2
                    c = (c << 1) + (m & 1);
                    m = m >> 1;
                    --d;
                }
            }
            Exp = e - 24;
            d = llStringLength(Mant = (string)c);
            jump go;
        }
    }
    d = llStringLength(Mant);
    if(1 + c = llSubStringIndex(Mant,"."))
    {//remove the decimal, this makes everything else easier.
        Exp -= (d -= 1) - c;
        Mant = llDeleteSubString(Mant,c,c);
    }
    @go;
    if(d>9 || llGetSubString(Mant,-1,-1)=="0")
    {
        if((e = d)>9)
            c = 9;
        else
            c = d - 2;//already checked the last digit.
        while(llGetSubString(Mant,c,c)=="0" && c)
                --c;
        Exp += e - (d = (c + 1)) + (c!=-1)*c;
        Mant = llDeleteSubString(Mant, c+1, e);
    }
    else if(d)
        Exp += d - 1;
    Mant = llInsertString(Mant, 1, llGetSubString("0.0", d>0, 1 + (d<2)));
    if(Exp)
    {
        if(Exp>0)
            Mant += "E+" + (string)Exp;
        else
            Mant += "E" + (string)Exp;
    }
    if(Feed<0)
        return "-" + Mant;
    return Mant;
}

default
{
    state_entry()
    {
        integer a = 1001;
        float b;
        string c;
        integer d;
        llOwnerSay("Starting Tests");
        llResetTime();
        while(--a)
        {
            if(b != (float)FloatToSci(b = llFrand(1.1754943508222875079687365372222e-37)))
                llOwnerSay((string)fui(b));
        }
        llOwnerSay("Test 1: " + (string)llGetTime());
        a = 1001;
        llResetTime();
        while(--a)
        {
            if(b != (float)FloatToSci(b = llFrand(3.402823466E+38)))
                llOwnerSay((string)fui(b));
        }
        llOwnerSay("Test 2: " + (string)llGetTime());
        a = 1001;
        llResetTime();
        while(--a)
        {
            if(b != iuf(fui(b = llFrand(1.1754943508222875079687365372222e-37))))
                llOwnerSay(FloatToSci(b));
        }
        llOwnerSay("Test 3: " + (string)llGetTime());
        a = 1001;
        llResetTime();
        while(--a)
        {
            d = (llFloor(llFrand(255.0)) << 23) | (integer)llFrand(16777216.0);
            if(d != fui(iuf(d)))
                llOwnerSay((string)d);
        }
        llOwnerSay("Test 4: " + (string)llGetTime());
    }
}

Hi Introvert, sorry for the delayed reply, I only just got pointed to this thread from a comment Blind added to my blog.

We have mapped the LSL API, all llFoo() methods can be called from scripts compiled to CLR assemblies.

We have found that everything from empty scripts to every script running on Abbotts and Themiskyra runs faster on mono.

API time dominates the script run-time of current scripts party because interpreted LSL runs so slowly. Heavy computation is currently painful, so very few scripts do much computation, but Mono is still faster when running current scripts regardless.

Mono should only break scripts that erroneously rely on the current execution speed. For example, if you send a message then start listening for a reply you might get away with it with interpreted LSL, but not when the script runs on the Mono VM. Of course you should start listening before sending the message to avoid this race condition, but I bet there are existing scripts that don't listen first.

Although interpreted and Mono scripts will run side by side while we iron out the issues, eventually we will recompile all existing LSL scripts from source code to CLR assemblies. Running scripts will be run to the end of their current event handler by the LSL interpreter and then have their globals and current state copied over to a new Mono script. No state will be lost and the script will continue to run as if it was still being interpreted.

Cheers,

Babbage.

Oops, sorry about that Andrew (I'd managed to break loops with the first version I put up).

On the plus side I had the problem fixed in a couple of hours and then the Mono simulator ran from December 23rd to today, which I was pretty happy about.

Cheers,

Babbage.

Well it appears as if I understood that exactly incorrectly. Please accept my apologies and my congratulations on what sounds like a significant porting and adaptaion job. I also thank you for correcting the incorrect conclusions I'd jumped to.

Here is babbages blog, I think: http://secondlife.blogs.com/babbage/

Hmmm... Mono CIL code is performed directly by the processor? NOPE. at best it is JITC and then cached, but still needs significant time to do the JITC. In a realtime application like RL, I don't know that thats much better.

Your comments regarding the uselessness of stats without a background of the environment is well taken.

From what I understand of the script environment, Linden have had to produce an architecture that can allow the scripts to run quickly in an uncompromising environment alongside many MANY other scripts at the same time. To do this in native code and threading is going to take some serious hardware if you think it aint going to bog down in swapfile.

I am all for being able to develop scripts in a nicer environment, the idea that cut/paste from your favourite editor is an acceptable solution is as far as I am concerned laughable. Too much cogito interruptus for me. Trying to code scripts in some other language may be a mistake as the overhead will probably (definitely?) be significant.

I can see that the running environment of LSL scripts is taylored to low overhead low latency, with none of this cycle munching OO business to get in the way of the refresh rate.

While I lament that the structure of the language is painful, I understand why its there. (and applaude the efforts of Linden in producing a system of this size that does not grind to a halt!)

Finally, a few questions.

Q1: Where do the scripts run? on the sim server or on the client pc? my guess would be up on Lindens HEAVY IRON.

Q2: Is it at all possible to find out where an agent has touched a prim within the prim? you can use llGetPos, but that returns the position of the avatar not the touch. Getting at where it was touched would enable SOO much in the way of constructor scripts.

Thanks, FG.

Ok A few questions to the Lindens:

The LSL Language wont change?

Will this increase the 16K per script Memory limit?

Becky