Advanced Scripts, CIL/Mono, Optimization - Proposal for "No State" flag

We need feedback on a "No State" flag and CIL compilation that has been talked about in the mail-list and other discussions. The text below is taken out of context from earlier discussion. I'll put the proposal first and append the details.

###

Subject: The proposal itself

I have a proposal that would make this implementation forward and backward compatible.

First, provide a new Edit->Preferences options to select the mode of optimization for the compilation. I suggest to have a check box that simple states "Advanced Scripter Mode". Without this checkbox selected, the LSL script compiles only to LSO and works as you now know it. Anotherwords, by default you will not noticed anything different and can expect your scripts to work as they have always worked.

With the "Advanced Scripter Mode" checked (turned-on), you will be expected to know LSL might perform several optimizations on the script, and you will need to know what triggers those optimization steps, as it may affect how your script works. With this mode set, your script may work very differently; however, the optimizations are for greater performance. With this mode set, you may have to write more code in order to handle events that are not automated for you like when this mode is off.

Second, when the "Advanced Scripter Mode" is on, if there is no state_entry()/state_exit() defined in a script and the script is compiled, the script then has the "No State" flag set. This means the stack is not always preserved for any instance of the script. You can expect the stack not to be preserved across sim boundaries or when an object with the script is rezzed in and out of inventory.

Third, a script with the "No State" flag set may be compiled to CIL instead of LSO. Note this only says "may" and not "shall."

Any further acknowledgments or concerns are welcome. I want to hear it! =)

###

Subject: script state vs. machine state

It's typically known as a machine state. There is no standard or specific structural definition for this kind of state. A saved machine state would be like a map coordinate with all necessary data (and camping gear) for a machine to return to that location of program execution. As a script crosses a sim boundary, these machine states are saved, transferred, and then restored.

When I referred to a state in the script syntax is when I talked about the "default" state definition, which encapsulates the definition of events written in LSL like state_entry(), state_exit(), on_rez(), touch_start(), etc. The "default" state definition being referred to there is not the same as the saved machine state.

An active machine state, being referred to here in LSO, is that 16KB chunk of memory that does contain areas for code, stack, and text. The code and text areas usually do not change as the program executes. The stack area changes, and that is where data (like your reference to global variables and etc) is stored. As the machine state changes over time, it is considered a computation.

For example if two sims already have the same script asset loaded in memory, there really is no need to transfer the code and text portions of the machine state between the two as it crosses over. That already would save some bandwidth, and it can be done internally without any extra flags being set by the user.

That leaves the stack. It has all that information spoken about previously in LSL syntax: global variables, permissions, and the ("integer that defines which 'state' the script is in" [sic]) scripts global switch statement (ala "state newState ;" syntax).

If the "No State" flag was set somehow by the user, the stack would not be saved. This then gives the possibility that none of that 16KB chunk of memory needs to be transferred as the script crosses sims.

Remember, it is just a flag that can be turned on and off, it would not be the new default for every script.

Even though I proposed a way to automatically set the flag with the lack of the state_entry()/state_exit(), it could also be set or unset with a new LLSomething() command. However, a new LLSomething() command would mean the need to keep compatibility with such command, and that may be harder in the long run. With the lack of the state_entry()/state_exit() in my proposal, it could also trigger the automatic compilation of an LSL script into CIL as well as the ability to not save the stack.

With the "No State" flag set (being the stack not being saved), the script would basically appear to be reset each time it each crosses sims. If you have a script as simple as this example below, why not? Why should it have to take 16KB chunk of memory?

default
{
touch_start()
{
llSay( "You touched me." ) ;
}
}

###

Subject: sim memory and the 16KB chunk

Currently under a LSO bytecode, each vendor script has its own 16KB of memory. If you have a wall full of these vendores, that can easily get over 100 vendors scripts. At the 100 vendors level, you can multiply that 16KB memory chunk by 100. That's over a megabyte.

If many of those in-place vendors use the exact same script asset and they each have the "no state" flagged somehow, there would be no reason to grab a seperate static chunk of 16KB of memory for each vendor. The CIL compiled (instead of the LSO compiled) scripts are able the share the same execution space and the stacks can be made dynamically as needed for each one. If none of them are active, they essentially take up no memory beyond the single execution space (think of it as the bytecode itself in memory). In this way even if you have a mall full of 1000 vendors and each share the same script asset, the simulator will use a lot less memory with the CIL compiled scripts than the 100 LSO compiled vendor script.

If you own a sim, I'm sure you'll want that extra memory available on your sim.

###

Subject: CIL, sim boundaries, and no state_entry()/state_exit() in LSL

It is ideally the difference between just a script language and a lower-level program language that is not taken into account in view. Right now you have general access to the script language. It automates many features for you. It automatically manages memory, stack, events, errors, states, and even propagation of such over the network. If you start at the lower-level, you would need to write more code to handle what was automatically done for you. However, with much lower-level control, you could make a very efficient program that works much faster than the provided automated tasks.

The argument here is shown in their Lang.Net video with the charted pain vs creativity. It would definitely more of a pain to have to write in a lower-level context. It would take the fun out of the script language.

There is now the hybrid model on the burner. With the video, we have the demonstration of the LSL script level and the Mono level. Noted, they did say there was a LSL-CIL compiler. Once the hybrid model works, one would ask why be limited to only LSL script. Noted, there is mention of the ability to use other languages like C# & VB. The ability to program SL at a much lower-level is possible.

It's even practical right now if CIL compiled LSL scripts are limited by default to not cross sim-boundaries if the script is in a run-state. How often does a sploder or tringo game cross a sim boundary? I've seen people only rez them from inventory. Many scripted objects that never cross sim boundaries can benefit now.

What if a script that has no state_entry()/state_exit() defined that the script will compile to CIL rather than LSO? In this case, you could use on_rez() to set some primitive properties or to start particles. If this stateless program crosses a sim-boundary, only the primitive properties are kept. The change() event could detect the crossover; however, globals would lose all updated information. For example:

integer sim_init = 0 ;

newSim() { same_sim = 1 ; doSomething() ; }

default
{
on_rez( integer n ) { setPrimitiveProperties() ; newSim() ; }
changed( integer n ) { if( sim_init == 0 ) newSim() ; }
}

With a "no state" script (being no states saved across sims), the sim_init will always return to value 0 when the object crosses sims.

I can easily right a timer event to that script, so that if the object exists within a sim for a certain duration that it will further initialize. That way, if, for example a jet plane load with avatar each with scripted objects, the script will know not to bother with initialization if the object doesn't stay stationary within the sim.

###

I'm glad you have taken the time to read this proposal and details. I look forward to your feedback.

Cory Linden suggested the "No State" flag in a discussion about serialization techniques. He mentioned there were some issues related to deployment with Mono. The video shows the hacks that do get features operable.

My proposal for the "Advanced Scripter Mode" is to help further deployment while also to enable the "No State" flag. As you see in the video, you select Mono if you went to compile to Mono. My proposal would not allow anybody to select Mono at this time. Instead, let the compiler automatically determine when it will compile to CIL instead of LSO.

With this proposal, it is easier to enable Mono on all the servers. Instead of a blanket switch over to Mono and the need to deal with all compatibity issues, just allow a select amount of scripts to compile to Mono. These conditions are laid out in this proposal. Basically, just the scripts that pretty much stay in one sim and don't cross boundaries often are chosen.

An update from another thread:



Please vote for prop#2286. One step closer...