Proposal/Discussion: Using Lua for scripting, instead of LSL

First and foremost, I apologise for the long post, but I try to state a sufficient amount of information for people to see what I'm getting at, and hopefully, convince any Linden devs that read this.

I believe that Linden Labs should implement Lua as the game's scripting language. Lua is a small and basic scripting language that is easy to learn and use but used in professional game development. Lua's virtual machine is very small and the entire package adds a minimal footprint (less than 300kb) and is lightening fast. This would, if propery implemented, allow a high number of scripts to run on a sim without introducing extreme lag from the sheer amount of running scripts.

Lua is tried and true, used in various games that are/were fairly well known such as Baulders Gate, MDK2, Escape from Monkey Island, Grim Fandango, Neverwinter Nights, Impossible Creatures, Blitzkrieg, Homeworld 2, and Far Cry, to name a few.

Rationale
Lua is lightweight, easy to learn, and although it is not Object Oriented on its own; it provides the facilities to fake it quite well using its built in "tables". Linden Labs would be able to implement Lua quite easily if they are using a language which supports C bindings. This is a Good Thing because it removes a great deal of workload from Linden Labs since they do not have to worry about parsers, syntax, etc; allowing them to focus on the meat of the scripting system such as access to in-game material (i.e. moving objects, using physics).

Another feature that many people (at least me and a few others) want, is Error Handling. Lua provides this which means Linden Labs would not have to implement it. Initial integration of Lua would be a bit of an inconvinience to the development team, because it would have to be hooked up in to the system and some-odd lines of code would need to be added depending on SL's design. After initial integration, things would be a piece of cake for the team to add new commands.

The Require command which is used to include other scripts in Lua could be rigged up to search the inventory of the object the script was in, permitting the use of script libraries and whatnot.

The way states appear would be different since Lua does not inherently support them. See the examples section for more information.

What to do with existing scripts?
All existing scripts could be turned in to a special LSL Script object and new scripts would be Lua Script objects. You could create either using the menus (instead of Create Script it would be Create Script->Lua and Create Script->LSL). A deprecation warning should be issued on creation of new LSL scripts. This allows legacy code to continue running using the current LSL virtual machine, but LSL would not be developed further.

Script Examples
Stateless hello world in LSL:

default
{
    state_entry()
    {
        // Greet the nearby peoples
        llSay(0, "Hello avatar!");
    }
}

Stateless hello world in Lua:

function OnEntry
    -- Greet the nearby peoples
    LL.Say(0, "Hello avatar!")
end

State controlled door in LSL:

default
{
    touch_start(integer num_detected)
    {
        // Perform operations to open the door here
        state open;
    }
}
state open
{
    touch_start(integer num_detected)
    {
        // Perform operations to close the door here
        state default;
    }
}

State controlled door in Lua:

states = {
    "default" = {
        function OnTouched(number_of_touchers)
            -- Perform operations to open door here
            state = "open"
        end
    },
    "open" = {
        function OnTouched(number_of_touchers)
            -- Perform operations to close door here
            state = "default"
        end
    }
}
state  = "default"

function OnTouched(number_of_touchers)
    states[state].OnTouched(number_of_touchers);
end

Final Words
States done in Lua are a bit different, but with Lua you get control and choice; with LSL you get what you get. Any input would be greatly appreciated; I will add this to the proposition list if there is some positive feedback and I will try to answer any questions that arise about this topic (ones that I can answer that is).

You may wish to check out the Lua homepage if you are not already familiar with the language. There you may find an online book named Programming in Lua if you want more in depth coding knowledge about the language.

You mean like OSMP does?
http://metaverse.sf.net
Visit our forums and stuff.

Hmm...

I'm starting to think posting this was a waste of a perfectly good half hour of writing and revising.

You should ask Cory about this at the townhall. The rumour is that the Lindens are looking at allowing us to slot in our own languages to support things like this.

Ok guess it's time to tell some of what i know. I was doing some research into LSL bytecode (mostly finished, have enough figured out that i could build a decompiler).

I sent my research to Cory, and we chatted about LSL. One of the things he mentioned was that all my work was in vain because they were moving over to Mono (no time frame was given for this).

What this means: LL is rewriting the bytecode, most likely making it a lower level language. What I think they are going to do is add a JIT compiler, and an optimized one at that .

What this change means to the language: Not much will change. Syntax will likely stay the same but we will likely get an better object model. Example: float a = llGetVel().x;

babbage just made a blog entry about it.
http://secondlife.blogs.com/babbage/

Wow, what a speed bump!

Also, a compiler sure would make the production of an external, validating editor a real possibility...

/esc

Personally, I would like to see LL rip the lid off the byte code so that the SL community can implement virtually any language for scripting.

- Ace

Fuck me gently with a chainsaw!
There is a god!
His name is Cory
I would bloody murder for a better object model... and more speed, too
Well, not that I really need more speed as much as I need more memory.
LSL's tiny 16K have provided me *ahem* hours of fun coding fancy distributed databases and stuff :\
From Babbage's blog:
"The initial figures look good, with Mono executing LSL scripts between 300 and 500 times faster than the current LSL interpreter."

The trouble with LSL right now it's not optimized in any form at all. The LSL bytecode is so similar to the language that there is little to gain by writing your own language. It would be easier to just write a compiler would make LSL code. The bytecode for LSL is high level. There are no low level commands the commands are interpreted to, it's one to one language (with the exception of ++, --, +=, -=, /=, %=, *=, which are expanded out). A bytecode decompiler can be easily acheived; one that will produce the original code with only the loss of variable, function and state names. I don't have much interest at present in writing one but maybe in the future.

The only interesting thing you could do with the bytecode is totaly abuse the function and state machines. By changing the routing tables used by functions and states it would be possible to have multiple states and functions point to the same bytecode or to jump from one part of the bytecode to another, totaly ignoring function and state boundries. Not that this would be useful.

Pass by refrence can't be made to work without adding new bytecode commands.

LSL feature suggestion I have sent to Kelly:
Inline if statements ((test)?(true):(false)). I have provided bytecode examples of how to do this, no changes to the language are needed, just a few to the compiler (things get a bit complicated with pushing types into lists, but very mangeable).

basic optimizations: don't push into the stack what isn't needed. if the return value of c++ isn't needed compile it as ++c.

new bytecode for getting companants of vectors and rotations in the stack. <bytecode><byte> where low order 4 bits is the compant wanted, high order 4 bits are the type or number of dwords to pop (this is the better object modal i was talking about).

Things i've been meaning to write up:
State Variables & functions - this would require multiple changes to the bytecode (new commands) and changes to the state machine.

Edit: This was originally meant to be under eggy's post

Well, my proposal wasn't entirely for speed and memory purposes. Lua is small, and low overhead yes. But you can do some really odd stuff in Lua that you can't do in LSL. One example is... bum ba ba... Object Orientation! Yea its not really part of the language, but you can rig it up using it's equivalent of arrays pretty easy. Then you could write code like this...

-- Global variables
....
GiantTrapDoor = ListenerDoor:new(30) -- Door listens on channel 30
....
    function OnTouch(toucher_count)
        -- Open the trap door
        GiantTrapDoor:Open()
    end
....

Where ListenerDoor would be made to send the message "Open" and "Close" to the channel given in the constructor. Personally I believe this looks cleaner and states what you're doing in code better than

....
    touch_start(integer num_detected)
    {
        // Open the trap door
        lllSay(30, "Open");
    }
....

And (not demonstrated above) we finally get error handling! That alone kills the possibility that it will be implemented!

I think the world would be in shock if this ever got implmemented, but you can actually bend parts of the language to make it do things it doesn't actively support. Not to mention it's useful in other realms than SL such as if you ever go to work on a Far Cry mod

That's just what they will be doing. Anything that can compile to Mono bytecodes will be able to stand-in for LSL.

By the way, in today's town hall Cory mentioned the timeframe as 2.0, which seems to be slated for Q1 of 2006.

Not sure how they plan to move to Mono, a lot of the form library is still unfinished. They could use Axiom, which would be sweet, since we would actually get the FPS we're supposed to be getting with brick graphics (it takes a $300 video card to get 12fps and the graphics are cheap primitives, no more than like 8,000 polygons on the screen most of which are the avatars)

i'm lost, what does a 300$ graphics card have to do with this?

I'm pretty sure that I'm speaking for at least a few people here, but it was a pain just to learn LSL (for me at least). If LSL is just 'thrown out' then that means the best scripters in SL who don't know other languages (I know LSL pretty well but not anything other than that and HTML) would e forced to stop what they are doing, and start from square one! In that jet lag, nothing is being accomplished, project that were previously in the middle of being developed would be end up being crap, and possibly, if LL decides to have both languages, NOTHING will change. People would just decide to use LSL because that is what they are most familiar with. It is just like a new restaurant from France, people dont get the escargot or any other French foods they've never heard of because they don't know if they will like it. I'm not usually one to rant or complain, but this is something that drastically affects me. One of the only things I do is script and spen money on my friend's blackjack! So, before LL decides to implement a new script language, they need to decide if it is really worth it since many people mihgt not know it.

A Java API would be nice...

...y'know, would automatically be a mature cross-platform bytecode compiler that leverages a large number of external editors and would provide an easy path for LL to offer the opportunity to "upload script"...

...just a thought.

/esc

(btw---- NOT JavaScript, God forbid, but... Java)

Thats because you may have never used anything like Lua, Python, Ruby, Perl, etc. If you come from a background of using real scripting languages it would be a piece of cake to learn until you start pulling your hair out from lack of vitals like error handling; not even the barbaric way C pulls it off.

It's one of those nasty circular dependencies; not many people will use Linux because there are virtually no games available for it (aside from a few ports) and nobody will make games for it because nobody uses Linux; if you get my point.

Exactly my point, not every person that knows LSL knew another programming languge before it. Sure, some of them did and it helped them, but what about the perhaps majority of people that didn't know a language. Would they need tu learn Lua, LSL, or both at the same time? It might be more beneficial to many if LL tries to speed up LSL, give scripts more memory, and require less resources while keeping the commands either the same or close. Its not like we get new functions all the time and they can't keep up with them.

to beat a dead horse:

State controlled door in LSL:

default
{
    touch_start(integer num_detected)
    {
        // Perform operations to open the door here
        state open;
    }
}
state open
{
    touch_start(integer num_detected)
    {
        // Perform operations to close the door here
        state default;
    }
}

State controlled door in Lua:

states = {
    "default" = {
        function OnTouched(number_of_touchers)
            -- Perform operations to open door here
            state = "open"
        end
    },
    "open" = {
        function OnTouched(number_of_touchers)
            -- Perform operations to close door here
            state = "default"
        end
    }
}
state  = "default"

function OnTouched(number_of_touchers)
    states[state].OnTouched(number_of_touchers);
end

the Lua code above would compile like:

string stat;

state_default_touch(integer num_detected)
{
    // Perform operations to open door here
    stat = "open";
}

state_open_touch(integer num_detected)
{
    // Perform operations to close door here
    stat = "default";
}

default
{
    state_entry()
    {
        stat = "default";
    }
    touch_start(integer num_detected)
    {
        if(stat == "default")
        {
            state_default_touch(integer num_detected)
        }
        else if(stat == "open")
        {
            state_open_touch(integer num_detected)
        }
    }
}

why?
here is how the state machine basicly works:
the sim generates the event, checks if any state in the script can handle the event, then it checks if the active state can handle it, finaly it runs the bytecode for that event. (i would say more on the mechanics as to how this is done but i that would be devolging trade secrets, and exact details aren't important).

The reasons the Lua code above doesn't compile nicely is Lua doesn't do state handling. Sure you can build a state handler, but it won't be as efficient. Lua is more basic then LSL in this way. It doesn't serve anyone to implement a language that can't utilize the full feature set of the architecture; especialy when you have a language that does.

Any language that is ported to SL will have be tainted by the unique features of the virtual machine. The language will have to modified to support states.

With the upcoming integration of mono underway, it's (almost) totally useless to ask for changes to the scripting engine in Second Life. Why?

Because once the thing run mono, all languages can be supported at once. You want Lua? here it is. Java? C#? Python? LISP? Maybe even Cobol?

Any language can be compiled to .net bytecode and then used as-is by Second Life.

[edit. typo]

Genuine question:... can that bytecode then be run on Mac/Linux/*nix?

/esc

I would like a triple scoop of Mono Basic or perhaps some Mono Pascal.

Wow, we will be able to use all those languages in SL when mono comes out?

Yes, that is the mono project. Microsoft provides it's implementation for Windows (the .Net Framework) and mono does the same thing for Linux/Mac/BSD.

But remember that with Second Life, mono is only used server-side. It still won't run scripts on the client.



Yep, that's one of the main idea behind .net/mono: language independance. Upload a compiled assembly ($L10) and put it in a prim. The server will never know what language or compiler you used.

That's great! One more reason to learn java