Same memory, better tools.

I apologize in advanced for the lengthiness of this post, but I feel the subject required adequate exploration, and description such that even the user unfamiliar with the subject may learn the issue involved. I thank you for your time!

I propose that that LL implement better tools to allow our scripts to utilize the memory they do have to better effect.

Currently, the only way to perform intra-state communication, other than by passing values, (not always possible in an event-driven language such as this) is to declare global variables. The problem with is is a big one, with a few facets.

Global variables are semi-permanent. The memory they consume will not become free until the script is re-written. If you only need a variable for one particular state, the memory is wasted while the script is in the others states.

There is a wholistic solution to this:


Implementing all three would be a nearly complete solution. Hopefully the warning and non-obvious enabler for advanced mode would prevent normal users from being uneccesarily frustrated by trying to use Dynamic Memory Allocation.

DMA (Dynamic Memory Allocation) would allow events and states to communicate between each other on an as-needed basis, with minimal overhead. By creating global pointer variables which are relatively small, global variables would only occupy their full memory footprint when actually in use.

For those unfamiliar with the terminology, DMA involves the use of a special "pointer" type variable that stores the address (just what it sounds like, the reference to a location) of a place in memory. Then, using other functions, you are able to make that address actually contain something, nothing, or even enter a state of near-nonexistence. (Meaning, though the memory spot exists, it stores nothing, not even information as to what type of information it should be storing. Normally variable always remember what type they are, even if they are set to zero, NULL_KEY, ZERO_ROTATION, etc.)

It is also important to understand that the language SL is written in natively supports these features. Developing the tools would be less "creating them from scratch" and more "bringing them to the surface."

In summary, increasing total memory allocation for each script is a political issue that is difficult to find a comprimise on, and one which the Lindens are unlikely to budge on in the near future. HOWEVER, adding the tools to allow us to more efficiently manage the memory we do have would bring much of the same benefit of increased memory allowance, without the increased demand on finite resources. Presumably, the only cost involved would be in development time for the tools.

Questions? Comments? Anyone? I'm sure I glossed over or missed alot that I should have included. I will edit as I think of things/things are suggested!

I'm hoping this will come as a part of the transition to another scripting language (mono is it called?)

I suppose it depends on your application, but some variables can be used by states for different purposes.

I've also been toying with a way to handle large scripted states by having several scripts in an object, and using link-messages to trigger the next state. Basically each script is given a number it listens for in it's link messages, along with the name of the state in the string part.

When a state is exited, the script in control sends a link message identifying the state to change to, and promptly stops working (ie it goes dormant).
An appropriate script in the object then picks up the state-change and takes control.

The disadvantage is that each script will need to have a copy of any commonly used functions, unless you can cope with handling these via link-message as well.

Mono isn't a scripting language, its an entirly new VM.
All LSL scripts will be converted or recompiled to run in the new VM.
While LSL will still be supported; there are hopes in the scripting community that LL will allow for the use of any CLR compatible language (that is a language with a compiler that outputs CLR bytecode).

I'm an old (by programming standards) Pascal/C/C++ developer. I've been leak-free for many years now, but it took C++ and a few years of unlearning and relearning to get that way. I am proud of my resource management, and abhor garbage-collectors.

BUT - how many LSL scripters have had the ooportunites I've had in this regard. Many scripters are vets, or firemen, or race-car drivers in RL - To give them unmanaged dynamic allocation would probably be grossly unfair.

As far as state variables are concerned - I couldn't agree with you more. Bring 'em on, please.

A thought: Speaking for myself only, the only times I've had memory problems with scripts were when I needed to store a lot of data. My code per script usually ends up being quite small as I find myself using multiple scripts when things get too complex (almost as much due to the LSL editor, than to any form of modularisation ). I would love to see better bulk-data handling tools where much of the storage could be handled in "data files" in my object's inventory, rather than within the 16K address space of the script.

Ben, you bring up a good point in reference to memory leaks. Myself, I was determined from the outset to harness the power of DMA. With great power, unforntunately, comes great responsibility. (Sorry! I had to say it. It was asking to be said. )

Perhaps there could be a toggle for advanced/basic scripting mode? Hmm... I'll edit my original post to reflect this, but this definitely bears thought.

I think that the potential outweighs the risk, personally, as LL could implement a means to hide this advanced feature from the average user.

It is precisely while coding scripts that are required to store large amounts of data that I run into the problem of memory efficiency. I too am forced to resort to multiple scripts which communicate amongst each other, but this isn't nearly as elegant as it could be.

LL seems greatly troubled by scripts using any more memory than they already do, however, when we are forced to split our scripts up, we are consuming two to sometimes 12 times more memory than the 16k alloted per script.

At the present rate of development (no insult intended) I suspect that using CSL compilers to facilitate the use of other languages will come quite a while AFTER mono is implemented. A short-term fix would be muchly appreciated.

In reference to the memory leakage, each script is only allowed it's 16k, which is pre-proportioned at run-time, yes? The worts that happens is you fill up your little 16k. Overall sim performance shouldn't be affected by it anymore than it would be affected by a leak-free script using all 16k.

As long as LL implemented a proper virtual memory interface, then damage would be minimalized to the scripts with the leaks. Free competition would hopefully sort out the scripters who could and could not manage their memory effectively.

Additionally, if some were to cry how unfair this would be, I would be happy, as I'm sure other existing tutors would be, to help teach proper memory management. As a side note, though, memory management not so much a separate skill as it is a facet of "knowing how to write code properly" Like anything else, if you don't know how to do it properly, then there is only one person to blame, yes?

At any rate, I would be happy to teach others memory management just to reap the benefits of having DMA implemented.

Thanks for all the intelligent comments so far! Keep 'em coming! I would love to make this a feature request on the voting tool once we get it nice and fleshed out.

I'll update the original post up top with the addendums as soon as I get a chance.