How to optimize code and when not to.

That doesn't explain how performing that assignment inside the function rather than before it actually saves anything.

I had been doing this for awhile, variable recycling, but I've stopped it in SL because of some very very painful bugs I created when I did it.

I've basically given up optimizing for memory (though, obviously, not for performance) and I work hard to develop message based architectures.

One advantage to message based architectures is that I now have function pointers, and therefore a type of inheritance / interface / classes, etc.

edit: I have a personal limit of 10K .. anything going over (data structures included) is in the 'red zone'.

I may be wrong here - but I'm about 97.2% confident that stripping whitespace and comments is completely unneccesary in SL.

1. Whitespace and comments do not cause an increase in size OR a decrease in speed of the compiled code.
   
2. The permission system prevents others from seeing your code if so desired, so obfuscation isn't needed.
   

Maybe you can simplify your workflow just a touch.

Hehe, I'm 99.9% sure comments don't add to the size of the compiled code and that the permissions system prevents others from seeing my code. But since LSL doesn't support /* ... */ style comments, I strip my comments since I've taken up using my multi-line comments again. And I can't get a whole lot simpler in my process anyway. I code in "ESL", hit the compile button and preseto, my esl file gets run through the C++ pre-processor (which is what actually strips my comments I've fount out), run through the white space stripper (which makes a lot less blank lines to go through improving readability a bit), giving me a LSL file that gets run through lslint to find out what mistakes I've made that won't compile in game.
Oh and it copies the entire script to my clipboard automatically for me too.

I wrote the stripper, because when you have #if blocks and such, you end up with huge amounts of whitespace (the C preprocessor, keeps the lines but not the data when it processes the file). You are unlikely to hit the code max in the editor (64k i think), but it's a pain to have to scroll past 20 or 30 white empty lines; wouldn't it make you wish you had some program that would strip them off?

As to readability, i don't strip comments when i use ESL (consequently i don't use C multi-line comments).

What it does... it leaves a single whitespace, where there was 1 or 4+ lines of whitespace, and leaves two where there were 2 or 3 lines of whitespace, it really only collapses the most egregious abuses of whitespace. If you want obfuscation, replace all the spaces and carriage returns with "#"

Time for some pure evil.

If you think strings or lists are pass by refrence, you are wrong*.
(* they are pass by refrence but when they are used (in an expression or as a paramater), they duplicate the list or string and push a refrence to that copy into the stack. Since the refrence doesn't point to the origional, it's best to consider it pass by value.)

Your script needs to work with big strings and lists, but they are temporary and need to be fed though many functions that are going to convert them.

list TightListParse(string in) {
    string spacer = llGetSubString(in,0,0);//saves memory
    return llParseStringKeepNulls(llDeleteSubString(in,0,0), [spacer],[]);
}

So we don't need the value of "in" after the function is called but what if we don't have enough memory?
When this function above is running it will in effect have to deal with three copies of "in" in memory by teh time it returns.

1. TightListParse's local copy
   
2. llParseStringKeepNulls copy without the first character
   
3. The result of llParseStringKeepNulls which is going to be bigger (it's a list)
   

What if we could release on of them... lets look at the facts

1. LSL is pass by value. (except function returns, which are by refrence, thank god)
   
2. Function paramaters are copied into memory in left to right order.
   
3. Function paramaters may be expressions.
   

What it means?
After we have passed "in" as the first paramater, we can do anything we want to "in" even if we hadn't deleted the first char of "in". This is because the value is copied into the stack.
So we set it to something smaller. This approch lets us parse 50% bigger stings.

list TightListParse(string in) {
    string spacer = llGetSubString(in,0,0);//saves memory
    return llParseStringKeepNulls(llDeleteSubString(in,0,0), [in=spacer],[]);
}

other ways of doing this... sure it's cheating but who cares
a + b
is in bytecode
push b,
push a,
pop 2 whatever, add them, push result

a() + b()
is in bytecode
call b() and the result is left in the stack,
call a() and the result is left in the stack,
pop 2 whatever, add them, push result (whatever is defined at compile time).

and likewise if a or b is an expression.
b is always evaluated first.
all operators work this way, comma is not an operator.

//because we know the order of operations we can do this
//and this is an example of abusing order of operations & pass by value.
list b = quickread((a="") + a);

Lets kill two birds with no ston- variable.

//Before
list quickread(string a)
{
    string c = llGetSubString(a,0,6);
    string b = llGetSubString(c,0,0);
    return llDeleteSubList(+
                        llParseStringKeepNulls(llDeleteSubString(a,0,5), [],[b,
                        llGetSubString(c,1,1),llGetSubString(c,2,2),llGetSubString(c,3,3),
                        llGetSubString(c,4,4),llGetSubString(c,5,5)]),
                        (llSubStringIndex(c,llGetSubString(c,6,6))==-1),1);
}
//After
list quickread(string a)
{
    string c = llGetSubString(a,0,6);
    return llDeleteSubList([a]+//a = llGetSubString(c,0,0) by this point
                        llParseStringKeepNulls(llDeleteSubString(a,0,5),[],[a=llGetSubString(c,0,0),
                        llGetSubString(c,1,1),llGetSubString(c,2,2),llGetSubString(c,3,3),
                        llGetSubString(c,4,4),llGetSubString(c,5,5)]),
                        (llSubStringIndex(c,llGetSubString(c,6,6))==-1),1);
}

If you haven't scoffed at any of these "optimizations" now is your chance.

How to save 1 byte

integer a;

if(a < 0)
becomes
if(a & 0x80000000)

if(a % 2)
becomes
if(a & 1)

the number doesn't have to be 2, it can be any power of two.
the & value is that power of two, minus 1.

if(a % 0x40000)
becomes
if(a & 0x3FFFF)

Huh, these are pretty obvious and not weird, obscure, or "scoffable" at all, but it's been 25 years since I've had to use them.

As i didn't really answer this question.
LL's assignment operators for everything but storing values at time of declaration of a variable result in a copy.

string a = "5"; //the value "5" that is currently off somewhere in memory, and the pointer to it that is in the stack, that pointer is directly stored as the value of a; this is the only time LL stores the original pointer. As a result of this assignment operator, the pointer is removed from the stack.
string b;
string c;
string d;

b = a;//the value of "a" is copied and a pointer to it is pushed into the stack, the memory used by "b" is released, the pointer in memory is peeked at, the string duplicated, and the pointer to that duplicate becomes the value of "b". finaly the original copy of the string is popped of the stack, by the string/key pop operator.
d = c = b; //see the bytecode below

d = c = b = a;

a;

0x60 "5" //null terminated string; push pointer to memory into stack
0x3B 0x00000000 //pop pointer from stack and store into local memory

0x60 "" //null terminated string
0x3B 0x00000004 //pop pointer from stack and store into local memory

0x60 "" //null terminated string
0x3B 0x00000008 //pop pointer from stack and store into local memory

0x60 "" //null terminated string
0x3B 0x0000000C //pop pointer from stack and store into local memory

0x51 0x00000000 //copy into the stack
0x31 0x00000004 //copy out of the stack
0x02 //pop string pointer and release memory from the stack

0x51 0x00000004 //copy into the stack
0x31 0x00000008 //copy out of the stack
0x31 0x0000000C //copy out of the stack
0x02 //pop string pointer and release memory from the stack

0x51 0x00000000 //copy into the stack
0x31 0x00000004 //copy out of the stack
0x31 0x00000008 //copy out of the stack
0x31 0x0000000C //copy out of the stack
0x02 //pop string pointer and release memory from the stack

0x51 0x00000000 //copy into the stack
0x02 //pop string pointer and release memory from the stack
//thats right, stupid null operations are actualy end up in your compiled bytecode.

If you dont' supply a default value when you declair a variable, the compiler will. It's a senseless waste not to.

When proforming the operation inside the function we are able to save a few bytes by not having to pop the value off the stack and then right back on.
As to the order the paramaters are evaluated, LL would drastricly have to change how thier code operated to screw this up, i see no advantage they would gain by doing so.

Strife -whats your take on:

a) to what length should we try to avoid using lists - I've heard people say they're quite memory hungry - but with the tradeoff they can be just what is required & their usage makes for readable code

b) use of functions to make code readable - does this increase memory usage much, noting you may be passing lists into them (stack get used no?)

c) performance impact on sim when using some of the alternative comms which may involve timers which short durations - i..e always checking/polling for updates

d) if there is a script which is a bit laggy - will this affect all other scripts in the sim in the same fashion, or would it have more of an impact on script within the same prim, object or objects of the same owner?

tks

Unless I really need those bytes, I suspect I'll be much better off with more readable code. And more readable code makes it easier to understand and find higher level optimizations.
All they'd have to do is implement compile time expressions and a peephole optimiser, and order of evaluation goes out of the window unless they explicitly insert sequence points within expressions... and every sequence point reduces the effectiveness of the optimiser.

a) Lists are very useful, and in many situations they are the best solution. In most situations the gain from using strings instead of lists is mitigated by the hassle. It depends how you are going to use the data. I only use strings when i have to.

b) Say you have a block of code that you use in 4 places in your code, (same code, but in for places in your script), and that block calls 4 functions, by making the block a function you reduce the number of functions in bytecode from 16 to 8. Of course you need to be conscious as to what exactly that function is parsing. If it's a huge list or string, you may run out of memory because of the local copy (i talk about wiping one from memory a couple posts up in the thread). Your code will be a bit slower because of the function call. When optimizing you code, there is almost always a cost.

c) I haven't done a full study on sim impact (that I remember). The last time i used llKey2Name for a transport was... 18 months ago. I cringe when i read some of my code written then.

d) If you write a laggy script it will impact other scripts. The impact will be hard to notice (in most cases). No one rain drop thinks he/she is responsible for the flood. If your script manipulates physics or other prims etc, you may find that the lag will be noticeable. As to how the script scheduler works exactly i do not know. Maybe some of the other scripters will chime in.

would you be willing to give lessons on load balancing? how much of the processing do I put in each script?

1. All depends where you are in the development process. And your right, you should look for the big ones first.
2. And thats why we have parentheses. Your compilers trash if it ignores parentheses. Your compiler is trash is it comes up with unpredictable results. Users don't like unpredictable results. The exception is floating point operations which if you have a good compiler will let you configure how predictable the results will be.

This just hit me while trying to cut a few more bytes out of a 14k script (thats 14k of bytecode, and this script is already pretty tight).

Instead of useing ZERO_VECTOR or ZERO_ROTATION or even a global variable storing them.
Use ((rotation)""
and ((vector)""
they each cost 4 bytes
Which is less then it takes to push a value into the stack.

This is the point that using ESL rocks. Simple way to shave a few bytes.

//ESL
#define ZERO_VECTOR ((vector)"")
#define ZERO_ROTATION ((rotation)"")

you're confusing order of evaluation with order of execution.

Parentheses effect order of evaluation, in the mathematical sense. There's no reason they need to effect order of execution. For example, changing the order of execution of function arguments can save re-pushing intermediate values. Or:


a + b(y) + c(x) + b(a).

This can be implemented, with the same order-of-evaluation results, as:

push a
push y
call b
push x
call c
push a
call b
add
add
add

Or:

push a
dup
call b
push y
call b
push x
call c
add
add
add

These are mathematically identical, they have the same "order of evaluation" in the mathematical sense, but if a dup is cheaper than a push the second one is faster. Now... what happens if b has side-effects?

Compilers do even stranger things than this.

1. LSL currently does not support a dup bytecode. Though i'm sure CLI does (is that the right acronym? i can never remember)
2. I totaly agree, a + b(y) + c(x) + b(a) is ambiguous with reguards to order of operation, but (((a + b(y)) + c(x)) + b(a)) isn't. If the compiler does the above optimization for my example i'd be annoyed.
3. LSL would evaluate a + b(y) + c(x) + b(a) like...(see below); it would not nessesarily result in the same answer as your code. LSL evaluates equations right to left, not left to right. For LL to change this would require changing the yacc file; resulting in horrible ongoing drama when scripts were recompiled (this script worked just fine before i recompiled it).

push a
call b
push x
call c
add
push y
call b
add
push a
add

There are very few programming languages in which parentheses do more than control the order of evaluation in the mathematical sense.

(a+b)*c means "multiply the result of adding a to b by c". It doesn't mean "execute this operation, execute that operation". What you're talking about is called a "sequence point". The C programming language is a fairly typical language where this kind of thing is concerned, and it defines semicolons and block boundaries are sequence points. Making every operation a sequence point would severly limit the possible optimizations.

Consider (a++ + a++ + a++).

A naive translation of this requires 5 adds, three fetches, three stores, three pushes.

In C this can legally be evaluated as:

push a
push 3
mul
push a
push 3
add
store a

Unless Linden Labs has explicitly stated that parentheses are sequence points in LSL, I'm not going to write code that depends on any stronger order of evaluation than C.

It would produce the same result as my first example.

I've been writing code using yacc since 1979, and the exact same grammar can produce all three of the code examples we're talking about, depending on how the code generator works. When they go to mono they'll have a new code generator, as well as gain the ability to use 3rd party tools to optimise their code. Also, there's likely to be some script breakage at that point and scripts that will have to continue to be run using the old interpreter... so the impact of one more change would be mitigated.

My hat is off for you; 27 years of experience.
I've spent the last 10 min doing some catching up reading (and the last hour revising my responce); my formal education in language & compile design, is non existant.
As two people who understand yacc & lex (ok so maybe yours is alot better then mine), we can both see that who ever wrote the LSL lex knew less then us (as is can be seen with how floats are read).

LL has stated, when confronted with some simple optimizations, that they don't plan on changing the language till after Mono. Until Mono, everything is up in the air.

We are tetering on the devide between LSL moving from a scripting language to a programing language. The way LSL is writen right now, it would probably be a simple excersize for yourself to rewrite the lex & modify yacc file to read the LSL bytecode and output LSL text. With the only loss being function, variable & state names (and not being able to tell the difference between ++a, a+=1 and a = a + 1, etc). I would find the task difficult but not impossible (i'm not to keen on doing it, being a quick study doesn't compair to experience).

I'm sure you will agree, when optimizing code writen in a scripting language doing so is very hackish. But with a programing language user level optimization is about how data is structured and organized.

I think we should start talking about how LSL3 should be designed (in a new thread).

New thread it is...

(though I don't see why the hell they can't handle compile-time evaluation of constant expressions... the negative number parsing glitch that makes expressions like "a-1" illegal is clearly a hack to try and work around it)

Twenty-seven years... nearly as long as i've been alive... i knew there had to be something behind that simple and elegant light switch code.

My experience with flex/lex is that it's really only a tokenizer, meant to take complex input patterns and tranlate them into simpler ones for bison/yacc. Its job is to tell bison/yacc, "okay, this glob of bytes is the reserved word 'float', and the next glob of of bytes is a string 'foo', and the next glob of bytes is an integer 65536, ..." Flex/lex is awesome for its regular expressions, but will it even do binary data? Argent?

What would be the point in developing LSL3 rather than just using any of the many languages that work with Mono?

Good point, i have no plan on using LSL once I can use C.



^_^ I suspect who ever wrote LSL didn't take anything past the 101 level compiler design course.

BTW i handn't heard of sequence points before you pointed them out (at which point i looked them up in the ISO C standard).

I'd love to use C++ in SL, but I think LL really need to continue supporting scripters who have learned LSL, but do not yet know any RL languages - as well as the large exisiting base of LSL library and open-source code.

IMO, LSL.MONO (or whatever) is a neccesary bridge before LL develop and release a MONO API.

Well, it uses "C" strings so you have to do something special for zero bytes, but other than that it really doesn't care what the input is. Also, using states and modifying the input stream on the fly allows all kinds of really amazing stuff in pure lex. On the other hand, you can use the default lexer with yacc and have productions on raw ASCII strings... but BOY is it inefficient.