Welcome to the Second Life Forums Archive

These forums are CLOSED. Please visit the new forums HERE

increase script memory

Argent Stonecutter
Emergency Mustelid
Join date: 20 Sep 2005
Posts: 20,263
03-07-2006 09:03
From: Ralph Doctorow
From: Me
reference counting
Essentially copy on change.
Not exactly, but close. If you appended to or truncated an object with a reference count of 1, it wouldn't need to be copied (you'd decrement the reference count, notice it was zero, and re-use it when you incremented the reference count again). That would take care of your llListRemoveList example unless foo was a parameter to the function calling llListRemoveList, or otherwise aliased... but in that case it wouldn't be a redundant copy.
Lex Neva
wears dorky glasses
Join date: 27 Nov 2004
Posts: 1,361
03-07-2006 09:42
From: Ben Bacon
Lex, you are quite right that a function that takes a list and returns a (modified) copy of that list has to create a copy.

The problem is that LSL creates two copies. So in your example, foo exists in memory. Then an exact copy is created to pass to llListRemoveList, which returns a third copy (minus one element) after which the first copy is dumped.

So llListRemoveList always requires free memory at least double the size of the relevant list.


Huh, are you sure? ... now I'm dredging back my memory from college computer language courses... there were two types of pass-by-value, one in which a copy of the argument was made right at the function call no matter what, and one in which a copy is made inside the function if the value is modified. Guess which one's easier to program from a language-design point of view? ;)

From: someone

The third example needs free memory to store a copy of bar, a copy of baz, the result of adding them together and the result of removing two items - all at the same time!


I actually don't see a way to get out of this. I told it to store bar and baz in variables, and it has to store the result SOMEWHERE for the function. But you're saying if I stored the result of bar + baz into a new variable "gleep", and then passed gleep in, gleep would be stored once in the variable, and once on the stack. And while this is kind of a pain... I can understand why it's done this way. It does make 16k look awful small, though.
Lex Neva
wears dorky glasses
Join date: 27 Nov 2004
Posts: 1,361
03-07-2006 09:45
From: Rickard Roentgen
I'm really not sure why the memory limit for a script is so small when there is no limit on the number of scripts used. Typically people who need more memory, use it in the form of more scripts, so it's not saving server memory.


But really, how typical is this? You probably do it often, becuase you'r a power-coder... but I would venture to say the vast majority of scripts on the grid DON'T hit memory limits. People tend to use more scripts because it's easier, and because we can't do things like llSetLinkTexture, and because they want multiple objects to interact. I don't think that using multiple scripts to avoid the memory limit is a significant source of memory usage across the grid.
Gus Plisskin
Registered User
Join date: 8 Feb 2005
Posts: 84
03-07-2006 10:34
From: Mark Busch
Could you PLEASE increase script memory? 16 is so little. 32 would be fine. I already divided the game I'm working on into 7 or 8 scripts but I got the feeling that when it's done I could get into trouble again (first I tried it with 2 scripts)
also why do you get a heap-stack collision when you still have 8 K free?
What about having two priority classes for scripts? A long script could hold more data. A short would execute faster. This could be easy or hard depending on how script throttling is implemented.
Rickard Roentgen
Renaissance Punk
Join date: 4 Apr 2004
Posts: 1,869
03-07-2006 14:35
From: Lex Neva
But really, how typical is this? You probably do it often, becuase you'r a power-coder... but I would venture to say the vast majority of scripts on the grid DON'T hit memory limits. People tend to use more scripts because it's easier, and because we can't do things like llSetLinkTexture, and because they want multiple objects to interact. I don't think that using multiple scripts to avoid the memory limit is a significant source of memory usage across the grid.


Alright that's a fair observation. But we might be saying the samething in a round about way. You're saying that not many people go around the memory limit with multiple scripts. The reason being not many people actually need more than the memory limit. That doesn't really conflict with my point, that the people who do need more memory (even if they're the minority) typically don't hesitate to use multiple scripts to get it. ie maybe only 10% of the scripters in sl hit the memory limit, but of that 10% most will multiscript to get the memory they need. So, the limit really isn't saving memory, it's just obfuscation.
_____________________
Rickard Roentgen
Renaissance Punk
Join date: 4 Apr 2004
Posts: 1,869
03-07-2006 14:37
From: Argent Stonecutter
Because most scripts are small, so most people don't need more memory, and doubling the size of script memory would end up requiring more total memory even if a few complex scripts are split up.


oh, maybe I'm not understanding... are 16k reserved and locked for each script whether it uses it or not?
_____________________
Greg Hauptmann
Registered User
Join date: 30 Oct 2005
Posts: 283
03-07-2006 23:35
My argument for additional memory would be along the lines:

(i) scripters will do what they have to to implement their system, so they will multi-script

(ii) the multi-script implementation of the same system problem takes more memory due to (a) inter-script overheads, (b) overheads of replicator common code across each script

(iii) the language should provide some basic ability to write readable/maintainable code, however with the memory limit (i.e. when you get to the point of breaking out into multiple script) the ratio of supporting common code (utils, comms framework, etc) to actual business logic code for what the script is trying to do, is quite high [this just supports item (ii)] so if you want to reduce this ratio you have to comprimise the readability/maintainability of the code

So overall I could image that if the script limit was raised that we could less memory required overall + provide scripts with the ablity to write more readable/maintainable code/systems.

BTW: I'm kind of making an assumption here that there should be a means to only allocate memory where it is required (i.e. if a script needs 8k, then this is all it will take up in the SIM)


Regards
Greg
Ben Bacon
Registered User
Join date: 14 Jul 2005
Posts: 809
03-08-2006 05:50
From: Rickard Roentgen
oh, maybe I'm not understanding... are 16k reserved and locked for each script whether it uses it or not?
bingo - correct.
From: Greg Hauptmann
BTW: I'm kind of making an assumption here that there should be a means to only allocate memory where it is required (i.e. if a script needs 8k, then this is all it will take up in the SIM)
see above :)

I realise how stupid and inefficient it may sound, but the decision to implement the LSL VM this way would have made things muuuuuuuuch easier for LL to code - and to code right. Without this LL would have needed what would have amounted to a a custom virtual memory manager, with inter-process security and some form of quotas.

Not only would it have taken much longer before we had a usable LSL, but scripts would probably also run slower due to extra memory reference indirection.

Understanding this should explain why increasing the allocation per script would waste an incredible amount of memory, only for the 1% of scripts that need it (/me stops sucking thumb).

My suggestion:
I believe (but could be way wrong) that the vast majority of scripts don't hit the 16k limit.
I also believe that of the ones that do - the vast majority do so trying to store a lot of data.
I realise that there may be scripts that struggle just fitting their code into 16k, but they must be an infentismal fractions of all scripts out there.

So - rather than pushing up the memory limit - I would ask for better data handling and storage mechanisms - so that your data could live in, for example, a data file in the prim, rather than in script memory.

Basically, I'm looking for something in between notecard/dataserver and script-ram.
Eggy Lippmann
Wiktator
Join date: 1 May 2003
Posts: 7,939
03-08-2006 06:25
LSL was not designed for fancy business logic or anything remotely like a "system".
LSL was (AFAIK) designed to allow users to add simple behavior to objects, by coding a large set of small indepent scripts that would be dragged and dropped into an object, allowing non-technical people to produce interesting content and customize its behavior.
So you would code a "follow owner" script, an "antigrav" script, and a "bob up and down" script... and anyone who wanted to make a pet could drag and drop those, according to whatever behavior he wanted.
SL was, during Beta and 1.0, all about the builds. It was not geared towards commerce at all, you were supposed to make your own stuff. Heck, LL claimed ownership over your creations and the L$ was not sellable. Permissions defaulted to allowing everything, and you couldn't even set them on textures and scripts, only objects.
The original idea was that SL would be this space, sort of like There, where people would hang out, see the pretty builds, attend the odd event, and most of them would not even need to own land.
Land was free back then, all you paid was a monthly fee of $14.95 and you could own whatever amount of land you wanted.
The focus on scripting fancy interactive content only started to appear in 1.1 and later moved towards an emphasis on using SL as a gaming platform, with LL creating the annual game dev contest, and selecting things based on perceived entertainment value, as measured by popularity, first with voting stations and later with dwell.
The move towards a capitalist culture and the promotion of SL as a place where you could make money happened mostly during 2004, after LL changed to the current tiered pricing plan in 1.2, and GOM popped up. Out of world comms came in 1.3 and were hailed as the final solution to the storage problem. Just get a database :)
Argent Stonecutter
Emergency Mustelid
Join date: 20 Sep 2005
Posts: 20,263
03-08-2006 07:38
From: Rickard Roentgen
oh, maybe I'm not understanding... are 16k reserved and locked for each script whether it uses it or not?
Yes.
Rickard Roentgen
Renaissance Punk
Join date: 4 Apr 2004
Posts: 1,869
03-08-2006 10:08
pfft. I realise I'm the minority, but I have a lot of scripts that barely fit into or don't fit into 16k. An AO and vehicles come specifically to mind. My vendor really doesn't push the memory limits but you're right it doesn't actually store much userinfo. Anything game related also takes up a lot of memory if the game logic is at all complex, keeps score, or needs to communicate. lol, okay so that didn't become a problem until 1.1. It's an issue now. Anyway, about the virtual memory management... how complex would it be to make the current system allocate in 16k blocks? essentially letting you put your code into one script but giving it another scripts worth (or more) of space behind the scenes. Maybe make people have to pay for it. Dialog: "This script is within 3K of the memory limit and may stack overflow, would you like to pay 100L to add another 16K of memory?" or "This script has exceeded the 16K memory limit, would you like to pay 100L to add another 16K of memory?" or "This script has exceeded 32K memory usage, would you like to pay 200L to add another 32K of memory?". This would still of course allow stored data to exceed the memory space, I think dynamic allocation would be too slow and to prone to exploit.
_____________________
Argent Stonecutter
Emergency Mustelid
Join date: 20 Sep 2005
Posts: 20,263
03-08-2006 11:43
Maybe it's time to think about active script quotas?

Give a parcel a script quota of 1/3 the prim quota (which should be ample, most prims aren't scripted and the number of active scripts per sim is rarely over 5000), with "fat" scripts of 32k costing 2x?
Rickard Roentgen
Renaissance Punk
Join date: 4 Apr 2004
Posts: 1,869
03-08-2006 11:55
From: Argent Stonecutter
Maybe it's time to think about active script quotas?

Give a parcel a script quota of 1/3 the prim quota (which should be ample, most prims aren't scripted and the number of active scripts per sim is rarely over 5000), with "fat" scripts of 32k costing 2x?


I like it, but what about vehicles?
_____________________
Argent Stonecutter
Emergency Mustelid
Join date: 20 Sep 2005
Posts: 20,263
03-08-2006 12:46
From: Rickard Roentgen
I like it, but what about vehicles?
Vehicles would be in the sim overhead just as they are now. If you fly your vehicle over a full parcel you're not derezzed, AFAIK, though you'd be ill-advised to UnSit while you're there. :)
Ralph Doctorow
Registered User
Join date: 16 Oct 2005
Posts: 560
03-08-2006 13:18
From: Argent Stonecutter
Not exactly, but close. If you appended to or truncated an object with a reference count of 1, it wouldn't need to be copied (you'd decrement the reference count, notice it was zero, and re-use it when you incremented the reference count again). That would take care of your llListRemoveList example unless foo was a parameter to the function calling llListRemoveList, or otherwise aliased... but in that case it wouldn't be a redundant copy.

Hmm, so the idea is that in any assignment, the LHS ref count is first decremented, then the call done?
CODE

list foo = [1,2]; // foo->1
list bar = [10]; // bar->1
list bar = llRemoveList(foo, 1, 1); // bar->0 before call, foo = 1, so foo is copied in function, new ref returned, bar->1
foo = llRemoveList(foo, 1, 1); // foo->0 before call, foo not copied in function, same ref returned, foo->1
Argent Stonecutter
Emergency Mustelid
Join date: 20 Sep 2005
Posts: 20,263
03-08-2006 15:25
In the example you gave, llRemoveList would still need to make a copy because the reference count would be 2: the constant [1,2] would be one copy, and the reference to it in foo would be the second.

But that's a good point. For appending, you can save a copy because the operation is atomic. For deletion, you would need to know that the assignment was to the argument to bar, because it wouldn't be safe to free the value before then. You'd still save copies on calls but I'd have to think about it more to see whether you could save copies on truncation...
1 2