My C-64 is a dream to program on.

Heya everyone,

I know this one has been probably covered many times before, but I got some pent up annoyance that I gotta let out. Needless to say, banging my head on my monitor doesn't seem to be alleviating it anymore.

We need more memory for the memory heaps on scripts. The current heap is just too small. Heck, I know guys who do SW deveopment for cell phones, and they even have way more space to work with than us.

Now, I know I'll probably hear some individuals say "Well, you can split it into multiple scripts" or "try cutting back on your variable name sizes" and stuff like that, but trust me, I've really tried to do that already.

Until RPC is improved for allowing multiple scripts to better chit-chat with each other (maybe shared memory spaces? OMG that'd be too cool), I'm kinda stuck where some of my projects are just next to impossible. LL seems intent on getting many of us to develop multiplayer content, but at the current state, they make it pretty tough. 16k is pretty bismal when the tool given to us is a meta language.

What you all think?

I think the word is 'abysmal'.

Yeah, that'd about sum it up when, 1 character in a variable name, or a single line of code busts the bank on you. Here's another, horrible

Luckily Cory is changing script bytecode debug stuff so long variable names dont affect memory usage anymore.

If I remember correctly, I think Kex or Francis did a study on variable memory usage in LSL, and the result was that the actual name of the variable took up more space then actually having a variable.

Usually, people have > 4 character variable names, so:

integer fubar;

4 bytes for an integer,
and 5 bytes for 'fubar'.

More mem taken up by the name then the actual var itself.

Fixed in v1.3!!


==Chris

I believe Cory is "looking into" improvements with memory and possibly making the size larger. I think it was mentioned in the town hall with Cory...

Ok first the link to the town hall log that Kex Godel did:

/invalid_link.html

And I went through myself looking for where it was mentioned and found it, so heres the qoute from the log of its mentioning:

"Haney Linden: Kex Godel: Q: At the last meeting you said you'd look into seeing if it's even possible to increase script memory size without a lot of trouble. Have you had time to look into this and make any conclusions yet?

Cory Linden: Kex: I've looked into it and it is too much work to try and do for 1.3, however I have gone trhough and removed most of the symbols which means that using long variable names no long hurts you."

---

So they're still working on it, but doing what they can as they go along.

Yeah but COME ON! A C-64 beats a SL object? COME ON!

Give us at least 32k PRETTY please?

Im completely for wanting more memory.

But lets look at it from their point of view. They have to allow enough memory for EVERY prim in the sim to have X # of scripts in it.

(Is there a limit on how many scripts you can have in an object? Thats an interesting fact I have never heard the answer to.)

Lets break it down:

givens:
1: 1k = 1024bytes so, 16k = 16384bytes.
2: 1m2 = ~4.37 allowable prims.
3: sims have 65,536m2, so each sim allows ~286,392 prims.

findings:
this yields (at 1 script per prim) the minimum requirement of
4,692,246,528 bytes per sim for scripting heaps/stacks.
(~4,475MB, 4.47GB)

Now - that figure times how many scripts are allowed in each prim.

This has to be wrong, that numbers way to high. What did i do wrong?

That's not a clear comparison.

To get around the memory limitations, people break up scripts into multiple scripts.

However, using multiple scripts requires communication between scripts.

If your project does not divide well, you will be wasting a *lot* of code just on communication.

So you'll need even more scripts because now you've filled half of your scripts with cross-communication code.

So, as an example I just gave in another thread, you now are consuming 64k of sim memory to run 4 scripts where you could have been just peachy with one 32k script.

So at this point in the curve of diminishing returns, we are *worse off* keeping memory limitations very low.

At increasing amounts of memory, we would spend less percentage of the script's functionality on communications with other scripts because 1) we won't need as many other scripts, and 2) there is more room for non-communication code.

I think 32k or 64k would be nice numbers. They represent signed and unsigned 16-bit integers, which might be what is used for memory addresses. It seems odd that we're stuck in 14-bit (16k) territory.

BTW, if each of all 15,000 prims in a sim were given 32k of memory to work with, that's still less than 512M of memory.

Oops - i had it backwards. =P
Its 4.37METERS per prim. I had it the other way around. so:

givens:
1: 1k = 1024bytes so, 16k = 16384bytes.
2: 4.37m = 1 allowable prim.
3: sims have 65,536m2, so each sim allows ~14,997 prims.

findings:
this yields (at 1 script per prim) the minimum requirement of
245,710,848 bytes per sim for scripting heaps/stacks.
(~234MB)

Oops.

Michael: I think its 255 inventory items per prim, 15,000 prims / sim.

Kex: First I do support larger memory sizes. I just know it isn't as easy as some people think.

14-Bits
Why would it use 14bits? I actually don't know. But I can imagine many reasons.
For example the first two bits could be used to map to a seperate memory space for linden functions or something.

One of my college courses was build your own OS from scratch. It was an amazingly fun and difficult class. One of the things I learned from it was that memory management is not easy or always intuitive until you really look at the underlying system. So something where we say 'geeze where did they pick 14 bits for addressing??' they could look and say 'well we need 2 bits to choose between these different memory areas...'.

Or they could look at is as a total addressing space of 32bits for memory, and just be doing a balancing act between number of scripts and script size - 18 bits to point to a specific script, the other 14 for where in that script. 18 bits allows for 262,144 scripts, or an average of 17 scripts per object. While a larger script size of say 32k would drop the total number of scripts in half and drop the average to about 8 scripts per prim. That may seem like an ok average, but what if they want to be able to up prim limits in the future? At 30,000 prims per sim (not entirely out of the realm of reason) we are down to 4 scripts per prim average allowed. Up the limit to 64k and you also are at 4 scripts per prim with current prim limits, and lower if those prim limits are raised.

But that ignores entirely that other things may need to be in that address space.

============

I do agree that a lot of memory is wasted in spliting scripts and a lot of server resources too. It is highly likely that an increased limit on script size would offer a tangible performance benefit.

It appears to be on cory's mind and to be a non trivial task. It will be great when it comes and I do support the idea.

Sorry, I don't really see it from their point of view. How often does that a player uses a script that requires 16k or more memory? Even more over, how many objects out there are even scripted? Even the basic default script in them?

As for effort, I just can't take Cory's response on this one. Sounds to me the bosses got him fiddling with other. Common, us females gotta watch how we sit! :|

I just am iked by the fact that we're given such a low memory amount, and LL's implmentation doesn't seem to really take that into account. Really, on so many occasions, I've had to scrap things all together!

Here's one for everyone. Which one would be more effecient in a perfect world in the way of memory usage?

example 1
------------
list somelist = [];
integer i;
for (i = 0; i < 6; i++) {
somelist += [ NULL_KEY ];
}

or

example 2
------------
list somelist = [];
somelist += [ NULL_KEY, NULL_KEY, NULL_KEY, NULL_KEY, NULL_KEY, NULL_KEY ];

To me, the first uses more memory, and more CPU resources. Allot more math to be done. The second should in theory prove to chew up less CPU cycles and less memory. Guess what, I found out quickly how wrong I was... The second took more memory! Appears that just raw code chews up your memory heap, and furthermore to that, constants chew up more memory. Excuse how I put this, but your screwed if you do and screwed if you don't...

So a question to LL, it seems to be the interest is in adding new eye candy, and little toys. I like them too, but considering how inadequate 16k has been, and the amount of other in-game dev's I've heard whine about it, they gotta really look at fixing that one up. Depending on the XML-RPC stuff, there may be some alleviation, but having seen how they've implemented other features, I doubt it will be as effective as we all hope it will be. All I have to say is "llDialog"... Hmph...

I do eventually forsee that even doubling the stack to 32k will not be the end of it, but as us in-game devs create bigger and better experiences for other players, so does the ram we need to use. If LL wants the "in-game technology" to move forward, they really gotta help support us. C-64 versus an IBM P-690... Little bit of difference in capabilities and what you can do with each, just too bad LL leaves us stuck.

For now, until they fix it, I'm opting out of any Multiplayer builds. Sorry, I don't feel like starting a build just to find out that either I have to rebuild it all-together, drop features or scrap it all.

Please oh omnipotent Lindens, grant us our wishes... Oh, and be ready for the next round

The whole premise is rather excessive. It would be better to handle this with inferential statistics than to assume the nastiest possible case, i.e. as many scripts running as possible.

Let us say that a sample of the max number of running scripts is taken over the course of a week, and it is found that, for 70 sims, the sample mean is 218 scripts with a sample standard deviation of 32.778. I have less than 100 samples, so I will use the Student's-T distribution to compute a confidence interval. (Some people only use T below 32 samples, others below 100. I think it is slightly safer to use them below 100. Your mileage may vary.)

T-dist confidence interval: μ=218 σ=32.778 n=70, α=.99
= (207.62, 228.38)

What this means in English is that we can be confident that there is a 99% chance that no more than 229 scripts are running in a given simulator.

Now, there are some things you can do to err on the safe side, such as discarding the 25% of simulators with the lowest script usage, or deliberately taking the census at a busy time of the day. You could also increase alpha in order to get a wider confidence interval. For example, at α=.999, the upper end of the confidence interval is 231.47 (we'll call it 232.)

Let's say that Linden Lab is willing to let scripts eat as much as 32MB RAM in a simulator. 32MB / 232 scripts = 0.138MB/script, with a 99.9% chance that they won't go over and have to deny execution resources to anyone's script. So let's make it a nice round number slightly less than the calculated value and say each script could have 128K of heap space. (I don't know that they can afford 32MB, and of course the statistics here are totally made-up, so don't assume what we could have is anywhere near this.)

This is just a very cursory example of how inferential statistical anlysis can be used in enterprise resource planning. In the time it's taken me to type this up, I've already come up with possible better ways of sampling the data.

Seeing that its nearly unanimous, except on account of some flawed mathematics, I have to put out the question then to Cory, or any of the other Lindens,

1/ Are there any real plans to increase our heap sizes?
2/ Other than the variable length name, what other plans are there to help us maximize our memory space allocated?

And if there is a yes to any of the above,

1/ When? Seriously, when? When can I start working on my MP and other heavy scripts?
2/ Why hasn't this already been addressed?

I think it's safe to say that SIM's already make sure they are not going to run out of memory when a script is loaded into memory. If it's a free for all and the sims just "crash" when they run out of memory I'd be shocked.

So bumping to 32K, actually 128k sounds a LOT better, should not be that hard. It would be easier than giving us more memory managment features. Then again, I have a scoreboard that uses like 100 scripts that would all fit in 2k. Thinking about all that wasted memory is depressing.

I'm stumped on my multiplayer builds too Trimda. I have to rez an object for each "player". Cheap hack of a malloc if you ask me. And slow.

Even more over, why is the memory shared for our code and variables? In 16k of memory space, it really doesn't leave us a whole ton of space to work in. What could even work, the code itself sits in one memory space (unlimited), and variables and any required runtime memory sits in its own space. At this point, 16k would be suffecient to work in.

Even more over, if the memory spaces were seperated, and the code was left to its own area, in theory, this space would remain very static. This could theoretically allow LL to "compress" this space through optimization or other meathods. By still keeping a cap on the "variable" memory stack, LL could ensure that no specific script gets out of hand and chews up all the memory on any given sim.


And while in the neighborhood, I would love to have facilities like malloc and free. I personally like to have control my memory space, and really dislike having to allocate memory at the begging of run time. Often, the footprint of a program can be kept to a minimum until it requires a little bit more. This would be great for the fact that any larger MP scripts could effectively stay in a much smaller footprint than otherwise would be usually taken. Often, I only need a variable for 30 seconds in certain states and otherwise, don't need them elsewhere. To be able to release the memory of this variable, and later allocate it needbe would be awesome. I do admit thou, this is not a function for the casual scripter out there, and would be mainly used by the hardcore scripters, but it would allow the "big boys" to really go to town...

Scripts should have a way to declare how much memory they need regarding variables and stack space. There should be a way for the script writer to declare whether or not he will be needing a larger stack than the "default maximum" will provide.

Similarly, if a script only needs a few bytes of stack space, why should the virtual machine give out so much more than it needs?

This is possible right now even on a Commodore 64 (a nod to Trimda, there) with one of the newer OS rewrites out there, like JOS/WINGS or even ACE. In ACE, a program can even set aside its own stack space, embedded within the program code if so desired!

Why can't we have programmable stack heaps, man? What is this, the stone age? I reckon LL needs someone from the embedded software world to come in and help them figure this stuff out.

-J

Rumor has it that they are about to start auctioning heap space in much the same way they are now doing with land.

Of course, I just started the rumor here, but that doesn't make it any less good, does it?

lol, darn rumour mongers... Shhh, don't give them (the Lindens) any ideas...

On another note, any Linden going to answer us, or once again are we to sit here listening to no official response I feel that this subject really needs a comment from LL as it really effects the people who wish to make MP content in game. From what I last heard from the developer contents, LL really wants to see more in-game MP content to keep the mass numbers of AVs entertained! I for one have really scrapped all my future MP projects as unfeasible. Maybe something may get be returning to the drawing board. SharedMemory spaces (ie handing pointers) support for malloc and the like, et cetra. Common! Something!!! The silences from LL is deafening me here, and I'm starting to get real board with small scale projects! Common!!!

Ok, can someone tell me if am I holding my breath for nothing here? Am I gonna dye from afixiation before we hear something?

Trimda: the only thing we know currently (as of the Cory town hall meeting) is:

- Expanded memory will not be in 1.3, but it is being looked into.
- 1.3 will bring some improvments to memory usage

Heya Kex,

I've too read the transcript of it all. I found Cory to be very non-comital to anything, and flagging it under the category of too-hard-too-complicated. What disappointed me about 1.3 was the fact that there was more concentration on converting the gui more to XML, and little snapshots on maps and the like.

As formentioned within the thread, I enjoy eye candy as much as the rest, but to myself, LL appears to really have gotten themselves into a pretzle. They wish to really have us, "The Scripters of the Universe" (in castle grey skull), to create more MP type of builds, but they have left us somewhat out in the cold.

A few memory optimizations as described still are just a little drop in the bucket. Many of us already do the 1 letter variable trick and other such little things. Earlier in the post, I gave a prime example of an odd-ball behavior which after showing several other indivduals in RL agree is totally backwords. It appears infact that there are many memory management issues within LSL that are prevalent.


To recap the questions I posed later in the post:


To recap what I have read, and heard:

1/ No concrete confirmation. "Its being looked into" is not a confirmation, its as exactly as it states, "we don't know yet and we promise nothing at this time". As the designers and programmers of this software, unkessall the original group of programmers and designers got hit by a bus on the way to the bank, then they should be fully aware of the scope of the work that needs to be done to make it happen. Sorry, no dice. It sounds like a "lets hope they forget" syndrome...
2/ Cory has eluded to some bytecode optimizations, but I really haven't heard anything else more concrete either there. As the question asks, what else can we expect.

And as for the yes questions,

1/ what 1.5, 1.6? It seems to be major releases are every two to three monthes. So 4-9 monthes, maybe?
2/ No answer whatsoever on this side.

I work for an IT outsourcing firm. When our organization shows a clear defeciency, our customers hold us responsible, and want to know why is it that way, and when to expect a correction or upgrade. In this situation, its exactly that, all of the people in this thread agree, and I assume including yourself, that this is an issue. Furthermore, in the real world, we do classify as customers. Do we accept the non-comital answer?

I would really like to know for sure, and even more over, if this has been a known short coming, why has it taken such a backseat to other issues? I wasn't around for 1.0, but 1.2 I feel it seemed to be the "abolish-tax, grab-the-RL-cash, and mac-turbo-cpu-load-builder" release, 1.3 seems thus far the "Eye-Candy and trinkets" release.

Furthermore, many of us are awaiting Havoc 2. No commitments there either. Increased server perfomance, smoother physics, and other benefits to boot. I can understand it taking time, but still no timeline at all. Not even a tentative release date!

Considering how many of in-game devs I've heard complain about this memory limitation, and the fact that many builds are just unfeasible in the current state, wouldn't this really be a higher priority? LL wants us, the in-game devs to build things for players to play with. We want to build them. Infact, we're paying LL to provide them content to help hook new players on the game, and keep some of us old-timers entertained between spats of in-game bordom. Hmm, kinda crazy sounding but, we pay Linden Lab to put content into their game, and they collect all the revenue... Wouldn't a subject such as this be in there best interest to reply to and set a target date to correct?

And as for my call for a comment from the Lindens, I feel that lack of response is really because they don't wish to commit to anything. Its very disheartening to never get even a comment or commitment from them. Needless to say, I added a little bit more to my tagline to maybe get noticed on how I feel. Even after I have put this post in place, I have very little confidence in seeing any official response to this post.

I just find it very fustrating to see core functionality of the game being ignored, and the lack of response from LL on any of our posts for the most part. It almost seems that LL has become uninterested in the players, and their issues. Sure they have town halls, land giveaways and events like the such, but, when a group of people cry out for a response they are left out in the cold. If this isn't a call for a response, don't know what is.

And a last point I just realized. Their slogan, "Your World. You Imagination." I like it, and when I first came to SL, I felt that way. Now, I feel with such a shortfall as this "Your World. Your Imagination. Your Limits".

SL is about the content we players we put into it. UI improvements and the such mean nothing if the content cannot be improved. How many variations of a door can one build. How many variations of a vending machine can one build? How many XYZ variations can I build of the same thing? S

L wants innovation and boasts of their technology, how would they like if their Linux clusters could only support 2megs of RAM, and Linus Torval and the rest of the Linux comunity said the same to them about it? Hmm, don't think they'd be running right now, would they?

Could a Linden please respond... I would love this thread to come to a close with a concrete answer... My head is starting to hurt from all this ranting

One thing that's bothering me, is I see the following taking place:

1. We have shortfalls in the scripting. Extreme shortfalls. (no arrays, limited data types, poor 'pulls' with cross script intercommunication, limited memory, variable name lengths take memory, compiler bugs, an editor that makes notepad look good, etc..)

2. They decide to ignore all of these, and add xml-rpc.


It starting to feel like: 'ok, we know the language is pretty bad, so why dont you do the work outside of SL and bring in the results? That fixes it from our end.'


Personally, I think they should give us a proper language and proper editing tools, and then worry about external IO.

I mean, shouldn't the inner-SL intercommunication needs be a higher priority?

Michael

Havok 2 is more important. I've got a sign.

Yeah, its pretty scarey. To me, its almost like I might as well use LSL as a midas style engine, and do all the real scripting from an external server. Heck, with the tools available to me outside, I could run loops around LSL. My only concern going this route is having myrads of in-game scripts out of order or lagging because of internet connectivity issues, downed remote servers, et cetra. It almost seems that LL really has implemented the XML-RPC stuff for the likes of GOM.

LSL as a language has many of its own issues, but some are so blatent that it often makes me scratch my head. If LSL is to be our tool to create content, then LSL alone really needs to be looked over very closely and have nearly its own release to improve it. LL improves LSL a huge amount, the content within the game will improve a drastic amount as well.

I would really recommend LL to have a "User's LSL Composium" and have us, the primary users of the language, help identify the required improvements.