Stack/Heap collision...

So, apparently my new Cutthroat Lotto Ball is more popular than I anticipated!

I got an IM last night from a club owner who said that with like 1600 in the pot and a minute left, the lotto ball crashed with this error. Oh noes! We were able to get it straightened out without an angry mob, but it was kind of scary.

From what I read on the wiki, this is basically an out of memory error, right? My guess is my list is growing too big.

When someone donates to the lottery, I'm storing their UUID and amount to the list. In fact, if they donate more than once (which is the nature of this kind of lottery), their name gets appended to the list again.

So I'm thinking that instead of storing their UUID, I can just store their name. That should reduce memory usage by a lot.

Also, if someone donates, I can check if they're already in the list, and if so, just add their new donation to their old... What is the perfomance like on llListFindList?

Thanks for any comments you can provide!
Adman

Just recently, I've found myself having to store a lot of data in LSL. One way to increase the amount you can store (without resorting to off-world solutions like email or xml-rpc) is to create database scripts. For example, I have a script like the following that does nothing but holds 50 keys:

list data;

default
{
    link_message(integer sender_num, integer num, string str, key id) {
        if (num < 50 && str == "read") {
            llMessageLinked(LINK_THIS, num, "found", llList2Key(data, num);
        } else if (num < 50 && str == "write") {
            data = llListReplaceList(data, [id], num, num);
        }
    }
}

I can then add an additional DB script to hold another 50 like so:

list data;

default
{
    link_message(integer sender_num, integer num, string str, key id) {
        if (num > 50 && num < 100 && str == "read") {
            llMessageLinked(LINK_THIS, num, "found", llList2Key(data, num - 50);
        } else if (num > 50 && num < 100 && str == "write") {
            data = llListReplaceList(data, [id], num - 50, num - 50);
        }
    }
}

..and so on. To read and write from all the DB scripts, I just have to issue a single link message:

llMessageLinked(LINK_THIS, num, "read", userkey);
llMessageLinked(LINK_THIS, num, "write", userkey);

... where "userkey" is the variable for the key you're reading or writing and "num" is the index of that user (from 0-100).

(I haven't checked this code so there may be a typo).

VERY clever!

Side question: If I have an empty list, and I do:

llListReplaceList(myList, ["adman"], 5, 5);

What do I end up with?

I do something similar for the 'Newbie Detector' at the Shelter.

New residents that arrive get a welcome notecard, and a public welcome in chat.

Thing is, I don't want the same new resident to get a notecard more than once, so I have to keep track of who's been greeted.

In my testing, I found that a single script could hold approxomately 220 keys without stack-heaping. Considering we get around 600 unique folks per week, and I define 'new resident' as under 3 weeks old, 220 wasn't going to cut it.

So, I seperated everything into 16 scripts - sorted by the last digit of the key. One script handles all keys ending in '1', another with '2', and so on. This afforded me appromately 3520 entries (altough the true number is much less, as the distribution of the last digit of one's key doesn't appear to be an even one). Still, plenty nonetheless

Hope this helps!

llListReplaceList(myList, ["adman"], 5, 5) will replace the sixth position of list "myList" with the contents of list "adman".

If there is no sixth position (or its empty), it will be appended to the end.

Note: (From the wiki) llListReplaceList doesn't actually manipulate "myList" directly, rather it returns a new list containing the results.

Great. This is what I wanted to know. Thanks!

Adman

Try the first digit of the key. I'd asked about this some time back, and someone posted stats for this. The distribution is pretty even using the first character, and that was measured over some 10,000+ keys (if I remember right).

Also, if you need to hold a lot of keys over a lot of time, one thing you can do is keep a parallel list that stores the result of llGetUNIXTime, then compare to the current time. If an avatar's key is like... 40 days old, it may not need to be there for your current application.

list keyList;
list ageList;

addKey(key id)
{
	if (llListFindList(keyList,id) == -1)
	{
		keyList += id;
		ageList += llGetUNIXTime();
	}
}

If your going to be storing just avatar keys in a list you should use a string instead, it will be faster (and use less memory).

string keyList;
list ageList;

addKey(key id)
{
    if (llSubStringIndex(keyList, id) == -1)
    {//and if you want to save another byte change "== -1" to "& 0x80000000"
        keyList += id;
        ageList += llGetUNIXTime();
    }
}

integer findkey(key id)
{
    integer a = llSubStringIndex(keyList, id);
    return (a / 36) | (a >> 32);//returns -1 if not found (the bitshift takes care of it)
}

wait, so storing keys as strings takes less memmory then storying keys as keys? O.o

no instead of storing them in a list....

A string is a sequence of characters limited in length only by available memory. Strings are enclosed in quotation marks. LSL uses UTF-8 for it's encoding standard for strings.

Now this is interesting. But whenever i tried to convert my list into a string and having it said in the chat, not all the items where displayed. it seemed to me like the string length was limited.

Keys ARE Strings (they just have a couple special operators and earmarked in lists differently but otherwise LSL treats them the same).

When feeding a string (or key) to any chat function, the chat will be truncated to 255 characters if it exceeds 255 characters.

Keys are always 36 characters, so working with a string full of keys is just as easy as working with a list.

Why would a string use less memory then a list? Because a list just encapsulates the it contained elements. A list of 20 strings is like having 20 strings as local or global variables. By combining them into a single string you replace the overhead required by those strings with that of a single string and you remove the overhead involved with them being in a list. On large lists it adds up.

Also lists are storing information about type of each value.

This always bothered me, since a key can efficiently and reliably be stored and manipulated as an 128 bit unsigned integer and only converted to a string for display.

Well, I expect that the internal representation is a 16 byte integer. They would be foolish indeed to store them internally using the printed representation.

Note that not all strings have to be human-readable. If you take a 32-bit integer, and store it into a 4-byte string, you've still got the same bits.

- Jon

Is there a limit to how long a string can be?
Is there a limit to how big a list can get?

Adman

Stack-heap is lsl's way of saying NO SOUP FOR YOU. Best of luck in finding a workable and consistent solution, if you are still having problems in say a week or two hollar at me in PM (figuratively not literally hollar) and I'll pass your problem along to ppl who I know are ninjas at dealing with this error.


Cheers - Calix

Strings and lists are limited only by memory limit of script, which is 16kb.

That's what I thought, but Strife says no... they're really strings internally.

Well, yeah, but strings can hold data in any form - they are basically bit buckets.

The question is not whether it is string or integer, the question is whether they use a binary or ASCII representation.

- Jon

Some optimizations I can think of for your implementation:

Since you're only storing avatar's keys, store their names instead. Split the first name and last name at a space, and store them seperately. Also, instead of storing the entire name again when a person re-enters, increment a counter for that person.

// Strides: [firstName, lastNameIndex, numEntrys]
list entryData; 
list lastNames;

addEntry(string name) {
    string firstName = llGetSubString(name, 0, llSubStringIndex(name, " ") - 1));
    string lastName = llDeleteSubString(name, 0, llSubStringIndex(name, " "));
    integer lastNameIndex = llListFindList(lastNames, [lastName]);
    integer entryIndex = llListFindList(firstNames, [firstName, (string)lastNameIndex]);
    if (entryIndex != -1) {
        integer prevEntries = llList2Integer(entryData, index+2);
        // Store the integer as a string to save memory space.
        entryData = llListReplaceList(entryData, [(string)(prevEntries+1)], entryIndex+2, entryIndex+2);
    } else {
        if (lastNameIndex == -1) { // Add a last name
            lastNames += lastName;
            lastNameIndex = llGetListLength(lastNames) - 1;
        }
        entryData += [firstName, (string)lastNameIndex, (string)1];
    }
}

==Chris

Thanks Chris... this is essentially what I've updated my code to do. Works like a charm.

Adman

YES, I KNOW, THAT IS THE POINT.

According to Strife (who has disassembled and decompiled LSL code), the key is stored as the 36-character ASCII representation internally.

Yeah, but how are they stored internally?

*ducks and runs*