Data Storage

HIyas Everyone,

I have a notecard reader that uses a llDialogue menu system to select and read any properly configured notecard in a prim's inventory. I can drop as many notecards into inventory I want and the menu adjusts itself. The way it is set up now is if I want to change my data set, I have to take the time out to read a different notecard....

404 error

404 errgggggggggg error


What I would like to do is to speed things up by having the notecard reader work in the background and dynamically store multiple notecard data into separate "storage' script(s) as notecards are added/deleted in the prim's inventory. And then just dump the data from those scripts when I need it.

I've looked at a few "storage solutions" that I have found thru the forums here but most are impossible for me to understand. It would be nice if people wrote in plain "English"... lol

I use the names of the notecards to populate the menu system. Each notecard is divided in exactly 5 named categories of data. The amount of data in each category (notecard lines) varies but doesn't exceed the max.

capability.

I can code a very basic static storage system but as the prim's inventory changes with diff notecards, I need to have the storage adjust "on-the-fly".

Any ideas, comments, help, etc... will be greatly appreciated. Thanx!

I guess you'll want to use the changed event and test with CHANGED_INVENTORY to trigger a re-read of the notecards. I don't think you can test for specific inventory changes such as a changed single notecard in LSL, though you might be able to use the description field of the notecard for this somehow.

Once triggered the llGetNotecardLine () function and the dataserver event that handles the result should trundle along happily in the background.

Is that any help?

PS Maybe llGetInventoryKey () could help IF the changed notecard is completely new, AND it is full perm (copy, modify, transfer). (Or does a notecard get a new key even if it's only edited?)

Thanx EF, I have this incorporated to keep my menu system updated. And will incorporate this into reading the notecards soon as I can figure how to store all the data.

Lists?

I'm just thinking out loud here...

Maybe create a function, a database function that has a name (notecard name) and 5 columns for the 5 categories. Each column would be the list of data. Would this be a "Table"?

And then create and delete Tables as notecards are added and removed?

What is an array?

There are no arrays or tables in LSL. There are "lists", and you can use these to similar effect with a bit of work.

So five lists of data, one for each notecard. Three questions need to be answered next: how much data that needs to be stored is there on each notecard, how is that data structured, and how do you need to access that data?

By structure, I mean is the data in the form of a simple list of items, or is it more complicated than that? And by access I mean do you need to get hold of a single item from a single list, a number of items from a single list, or a number of items from across several of the lists?

An array is a table with only one row or column, unless it has more than one dimension, in which case it is the same as a table.

no way to read description fields of inventory items BUT including a forced change counter as the first line in each notecard that you increment (or just change) should allow you to do version checking to see exactly which notecard was updated. simply store that number separately and then compare it when you get CHANGED_INVENTORY... if it's different, re-read the whole thing.

Right now, I have 7 global lists that I maintain:

list NotecardsNames;
list CategoryNames;
list CategoryName1Data;
list CategoryName2Data;
list CategoryName3Data;
list CategoryName4Data;
list CategoryName5Data;

And I use and manipulate these lists for what I need to accomplish. As I index a NotecardName, that notecard is read and the data is parsed to other scripts which is used and stored temporarily until I call another notecard. Pretty simple really. The trick now is having the info stored already without having to read the notecard each time.

I can saturate a script with fixed lists but I need to add, delete and sort these like a database as I add, delete, and sort notecards. As I drop a new notecard into the prim's inventory, the index position of all the other NoteCardNames change... the same if I delete a notecard. The list NoteCardNames is what drives my menu system and I need to make sure my lists index the same on the storage script. I want make one call to the storage script using a NoteCardName and parse all the corresponding data out. This is not a real easy thing for me to figure out. It's like I don't have the "Big Picture" yet... lol

Thanx Void for the tip... I am planning on using NoteCardNames to do the compare and prevent re-reads but, if the notecard gets deleted from the prim, I want that data deleted too. I'm willing to suffer a new notecard read if it is again dropped into inventory at a later date.

[Example script given in-world; cannot post it here and cannot see what's in it that's upsetting the forum software]

lol

But your latest post kind of nullifies that, anyway. It's way to simplistic as an example, I think.

So do you want to extract data item 7 from category 3 (say) by calling a function something like this data = GetItem (7, 3), and add or replace it it by calling a function like SetItem (7, 3, data)?

If so, you could do it by first creating a list that is long enough to contain the maximum number of items (say 5 categories x 12 buttons = 60)

list data_items = [];
integer index = 0;
while (index++ < 60) data_items += "";

and then coding the get and set functions like this

string GetItem (integer item, integer category)
{
return llList2String (data_items, (item * category) - 1);
}

SetItem (integer item, integer category, string data)
{
data_items = llListReplaceList (data_items, [data], (item * category) - 1, (item * category) - 1);
}

The -1s are there because lists are indexed from 0; thus item 1, category 1 is at position 0 in the list, item 12, category 5 is at position 59 and item 12, category 2 is at position 23. This is using a single list rather like you might a table.

Am I any closer to what you're looking for with that? If so, we can look at sorting the data next

So do you want to extract data item 7 from category 3 (say) by calling a function something like this data = GetItem (7, 3), and add or replace it by calling a function like SetItem (7, 3, data)?

If so, you could do it by first creating a list that is long enough to contain the maximum number of items (say 5 categories x 12 buttons = 60)

list data_items = [];
integer index = 0;
while (index++ < 60) data_items += "";

and then coding the get and set functions like this

string GetItem (integer item, integer category)
{
return llList2String (data_items, (category - 1) * 12 + item - 1);
}

SetItem (integer item, integer category, string data)
{
data_items = llListReplaceList (data_items, [data], (category - 1) * 12 + item - 1, (category - 1) * 12 + item - 1);
}

The -1s are there because lists are indexed from 0; thus item 1, category 1 is at position 0 in the list, item 12, category 5 is at position 59 and item 12, category 2 is at position 23. This is using a single list rather like you might a table.

Am I any closer to what you're looking for with that? If so, we can look at sorting the data next

each time a notecard is saved, it's key is changed, couldn't you also use that to figure out the changed notecard?

like say keep a list of the notecard names and their keys, run thru a loop and compare the current notecard's key to the one saved in the list, if it's different, that's the one that was updated

ETA:
well here's the quick script i cooked to test that theory, doesn't seem to work, though it is still early for me, maybe i did something wrong?

ETA2:
fixed it! i had the llListFindList arguments backwards, and forgot an equal sign in the index test, added a test to see if a notecard was removed

list notes;
list ids;
integer note = INVENTORY_NOTECARD;

default
{
    state_entry()
    {
        integer tot = llGetInventoryNumber(note);
        integer i = 0;
        for(i;i < tot;++i)
            {
                string name = llGetInventoryName(note,i);
                key id = llGetInventoryKey(name);
                integer index = llListFindList((list)name,notes);
                if(index = -1 )
                {
                    notes += name;
                    ids += id;
                    llOwnerSay("Added notecard named: " + name);
                }
            }
        }
    changed(integer change)
    {
        if(change & CHANGED_INVENTORY)
        {
            integer tot = llGetInventoryNumber(note);
            integer i = 0;
            for(i;i < tot;++i)
            {
                string name = llGetInventoryName(note,i);
                key id = llGetInventoryKey(name);
                integer index = llListFindList(notes,(list)name);
                if(index == -1 )
                {
                    notes += name;
                    ids += id;
                    llOwnerSay("Added notecard named: " + name);
                }
                else
                if(index >= 0)
                {
                    if(llList2Key(ids,index) != id)
                    {
                        ids = llDeleteSubList(ids,index,index);
                        ids = llListInsertList(ids,(list)id,index);
                        llOwnerSay(name + " was updated");
                    }
                }
            }
            integer len = llGetListLength(notes)-1;
            for(len; len >= 0; --len)
            {
                string name = llList2String(notes,len);
                integer type = llGetInventoryType(name);
                if(type == -1)
                {
                    llOwnerSay(name + " was removed.");
                    notes = llDeleteSubList(notes,len,len);
                    ids = llDeleteSubList(ids,len,len);
                }
            }                
        }
    }
                        
}

If a have something Like this in my Global:

//Storage Lists
list StoreStore = ["Store0","Store1","Store2","Store3","Store4","Store5"];
list Store0 = ["Store0_1","Store0_2","Store0_3","Store0_4","Store0_5"];
list Store0_1;
list Store0_2;
list Store0_3;
list Store0_4;
list Store0_5;

Can I cast list names to work like this:

state Send
{
state_entry()
{
integer indexMain=llListFindList(MasterStore,[NoteCardName]);
string send=llList2String(StoreStore,indexMain);
llMessageLinked( LINK_SET, 1, "Config", llGetKey());
llSleep(0.5);
llMessageLinked( LINK_SET, 161, llDumpList2String([send,0], ";", "" );
llMessageLinked( LINK_SET, 162, llDumpList2String([send,1], ";", "" );
llMessageLinked( LINK_SET, 163, llDumpList2String([send,2], ";", "" );
llMessageLinked( LINK_SET, 164, llDumpList2String([send,3], ";", "" );
llMessageLinked( LINK_SET, 165, llDumpList2String([send,4], ";", "" );
llMessageLinked( LINK_SET, 0, "DataEnd", "" );
}
}

So that

llMessageLinked( LINK_SET, 161, llDumpList2String([send,0], ";", "" );

will be sending the data from list Store0_1;

???

I decided to delve into Very Keynes VK-dbms-VM system. I had glanced at it initially and my first impression was that it was way too hard. But it seems to have everything I need. Just takes a little bit of reading and the examples really help.

I think vk's dbms would be a bit of overkill in this situation... I could be wrong, since I don't know the extent of the information you're saving/using

Ohs Well,

I got the vk-dbms to work great! The Database/Storage script I created to work in conjunction with my Main, Controller, and Reader scripts compiled with about 50k of freemem left.

Unfortunately, the data from "one" of my notecards is about 40k. No chance of using this to store multiple notecards to one script! I guess I'll stick with my previous version and read notecards into memory only when I'm ready to use that data.

Learned quite a few things LSL-wise so not a loss... lol

If one Note Card is 40K and the LSL total memory allocation in mono is only 60K you will never store more than one. But I am pleased you tried VK-DBMS
What I suggest you try, and the way I work with VK-DBMS where I need more storage, is to create a small table in the main script that acts as a system table that cross indexes the notecard name to the name of an external script containing that notecards actual data, also in the the form of a table. That way you can have hundreds of cards stored in scripts and a single index to them. I have beta code I was working on that can do a Union or Join across scripts, but found it needlessly overcomplicated most applications and that passing a query and reading a result via linkmessages worked reliably. you would need to use linkmessages, but from what you have said so far in the thread I get the impression that you would have no problems with that.

HIyas Very,

I do have my own system worked out but your system would work well for cross indexing to separate scripts. Unfortunately, I would need many many scripts (one for each notecard of data) for my project.

Hmmm.. just thinking a bit. Most of those scripts would be sitting idle most of the time. Even if the mem was maxed out in them, would it really matter if I had 40 or 50 "storage" scripts?

I do wish I found your system before I eeked out "my" menu system though... listing notecard names as buttons in alpha order from top to bottom with "Next" , "Option", and "Back" buttons and having all that adjust with change in inventory was an exercise in LSL... lol

If it was this much data and that many notecards, then the last place I would doing all of the heavy lifting would be in world. Why aren't you just uploading to a server and have the server parse the data and send back the menu lists, etc?

I know it's possible to do something like this. My main goal is to increase data transfer speeds within the system for what I have right now. Wouldn't I be adding more "wait" time by going off-world for data?

There is no harm in having 40 or 50 scripts in an object, I have some with even more than that and have them generated as needed at run time, so i have no idea how many are running in the larger databases. But the trick to avoiding server load is to have the indexing script only activate llSetScriptState(scriptname, Running) the slave script it requires and then turn it off when it is done.
Think of it as a file system with Open and Close commands and a file handle.
The added benefit of that, in mono, is that if the scripts are identical, it only ever loads one copy of the script and then uses context switching to place the correct data into it.

If I have one mono script but I change the name of it, doesn't that change the asset id? I tried testing this but all I get are zeroes for UUID... lol

That is a very good question. I had assumed (always a bad idea) that it worked on the CRC of the object code to determine if it's the same, rather than the script name or UUID. But you could well be correct.

If I keep dropping multiple copies of the same named mono script into a prim, it automatically gets renamed to Script 1, Script 2, Script 3, etc... Does the UUID (asset id) change for each iteration?

Does anyone have the definitive answer for this?

I reference the 4th paragraph under MONO Benefits: