DataStore script and functions (realtime storage of data with just a function)

A lot of people have askeed for a way to transfer data between scripts EASILY. Seems myself and a friend have figured out how:

I would like to point out I did not do this alone. I created this with help, and hope that this will be useful to EVERYONE. Not just myself.

Think of it like a VERY primitive database system.

This systems DOES have a few limits:

1- It must be able to edit the object description field. If you need to keep that unsullied, then you will have to pul all of these sections into a child prim in the object.

2- ALL of the script must be in the same prim, unless you can get the prim description from a prim other than the one it's in? I found nothing on this.

3- There is a delay, on average, of 0.1 second to use each of thse functions while communication takes place. Still much faster than notecard reading, or HTTP calls. This means it's great for storing data that is not needed over an over. If you have to keep track of the data ALOT, then store it locally and update as needed.

4- The system is untested with multiple scripts trying to access and edit the same data store. Simply, this would probably take a timestamp function to see when an item was last changed, and I MAY add it if it's needed.

5- I have NOT fully tested the memory needs to make sure the email function will not crash it with a memory overflow. If you run into that, then change the memory check to a higher number.

6- This is really an example that it's POSSIBLE to connect scripts in real time. This, in many cases, is not the best way to store data unless you need: A- to store a LOT of data, or B- to store data EASILY accessed by multiple scripts.

The functions are:

string ff_WriteData(string name, string data, integer nodenumber)
Either creates an entry with the name of 'name', and stores 'data' inside of it. The item 'nodenumber' is the number of the datastore, just as in all of the other functions.
RESPONSES:
'ADDED' - The entry was created from scratch.
'CHANGED' - The entry was updated with new information.
'LOW_MEM' - The write failed due to memory limits being reached.


string ff_GetData(string name, integer nodenumber)
Returns the data stored in entry 'name'.
RESPONSES:
'NOT_FOUND' - The item name requested was not found.
Otherwise, the data will be returned.


string ff_DeleteData(string name, integer nodenumber)
Deletes the entry 'name' and any data it contains.
RESPONSES:
'NOT_FOUND' - The item name requested was not found.
'DELETED' - The entry was found and deleted.

string ff_WipeData(integer nodenumber)
This one forces a total reset of the datastore.
RESPONSES:
'WIPED' - Data was wiped.

string ff_EmailData(string emailaddy, integer nodenumber)
Sends all data stored in it to the email address 'emailaddy'. This stops both the datastore and the sending script for 20 seconds.
RESPONSES:
'MAILED' - The data was sen to the email address.

All of these functions return a response


The following is the script for the datastore. You really don't have to edit anything but the 'nodenumber'. this should be differant for every datastore.



This next section of code shows the functions that allow you to access the datastore and how the functions work. It also acts as a speed test.

Hi Merry,

This is the version that I have been useing, looks like we are on a simmilar track.
I publish this one as it
a) wont use the name/description field
and
b) can expand the storage automaticaly up to the limit of scripts allowed in an object.

some notes on use:
create as many copies as needed in one prim and set the string inst = "0xf0" to the instance number for that copy, they will fill sequentaly from 0xf0 to 0x00 so the range can accomodate 16 instances (the last digit is the command) but could be expanded by using 0xf00 as the first instance thus 265 istances could be handeld with a moinor code edit.

The commands passed in the number field are also represented in hex:
0xfff1 = PUT
0xfff2 = GET
0xfff3 = DEL

The Field name of the data is passed in the ID field and the data is passed in the message field.

0xfff0 is the return value to identyfy the calling script

This is the simple implimentation and I include a test script to show how the interface works.

my main CPU script, that normaly utalises this data store, also includes code to write the storage contents to a SQL Database or send it offline as email and then flush or reset any slice to free up space, the data store is effectvly used as a cache by my systems.

Inter object communications are done using llRegionSay or email and are handled by the main CPU script.

hope it helps.

// VKF-Datastore
string inst = "0xf0";
integer FULL = FALSE;
list datafile = [];
integer recptr;

default
{
state_entry()
{
llOwnerSay("VFK-Datastore Slice "+inst+ " ready. "+(string)llGetFreeMemory());
}
link_message(integer sender_number, integer number, string message, key id)
{
if (number == (0xff00+(integer)inst+1)) // put - store
{
recptr = llListFindList(datafile,[id]);
if(~recptr)
{
datafile = llListReplaceList(datafile,[message],++recptr,recptr);
}
else
{
if(FULL)llMessageLinked(LINK_THIS,number - 0x10, message,id);
else datafile = [id]+[message]+datafile;
}
if(llGetFreeMemory() < 2560)FULL=TRUE;
}
else if (number == (0xff00+(integer)inst+2)) // get - retreave
{
recptr = llListFindList(datafile,[id]);
if(~recptr)llMessageLinked(LINK_THIS,0xfff0,llList2String(datafile,++recptr),id);
else{llMessageLinked(LINK_THIS,number - 0x10, message,id);}

}
else if (number == (0xff00+(integer)inst+3)) // del - delete
{
recptr = llListFindList(datafile,[id]);
if(~recptr)datafile= llDeleteSubList(datafile,recptr,++recptr);
}
else if (number == (0xff00+(integer)inst+4)) // fmt - delete
{
datafile = [];
}
}
}

and the test script, this is set up to use channel 0, which is not a good idea change it to a channel of your own choice.

//VFK-Datastore test

default
{
state_entry()
{
llListen(PUBLIC_CHANNEL,"",NULL_KEY,"";
}
listen(integer channel, string name, key id, string message)
{
list data = llParseStringKeepNulls(message,[" "],[""]);
string cmd = llList2String(data,0);
if(cmd == "put"
{
llMessageLinked(LINK_THIS,0xfff1,llDumpList2String(llList2List(data,2,-1)," ",llList2String(data,1));
}
if(cmd == "get"
{
llMessageLinked(LINK_THIS,0xfff2,"",llList2String(data,1));
}
if(cmd == "del"
{
llMessageLinked(LINK_THIS,0xfff3,"",llList2String(data,1));
}
}
link_message(integer sender_number, integer number, string message, key id)
{
if(number == 0xfff0)llOwnerSay(message);
}
}

True, but the point of the system displayed is to have a datastore accessable with ONLY a function call. No need to listen with link_message events or other round about kludges. It's effectivly invisible to the designer's progam code, so long as you don't play with the description field for the prim.

Your design is the style I used before we figured out this one. We just wanted something to allow script 2 scripts communication in real time.

Either way, there are also a LOT of times to use one design or the other at times. It just depends on what you need.

Can you explain why you consider link messages to be a round about kludge? I use them extensively to communicate between scripts and have had no issues doing so. I guess I'm curious what the advantage of your system is over Link messages.

Simple. Mine does not need the script accessing to datastore to be built around the idea of getting responses back. Normally, you have to use a linkmessage like the http call or notecard reading calls.

Mine just uses a functon to ask for data, and get's it back. Simple.

As to why I think of it as a kludge, I programmed for years, and am used to being able to use functions to access outside data. PHP has functions to pull data like the design shown above. So do most programming languages. I just figured out how to make LSL follow the designs I'm used to.

And, if you want to know the truth? I think MOST of LSL is a kludge.

So... that means that you have to include your function(s) in every script which would need access to read/write the data, correct?

So, the trade-off is the extra script space for redundant code so you can have "quicker" access, right?

Plus, how much data can you really store in an object description field?

Interesting stuff...

Has anybody looked at the cost of using this over using link messages? I'd think that a description change would cause an update to, though probably not a full update, to be sent to clients.

Also, does anybody know if changing the description cause an asset update?

Good question. As far as I know, you don't get description info unless you hover the curser over it or edit it.

As to the benifits? You can have as many datastore scripts as you need, so it scales easilly as ou need more data space and add more datastores.

Not to mention, several scripts can read from the datastore as well.

Show Updates doesn't show anything when a description changes.

Thanks for the reply. I agree with your assessment of LSL, it could be better.

The wiki states 127 characters for the Description field. This may or may not still be valid. As in my application I'm dealing with strings well over 1024 bytes (don't ask what), this probably wouldn't work for me. But for some applications I'm sure it would be beneficial. Thanks for sharing

see here: http://wiki.secondlife.com/wiki/Prim_Attribute_Overloading

Temporary Extra Memory Storage in Running Scripts

At the current time, you can hold up to 8034 characters in a prim's description while a script is running. Once the script ends, whatever data is written into the description will be truncated to 127 - 255 characters. While this is not stable for long-term data storage like it used to be, at the present time it can be a useful trick for dealing with larger amounts of data in a script.

The following scenario is one that demonstrates this effect:

Increment a string up to 7127 characters and then write that to the prim desc using llSetObjectDesc().
Clear the variable, desc_data = "",
Read the data back into a different variable, string tmp; tmp = llGetObjectDesc();
Show the length of tmp, it shows 7127,


Test this with a timer(has been proven up to 4 minutes at least, then clear tmp = "";

Read in the data again using a 3rd variable, string new; new = llGetObjectDesc(); then it shows the same length, 7127.

Now, comment out the code that writes the initial data, and just read the description data with llGetObjectDesc(), it only comes back 255 as the string length,

So it seems, as long as you write and read the data within the same occurrence of the script, its fine. But if you try and read the data back in a different script iteration, its limited to 255.

The assumption is that when a script is finished running, the state of the prim description and name are being updated, cutting it off at 127 to 255 characters. --Geuis Dassin 6/16/07

I've also discovered (the hard way) that when using the description as a datastore, if the object is taken back into inventory and then rezzed again, the description (and name) fields are both truncated to 64 characters.

I understood the benefits - like I said, it's interesting stuff. Was just saying you've got to consider the costs, too.

I know llSetText used to cause a full object update. If changing the description also did that, having these kind of scripts in a few complex objects could do bad things to sim performance. Sindy verified that changing the description doesn't cause any update, not even a partial one, so that's no longer a concern.

I'm still curious if anybody knows if this kind of script hits the asset server. /me wonders if one of the open source people around have any insight to that..

Well, the text above points to the idea that it does not update until the scripts stop for some reason.
Also, I can honestly say: I see nothing in the sim stats to say that several of these running are doing much to the sim.

I guess that we should see if any of the other mad-scientist types reading this thread find a major glitch. I certainly hope not, but would rather know if that's the case.

Thats interesting, but I'm not sure I understand what is meant by "different script iteration", and talking about the script "ending". in LSL there is no end, scripts are always in one or other event handler. Would "ending" mean reaching a point where no events are handled? And by "different script iteration", do they mean you can't pass values between scripts? The terminology has me confused. I need to pass the long strings from one script to another within the same object.

I'm guessing it means it's NOT going to shorten until it stops the scripts by editing the item in some way.
Hard to say, really. How about trying it with long strings and seeif anything makes it break?

Hi!! noob intrusion...

i'm trying to use Very's modular datastore script, but there's something fundamental about these things i'm just not getting...

I want to manage very long lists, that would overflow between banks.. do i have to write some kind of function that imitates llListFindList and similar for dealing with the stored data? I mean i wind up with strided lists of reference/data in a single list in each script...

I assume the answer is yes, but i'm lost as to how to pull it off *sigh*

Hi Chrysala,

It is not intended to store lists as such, rather it stores variables within a list.
It should have the same effect as you want, however, without you having to define a list find list function.

If it is a list of names and IDKeys for example

use Put Name,IDKey each time you you want to add a name to the list and Get Name to reteave the maching IDKey. The orrigonal scripts will do the searching and spanining for you.

Hi!

Thanks, yes, I did get that it's for grabbing single elements ... my problem was that I am using it for something (menu driven warpoPos transport) that uses list for access and dialog menu choices (creates button list, integers, coupled to destination list position for choices. extrapolates description list and final choice from same source -- lists at the moment, in 3 scripts, 1 for each type of access. Yes, redundant info and I'm looking in to how to abolish it with this and other ideas).

I suppose making my adress strings be ListnameElementnumber, keeping an integer on list length when I put the info, and using a for loop will take care of ListFindList for access list. now just fighting off some bad laziness to get started on what i think SHOULD be simple.

Only doubt I have now.... when i'm done will it be faster than Dataserver or will I "iz be teh fail" and have stressed for nothing?

As of Feb 1st 2008, this method may prove unreliable due to the implementation of SVC-674

http://blog.secondlife.com/2008/02/01/fix-may-cause-problems-for-scripts-relying-on-long-object-names-and-descriptions/

There was a point in time when setting a prim/object name or description to over 255 characters could or would corrupt the object to the point where it could not be rezzed from inventory. I bug reported this back in the Summer of 2006. PROBABLY that's been fixed by now. LOL.

Anyway, as there is only one (or a fixed number if you have multiple prims) of descriptions, I think you'll find that resource contention (if nothing else) is going to raise a great number of concurrency issues that you can easily avoid with an event-based query/response link message implementation.

And yes, you can now get the description for a prim other than the one the script is in. Try 'llGetObjectDetails(..., [ OBJECT_DESC ])'

Well, I was using this for months. Somehow, I seemed to have missed all the info on this being changed until I started noticing several of my devices failing to function.

I would advise avoidance of this entire codebase, since there is no way to know what will be changed in the future.

I do have to say, I am VERY disappointed at losing weeks of work with one "Fix" by Linden Labs. That's all that has to be said on the matter, actually.

Anyway, THIS is an example of why scripters are so cranky. We keep making the hidden gears of SL turning, and they keep breaking our systems and making us do the same things over in a different way.

Just to followup on Darien's post above to be very very clear: the fix to SVC-674 caused the following.

As of 2/1/2008, all object descriptions are limited to 127 characters; data over that will be truncated.

Object names are limited to 63 characters.

The wiki page cited in post 11 has already been changed to reflect this.
.