Welcome to the Second Life Forums Archive

These forums are CLOSED. Please visit the new forums HERE

Library: VK-dbms-VM

Very Keynes
LSL is a Virus
Join date: 6 May 2006
Posts: 484
10-31-2008 11:57
Data Storage and retrieval has long been an issue with LSL and many people including myself have developed somewhat convoluted methods over the years of overcoming or at least alleviating the problem by means of lists and link messages.

The advent of MONO on the main grid has opened up the possibilities considerably by enhancing both the performance and memory available to LSL scripts. Whilst I have posted data storage scripts in the past, they were often difficult to use, especially for new LSL scripters so I have taken advantage of the new MONO capability's to combine my data storage scripts into a form that is functional, logical and accessible to scripters of all levels.

VK-DBMS is a Multi-Table database management system written for and in LSL, it has no dependencies on external databases and is totally embedded in the users script. The embedded code is called VK-DBMS-VM and will compile to LSL with 11920 bytes free and to MONO with 46324 Bytes free, the latter is the obvious choice for data storage but it still may prove useful as a applications environment in the former.

Although not fully optimised I have decided to release the code to the community at large in the hope that beginners will benefit from the relative ease of storing and retrieving data and that advanced scripters will improve on my work or indeed find new uses for it as my current beta testers have done.

Living Documentation is posted at:



CODE

//Very Keynes - 2008
//
// 2008-10-31, 22:24 GMT
// Release Candidate
//
// For usage information and examples please see
// http://docs.google.com/Doc?id=d79kx35_26df2pbbd8
//
//------------------Begin VK-dbms-VM----------------------------\\
//--------------------Introduction------------------------------\\
//
// Very Keynes - db - Virtual Machine
//
// Implements a core set of registers and root functions
// to create and manage multi-table database structures as
// an LSL list. Although intended to under pin higher level
// database management tools such as VK-SQL it is usable as
// a small footprint database facility for system level
// applications.
//
//
// Naming Conventions and Code Style
//
// This Code is intended to be included as a header to user generated
// code. As such it's naming convention was selected so that it would
// minimise the possibility of duplicate names in the user code portion
// of the application.
//
// Licencing
//
// The concept and code are hereby released to the public domain on an honer system.
// If expanded upon, or altered in any positive way, please post the update,
// else use it in any way you wish for non commercial applications. If used in a
// commercial application then please be mindful of the time spent to develop this system
// and contact the author about a distribution agreement :)
//
// Table Control Registers
//
integer th_; // Table Handle / Index Pointer
integer tc_; // Columns in Active Table
integer tr_; // Rows in Active Table
integer ts_; // Active Table Start Address
//
list _d_ = []; // Database File
list _i_ = [0]; // Index File
//
// Exposed Variables
//
integer dbIndex; // Active Row Table Pointer
list dbRow; // User Scratch List
string dbError; // System Error String
//
// Temporary / Working Variables
//
integer t_i;
string t_s;
float t_f;
list t_l;
//
// System Functions
//
// string dbCreate( TableName, ColumnList )
//
// Creates a new Table in the database
//
string dbCreate(string tab, list col)
{
if(dbOpen(tab)) // Integrity test
{
dbError = tab + " already exists";
return "";
}
// Create a new index record for the table
_i_ += [tab, tc_ = llGetListLength(col), 0, 0, 0];
th_= 0; // release current handle
dbOpen(tab); // Activate the new table
dbInsert(col); // insert the header record
return tab; // Return the alis
}
//
// Helper function returns Col# for Column Name
//
integer dbCol(string col){return llListFindList(dbGet(0), [_trm(col)]);}
//
// integer dbDelete( Row# )
//
// Deletes the specified Row of the Active Table
//
integer dbDelete(integer ptr)
{
if(ptr > 0 && ptr < tr_) // Bounds test
{
_d_ = llDeleteSubList(_d_, t_i = ts_ + tc_ * ptr, t_i + tc_ - 1);
--tr_;
return tr_ - 1;
}
else {
dbError = (string)ptr+" is outside the Table Bounds";
return FALSE;}
}
//
// integer dbDrop( TableName )
//
// Complement to dbCreate - Removes a Table and its data
//
integer dbDrop(string tab)
{
if(-1 != (t_i = llListFindList(_i_, [tab])))
{ dbOpen(tab);
_d_ = llDeleteSubList(_d_, ts_, ts_ + tc_ * tr_ - 1);
_i_ = llDeleteSubList(_i_, th_, th_ + 4);th_= 0;
return TRUE; }
else {dbError = tab + " : Table name not recognised"; return FALSE;}
}
//
// integer dbExists( condition list )
//
// searches from last row towards row 1 stopping and returning the first match
//
integer dbExists(list cnd)
{ for(dbIndex = tr_ ; --dbIndex > 0 ; ){if(dbTest(cnd))return dbIndex;}
return FALSE;}
//
// list dbGet( row# )
//
// returns the specified row
//
list dbGet(integer ptr)
{
if(ptr < tr_ && ptr >= 0)return llList2List(_d_, t_i = ts_ + tc_ * ptr, t_i + tc_ - 1);
else dbError = (string)ptr+" is outside the Table Bounds";return [];
}
//
// Internal function to calculate a tables start address
integer _idx(integer hdl){return (integer)llListStatistics(6, llList2ListStrided(_i_, 0, hdl, 5));}
//
// integer dbInsert( list Row Data )
//
// Creates a new row at the end of the table, returns rumber of rows
//
integer dbInsert(list val)
{
if(llGetListLength(val) == tc_){_d_ = llListInsertList(_d_, val, ts_ + tc_ * (dbIndex = tr_++));return dbIndex;}
else{dbError = "Insert Failed - too many or too few Columns specified"; return FALSE;}
}
//
// integer dbOpen ( Table Name )
//
// saves current registers and opens a table if it exists
// returns Number of rows in opened table
//
integer dbOpen(string tab)
{
if(th_)_i_ = llListReplaceList(_i_, [tr_, dbIndex, tc_ * tr_], th_ + 2, th_ + 4);
if(-1 == (t_i = llListFindList(_i_, [tab]))) //if tab does not exist abort
{dbError = tab + " : Table name not recognised"; return FALSE;}
else if(th_ != t_i){th_ = t_i++;ts_ = _idx(th_);
tc_ = llList2Integer(_i_, t_i++);
tr_ = llList2Integer(_i_, t_i++);
dbIndex = llList2Integer(_i_, t_i);}
;return tr_ - 1;
}
//
// integer dbPut( list Row Data )
//
// Replaces the currently Active Row of the Table with the new data
integer dbPut(list val){if(llGetListLength(val) == tc_)
{
_d_ = llListReplaceList(_d_, val, t_i = ts_ + tc_ * dbIndex, t_i + tc_ - 1); return dbIndex;}
else {dbError = "Update Failed - too many or too few Columns specified"; return FALSE;}
}
//
// integer dbTest( condition list )
//
// tests condition against current row, returns true or false
//
integer dbTest(list cnd)
{
if(llGetListEntryType(cnd,2) >= 3){
t_s = llList2String(dbGet(dbIndex), dbCol(llList2String(cnd, 0)));
if ("==" == llList2String(cnd, 1)){t_i = t_s == _trm(llList2String(cnd, 2));}
else if("!=" == llList2String(cnd, 1)){t_i = t_s != _trm(llList2String(cnd, 2));}
else if("~=" == llList2String(cnd, 1))
{t_i = !(llSubStringIndex(llToLower(t_s), llToLower(_trm(llList2String(cnd, 2)))));}}
else{
t_f = llList2Float(dbGet(dbIndex), dbCol(llList2String(cnd, 0)));
t_s = llList2String(cnd, 1);
if ("==" == t_s){t_i = t_f == llList2Float(cnd, 2);}
else if("!=" == t_s){t_i = t_f != llList2Float(cnd, 2);}
else if("<=" == t_s){t_i = t_f <= llList2Float(cnd, 2);}
else if(">=" == t_s){t_i = t_f >= llList2Float(cnd, 2);}
else if("<" == t_s){t_i = t_f < llList2Float(cnd, 2);}
else if(">" == t_s){t_i = t_f > llList2Float(cnd, 2);}}
if(t_i)return dbIndex; else return FALSE;
}
// Internal function encapsulates llStringTrim
string _trm(string val){return llStringTrim(val, STRING_TRIM);}
//
// dbTruncate ( table Name )
//
// removes all rows from a table but leaves the header intact
//
dbTruncate(string tab){dbIndex = dbOpen(tab);while(dbIndex > 0)dbDelete(dbIndex--);}
//
// dbSort ( integer Order )
//
// permanently sorts a table by the first column, ascending or descending
//
dbSort(integer dir){_d_ = llListReplaceList(_d_,llListSort(llList2List
(_d_, t_i = ts_ + tc_, t_i + tc_ * tr_ - 2), tc_, dir), t_i, t_i + tc_ * tr_ - 2);
}
//
// dbFn( Function, Column )
//
// Preforms Statistical functions on the Column Specified
//
float dbFn(string fn, string col)
{
t_l = llList2List(_d_, t_i = ts_ + tc_, t_i + tc_ * tr_ - 2);
if(dbCol(col) != 0) t_l = llDeleteSubList(t_l, 0, dbCol(col) - 1);
return llListStatistics(llSubStringIndex("ramimaavmedesusqcoge", llGetSubString(llToLower(fn),0,1)) / 2,
llList2ListStrided(t_l, 0, -1, tc_));
}
//
//--------------------------- End VK-dbms-VM ---------------------------\\
//


Summary of commands in alphabetical order:

string dbCreate ( string TableName, list ColumnNames );

integer dbDelete ( integer Row# );

integer dbDrop ( string TableName );

integer dbExists ( list Condition );

float dbFn ( string Function, string ColumnName );

list dbGet ( integer Row# );

integer dbInsert ( list Values );

integer dbOpen ( string TableName );

integer dbPut ( list Values );

void dbSort( integer Direction );

integer dbTest ( list condition );

integer dbTruncate ( string TableName );

Helpers:

integer dbCol( string ColumnName );
string dbError - Description of last Error encountered
integer dbIndex - Table Index pointer
list dbRow - user variable
Pavcules Superior
Registered User
Join date: 30 Oct 2006
Posts: 69
11-01-2008 07:42
Hi all,

As one of the Beta testers of Very's DB work, I would like to say a few points about the hard work Very has put into this, and how it can make programming in LSL easier.

The DB code could be used in a variety of products, ranging from Tip Jars, Visitor trackers, Voting systems, Competiton Boards, etc. Basically if you want a list of things to report or maintain in-world, the DB code will definitely help you in many ways.

I have recently released a new Xmas Tree product which is using Very DB code to maintain a list of people who have touched the tree. When I wrote last year's Xmas Tree, the tracking feature I created, I had to manipulate a list in updating figures and names. After trying out Very code, I was able to code this feature much more faster than before. There are loads of functions that quickly aided the development of this feature. Once I understood how it worked, my code was much more simpler than before.

For those interested, the product can be found at the link below As this is a commerical product, an agreement has been reached with Very.
http://www.xstreetsl.com/modules.php?name=Marketplace&file=item&ItemID=992326

After understanding and using Very DB code, I had further ideas on how this could be extended on. I have developed two new concepts that are not present in the LSL language. The concepts are currently being refined, but once they are released to the public domain, I would like to see how they could be exteneded or refined upon.

Personally I would like to see more feedback on Very work, because there is big opportunities with this code here. Maybe not everyone will see it first, but it may click in later once you have used it.

Very, you did a very fine piece of work, lets hope people appreciate it. :)
Zerlinda Boucher
Registered User
Join date: 11 Oct 2008
Posts: 23
12-06-2008 23:35
i try to compile but i receive and error after this:

float dbFn(string fn, string col)
{
t_l = llList2List(_d_, t_i = ts_ + tc_, t_i + tc_ * tr_ - 2);
if(dbCol(col) != 0) t_l = llDeleteSubList(t_l, 0, dbCol(col) - 1);
return llListStatistics(llSubStringIndex("ramimaavmedesusqcoge", llGetSubString(llToLower(fn),0,1)) / 2,llList2ListStrided(t_l, 0, -1, tc_));
}
Very Keynes
LSL is a Virus
Join date: 6 May 2006
Posts: 484
12-07-2008 01:43
Remember it is not an entire script, but rather a function.
You will still need a default state and at least one event in order for the code to compile inworld.
Zerlinda Boucher
Registered User
Join date: 11 Oct 2008
Posts: 23
12-07-2008 02:35
From: Very Keynes
Remember it is not an entire script, but rather a function.
You will still need a default state and at least one event in order for the code to compile inworld.


ops my fault :)

xoxo
Zer
Zerlinda Boucher
Registered User
Join date: 11 Oct 2008
Posts: 23
12-08-2008 09:39
the script and db system works very good.. there is any way to expand the storage capacity?

ty
Zer
Very Keynes
LSL is a Virus
Join date: 6 May 2006
Posts: 484
12-08-2008 11:38
Yes and No.
In the form that it is published it relies on the total memory allocated to a script in the LSL VM, or MONO. so within a single script the only way to maximize storage is to minimize the support code. However you can use it as a building block with several instances running as storage scripts, one per Table, and one instance acting as a control table that points to the script containing the actual data.
I have done some work in that direction but it becomes very application specific and forces the use of link messages and some rather borderline synchronization hacks that use prim property's in ways that nature (or LL) never intended.
Talarus Luan
Ancient Archaean Dragon
Join date: 18 Mar 2006
Posts: 4,831
12-08-2008 14:54
The version of script dbms I made a while back, adding storage capacity is about as easy as dragging another copy of the storage script into the object. It mostly self-configures, but it still requires a tiny bit of manual massaging, mainly in the way the script is named.

However, mine doesn't implement high-level functionality like this, pretty much just "key=data" pairs with simple add, get, replace, and delete functionality.
Zerlinda Boucher
Registered User
Join date: 11 Oct 2008
Posts: 23
12-08-2008 16:14
From: Talarus Luan
The version of script dbms I made a while back, adding storage capacity is about as easy as dragging another copy of the storage script into the object. It mostly self-configures, but it still requires a tiny bit of manual massaging, mainly in the way the script is named.

However, mine doesn't implement high-level functionality like this, pretty much just "key=data" pairs with simple add, get, replace, and delete functionality.


Have the link so i can take a look?
I'm a script novice i try to undestand how i can work with it :)

ty
Zer
Talarus Luan
Ancient Archaean Dragon
Join date: 18 Mar 2006
Posts: 4,831
12-08-2008 20:57
From: Zerlinda Boucher
Have the link so i can take a look?
I'm a script novice i try to undestand how i can work with it :)

ty
Zer


Unfortunately, it's not published anywhere, as I only use it in a few projects, but I am happy to answer questions about the subject. :) Feel free to contact me in-world.
Zerlinda Boucher
Registered User
Join date: 11 Oct 2008
Posts: 23
12-10-2008 20:31
i still testing the database with the guest script u put as sample...

this part of sensor have the peak calculation:

sensor(integer total_number)
{
integer i;
integer count;
dbOpen("guests";);
for (i = 0; i < total_number; i++)
{
if(dbExists(["name", "==", llDetectedName(i)])){
dbRow = dbGet(dbIndex); // if guest exists increment duration
dbPut([llList2String(dbRow, 0),llList2Integer(dbRow, 1) + 1]);}
else dbInsert([llDetectedName(i), 1]); // else add guest to table
}
if(scan = !scan)// if this is the first scan rotate and rescan
{
count = total_number;
llSetRot(llEuler2Rot( <0, 0, 180.0 * DEG_TO_RAD> ) * llGetRot());
llSensor("", NULL_KEY, AGENT, 25.0, PI_BY_TWO);
}
if(scan)// if this is the second scan calculate the peak
{
i = count + total_number;
if( i > peak ){
peak = i;
peaktime = llList2String(time, 1) +":"+ llList2String(time, 2);}
}
}

But thre is something strange, so i put the script with only my avatar in rabge for more then 2 hours and i receive the email:

2008-12-09,08:00 < 1 | 2 @ 07:06 | 1 | 1 >

date,time < 1 avatar detected | peak 2 avatar??? @ peak time | max | med

so with 1 avatar is not possible the peak is 2 right? :)
Don't know the utility to rotate and rescan... whi not use the 360° feature on sensor?
sorry for the question still learn :)

Zer
Very Keynes
LSL is a Virus
Join date: 6 May 2006
Posts: 484
12-12-2008 06:12
Hi Zar,

Yes, there is something strange with the peek calculation and I have not managed to pin it down, I suspect it is something to do with the scan function but only occurs when only one avatar is present.

The reason to rotate and scan is that llSensor() can only detect the closest 16 avatars and in most situations, where you need to count visitors, 16 is too few. By scanning twice at 180 degrees you can detect up to 32 visitors. My production version scans 4 times at 90 degrees, but it introduces some very complex rotation mathematics in order to make 4 scans appear as a hemisphere as the prim has to move in all 3 dimensions, so I opted to post the simpler example code as It will be useful to most small club/shop owners but not over complicate the example script.
Xeaustin Twine
Registered User
Join date: 31 Jul 2008
Posts: 15
that code doesn`t work
12-18-2008 01:36
Hi,

Am very interest to connect database with SL.But am new to LSL.If I copy that code in simple cube in sl which you gave.it does not work..would you pls., tell me the procedure to get work that code..
Thanks in advance.
Pavcules Superior
Registered User
Join date: 30 Oct 2006
Posts: 69
12-18-2008 05:44
Hi Xeaustin,

I have sent you in-world Very's DB script that compiles. As you are new to LSL, the code in the first post does work, but it needs to copied and pasted above the 'default' state in a new script.

---------------------------------
< Very's DB Code >

default
{
state_entry()
{

}

}
----------------------------------

You will need to look at the documentation for examples in using the DB code.
Xeaustin Twine
Registered User
Join date: 31 Jul 2008
Posts: 15
I have recieved your script..but still stuck!!
12-19-2008 03:45
Hi,
first of all thanks for your valuable help..Am very new sl?so,I didn`t know anything about scripting.So,what function I have to call it in inside of the state_entry funtion..tell me clearly...
Thanks in advance.
Rolig Loon
Not as dumb as I look
Join date: 22 Mar 2007
Posts: 2,482
12-19-2008 07:25
Everything in the code block labelled

CODE

default
{
state_entry()
{

}

}


is your own application. It depends on what you want to use the database functions for. Very's DB Code is a collection of functions that you call from within whatever you write in that code block. As Pavcules Superior suggested, look at the examples posted at http://docs.google.com/View?docid=d79kx35_26df2pbbd8. They are very clear.
Pavcules Superior
Registered User
Join date: 30 Oct 2006
Posts: 69
01-18-2009 14:41
For people who use Very DBMS system, here is additional code that extends the system further. Arrays and collections are another way of storing/retrieving items in memory, each having their pros and cons.

Arrays:
/54/81/303075/1.html

Collections:
/54/77/291106/1.html

OO / Classes:
/54/a7/304708/1.html

Hope it helps. :)
Pavcules Superior
Registered User
Join date: 30 Oct 2006
Posts: 69
02-27-2009 15:04
For anyone interested in obtaining live copies of the VK-DBMS-VM library code and its examples, you can obtain them for free by visiting PK Pounceworks at:

http://slurl.com/secondlife/Halfmoon/130/207/138/

The code samples are on the top floor.