Welcome to the Second Life Forums Archive

These forums are CLOSED. Please visit the new forums HERE

Discussion: Safe list-to-string conversion script

Keknehv Psaltery
Hacker
Join date: 11 Apr 2005
Posts: 1,185
01-12-2006 16:22
Hello everyone, I made this pair of list-to-string and string-to-list functions. I thought they might be interesting mainly because the way the string is set up, it will always split into the same number of pieces -- it doesn't use separators, and it retains the types of the original list elements.

Well, have a look, may someone find it useful!

CODE

// list2string and string2list functions...
// For the safest transfers of lists with strings

// By Keknehv Psaltery, 1/12/06 -- Public domain, do whatever you wish (Although I'd prefer if I still get credit)
// Revised By Christopher Omega, 1/28/06
// Revised By Strife Onizuka 1/28/06

// Updated so it can perserve floats, overhead 900 instructions per float conversion also optimized some of it's code.

//How it works:
// Instead of using something like llParseString2List with an uncommon separator, these list-strings
// actually have an index of where the strings are, and, as a bonus, it also preserves data types
// The data structure of the string looks like this:
// [index1][type1] [index2][type2] [index3][type3]...[indexN][typeN] [data1][data2][data3]

// So, if you had a list like this: [The, 1, <1, 5.3, 6>, dog], it would look like
// this (without quotation marks) when it passes through the list2string function:
// "00163001910020500503The1<1.000000, 5.300000, 6.000000>dog"
// ====^====^====^====^---_------------------------------___
// ind1|ind2|ind3|ind4|data1 data3 data4
// typ1 typ2 typ3 typ4 data2
// This formatting allows for **any** conceivable data to be transferred.

string hex = "0123456789ABCDEF";//it's faster this way *shurg*

string list2String( list inList ) //Converts a list to a string -- with no possible chance of error
{
string outputString;
integer listLength = llGetListLength( inList );
integer offset = listLength * 5;
string prefixString;
string part;
integer type;

integer listElementNum = -listLength;
for (; listElementNum; ++listElementNum )
{
type = llGetListEntryType( inList, listElementNum );
if(type == TYPE_FLOAT)
part = Float2Hex(llList2Float( inList, listElementNum ));
else if(type == TYPE_ROTATION)
part = Rot2Hex(llList2Rot( inList, listElementNum ));
else if(type == TYPE_VECTOR)
part = Vec2Hex(llList2Vector( inList, listElementNum ));
else
part = llList2String( inList, listElementNum );
prefixString += llGetSubString("0000" + (string)offset, -4, -1) + (string)type;
offset += llStringLength( part );
outputString += part;
}

outputString = prefixString + outputString;
return outputString;
}

list string2List( string inString )
{
list outputList;
integer elementStart = (integer)llGetSubString( inString, 0, 3 );
integer listLength = elementStart / 5;

integer listElementNum = 0; //your wasting bytecode by not doing this here (as in relaity if you dont' specify a value the compile does for you
integer ti; //save a few bytes this way
string ts;
while(listElementNum < listLength)
{
//sure it's ugly but its fast and effeciant. trimmed maybe 200 byte off the bytecode, and it isn't too ugly really
integer elementType = (integer)llGetSubString( inString, ti = (listElementNum * 5 + 4), ti);
if ( ++listElementNum < listLength ) // Strife: I agree with Chris, might as well put it to some use.
ts = llGetSubString( inString, elementStart, (elementStart = (integer)llGetSubString(inString, ti + 1, ti + 4)) - 1 );
else // Chris: I dont like checking for this every iteration :-(
ts = llGetSubString( inString, elementStart, -1 );

if ( elementType == TYPE_INTEGER ) //1
outputList += (integer)ts;
else if ( elementType == TYPE_FLOAT ) //2
outputList += (float)ts;
else if ( elementType == TYPE_STRING ) //3
outputList += ts;
else if ( elementType == TYPE_KEY ) //4
outputList += (key)ts;
else if ( elementType == TYPE_VECTOR ) //5
outputList += (vector)ts;
else if ( elementType == TYPE_ROTATION ) //6
outputList += (rotation)ts;
}
return outputList;
}

string Rot2Hex(rotation a)
{
return "<"+Float2Hex(a.x)+","+Float2Hex(a.y)+","+Float2Hex(a.z)+","+Float2Hex(a.s)+">";
}

string Vec2Hex(vector a)
{
return "<"+Float2Hex(a.x)+","+Float2Hex(a.y)+","+Float2Hex(a.z)+">";
}

string Float2Hex(float a)
{// Copyright Strife Onizuka, 2006, LGPL, http://www.gnu.org/copyleft/lesser.html
if(a != 0)
{
float b = llFabs(a);
string f = "";
integer c = llFloor(llLog(b) / 0.69314718055994530941723212145818);//floor(log2(b))
integer d = (integer)((b / llPow(2,c)) * 0x1000000);//shift up into integer range
c -= 24;//the extra c used to make it an integer
while(!(d & 0xf))
{//strip extra zeros off before converting or they break "p"
d = d >> 4;
c+=4;
}
do
f = llGetSubString(hex,15&d,15&d) + f;
while(d = d >> 4);
if(a < 0)
return "-0x" + f + "p"+(string)c;
return "0x" + f + "p"+(string)c;
}
return "0";//zero would screw up the log.
}

default
{
state_entry()
{
list testList = [ "The", "red", "dog" ];
llOwnerSay( "testList = [" + llDumpList2String( testList, ", " ) + "]" );
string testList2String = list2String( testList );
llOwnerSay( "testList2String = \"" + testList2String + "\"" );
list reversedL2S = string2List( testList2String );
llOwnerSay( "Reversed list = [" + llDumpList2String( reversedL2S, ", " ) + "]" );
}
}
Nada Epoch
The Librarian
Join date: 4 Nov 2002
Posts: 1,423
Original Thread
01-14-2006 07:20
/15/e7/82185/1.html
_____________________
i've got nothing. ;)
Christopher Omega
Oxymoron
Join date: 28 Mar 2003
Posts: 1,828
01-28-2006 15:42
Whoa, has this remained flawed for so long? Kek, your test case is telling you the code works when it doesnt; you dont print reversedL2S after you compute it. There's a minor flaw in list2String, where you compute baseOffset (should be * 5, not * 4), and major flaws in the string2List loop - the protocol doesn't store lengths in the prepended metadata, only offsets. To use lengths you'd need to do the end - start computation somewhere else.

Here's a correction that should work:
CODE

// list2string and string2list functions...
// For the safest transfers of lists with strings

// By Keknehv Psaltery, 1/12/06 -- Public domain, do whatever you wish (Although I'd prefer if I still get credit)
// Revised By Christopher Omega, 1/28/06

//Known limitations -- It doesn't preserve a small amount of the floating point data.
//Of course, there is the memory limitation. (I could do some float trimming, but that would make it much slower)

//How it works:
// Instead of using something like llParseString2List with an uncommon separator, these list-strings
// actually have an index of where the strings are, and, as a bonus, it also preserves data types
// The data structure of the string looks like this:
// [index1][type1] [index2][type2] [index3][type3]...[indexN][typeN] [data1][data2][data3]

// So, if you had a list like this: [The, 1, <1, 5.3, 6>, dog], it would look like
// this (without quotation marks) when it passes through the list2string function:
// "00163001910020500503The1<1.000000, 5.300000, 6.000000>dog"
// ====^====^====^====^---_------------------------------___
// ind1|ind2|ind3|ind4|data1 data3 data4
// typ1 typ2 typ3 typ4 data2
// This formatting allows for **any** conceivable data to be transferred.

string padToFour( string inString ) //Outputs the string padded to four characters with spaces on the left
{ //( or cut to four characters -- pad("01234")="1234")
//Note: this function uses a clever trick for efficiency
integer inStringLength = llStringLength( inString ); //The length of the input string
if ( inStringLength < 4 )
return ( llGetSubString( "0000", 0, 3 - inStringLength ) + inString );
else if ( inStringLength == 4 )
return inString;
else
return llGetSubString( inString, inStringLength - 4, inStringLength );
}

string list2String( list inList ) //Converts a list to a string -- with no possible chance of error
{
string outputString = llDumpList2String( inList, "" );
integer listLength = llGetListLength( inList );
integer baseOffset = listLength * 5;
integer lastOffset;
string prefixString;

integer listElementNum;
for ( listElementNum = 0 ; listElementNum < listLength ; ++listElementNum )
{
prefixString += padToFour( (string)( baseOffset + lastOffset ) )
+ (string)llGetListEntryType( inList, listElementNum );
lastOffset += llStringLength( llList2String( inList, listElementNum ) );
}

outputString = prefixString + outputString;
return outputString;
}

list string2List( string inString )
{
list outputList;
integer baseOffset = (integer)llGetSubString( inString, 0, 3 );
integer listLength = baseOffset / 5;
integer elementStart = baseOffset;
integer elementEnd;
integer elementType;

integer listElementNum;
for ( listElementNum = 0 ; listElementNum < listLength ; ++listElementNum )
{
elementType = (integer)llGetSubString( inString, listElementNum * 5 + 4, listElementNum * 5 + 4 );
if ( listElementNum + 1 < listLength )
elementEnd = (integer)llGetSubString(inString, (listElementNum + 1) * 5, (listElementNum + 1) * 5 + 3) - 1;
else // Chris: I dont like checking for this every iteration :-(
elementEnd = -1;

if ( elementType == 1 )
outputList += [ (integer)llGetSubString( inString, elementStart, elementEnd ) ];
else if ( elementType == 2 )
outputList += [ (float)llGetSubString( inString, elementStart, elementEnd ) ];
else if ( elementType == 3 )
outputList += [ llGetSubString( inString, elementStart, elementEnd ) ];
else if ( elementType == 4 )
outputList += [ (key)llGetSubString( inString, elementStart, elementEnd ) ];
else if ( elementType == 5 )
outputList += [ (vector)llGetSubString( inString, elementStart, elementEnd ) ];
else if ( elementType == 6 )
outputList += [ (rotation)llGetSubString( inString, elementStart, elementEnd ) ];

elementStart = elementEnd + 1;
}

return outputList;
}

default
{
state_entry()
{
list testList = [ "The", "red", "dog" ];
llOwnerSay( "testList = [" + llDumpList2String( testList, ", " ) + "]" );
string testList2String = list2String( testList );
llOwnerSay( "testList2String = \"" + testList2String + "\"" );
list reversedL2S = string2List( testList2String );
llOwnerSay( "Reversed list = [" + llDumpList2String( reversedL2S, ", " ) + "]" );
}
}


I do have a question, for Kek and anyone who reads this: look at the first few lines of string2List's loop. Do you see any way to get rid of that "if (listElementNum + 1 < listLength)" conditional without duplicating code?
==Chris
Strife Onizuka
Moonchild
Join date: 3 Mar 2004
Posts: 5,887
01-28-2006 19:00
This infact isn't safe. floats, vectors and rotations all have the possability of being corrupted by this function.
To safely store the floats you need to use a function such as Float2Hex
instead of a standard (string) typecast or llList2String.
_____________________
Truth is a river that is always splitting up into arms that reunite. Islanded between the arms, the inhabitants argue for a lifetime as to which is the main river.
- Cyril Connolly

Without the political will to find common ground, the continual friction of tactic and counter tactic, only creates suspicion and hatred and vengeance, and perpetuates the cycle of violence.
- James Nachtwey
Christopher Omega
Oxymoron
Join date: 28 Mar 2003
Posts: 1,828
01-28-2006 19:47
From: Strife Onizuka
This infact isn't safe. floats, vectors and rotations all have the possability of being corrupted by this function.


Corrupted? Surely that's a little too harsh a word to use in this context. Floats, vectors and rotations only loose precision when converted to a string, unless Im mistaken. (Please correct me if Im wrong)
==Chris
_____________________
October 3rd is the Day Against DRM (Digital Restrictions Management), learn more at http://www.defectivebydesign.org/what_is_drm
Strife Onizuka
Moonchild
Join date: 3 Mar 2004
Posts: 5,887
01-29-2006 01:13
well all floats less then 0.000001 will become 0, i consider that corruption. We are talking more then half the spectrum of floats get rounded to zero. Any float that is less then 16 has a chance of being corrupted.

Anyway... i've made a version that won't, also it will be much more friendly with very large floats. It adds about 900 instructions per float conversion.

CODE

// list2string and string2list functions...
// For the safest transfers of lists with strings

// By Keknehv Psaltery, 1/12/06 -- Public domain, do whatever you wish (Although I'd prefer if I still get credit)
// Revised By Christopher Omega, 1/28/06
// Revised By Strife Onizuka 1/28/06

// Updated so it can perserve floats, overhead 900 instructions per float conversion also optimized some of it's code.

//How it works:
// Instead of using something like llParseString2List with an uncommon separator, these list-strings
// actually have an index of where the strings are, and, as a bonus, it also preserves data types
// The data structure of the string looks like this:
// [index1][type1] [index2][type2] [index3][type3]...[indexN][typeN] [data1][data2][data3]

// So, if you had a list like this: [The, 1, <1, 5.3, 6>, dog], it would look like
// this (without quotation marks) when it passes through the list2string function:
// "00163001910020500503The1<1.000000, 5.300000, 6.000000>dog"
// ====^====^====^====^---_------------------------------___
// ind1|ind2|ind3|ind4|data1 data3 data4
// typ1 typ2 typ3 typ4 data2
// This formatting allows for **any** conceivable data to be transferred.

string hex = "0123456789ABCDEF";//it's faster this way *shurg*

string list2String( list inList ) //Converts a list to a string -- with no possible chance of error
{
string outputString;
integer listLength = llGetListLength( inList );
integer offset = listLength * 5;
string prefixString;
string part;
integer type;

integer listElementNum = -listLength;
for (; listElementNum; ++listElementNum )
{
type = llGetListEntryType( inList, listElementNum );
if(type == TYPE_FLOAT)
part = Float2Hex(llList2Float( inList, listElementNum ));
else if(type == TYPE_ROTATION)
part = Rot2Hex(llList2Rot( inList, listElementNum ));
else if(type == TYPE_VECTOR)
part = Vec2Hex(llList2Vector( inList, listElementNum ));
else
part = llList2String( inList, listElementNum );
prefixString += llGetSubString("0000" + (string)offset, -4, -1) + (string)type;
offset += llStringLength( part );
outputString += part;
}

outputString = prefixString + outputString;
return outputString;
}

list string2List( string inString )
{
list outputList;
integer elementStart = (integer)llGetSubString( inString, 0, 3 );
integer listLength = elementStart / 5;

integer listElementNum = 0; //your wasting bytecode by not doing this here (as in relaity if you dont' specify a value the compile does for you
integer ti; //save a few bytes this way
string ts;
while(listElementNum < listLength)
{
//sure it's ugly but its fast and effeciant. trimmed maybe 200 byte off the bytecode, and it isn't too ugly really
integer elementType = (integer)llGetSubString( inString, ti = (listElementNum * 5 + 4), ti);
if ( ++listElementNum < listLength ) // Strife: I agree with Chris, might as well put it to some use.
ts = llGetSubString( inString, elementStart, (elementStart = (integer)llGetSubString(inString, ti + 1, ti + 4)) - 1 );
else // Chris: I dont like checking for this every iteration :-(
ts = llGetSubString( inString, elementStart, -1 );

if ( elementType == TYPE_INTEGER ) //1
outputList += (integer)ts;
else if ( elementType == TYPE_FLOAT ) //2
outputList += (float)ts;
else if ( elementType == TYPE_STRING ) //3
outputList += ts;
else if ( elementType == TYPE_KEY ) //4
outputList += (key)ts;
else if ( elementType == TYPE_VECTOR ) //5
outputList += (vector)ts;
else if ( elementType == TYPE_ROTATION ) //6
outputList += (rotation)ts;
}
return outputList;
}

string Rot2Hex(rotation a)
{
return "<"+Float2Hex(a.x)+","+Float2Hex(a.y)+","+Float2Hex(a.z)+","+Float2Hex(a.s)+">";
}

string Vec2Hex(vector a)
{
return "<"+Float2Hex(a.x)+","+Float2Hex(a.y)+","+Float2Hex(a.z)+">";
}

string Float2Hex(float a)
{// Copyright Strife Onizuka, 2006, LGPL, http://www.gnu.org/copyleft/lesser.html
if(a != 0)
{
float b = llFabs(a);
string f = "";
integer c = llFloor(llLog(b) / 0.69314718055994530941723212145818);//floor(log2(b))
integer d = (integer)((b / llPow(2,c)) * 0x1000000);//shift up into integer range
c -= 24;//the extra c used to make it an integer
while(!(d & 0xf))
{//strip extra zeros off before converting or they break "p"
d = d >> 4;
c+=4;
}
do
f = llGetSubString(hex,15&d,15&d) + f;
while(d = d >> 4);
if(a < 0)
return "-0x" + f + "p"+(string)c;
return "0x" + f + "p"+(string)c;
}
return "0";//zero would screw up the log.
}

default
{
state_entry()
{
list testList = [ "The", "red", "dog" ];
llOwnerSay( "testList = [" + llDumpList2String( testList, ", " ) + "]" );
string testList2String = list2String( testList );
llOwnerSay( "testList2String = \"" + testList2String + "\"" );
list reversedL2S = string2List( testList2String );
llOwnerSay( "Reversed list = [" + llDumpList2String( reversedL2S, ", " ) + "]" );
}
}
_____________________
Truth is a river that is always splitting up into arms that reunite. Islanded between the arms, the inhabitants argue for a lifetime as to which is the main river.
- Cyril Connolly

Without the political will to find common ground, the continual friction of tactic and counter tactic, only creates suspicion and hatred and vengeance, and perpetuates the cycle of violence.
- James Nachtwey
Christopher Omega
Oxymoron
Join date: 28 Mar 2003
Posts: 1,828
01-29-2006 17:35
From: Strife Onizuka

CODE

string list2String( list inList ) //Converts a list to a string -- with no possible chance of error
{
string outputString;
integer listLength = llGetListLength( inList );
integer offset = listLength * 5;
string prefixString;
string part;
integer type;

integer listElementNum = -listLength;
for (; listElementNum; ++listElementNum )
{
type = llGetListEntryType( inList, listElementNum );
if(type == TYPE_FLOAT)
part = Float2Hex(llList2Float( inList, listElementNum ));
else if(type == TYPE_ROTATION)
part = Rot2Hex(llList2Rot( inList, listElementNum ));
else if(type == TYPE_FLOAT)
part = Vec2Hex(llList2Vector( inList, listElementNum ));
else
part = llList2String( inList, listElementNum );
prefixString += llGetSubString("0000" + (string)offset, -4, -1) + (string)type;
offset += llStringLength( part );
outputString += part;
}

outputString = prefixString + outputString;
return outputString;
}


The second else if should be "else if(type == TYPE_VECTOR)"
_____________________
October 3rd is the Day Against DRM (Digital Restrictions Management), learn more at http://www.defectivebydesign.org/what_is_drm
Strife Onizuka
Moonchild
Join date: 3 Mar 2004
Posts: 5,887
01-30-2006 13:05
your correct, i posted the wrong version; i was testing it and forgot to update the version outside of SL :(
_____________________
Truth is a river that is always splitting up into arms that reunite. Islanded between the arms, the inhabitants argue for a lifetime as to which is the main river.
- Cyril Connolly

Without the political will to find common ground, the continual friction of tactic and counter tactic, only creates suspicion and hatred and vengeance, and perpetuates the cycle of violence.
- James Nachtwey
Strife Onizuka
Moonchild
Join date: 3 Mar 2004
Posts: 5,887
11-03-2006 13:48
Every so often I revise Float2Hex (which is used by this)

Check for updates at (or where ever the wiki is currently hosted):
http://lslwiki.com/lslwiki/wakka.php?wakka=LibraryFloat2Hex
_____________________
Truth is a river that is always splitting up into arms that reunite. Islanded between the arms, the inhabitants argue for a lifetime as to which is the main river.
- Cyril Connolly

Without the political will to find common ground, the continual friction of tactic and counter tactic, only creates suspicion and hatred and vengeance, and perpetuates the cycle of violence.
- James Nachtwey