List to String for transport

I know this has come up before, but what is the most effecient method for converting lists to strings for transport... just casting as string doenst place an seperators, which makes it really fun(read impossible)to seperate numerical data... any thoughts?

llList2CSV
llDumpList2String (which lets you place a seperator)
or a for loop and mutiple messages

:macks self in forehead::

thanks

::hides in shame::

This is, unfortunately, broken. As long as you have no strings in the original list it should work, but it doesn't quote strings correctly (and uses an incompatible hack to avoid quoting vectors and rotations) so you've gotta be careful what you put in. When it's applicable, though, it's the best way to go.

Safe: keys, integers, floats, vectors, rotations, string versions of these, avatar names, region (sim) names.
Not safe: group titles, or any other arbitrary strings.

I just posted a feature request for a fixed version of this.

actually I found the library function for List2TypeCSV and TypeCSV2List... but just noticed a possible problem with it as well.... in strings that contain "words, more words".... since it cuts on ", "[comma+space] I suppose I could tweak it to use something else, like "` "[accent+space] and replace the ll list<->CSV functions with llDumpList2String and llParseString2List.... or tweak the alternate version in the library... which may be similar... oh well just thinking out loud

btw what exactly are the problems, besides seperators, with llList2CSV?

Separators are a result of it not quoting strings.

Converting a list to a CSV it should:

1. Replace all quotes with doubled quotes.
2. Put quotes around all strings containing commas or quotes.
3. Put a single comma between list elements.

So ["hello", "there, joe", "says \"fred\"] would come out as hello,"there, joe","says ""fred""".

In reverse... split on unquoted comma, remove quotes around strings, convert doubled quotes to single quotes.

I use the pipe symbol and use llParseStringKeepNulls

making the string is easy. Use a for loop and cast everything to a string.

The only problem is that the resulting list at the destination is entirely made up of strings and this has to be cast to x when used.

An age ago i wrote a class of functions for safe transport of lists though strings (and more recently a version that can preserving types).

You can of course replace the alg for generate the spacers and not have any compatability issues with other versions of the functions.

string TightListDump(list a, string b)
{
	string c = (string)a;
	integer d = -38 - llStringLength(b);
	if(d == -39)
		if(llSubStringIndex(c,b) == -1)
			jump end;
	b += "|\\/?!@#$%^&*()_=:;~`'<>{}[],.\n\" qQxXzZ";
	while(1+llSubStringIndex(c,llGetSubString(b,d,d)) && d)
		++d;
	b = llGetSubString(b,d,d);
	@end;
	c = "";//save memory
	return b + llDumpList2String(a, b+(string)(a = []));
}
 
 list TightListParse(string a) {
     string b = llGetSubString(a,0,0);//save memory
     return llParseStringKeepNulls(llDeleteSubString(a,0,0), [a=b],[]);
 }

You can find type version of these functions in TLML.

Typical twisty Strife code. I'll bet there's not a single wasted opcode...

Looks like it's got a list of possible separator characters, and it looks for one that's not in the list already, and it generates a new string with that separator character at the beginning.

I'm not sure I understand the "string b" argument or what you're doing with it, or what the point of the extra 38 bytes in that string are for. Can you elaborate?

the second paramater lets you provide extra characters for it to use as seperators. If you provide a single character and it's not used, it will use that one, otherwise it will add your seperators onto it's internal set of seperators and run though all of them. The 38 provides and the string length are used for providing the negitive index for the llGetSubString.

The goal was to have a fast function that would be light on opcodes. Fast being the main goal.

I actualy store all of my TightList functions in an ESL header for including in code i write. I'd post it but it makes this code look easy to understand.

I also wrote another version that doesn't take a second paramater.
(the advantage of the second paramater is so if you know your data is going to bork the internal seporator list)

 string TightListDump(list a)
 {
 	string b = "|\\/?!@#$%^&*()_=:;~`'<>{}[],.\n\" qQxXzZ";
 	string c = (string)a;
 	integer d = -39;
 	do; while(1+llSubStringIndex(c,llGetSubString(b,d,d)) && ++d);
 	b = c = llGetSubString(b,d,d);
 	return b + llDumpList2String(a, b);
 }
 

Why, Strife, I'm amazed.

I don't think you're actually doing anything I'd consider "dodgy" in this one, though the negative substring index is a bit confusing at first.

You don't handle the case where all the separators are already used in the list, though.

 string TightListDump(list a)
{
    string b = "|\\/?!@#$%^&*()_=:;~`'<>{}[],.\n\" qQxXzZ";
    string c = (string)a;
    integer d = -39;
    do; while(1+llSubStringIndex(c,llGetSubString(b,d,d)) && ++d);
    if(!d) return ""; // Stercus stercus stercus moriturus sum.
    c = llGetSubString(b,d,d); // don't need to override b because we're about to return
    return c + llDumpList2String(a, c);
}

If this routine returns an empty string that's an error, you DO need to check that in the calling routine. Also, isn't this version a bit tighter than your original? It should be just about as fast because it puts the new characters at the beginning... so if you specify a single character that works it'll hit the first pass through the loop, and if you don't it'll fail gracefully.

string TightListDump(list a, string b)
{
    integer d = -39-llStringLength(b);
    b += "|\\/?!@#$%^&*()_=:;~`'<>{}[],.\n\" qQxXzZ";
    string c = (string)a;
    do; while(1+llSubStringIndex(c,llGetSubString(b,d,d)) && ++d);
    if(!d) return ""; // Stercus stercus stercus moriturus sum.
    c = llGetSubString(b,d,d); // don't need to override b because we're about to return
    return c + llDumpList2String(a, c);
}

We handle the failure by using the first character as a separator. I decided when writing the function that it was up to the user to make sure they didn't corrupt thier data. I'd rather have no error checking & corrupt data then have no data. Personal prefrence i guess.

The reason for overwriting b is to release it's memory (most of it at least). But I suppose it's ok.

These functions are actualy pretty old, I fiddle with them every so often. I did alot of research for a combat system soon after I joined. Never finished the combat system (long story), but TightList was something I wrote for it. It's really useful, it's rohbust, and you can write your own encoder function and not have to worry about decoding.

At one point in time i toyed with the idea of using the my unicode character generator, to generate a random multi-byte UTF-8 character as a separator. Figured it would be too slow.

For TLWAPI I use 2byte characters to represent link numbers for hierarchal linking and TLML for rendering.

Here is the parsing functions for TightListType, they haven't been optimized, i have a version (not posted here) that is much lighter on bytecode (i wrote a function for building the separator list, that uses recursion, resulting in reduced total bytecode, but running slower and at a greater memory expense). It should be noted that i dropped support for the leading integer hack that this version of the Parse command supports.

    list TightListTypeParse(string a) {
    	if(llStringLength(a) < 7)  return [];
    	string m = llGetSubString(a,0,6);
   	list b =  llList2List([a]+//a = llGetSubString(c,0,0) by this point
    						llParseStringKeepNulls(llDeleteSubString(a,0,5), [], [a=llGetSubString(m,0,0),
    						llGetSubString(m,1,1), llGetSubString(m,2,2), llGetSubString(m,3,3),
    						llGetSubString(m,4,4), llGetSubString(m,5,5)]),
    						(llSubStringIndex(m, llGetSubString(m,6,6)) < 6) * 2,-1);
    	integer c = -llGetListLength(b);
    	list f;
    	integer d;
    	do
    	{
    		f = [a=llList2String(b,c + 1)]; //TYPE_STRING || TYPE_INVALID (though we don't care about invalid)
    		if((d = (1 + llSubStringIndex(m, llList2String(b,c)))) == TYPE_FLOAT)
    			f = [(float)a];
    		else if(d == TYPE_VECTOR)
    			f = [(vector)a];
    		else if(d == TYPE_ROTATION)
    			f = [(rotation)a];
    		else if(d == TYPE_KEY)
    			f = [(key)a];
    		else if(d == TYPE_INTEGER)
    			f = [(integer)a];
    		b = llListReplaceList(b, f, c, c+1);
    	}while((c+=2) & 0x80000000);
    	return b;
    }
    
    string TightListTypeDump(list a, string b) {
    	b += "|\\/?!@#$%^&*()_=:;~`'<>{}[],.\n\" qQxXzZ";
    	string c = (string)a;
    	integer d = 0;
    	do
    		if(1+llSubStringIndex(c,llGetSubString(b,d,d)))
    			b = llDeleteSubString(b,d,d);
    		else
    			++d;
    	while(d<6);
    	b = " " + c = llGetSubString(b,0,5);
    	if((d = -llGetListLength(a)))
    	{
    		do
    		{
    			integer e = llGetListEntryType(a,d);
    			c+= llGetSubString(b,e,e) + llList2String(a,d);
    		}while(++d);
    	}
    	return c;
    }
    

You can see TightList & TightListType used in TLML.