transmitting certain chars through llHTTPRequest

i've discovered that when ampersands ("&" are embedded within notecard lines and these are subsequently transferred to a server using llHTTPRequest, the first ampersand, scanning left to right, stops the transmission and only the prefix of the line up to that point is sent.

(1) is there some builtin slashification mechanism i can appeal to to help fix this? i thought i'd ask before i rolled my own. any standards? there's llEscapeURL, but it grows line lengths a lot.

(2) are there any other characters which similarly break the transmission?

UPDATE: oh, yeah, i'm already using a MIME type of application/x-www-form-urlencoded.

that's true if i want a completely general solution, but i don't need that. i can concoct a substitute character or substring for this special case.

what's curious is that a transmission from a server -to- the HTTP response event doesn't suffer the same problems. it carries ampersands just fine.

thanks.

You should also be aware that llEscapeURL can only return 255 characters; if you are dealing with utf8 characters of unknown origin then you can only parse... 14 characters at a time through the function (14 * 6 * 3 = 252). Report this as a bug, it's a stupid limitation (if i ever meet the linden who thought 255 was a good limit...). If it's regular 7bit ascii, then you can pass 85 characters safely at a time (notecards only support 7bit ascii).

I just loop through the data till pos > len.
In one situation i knew i had 2 byte utf8 characters

	t_int = 0;
	path = "";
	do//stupid llEscapeURL 255 char output limitiation *grrrr*
		path += (string)llParseString2List(llEscapeURL(llGetSubString(children, t_int, t_int + 41)),["%"],[]);
	while((t_int += 42) < count);
	children = path;

vs.

	children = (string)llParseString2List(llEscapeURL(children),[children=path = "%],[]);

If you just want to fix "&" characters it would be easier to just do...

string escape(string str)
{//have to escpae % too, or you cannot unescape this.
	return str_replace(str_replace(str, "%", "%25"), "&", "%26");
}

string str_replace(string str, string from, string to)
{
	integer len = ~-llStringLength(from);
	if(~len)
	{
		string  buffer = str;
		integer b_pos = -1;
		integer to_len = ~-llStringLength(to);
		@loop;//instead of a while loop, saves 5 bytes (and run faster).
		integer to_pos = ~llSubStringIndex(buffer, from);
		if(to_pos)
		{
//			b_pos -= to_pos;
//			str = llInsertString(llDeleteSubString(str, b_pos, b_pos + len), b_pos, to);
//			b_pos += to_len;
//			buffer = llGetSubString(str, -~b_pos, 0x8000);
			buffer = llGetSubString(str = llInsertString(llDeleteSubString(str, b_pos -= to_pos, b_pos + len), b_pos, to), -~(b_pos += to_len), 0x8000);
			jump loop;
		}
	}
	return str;
}

i replaced it with a caret symbol ("^" on the SL side, and turned it back into an ampersand on the server.

the error of the design is that it seems to me that

X on SL ----llHTTPRequest----> X on server -----HTTP response----> X on SL

ought to preserve whatever X is within *reasonable* limits of capacity.

the problem with str_replace in SL, or anything like it, is that it uses up so much of the precious 16 KB, just for the code!

i mean, we could use llEscapeURL but then we need something like Burrows-Wheeler transform on both sides to fit within the dumb space limits.

but, then, *that* would use up a lot of the space, too.

the str_replace function above is only about 300 bytes (give or take 10 bytes; not logged in presently so can't give an exact number). Which is pretty light for what it does (you cannot optimize anymore out of it without axing features). It's only about 1.8% of script memory.

that is light. thanks.

but since i'm working on the server side anyway, i might as well do it there.

also, despite being thankful for your efforts, the availability of a str_replace function doesn't fix the blight such a problem is in LSL's design: it isn't so much the failure of ampersand to transmit, but the 255 byte limit on the amount which can be sent, and the effective 80-100 byte length after llEncodeURL. i understand that failure to transmit the ampersand might be the result of the protocol and encoding. nevertheless, there we are.

LSL is usable for lots of things. but to try to do so with these tools as implemented both demands far more ingenuity than it should and strongly crimps what might be available for Residents within SL otherwise.

such tools are good considering they were thrown together in less than a month. i'm not sure that's something of which anyone should be proud. now Linden Labs has saddled themselves with compatibility problems in any future changes, and, by capping the capability and self-extendability of LSL, violated the self-bootstrapping premise of their business model. in all other areas, Residents can help build SL. but noone can help them make LSL better.

now, lslint, that's an achievement. it's true it may be easier since the canonical copy semantics Linden Labs needs to achieve script state portability makes type analysis easy. lslint is a pretty piece of work. i wish i had something similar for PHP. but that's not likely: having mutable references, such as true arrays with L-values, or reference parameters makes such analysis difficult.

don't get me wrong: a lot can be done in 16 KB. but to do so requires tools that use implementations of data structures and semantics which aren't profligate in use of memory.

what LSL needs is a self-extension mechanism. there's talk of Mono but that's not an answer here. i'm thinking of something like Lua.

but nothing can be done without references and real arrays. and i don't know if SL's implementation can deal with those.

I'm very pleased to have found this thread having just come up against the same problems and pondering the same solutions

I'd say we need the LSL equivalent of PHP's addslashes, stripslashes and htmlspecialschars

Thx for the tips!