String memory in a Mono world

I'm late to the game, as I haven't done much strict testing with Mono, but I'm a bit confused by my tests with strings under Mono. I'm not at all familiar with Mono, it's VM or memory architecture. I'm just going off of my observed tests in LSL.

It seems that strings are not stored as UTF-8 in script heap? Storing a standard low-ASCII string (<0x12 consumes twice as much memory as under the LSL VM, according to llGetFreeMemory(). (ie: A 2000 character string of repeating "012456789" takes 4000 bytes, versus 2000 bytes as under the older LSO VM.) I would expect multi-byte UTF-8 to take more memory, but low-ASCII too? Am I not understanding the way Mono implements strings?

Also, the new way that Mono handles immutable strings is nice, but I can't quite understand the method it uses to determine when a string should be copied into heap from an external source. For example, in my tests it seems that a link_message receiving a 20k-character string would consume heap space if the script contains 20k or so free memory (according to llGetFreeMemory), yet if it contains LESS it appears I am manipulating a series of references to that string data. (ie: I can touch the 20k-char string without causing a heap-stack collision.)

I can copy the string, split it up, even read the data without causing the collision. But what I don't understand is that it appears this copy-to-heap or use-by-reference logic is based on the string character LENGTH versus its actual BYTE SIZE. Therefore, it's possible to cause heap collisions by the VM attempting to copy in a 20k-character string that actually takes up 40k, when there's 25k of free memory available. Yet if I fill up the heap to push it to less than 20k free, I can manipulate the external string without issue.

I must be missing something basic and obvious here.

Or am I see artifacts of the of SVC-3679 and a loose free memory estimation?

Any ideas?

well sadly, testing with llGetFreeMemory in MONO is highly inaccurate at this time... I observed massive reporting changes on changes of 1 character for a compiled undeclared instance, and even with changes in whitespace... ie the function is essentially useless in MONO. (or the compiler is doing some REALLY funky stuff). also seen changes in reported free memory when a script is recompiled with no actual changes.

any of those might be affecting you, and will seriously mess with your ability to guess when a variable is being called by reference.

However, I believe it's true that each character takes two bytes in MONO. I don't remember who told us that first; perhaps it was Argent, but it seems to be true in my experience.

Void, you won't see a difference due to changing whitespace or comments in the script source, so I'm not sure what you meant by changing whitespace, unless that was whitespace in a string. It's possible that simply recompiling causes a change -- hopefully a rather small one.

about 30 bytes, and no I meant outside a variable... wish I saved the stupid script because it seriously blew my mind at the time.

and yeah, MONO uses unicode characters, although I found some strange string behavior in the evaluation of LSO undeclared string literals (and probably variables too during concatenation) where it's using 2 bytes per character after the second when placing values on the stack... still trying to work out the exact math of what's occurring, but at the basic level it looks like the termination characters are being ignored for stack movements.

from the looks of it though, there may be situation is which compiled bytes size may be smaller, but running byte size is larger... for slightly differing versions of a script (essentially in how the variables are referenced).... not a comparison I was wanting to find at that level....

Mono uses UTF-16.

/54/9a/250229/1.html
(not everything in the thread is entirely accurate, it was written before Mono hit the grid)

Thanks. So we got 64k memory and dynamic allocation in Mono -- but we're using UTF-16 strings with twice the storage requirements for almost %90 of the script cases on SL (with less overall glyph space), VM instructions that take 4/5x as much space, and the same kind of memory-intense list structures.

I find my Mono scripts aren't able to handle much more than my original LSO-compiled scripts and I still need to split them up into multiple parts. But by golly, they're faster! I guess that's something.

Anyway, as for the rest, I can't figure it out, and the more I read the more it appears to be part of wonkiness with llGetFreeMemory() and free space calculations. I just can't accurately predict when my script will crash if has to handle an abnormally large link message work load. Time to shelve this and come back to it later.

Fenrir, I find that Mono scripts hold considerably more data than LSL scripts. For example, MLPV2 scripts that used to peak out at around 80 poses now peak out at around 250. Much of the data is stored as character strings, including animation names and pose positions.

I don't advise loading MLPV2 with 250 poses, though. Memory limits will discourage that.

all depends on usage... if it's string heavy and that what it processes most, it's not gonna be much more space... but if it's function and math heavy, it's a huge boon... and the speed of most things is blistering compared to LSO... although a few things take longer now (state changes are one, list manipulations are being played with and may end up being another)

I've found this also, in my testing with VK's VK-DBMS..it'll hold almost twice as much data in the list table. Never considered that the command strings being passed back and forth might create a memory problem though.

I believe this is a side effect of not needing the memory hacks that are in use for LSO (which effectively double the usable handling space for lists and strings in LSO).

the logic for mono doesn't double the size of lists/strings when modifying them whereas LSO did.

without memory hacks using a function that doubled the starting list (10 keys) I could get 160, using the memory hack I could double that to 320 before stack heap collision.

ETA:
some of the changes they are intending for string/list handling may cause the the reinstatement of the the usefulness of the memory clearing hack within MONO. depending on implementation.

Not in my case, because the LSL code used the hack. With mono, the results were the same with or without the hack.