I've voted and commented on SVC-3679, it's an incredibly bad change for the worse! What I don't understand is why the garbage collection isn't kicking in immediately for scripts that are low on memory? You know, like garbage collectors are supposed to?
If we'd had fecking arrays from the start this wouldn't even be an issue; it's only the fact that changing a single item in a list is the single most inefficient thing I've ever seen in a programming language that we can even run into these problems!
Welcome to the Second Life Forums Archive
These forums are CLOSED. Please visit the new forums HERE
Beta Testing 1.25.3 |
|
|
Haravikk Mistral
Registered User
Join date: 8 Oct 2005
Posts: 2,482
|
01-23-2009 04:39
_____________________
Computer (Mac Pro):
2 x Quad Core 3.2ghz Xeon 10gb DDR2 800mhz FB-DIMMS 4 x 750gb, 32mb cache hard-drives (RAID-0/striped) NVidia GeForce 8800GT (512mb) |
|
Liandra Ceawlin
Registered User
Join date: 16 Apr 2007
Posts: 3
|
List processing -much- slower on 1.25 simulator
01-23-2009 06:55
Not only are we running out of memory where we weren't before... List processing is ludicrously slow now. Here's a profile of the inside of a list manipulation loop on a 1.25 simulator with about 16ms script time:
CODE
And here is a profile of the exact same code running on a 1.24 simulator with about 20ms (-higher-) script time: CODE
As you can see, there is some -major- slowness going on here, making this interactive object totally unusable. |
|
Liandra Ceawlin
Registered User
Join date: 16 Apr 2007
Posts: 3
|
01-23-2009 06:59
I'm getting similar results on all the 1.25 simulators that I try, so I am pretty sure this isn't just a problem with the simulator my workshop is located in. >_> It's actually the fastest of the 1.25s I've tried so far, lol.
|
|
Basil Wijaya
Registered User
Join date: 25 Feb 2007
Posts: 1
|
stack heap collisions
01-23-2009 08:30
There is a timing problem with the garbage collector. Here is an important discussion I had with TheBlack Box and HeZkeZl Slade. It could be helpful till a global solution is found :
[20:20] TheBlack Box: anyone having trouble with large lists producing stack-heap-collision too early ? https://jira.secondlife.com/browse/SVC-3679 [20:28] Basil Wijaya: damned stack-heap collisions [20:32] Basil Wijaya: I'm still on Second Life Server 1.24.10.106829 and I tried the script in the jira with 356 and I get the stack-heap collision [20:34] TheBlack Box: Basil .. what is the last output before the collision ? [20:34] Basil Wijaya: [20:31] Object: llGetFreMemory() says: 22860 [20:31] Object [script:New Script]: Stack-Heap Collision [20:31] Object [script:New Script]: Script run-time error [20:34] TheBlack Box: did you try 256 and consecutive runs ? [20:34] Basil Wijaya: ya with 256, it was ok [20:35] Basil Wijaya: I can try again [20:35] TheBlack Box: ok .. so 22860 memory lesft and replacing the first item of a long list with an item of equal length causes heap-collision [20:35] TheBlack Box: with 256 .. try multiple runs [20:36] TheBlack Box: i see the free memory sudenly dropping after 2-3 runs .. not sure what causes it [20:38] Basil Wijaya: goes from 33460 to 8292 [20:38] TheBlack Box: suddenly .. for no apperent reason .. right ? [20:38] Basil Wijaya: then stays at 8292 [20:39] Maldavius Figtree: odd [20:39] Basil Wijaya: I will reset and restart [20:44] Basil Wijaya: [20:43] Object: Touched [20:43] Object: Before doing list : llGetFreMemory() says: 60012 [20:43] Object: After doing list : llGetFreMemory() says: 32948 [20:43] Object: llGetFreMemory() says: 32948 [20:43] Object: llGetFreMemory() says: 32948 [20:43] Object: llGetFreMemory() says: 32948 [20:43] Object: llGetFreMemory() says: 32948 [20:43] Object: llGetFreMemory() says: 32948 [20:43] Object: llGetFreMemory() says: 7780 [20:43] Object: llGetFreMemory() says: 7780 [20:43] Object: llGetFreMemory() says: 7780 [20:43] Object: llGetFreMemory() says: 7780 [20:43] Object: llGetFreMemory() says: 7780 [20:43] Object: After changing list [20:45] Basil Wijaya: I added some memory check [20:45] TheBlack Box: yes ... thats whats biting my scripts currently [20:47] Basil Wijaya: replacing 5 keys does not change anything, then memory drops. If I retouch, the memory stays to 7780 all the way [20:48] TheBlack Box: well llGetFreMemory() isnt supposed to go back up again [20:48] Basil Wijaya: it is the lowest level of memory, not the actual level of free memory [20:49] TheBlack Box: exactly .. but why does it drop from 32948 to 7780 ... sometimes ... [20:49] Basil Wijaya: after the 5th key change [20:49] TheBlack Box: thats different on different runs [20:49] Basil Wijaya: I will reset and redo to see [20:51] Basil Wijaya: your right [20:51] Basil Wijaya: not stable [20:56] Basil Wijaya: i initialised the list and on the first try, the memory stays to 32948. [20:57] Basil Wijaya: list keys = []; [20:57] HeZkeZl Slade: are you passing the list into any other functions? [20:57] HeZkeZl Slade: or using any list functions on it? [20:57] Basil Wijaya: the list is used in a function but not passed in the function parameter [20:58] TheBlack Box: we are testing with the script from https://jira.secondlife.com/browse/SVC-3679 [21:00] HeZkeZl Slade: thats an awful lot of lists on the stack in that funtion [21:01] Basil Wijaya: ya TheBlack Box is testing stack-heap collisions on the new server [21:03] TheBlack Box: and i would assume there only being 1 list on the stack ,,, not right ? [21:04] Basil Wijaya: I think there is only one list [21:04] Basil Wijaya: I thought would explain what he means [21:04] Basil Wijaya: Hezkezi [21:04] HeZkeZl Slade: no, because lsl does a copy by value, of lists, so when you assign a list to a list, it creates a temporary list [21:04] HeZkeZl Slade: so you have 2 lists on the stack at once [21:05] HeZkeZl Slade: still doesn't explain why running it twice would break it tho [21:05] Basil Wijaya: You mean llListReplaceList makes 2 lists in memory? [21:06] HeZkeZl Slade: yes.. although after the assignment is complete, the second list should go away [21:06] Basil Wijaya: good to know [21:06] HeZkeZl Slade: the same thing happens when you pass a list as an argument to a function [21:06] Basil Wijaya: sure if no pointers are used, the list has to be copied [21:06] HeZkeZl Slade: although that's not happening here [21:07] TheBlack Box: and the memory-need for that sometimes is reflected in llGetFreMemory() and sometimes not ? [21:08] HeZkeZl Slade: nono.. thats the part that seems very broken.. Unless they added some sort of deferred garbage collection to lsl with this latest release [21:08] Basil Wijaya: it would be so useful to have a function that shows the actual memory used [21:08] TheBlack Box: or what cause it to go from 32948 to 7780 all of a sudden ? [21:08] Basil Wijaya: ya [21:08] Basil Wijaya: and in an unstable manner [21:09] TheBlack Box: Hez: that was what i was thinking .. it could have been changes to the garbage collection ... [21:09] Basil Wijaya: what kind of changes? [21:10] Basil Wijaya: But the version is not in cause [21:11] TheBlack Box: no idea  [21:11] Basil Wijaya: on 1.24, it does the same thing [21:11] TheBlack Box: no but the version-shift also changes the effects of this problem [21:12] HeZkeZl Slade: it's probably allocating things on the heap that it used to on the stack, and then reference counting.. with a periodicprocess that goes through and frees any heap memory that isn't referenced [21:12] Basil Wijaya: stack heap collision with 356 [21:15] Basil Wijaya: so the periodic cleaning process does not always fall at the same place in the function process? [21:16] HeZkeZl Slade: no.. it's usually a seperate thread that is timing based, the process sandbox would run it. This is the way java works, and mono is very similar to java [21:21] HeZkeZl Slade: definately looks like whatever memory is allocated in set isn't being freed sometimes.. based on your numbers above, set uses approximately 27k (you jump from 60 to 33), then jump from 33 to 7.. [21:22] TheBlack Box: ha ... anf if you replace the set function with keys = llListReplaceList((keys=[])+keys,[id],i,i); [21:22] TheBlack Box: you get 356 running though [21:22] TheBlack Box: at least once...then you should wait a bit [21:23] TheBlack Box: thats what the vodoo-list stuff refers to i guess ... [21:23] Basil Wijaya: hehe [21:23] HeZkeZl Slade: Try putting an infintesimal sleep after your set (like .05) [21:24] HeZkeZl Slade: and see if it still happens.. the sleep should relent the process and give garbage collection time to clean up (if thats in fact what is happening) [21:24] Basil Wijaya: interesting [21:24] HeZkeZl Slade: i can't compile mono on this client, or else i would try it [21:25] TheBlack Box: HeZ .. your good ! .. this works good for lists that dont fit into the memory twice [21:25] Basil Wijaya: it works... no collision and i used 356... I had before [21:26] Basil Wijaya: wowow [21:26] TheBlack Box: learned a lot ... thank you ! [21:26] Basil Wijaya: me too... thks [21:26] HeZkeZl Slade: yw [21:26] TheBlack Box: ... closing the jira-entry Here is the script that worked after adding a sleep : Important : The stack heap collision happens too in Second Life Server 1.24.10.106829 But the solution could be helpfull in many occasions. list keys = []; set(integer i,key id) { keys = llListReplaceList(keys,[id],i,i); } default { touch_start(integer bla) { llOwnerSay("Touched"  ;keys = []; integer i; llOwnerSay("Before doing list : llGetFreMemory() says: "+(string)llGetFreeMemory()); for (i=0;i<356;i++) keys = keys + [llGetOwner()]; llOwnerSay("After doing list : llGetFreMemory() says: "+(string)llGetFreeMemory()); for (i=0;i<10;i++) { llOwnerSay("in the for" + (string)i); set (i,llGetOwner()); llSleep(.05); } llOwnerSay("llGetFreMemory() says: "+(string)llGetFreeMemory()); llOwnerSay("After changing list" ;} } |
|
Prospero Linden
Linden Lab Employee
Join date: 6 Aug 2007
Posts: 315
|
01-23-2009 09:10
Hey guys -- make sure to put all relevant information that might help track this problem down into the JIRA issue.
|
|
Prospero Linden
Linden Lab Employee
Join date: 6 Aug 2007
Posts: 315
|
01-29-2009 12:30
I have updated the release notes for 1.25.5 ; the latest 1.25.5 will be going to aditi "Second Life Beta Server" momentarily.
https://wiki.secondlife.com/wiki/Release_Notes/Second_Life_Beta_Server/1.25 |
|
Vittorio Beerbaum
Sexy.Builder Hot.Scripter
Join date: 16 May 2007
Posts: 516
|
01-29-2009 16:18
_____________________
 Beerbaum Music Department http://slurl.com/secondlife/Beerbaum/127/131/23 http://www.vittoriobeerbaum.com |
|
Prospero Linden
Linden Lab Employee
Join date: 6 Aug 2007
Posts: 315
|
01-30-2009 13:01
WOAH. You can get acutal links to show up in these fora? How did you do that?
|
|
SuezanneC Baskerville
Forums Rock!
Join date: 22 Dec 2003
Posts: 14,229
|
01-30-2009 13:39
You add a question mark to the end of the url and enclose the url in bbcode style image tags.
However, the result does not necessarily show up at all on some browsers. Firefox and Chrome users can install Greasemonkey and use the "BBCode in Text" Greasemonkey script to replace our missing VBCode, if they want. This can also be done in Opera, with a small modification to the script. The Greasemonkey script is described in a sticky in the Resident Answers forum. _____________________
-
So long to these forums, the vBulletin forums that used to be at forums.secondlife.com. I will miss them. I can be found on the web by searching for "SuezanneC Baskerville", or go to http://www.google.com/profiles/suezanne - http://lindenlab.tribe.net/ created on 11/19/03. Members: Ben, Catherine, Colin, Cory, Dan, Doug, Jim, Philip, Phoenix, Richard, Robin, and Ryan - |
|
Vittorio Beerbaum
Sexy.Builder Hot.Scripter
Join date: 16 May 2007
Posts: 516
|
01-30-2009 16:20
WOAH. You can get acutal links to show up in these fora? How did you do that? Here: /348/f8/300912/2.html#post2307914 _____________________
Beerbaum Music Department http://slurl.com/secondlife/Beerbaum/127/131/23 http://www.vittoriobeerbaum.com |
|
Prospero Linden
Linden Lab Employee
Join date: 6 Aug 2007
Posts: 315
|
01-30-2009 17:15
OK, this is going to be an off topic rant, so I encourage the moderator to delete this message.
But. WTH is it with BBCode anyway? Or Wiki markup? Or the other million markups that get invented every time somebody writes something new? It all has to get converted to HTML in the end, and *rarely* are these markup languages easier to use than HTML anyway. Why not just use bloody HTML? Geez. Oh, and, back on topic, er, um, yeah! There's a new deploy to aditi as of about 1/2 hour ago, fixing some things that people will be happy to see fixed. I'll get the release notes updated later tonight. |
