Rolling Restart : server version 1.25.3 / 1.25.4

If you think the change may be related to the rolling restart, try the following:

* Identify a region that's on 1.25.4, and one that's on 1.24.10. You can figure this out by looking at Help:About Second Life -- near the top is the version of the server of the region you're currently in

* Do the problems only happen in a 1.25.4 region, or do they also happen in a 1.24.10 region?

However, chances are that it's not related to the roll. When a new version causes problems that show up client side, there are so many people in Second Life that we'll see a pattern of several people having those problems. Most likely it's an individual problem; start at http://www.secondlife.com/support to try to resolve it

I own some land there, if I log in there, the land title at the top of the screen shows location but nothing else. If I click it, all the land details are blank.

If I try and TP out, I just get logged out of SL. EVERY time. You cannot TP out of Bhima.

If I log in somewhere else, land title works fine. If I TP to Bhima (TP's inwards to Bhima seem to work ok) the location at the top of the screen is correct, but land details are of the place I just TP'ed from.

I've put in a ticket, might hear something in a day or two.

Hi all,

I just found out that when a script calls itself to go back to its state_entry, the new server version seems to hang the code. The previous one worked fine. I filed a Jira entry. Can anyone running on a new server test this code?

default
{

state_entry()
{
state test;
}

}

state test

{
state_entry()
{
llOwnerSay("working!";
}

touch_start(integer total_number)

{
state test;
}

}

Put this in a new object and compile. What is shoud do it everytime you touch it, it should send a message back to you. In old server, it works but not with the new one. Just hangs there.

Hope this gets fixed soon as I have games with calls like this and they stop working

Thanks for your support.

o_O;;;

It never occurred to me that state_entry() would fire if you didn't change state.

I've had "state reenter_other_state" sections like forever.

I'm glad I wasted the time and opcodes.

However, LL is generally pretty good about fixing things like this.

Have you filed a JIRA on it?

I don't think its a bug. From the wiki:

Calling a state from within itself does not cause a state change, and will not trigger state_entry.

He has state "test" calling itself, so state_entry shouldn't fire.

If you change the last state line from "state test" to "state default" it would work.

If it works in the previous server code, that is actually the bug that has been fixed. I imagine it was related to the multiple touch_start() problem (fixed in V1.25.4).

Thank you so much!

Amber from LL has restarted the region, problem fixed...

This update needs to be rolled-back, it's changed something in the garbage collection which makes Mono lists even less efficient (and we though that wasn't possible!), it causes stack/heap collisions in scripts that manipulate even relatively small lists. How the hell this gets by testing I can't understand!

https://jira.secondlife.com/browse/SVC-3679

I agree, this is a serious issue that's really causing problems and has to be look at since it's breaking content. I'm actually honestly surprised that this isn't critical or even showstopper and anyone who's been around the JIRA awhile knows how serious it is when I actually am advocating for an issue being a showstopper.

Hi everyone first time I've posted on here

Is anyone else having problems with voice chat since these restarts started? Mine was perfectly fine till Tuesday, then it suddenly stopped working.

Talk button clicks itself on and off randomly, and even when it shows as connected, i either have no sound at all,or it's so bad it's useless

Have sent in a support ticket, but was wonderin if anyone else can concur?

I'm not even gonna start with the rest of the problems been having as they are numerous,but having read last couple days posts, seems it's not just me.

Sparkie

Mono treats deallocated but not garbage-collected lists as still part of the heap? I thought Mono was going to have a soft memory limit... that is, it wouldn't have a fixed size heap but rather a per-script quota.

If they're going to do GC, they need to modify the memory allocator to stall the operation and perform an immediate GC when they hit the quota.

Something I don't think is discussed enough is how a rolling restart greatly increases grid-wide lag. The technical reasons behind it I'm only guessing, but perhaps additional network strain as a sim image is pulled over the network to thousands of regions as they restart, and perhaps database stress on top of that.

In any case, LL, please don't underestimate this. A rolling restart affects much more than just guinea pig pilot regions, it makes the grid unbearable for a couple days while this is going on, and those of us who manage a large number of regions always have to deal with the finger pointing towards us during these times. It's not fun.

Regardless of the excuses provided, your release engineering procedures should aim for less rolling restarts, period. The current situation is pretty depressing.

Hi reacher,you know me,my homestead for a week had absolutely bad lag,it was hard to walk anywhere,it was so frustrating,i call it walking through mud chest high because that what it was like
I was on my island when i got the message for the rolling restart,when i came back 10 minutes later it was wonderful,i could walk again,hopefully it will last,

The collisions are greatly improved! However sometimes longer objects pierce through others when they are moving at higher velocity and preform a collision on a object or avatar on the other side.

The collision improvements are greatly improved, but until that is piercing problem is fixed, its still not as good at stopping things as H1.

Mainly this is an effect on the combat community, though I am sure they will be pretty happy that objects at least stop other objects a lot better now. But why settle for half way while your busy fixing things.

On that script memory thing, I'm seeing some real freakiness. Some scripts I've been using for months suddenly stopped refusing even to recompile, with a not enough memory error, under two different avatars under both 1.25 and 1.24, both this evening. Commenting out some stuff, I was able to get them to compile but free memory was way too low, making be wonder about the bug mentioned upthread (but this stuff was all ok on beta, so it was still puzzling).

FWIW, the pre-dinner compiles were being done while logged in with the Kirsten S16 viewer, which was doing fine yesterday.

After dinner the new RC came out, so I loaded up that. what I noticed after re-opening the affected scripts is that the Mono checkbox was undone, but it had been enabled on this stuff beforehand.

Going back now to the Kirsten viewer, I find the same problem, that the Mono checkbox seems to be getting ignored, it appears to be checked there, but back in RC6 I see that Mono is disabled.

So, if you are having sudden script memory problems today starting with a recompile, give RC6 a spin and see if that helps.

(ETA: 1.21.6 viewer does not seem to be stumbling on this, only the Kirsten viewer is breaking Mono compiles for me tonight.)

I'm seeing problems with scripts too. I've got a door script that does its calculations for where to move to, every time it opens/closes, and that seems to be dying a terrible death. Changing it to calculate on reset only fixes, but then it doesn't handle being moved...

Edit: Sorry if that seems vague, I'm having trouble getting anything that is readily reproducable. If I figure it out, I'll write a JIRA.

Edit 2: My fault, nevermind.

Once I get the new RC downloaded*, I'll be in-world on my Island, and the limited edition bear dispenser will be out again.

This is your last chance to get this limited edition rolling restart commemorative bear! Well, unless we have to do another 1.25 point release....

[ * plan ahead FAIL ]

The roll is done.

Here's the deal with SVC-3679. There was a change in the way that the scheduler for Mono scripts works, and that affected how memory usage is calculated. It turns out that previously, you were able to get away with briefly using *more* memory than you are supposed to be able to use in a script. E.g., by assigning a long list through a function call with another long list as the argument (list = foo(list)), you could briefly use more memory than you were supposed to be able to due to the temporary storage needed for the assignment. Previously, the scheduler was such that you wouldn't swap out during that assignment, and the memory usage calculation done while you were swapped out wouldn't "catch" you using too much memory. Now, you might swap out while the assignment is happening, and hit the out of memory condition.

Fixing this is non-trivial, and it's Friday, so we're going to think harder on over the weekend and on Monday. Options will include fixing it one way or another, or, short term, reverting the scheduler behavior while we take more time working on a fix. (Note, however, that this scheduler change fixed *other* problems, so it's not blatantly obvious that we want to revert the scheduler.) In any event, we are not going to revert back to 1.24.10-- 1.25.4 will stay on until we figure out what to do next. We may well do a 1.25.5 that addresses this in addition to adding more database fixes. Watch the status blog for more updates.

Quick question about SVC-3670, why is the key for llGetInventoryKey() on scripts and objects hashed now?

I know the double touch bug accidentally re-surfaced in 1.25.3, however I was under the impression it was going to be fixed in 1.25.4. Was it fixed or not? I am still seeing double touch behavior under 1.25.4.

the region of brussels went down with the restart and has now been down for 5 hours or so pleasea advise

hmmm i've noticed this too and i thought it was just my mouse going wonky.

llGetInventory() for objects and scripts was able to be used to make copies of objects that aren't supposed to be copyable, I *think*. I do know that we've had multiple security vulnerabilities in the past that wouldn't have been a problem if this information wasn't available. The problem is that a bunch of the code was written assuming the UUIDs weren't generally available.

Originally in 1.25, exploits were closed by returning NULL. Hearing that some people used this to see if the right set of scripts and objects were present, we put in the ability to get a hashed UUID. That can't be used for exploit purposes, but can be used for fingerprinting purposes.

Re: double touching events, sigh. That *was* supposed to be fixed in 1.25.4. We'll look into it.

This was described as a security fix earlier in this thread, yesno? We were supposed to get an explanation after the rollout.

Oh, Prospero...?

I have an object that was a pretty reliable victim of the double-touch bug, it's working correctly for me. That said, we had observed that that problem could be very load-sensitive.