Rolling Restart for 1.32

I'm going to give you the benefit of the doubt and assume you have never dealt with computer programming. If they wait until ALL of the bugs are fixed, then we'll be waiting a really, really long time - like, 10-20 years or so. Can you wait that long between releases? There's also the matter of being able to find and duplicate all of the really obscure bugs that only trigger for someone with a specific series of video card with a specific range of graphics drivers on a specific version of Windows, between 2:00pm and 2:02pm, on the fifth Tuesday of every month. Without an unlimited beta testing group, these bugs very probably will not manifest themselves until released to the public at large - and *someone's* bound to come along with that really obscure motherboard chipset that SL plays pure hell with. At some point of time, the project manager has to decide how many of each category of bug is acceptable for release, and roll with it, otherwise our children's children's grandchildren's children will die of old age waiting for SL to be released.

On the subject of bugs: I am not experiencing teleporting issues. In fact, my teleports seem to be snappier than they have been, which is a good thing. My problems occur *after* a teleport; namely, it takes far longer than normal for textures to rez, and for whole prims and linksets to even materialize. Since 1.32 hit my region, I can teleport home, find my couch, sit on it, turn on my wall-mounted television, and be most of the way done with a music video before my walls, floor, and ceiling even show up! And texture loading priority seems to be out of whack again, and really, really slow. I teleported to Bare Rose recently. I was able to get up, go to the bathroom, wash my hands, go to the kitchen and make myself a sandwich, and come back to the computer to find that most of the textures still have not completely loaded! I'm also encountering more and more "invisible" avatars: Avatars whom the only thing I can see of them are their prim attachments, or a default brown skullcap. No white clouds, no Ruths. Just, simply, no avatar at all! Asking the mostly-invisible avatar to rebake or to hit Appearance usually fixes this.

The "deruth" option on Flight Feather should still work as well. It just takes longer under Havok4.

Anyone notice they took out from the grid status page when the rolling restarts will be?
Theres one for the 17th and 18th well there was one.

Also notice each time I goto the forums I have to relog in... each time ggrr

Having the Second Life Planned Outages Calendar was kinda handy ya know why did you guys take it out?

hi,, love you friends..!!
& plzzzzz help me coz I'm not able to login.. dont know why coz this site having problem with me...!!!

OK, about since when the new server code was pushed out, I started experiencing huge issues with SLOOOW rez times. Going places and it takes 5-10 times the usual length of time for prims to appear. Or they will texture in part, with the texture remaining blurry for the entire duration of my stay.

At first I thought it was just me, but I'm heavily involved in a grid-wide hunt, and *everyone* is reporting the same thing. Not a single person has said, "no, everything seems just the same as it was". This is countless locations, from near empty to totally full, from dead quiet (low traffic) to highly busy. I have now been standing in one location for 10mins, waiting for a deck to appear and it still hasn't.

Furthermore, it seems that bandwidth values have been clobbered: I used to have decent bandwidth, but now it is dropped to below half, and the meter shows constant packet loss. Again, I thought this was just me, but everyone I talk to is reporting exactly the same thing.

Naturally, I have no clue whether the new server code is responsible, or whether it is something else entirely. It just all seems to have started about when 1.32 was pushed out. And frankly, it is driving me nuts.

First a quick caveat - I don't understand most of the scripting talk in here.
However: I also see the rezzing of "invisible" avatars as an issue. I also frequently only see prims until I zoom in on a new teleport and unprimmed avatars are often totally invisible at first. I thought it was just an artefact of my graphics, but others are reporting it too. It seems only to have arisen recently, but what is it and why has it not been dealt with?
Also textures can take ages to rez and clothing rerezzes at leat twice when it is worn - that surely is anomalous behaviour.
Some sims rez very quickly, others take ages or sometimes never fully rez unless I stay for hours. It does not appear to be related to script or prim numbers - one heavily laden sim rezzes almost instantly. What is going on?
Finally, I notice this (GMT) morning that approximately 750 sims are having a rolling restart - what to and why?

No clue - and I am also a bit surprised not to see more mentions of this, given that *everyone* I have spoken to has been reporting issues of snail-pace and often incomplete rezzing, over the last few days. Sometimes relogging helps the situation, for a while, but that isn't guaranteed.

And yes, I'd love to know why we have a new rolling restart going. No reason is given on the status page. New version deploy, or... what?

Ever since 1.30 came out, my sim has been laaaaaaaaaaaaaaaaaaaaaging badly, and calls to support give the unhelpful "there is not enough free script time, we don't see anything wrong." After nuking the top 100 object that show up in top scripts, guess what - NO CHANGE AT ALL!!!!! In fact, lag is even WORSE than before now that 1.32 has pushed. My sim has NEVER performed this poorly, even when I was at around the 15,000 object max (I'm down around 10K now.) I guess the new max limit for objects in a full standard region running new server code is only, what, 5000 objects before lagging badly????

If you (LL) don't get a handle on this soon, I'm going to shut my sims down because at this rate, they will be unusable in a month.

We (content creators and sim owners) have been begging for better script profiling tools for a LONG time and we still only have the USELESS instantaneous "top scripts" view which tells us exactly NOTHING!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! We must, MUST have the ability to view TOTAL script time used by an object since sim restart. That is the ONLY way we can find poorly behaved items and manage our sim's script usage. We also NEED to be able to limit the CPU time based on a policy framework WE can define of an avatars scripts (worn attachments AND rezzed objects, separately.) I need QoS features. I need the ability to say that a landowner with 16Km is guaranteed, say, 20% of the script time while a landowner with 1024m may only be guaranteed 2%.

So....

We NEED:
Script profiling tools giving content creators the ability to analyze their scripts (llGetCPUusage() - give total script CPU time since last rezzed or last sim restart)

Another column in Top Scripts for TOTAL CPU time since rezzed / sim restart

Sim-owner definable script limit policy controls based on group membership, prim allocation, or other criteria. (Hell, I'll manually upload an XML file. No GUI needed.)

LL to analyze what they broke in the recent releases that have such a severe impact on performance

Fixes to the "sim freeze" problem that occurs when a heavily scripted avatar TP's in or out of a sim (Resource limits would fix this people!!!!!)

No, that's expected. Each clothing layer is first downloaded and rendered for you, then baked and uploaded and set as your baked layers... and then that gets downloaded and displayed.

I'ved noticed the same thing as Ayesha Askham most of the time i try force loading ( ie: right clicking and editing each thing to get them to load ) and avatars with just prims loading until you login I believe this has been discussed for sometime

I've been in SL going on 6 years now and for a while it was getting better but now it seems like they just keep breaking stuff over and over unless they are trying to introduce new code
that will work with the new viewer but since it's not complete it looks broke to us


LL serious wake up and actually read what we type in here we sure are not doing it for our health

Your making what around 30-40 Million USD a month just off server rental you think that would pay for some kind of tech support

Any news on todays rolling restart, Lil? Ugly security stuff? Date/time weirdness? Babbage make progress on SVC-4196/SVC-3895??

I'm curious too about why there is no information about the ongoing Rolling Restart.

Also the way groups of regions are being restarted is different than usual. They're restarting in groups of 640-750 regions. What does the size of these groups correspond to?
Racks in the server farm maybe?

O.o is my EngRish understandable at all?

With either 4 or 16 sims per host server, this sounds like more than a rack. They might be doing groups of racks, though.


Yep!

Could they be rolling all the sims on older servers to class 5's?

Not if mainland regions are being restarted - those have all been class 5 for a while now..

Thats what I was thinking.
It would be nice if they said what the rolling restart is for. I guess predictability isnt needed any longer?

Maybe - but in that case, why not say so?

Maybe they're opening up the new east-coast colo and moving a few regions over there...

If it's for a security problem they're not going to tell us until it's over.

That's probably the most likely reason but even then, we usually get a bit more of a heads-up.

They're also going a little differently this time, doing 750 regions or so at a time, instead of the normal small pilot followed by two big rounds. Seems like LL is being a little extra-cautious with whatever it is they're doing.

I hope they get the asset pipeline issue solved. Just watch sims pending downloads. been getting hosed up since the previous rolling restart. Getting old having to restart sims just to be able to attach something or rez a texture.

Lil, I would LOVE to hear some sort of explanation as to what is going on. Here are the facts I have gathered, but they make no sense to me:

(1) Sims are being restarted in small batches (very unusual), and when they come back up the server version is the same

(2) There are no blog posts as to why restarts are occurring, and no restart schedule posted (very unusual)

(3) The restarted sims are coming back up in less than 2 minutes, regardless of how many prims/scripts there are (very unusual)

(4) Two of the mainland sims I've been watching were moved from Dallas to the Phoenix datacenter after restarting (seems unusual also)

(5) Sims which have had no changes made at all, and are devoid of all avatars (except me), now show MUCH higher total script time and little or no spare time, whereas before there was ample spare time

I guess I am most curious as to how the reported script time and spare time can change so drastically when it is the same version of the server code running??? All the other sim stats seem normal -- it appears scripts in general are now using much more time than they were just a couple of days ago.

Thanks for any light you can shed on the subject.

Seeing this as well. Things were fine with about 8ms spare time, then a heavily scripted AV TP'ed in, brought spare time to 0.0ms, TP'ed out, and the spare time has not recovered even after 30 minutes!

I think there's been a few issues with correctly reporting things like script time lately.. Do you think these could be reporting errors (displaying the wrong number) or do you think that the sim is actually getting more load on it?