Sim rest on New Years Eve?!?!?!?

Wtf? Thank You Sl!!!!!

It's an emergency!

https://blogs.secondlife.com/community/technology/release/blog/2009/12/31/server-1342-emergency

/348/30/358040/1.html#post2655138

only new years for some.

Go out, have a blast in that other high rez world called RL! (and its only a rolling restart, no biggie, cept for shoes up bums and things *laughs*)

There sure have been a lot of "emergency" fixes lately.

/me suggests maybe a bit more testing of server releases before they are actually deployed.

Some things just don't show up until the server version is under stress.

Some several billion people.

No biggie? Tell that to the people like me who have jobs in SL that have seen their income and their ability to do their job thrown in the garbage by this very poorly planned and executed bout of stupidity from Linden Lab.

I am a DJ. That's what I do. Providing entertainment to everybody. Even you are welcome if you choose to arrive. However glib you want to be about the idea that I should be in the real world, this is still a job. This IS the real world, because I am a PROFESSIONAL DJ. Thanks to this wonderful piece of work the Lindens have foisted on us I got to see a lot of planning be reduced to absolutely nothing.

I and many others have some extremely valid reasons to be angry about the way Linden Lab has chosen to treat their customers.

Happy freakin' new year.

Oh get a life. I have a business or two to run as well, do you see me screaming?

You were unlucky, it got you this time, next time it might be me.

I have no doubt that they would not of done this rolling restart unless it was extremely urgent.

For all we know they may have found a bug that was being actively exploited to do something very bad, how much would you of screamed if they didn't do the restart and it caused you to lose, oh I don't know, all your L$, or maybe your inventory.

As the saying goes, "sh*t happens, deal with it"

It would be nice if Operations were allowed to be a bit more forthcoming about what specifically was the threat being averted by the update. I'm pretty sure they didn't do it to amuse themselves on New Year's Eve, nor out of some Y2K nostalgia. Given the calendar, the short notice, and the compressed scheduling of the restarts, it must have been a Very Big Deal. I'll bet that if we knew more precisely what was going on, nobody would be complaining about it.

On the other hand, it's not difficult to imagine reasons that would preclude talking about it.

Got me wondering... considering how disruptive to central services is even an accelerated rolling restart, what if the entire grid went down? How many hours would it take to bring it all back up again? Assuming the fix were already loaded everywhere, could they actually do any better than the 8 hours it took for this rolling restart without crashing the asset service, central DB, etc? That would be a pretty long outage for folks on the end of the queue.

wow the Y2K thing...Id forgotten all about that fandango..lol

I just know that they could have made an effort to NOT disrupt thousands of people at midnight. This emergency miraculously by-passed the Linden celebration in Voss. Because they made arrangements. The same amount of consideration could have been applied when dealing with the customer's worldwide celebrations.

I did not know beforehand this was going to happen. When my host contacted me and said the sim was closing exactly at midnight I called Live Help. Nicole Linden said 'if you have a complaint then post it. Good Night.' Just pathetic.

Instead of advertising the Linden New Year's party in the Message of the Day it should have been changed to let the customer base know about the pending update that would negatively impact thousands of celebrations around the world.

The MotD today should be a blanket apology for what occurred last night. But of course, we cannot get an apology.

I posted in another thread, but I'll repeat it here -- I can't think of any reason they would have gone to such extreme measures on such short notice if it were not absolutely necessary. And the only thing I can think of that would require that kind of emergency is the calendar rollover to a new year. So yeah, a mini Y2K event.

As someone else mentioned, imagine the entire grid going down because of a date problem. Not good.

Not buying it. Linden Lab would not have organized a New Year's Eve event advertised on the MotD and go ahead with it if they were not certain this 'emergency' rolling restart would not effect their festivities. If they could make arrangements for their event to go ahead at Midnight without a hitch they could have worked harder to figure out a way of reaching their goal without putting all of the inconveniences on their paying customers.

2010 did not jut creep up on all of the technicians in Linden Lab. If it was a problem they could not collective solve all year then they needed to make a better effort to let their customers know that they should not waste their time and money developing elaborate New Year's Evev Celebrations in Second Life this year. But that would have been too much like doing the right thing.

some people have so little imagination...

how about a sim crasher that causes the entire region to be mangled/lose content.. and oh it seems certain griefing idiots are using it to target anyone they feel like it'd be funny or vindictive to do so...

not good enough for you? how about a loop hole that exposes your account, or at the very least access to your L$ allowing the knowledgeable person to redirect your funds any way they see fit?

not only have both of those very specific things happened in the past, they've happened more than once.....

direct loss trumps incidental loss, and they'd be legally liable for the former if they delayed simply because it was an inconvenience for the holiday...

Void,

I'm sure those things may have happened in the past. But that's the past. We're talking about now.

What information have we been spoon-fed by our benevolent Masters? Just that there's some kind of a bug they discovered that "may" allow for some things to happen. That's it. Just a "maybe."

Guess what? On a night like last night, with so much real money at stake, a "maybe" is a far cry better than an absolute guarantee of massive, large-scale, grid-wide disruptions. And that is exactly what Linden Lab gave us with their stupid rolling restarts.

It is well known that rolling restarts have always caused problems for the entire grid, without any exceptions whatsoever. And this time for the first time that I've ever seen, those disruptions even extended into forcing prims in sims that were otherwise running along just fine, to suddenly turn invisible the instant they started those restarts. Not just a prim here and a prim there either, but (and I am not exaggerating) half the prims in the entire sim. Prims were still there, you could certainly walk into them and know you had done so, but not visible to anybody.

That still doesn't include the fact that people did not attend planned parties in the numbers that are typical of New Year's Eve, or that they were not tipping or spending the normal amounts, or the disruption of people's parties being shut down just before Midnight by Linden Lab by sim restarts at 11:47 PM which had absolutely no prayer of being completed before Midnight.

Compare all these real problems to what? Just a vague "might have been." That hardly constitutes an emergency that had to be dealt with immediately. If it was really that time critical then where is the real information that actually means something more than vague threats of god-only-knows-what "might have been" trouble?

do you REALLY think that a company that so loves it's holidays and time off during them is going to call in the extra folks necessary to not only do a rolling restart of the grid, but to also accomplish it in double time.... just for "a vague possibility" ? that's just plain silly.

as for information, I can almost guarantee you that there won't be any details... which generally only occurs in one situation... when it's a big deal security problem. otherwise there's usually a convenient list of what was changed and why...

as for the why they don't list those things? same reason microsoft puts generic labels on their security patches... because even though they've patched one hole, other holes related to it may still exist, and they're hoping not to give people a good place to start looking.

If you're subscribed to the JIRA mailing list, you may have noticed someone reposting two issues, a sim crasher and a permission glitch, in the public section of the JIRA in the early afternoon of new year's eve, and then another user shortly moving them into the security section.

No, I expect that if it's the "vague possibility" that Linden Lab themselves said it is, they'd wait and do the job right, without tearing the living daylights out of activities on the grid which are time-critical. I notice they didn't have any problems making their own festivities go off without the same headaches they pushed onto the other planned events. Imagine that. I guess it's of grid-wide importance that their party wasn't disrupted.

Whatever it is they were trying to fix, the whole bloody thing stinks to high heaven.

And as for the next poster's references to Microsoft, at least that company doesn't severely compromise the functionality of your computer while they're sending updates to other people's machines. That's why we need to be told more than people like Microsoft tell us. At no time has Linden Lab ever bothered to actually give us any information supporting their contention that it was an emergency, just a "trust us, it is." Their credibility is already damaged beyond the point where a "trust us" is sufficient. They need to restore that trust or I'm going to act like the people from Missouri: SHOW ME!

I can understand it, no worries. HAPPY NEW YEAR

I think it was poorly planned, well except for their event of course...seems like it could have been handled ALOT better with the customer in mind, more so than it was. There's an excuse for everything and if "Emergency" works so be it.

So for those of us who logged out before then, what actually happened?

I had a countdown thingy set up to do stuff at midnight but no telling if anyone would've been close enough to see/hear it anyway.

Glad now though that I didn't try to be in SL at SL midnight if I understand correctly that the grid went down just then.

I too wonder if sometimes it's a grid attack or something and they aren't allowed to say.

No it was preplaned to take the servers down the day of, there was a notice that went out about it in the blogs by Lil Linden. I think i read it mid day or so of New Years eve.

Heres the link for it:

http://status.secondlifegrid.net/2009/12/31/post853/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+SecondLifeGridStatus+%28Second+Life+Grid+Status+Updates%29

That DOES seem bad timing then, a lot of places and people were in SL counting down to midnight. (To celebrate, not to put the lights out. Lol)

I've read that entry. I was logged in to SL when that entry was posted. Originally the restarts were slated to begin at 6 PM and they would be restarting every sim. But then they realized they had things ready early so they started sooner than that.

It still fails to address how this was so crucial to completely disrupt the entire grid based on this supposedly critical bug which was still so vaguely defined as to be a "maybe something unpleasant might happen."

Facts:

Prior to this posting and the start of the rolling restarts, there may have been some jira reports of some issues as somebody else pointed out, I don't know, I don't have every single jira entry created being e-mailed to me. But those of us who were in-world doing our thing certainly were not having so much as a single problem that we were not having on Wednesday, Tuesday, Monday, Sunday, Saturday...... However critical this bug may have been, it still was not having a negative impact on most of the people who were logged in, or else I would have been hearing about it in multiple technical oriented groups I belong to.

At the time of the very beginning of the rolling restarts when they did begin deploying this new server, once they actually commenced rolling this code out half of the prims in the sim I was in became invisible. Still there, but impossible to see them. Relogging did not fix it, some people who were present tried that. I found the same phenomenon occurring in my home sim when I teleported home as well. Had this been caused by this supposedly uber-critical bug they were supposed to be squashing, I would not be complaining about that, but it never was an issue in any sim I had ever been in on Thursday UNTIL Linden Lab started their rollouts. The fact that it began instantly as soon as they restarted their first sim is no coincidence. It was caused BY the updates, NOT the bug they were fixing in the first place.

Less verifiable because there's nothing technical anybody could point to in order to prove it, is the question of attendance. This year was absolutely the lowest attended party I have ever DJ'd on New Year's Eve. Even though there were more people logged in to the grid than any other New Year's Eve I've seen in my time in SL. It was also far and away the worst paying gig I've ever done on New Year's Eve. The number of competing parties planned elsewhere was not particularly greater than in years past either. So where were all the people and why didn't they come to the party? All the evidence points to one thing: They stayed away because of the restarts and the other issues that came along with it, such as the invisible prims baggage I mentioned above.

Had this truly been such a critical issue that it was so time-critical as to disrupt the holiday plans of many people on the grid then I maintain they owe us some more useful explanation than just there was some vaguely hyper-critical bug they needed to squash and that we should trust their assertion that this was an emergency. Linden Lab has done far too much to damage their own credibility for a mere "trust us, it's for your own good" to cut it anymore. We need real facts that they're refusing to provide.

In the meantime the discontent and dissatisfaction with how this was handled remains, and their continued silence proves that they don't care if their paying customers are displeased.