Equipment Fails..Give it a break Whiners

There is such a thing as equipment failure....Nothing in this world is perfect (INCLUDING YOU) ... so the grids down, its not like the purposely shut it down on you..

The obviously didn't expect it...

Patience is a virtue..

*Is disapointed to learn she isn't perfect and is in total shock, runs to hide her shame*

So you felt the need to create a post just to berate people and whine yourself eh?

Hey ppl are bored, ya gotta expect random posts...

lol...

other virtues include; dedication, persistance, and devotion, all of which we can't practice while SL is down

There is also such a thing as redundancy, and anticipating problems before they happen. Monopolies are evil things, but one thing that can be said for the old phone system monopoly is that they did an admirable job of designing for reliability.

Exactly!

If it was a hardware issue, I'm surprised...no, shocked! that they do not have back-up equipment for critical hardware.

Perhaps they have learned another lesson today...."ooo, this is a critical piece of equipment apparently....perhaps we should have back-ups in the future"

hehe


oh, well

The problem is, they keep calling this a GRID....

Usually in the realm of networking, a grid can also be referred to a mesh network topology. In a mesh network topology, theoretically over half to seventy five percent of node (server) failures can happen and the rest of the mesh (grid) will remain operable. This mesh layout should also cover the actual networking devices when implemented correctly, there could be a 75 percent failure of networking device failures and the grid would remain operable, it would be really poor in performance but operable.

Perhaps it may be better if they told us it more or less resembled an old token ring network topology, at least then having to bring the entire grid down for 1 system failure would be more accurate.

It is true that equipment fails, the thing is to be prepared to deal with a failure, have backup plans in place, redundant systems ready to spring into action. It isn't like people hide the fact that yes, that cheap Wal-Mart hard drive can fail. What needs to happen is to actually have a plan ready to deal with it when the drive does fail. To not expect system failure is the same as not expecting a thunderstorm in Florida, a very ignorant thing to do.

That's why you set up redundant servers.

Redundant front ends in front of a shared SAN aren't redundant servers.

Last time we had a san failure where I work at I didn't leave my desk for literally 24 hours. After we brought the new hardware online--roughly 8 hours after the initial failure, security was hosed. I had to throw together an app that took ownership of every single file on the san and then set rights for the appropriate users. Of course that was on a windows based system.

My point is, I have a feeling it's going to be awhile for this problem to be fixed--longer than it takes for just the hardware replacement.

We built in redundancy at around 10,000 total users. SL has over 2 million. I'm reeaaallly surprised a single point of failure exists in a system this large.

I just wanna get in =/

The grid is up and operational, and many of us are in it... which means there is redundancy in the system (to some level.) However, when there is a failure like this, systems operate in "reduced capacity" mode.

For example, when part of a RAID array fails, the system is still up but the load has to be reduced because a lot of I/O and CPU is needed to rebuild that part.

Thus logins are disabled.

-peekay

From what I have read a hardware failure was encountered during the **emergency** maintenance on the asset servers, not the "grid"...

Maybe the emergency maintenance was to fix problems that were at the time not known to be the precursor to a storage device failure on one of the servers?

So, to keep organization and integrity of data on the asset servers they took grid down until the server is in operational order.

Or, at least thats my personal opinion on what is going on with the data I have gathered.

Wow has had entire clusters of 'realms' go down due to failure of one part... a router.

Backups don't help on the component(s) that actually funnel or protect your bandwidth from going out.

Servers can be backed up, but I highly doubt LL is paying for more then one t3 line going out of the building. They don't make that much money.

So you're telling me that you are in SL right now and only logins are disabled?


hmmmm...so SL is occupied only by campers?

hehe

And the hypocritical award of the day goes to .....

It is simply impossible to anticipate every single disaster scenario.

The Empire certainly didn't expect some kid from Tatooine and his smuggler buddies to shoot up a 4m wide thermal exhaust port...

you think all of SL runs on a single DS3??

oh wow.... just..... wow....

you realize that's only 45mbps right? That would be enough to sustain *maybe* 5 users with my downstream speed (10mbit). I assure you that while they certainly don't have the quality and quantity of bandwidth they really *do* need, they have *way* more than a single DS3.

Back on topic, a few of the posters here in this thread are right on the mark. It's very apparent that this asset server is a very critical part of SL's infrastructure, and there's NO reasonable explanation as to why they don't at least have a cold spare (they really should have a proper cluster). You can put all kinds of redundant parts in a good server, but you can never have redundant RAID controllers or motherboards. There's absolutely no excuse for a server of this importance not to have *some* kind of backup ready to put into place.

Yeah, but the empire had a back-up plan...RUN AWAY!! hehe


You mean they didn't have a back-up Deathstar?
~gasps~


You always have a "spares" list in ANY mechanical assembly. The spares list should contain any device or mechanism that is not easily replaceable, and is critical to the function of the main assembly.
Especially in a system that is designed and "expected" to run 24/7.

Congradulations, and what address do we send your award to..