Much of the grid was down... now back up.

Well, its too late to announce that much of the grid is down because is is back up.

As far as we can tell, some Internap routers that we rely on for routing traffic between two of our subnets went down for a while. When the simulators lost connection with the rest of the world they shut themselves down.

We don't manage the core router between out two subnets because the theory is that Internap would have more reliable hardware and service than our own, given our own experience in that area. It appears that may not be true.

Shortly after everything went down the very frontline of our response team lost their network from home for unrelated reasons and it took us longer than it should have to bring everything back up.

Very sorry for the inconvenience.

We found out this morning that Internap had scheduled a card replacement on the router that connects our two subnets. Unfortunately they forgot to tell us anything about it. They're currently trying to figure out why Linden Lab was not included in the list of alerts that were sent out.