Grid Emergency Maintenance 1/22/09 5:30-6:30 AM SLT

Good Job FJ Keep up the good work!

Mar

it looks to me like they did say that at the end of their message. just think about this, if you could run a virtual world better than they can, you probably wouldn't be here to bitch about theirs

Thank you FJ for the information. I really appreciate it to be informed before something like this happens. Now I can do some work in Photoshop while you are working on the database instead of trying to script and build stuff with unpredictable results.

A little thought of mine, which you could discuss with your fellow Lindens, please:

There are quite a few europeans (and don't forget our african friends) on the grid now. I guess, a lot of us would be happy, if you could give times in UTC / GMT as well as in PDT / PST in your posts.
Something like: "We are scheduling a 60 minute maintenance window beginning at 5:30am PST (13:30 UTC/GMT)." We still would have to do the calculations according to our very own timezones, but believe me, that's a lot easier because the time offset is smaller and we are all used to UTC/GMT calculations. Thanks for considering this idea.

Good luck

Frank, you're doing a great job. I know the headaches and difficulties associated with server migrations, etc. Good to see somebody that knows what they're doing. All the best!

You guys buy SLX and now it seems to be a mess also tried for a funds transfer tonight and it failed. In 2 years I have never had a funds transfer fail. Can I say typical Linden management. Touch it and break it...

FJ, two thumbs up to you for actually listening and understanding what needs to be done. Migration of services are never easy, but often a must as the business progresses and increases in growth. Maintaining extensive clusters with constant heavy-loads has impact on more than one sector, doing preventive steps in good time is crucial. Even more so, informing users who are using the service of a planned outage is essential as to show signs of communication. Well done and you certainly showed us that there still is hope for a working communication between a company and it's user-base.

Thank you Mr Linden..
Thank you.. thank you... holy lab..
Right from our hearts.. we thank you..
Thanks again... oh mighty lab.. thank you..

For doing 1 good thing.. XD

hi
i m soumya diesel .since the day i have downloaded the secondlife new version that is 1.21.6.99587. . . .i m facing huge problem.i cant stay for a while and i cant teleport to other places ,i keep crahing all the time in very short duration like 5 mins,help me please

Nice to see that Frank Ambrose (FJ Linden) is a serious person.

In my book (and since my rezz day) Linden employees have managed to put themselves in the blacklist of technical incompetence until some bright examples proved themselves different. Not that I wanted to... but that was just my view of things based on tech facts.

Only a few Lindens have survived and Frank Ambrose (FJ Linden) has been added recently to my whitelist also. (Thank you FJ Linden)

Starting to consider the grid not as a small garage ISP of the 90s but as a business environment that generates millions, thus technically starting to treat it as such, is a huge step compared to the previous technical approach of Lindens.

Those who have read my blog comments the last year know what I'm talking about. I am sure in a year or less (when other things have settled down) LL will switch to an RDBMS good enough for the job. I'm pretty sure (just guessing) that FJ is already in the process of examining the next database system to use and is in discussions with vendors.

And perhaps in 2 years from now or less SL will be a place that businesses can consider stable (both technically and management decision wise) enough to invest in it and start an SL business with RL monetary projections.

Cause right now, most SL business transactions in here are just a joke and only for lemonade stands.

Soy Nakamori

I know this is a little bit off topic, but... Isn't it about time that you also put a clock on the website?

That would make it so much easier to check if the blog posts we read faithfully every day... have any reference to what you are doing right now.

Time and Time again we see blog post (by the way thanks for them) with references to Time slots. But you have to log in to see what time it is in PDT...

Good luck though with the work ahead... eh.. work underway... eh... work you have done..

Good to see some positive action on the database problems,

Could you possibly put out an in world notice shortly before this happens so that we dont get people trying to relog and then being locked out for an hour.
Thanks

Do you want them to tuck you in at night also? Sheesh.

Ooo yes thankyou fj for the short notice!! Sooooo kind!!! Ermm what about those of us with time critical businesses to run, eg weddings? No matter I guess as our little businesses are just play fun in your multi million dollar scheme of things.... Joking aside we need more notice and this information is critical and needs to be on the login screen. Only people with spare time can read the blog before logging in.

Still I see that those that know what this all means are happy to see the maintenance and I'll trust that it'll mean and end to the daily "logins have been disabled" and that, fj, might mean you can continue to completely ignore all the references to system resource abuse and traffic rating abuse by multiple logins (ie bot farms). Oh yes you listen to us, but not answer what's of obvious concern but not directly linked to blog.

Please as asked by other forum posts will you now consider a blog on the issue of bot farms, LL's views, situations where we can presently file an AR and initiate a discussion so we can all express our views in the right place?

Best wishes on the maintenance!
I'll be bunkering down in my hide hole since I won't be able to escape the sim lol

I don't have a problem doing the conversion to SLT in my head (subtract two hours), but a clock on the website isn't a bad idea and wouldn't be hard to implement. (You'd think.)

The challenge of dealing with an RDBMS that can handle the enormous traffic levels is huge.
To be honest however, the database they are using is one of the fastest out there because it makes compromises in some areas that other commercial databases (Oracle) don't do. This isn't an issue for SL since it's not a traditional transaction oriented application.

You are dealing with a system where every user, every prim, every object, every texture, every parcel, every group, every notecard, etc. is an asset in the database, with permissions, owners, and tons of other attributes. All these things are available grid wide in multiple datacenters nearly instantly.

So how do you handle that kind of load? Well, there are many ways but some common techniques are to segment DB's by function, and also partition where asset ID's are hashed and directed to different mini-clusters. But as outsiders we don't really know what SL is doing. We do know however that it is no easy task.

It's so easy to armchair quarterback and say "use Oracle" when that would be one of the worst decisions EVER due to the HUGE performance penalty, not to mention that Oracle would charge MILLIONS in licensing fees that couldn't be used for more programmers to fix bugs / add features.

How about reading it while you're logging in? Or does it instantly pop up with everything rezzed when you click the button? Because it doesn't for me or most other people.

Solution to stability, and future growth.. etc etc

Decentralize the grid asap.

It was announced officially by former ceo Phillip (somewhere in 2007 on a virtual world congres), that LL would openup their server codebase.. and guess what.. still nothing (missing a big oppertunity here!!)..

Now LL is busy with a instable reversed enginered (based on .net ) project instead of opening their own stable server code. It's nice to be openminded to other interoperabilities.. but this is a way to long term project.. why not also look at terms closer to today?

so.....

Really, what's the holdup for not releasing your own region/estate code, that's been proven stable. And has all the features we use/know. Is it licensing? why? Havok runtimes/libs are allowed to be distributed to users freely (compiled ones ofcourse), just like games do (when they use it). Vivox? same, although it's hosted remotely. The rest is already opensource based code merely, as i understood, thus distributable..
You could also think of a 2 way method.. one opensourced (excl. the propriatary codes.. but those propriatary things could be provided as libs compiled)
and another method: all precompiled, in packages like rpm, deb, and tgz.
all one needs is signing up, with the IP address(es) of the remote region(s) running, and an automated check tool, to see if everything is configured correctly. (local and internet wise).
If configured properly, there will be a firewall session between the two parties, maybe a vpn too. So only that machine can use the database services.. as an expension to LL's own hosted regions.

Support, is on payed basis.. (directly from LL)
and community support (forum, only accessible for signedup remote hosters)

i really think LL should quickly release the code of their own serverbase, asap. This way the grid is decentralized, and if LL has a problem, still people would be able to use the decentralized grid, impacting less customers/users.
And the database servers has less overload, since it's spread globaly. (pick trusting partners to replicate some services data, like IBM you work already close with)
load balance it, throughout the partners... so users are spread among serveral clusters, instead of ONE owned by LL..
if this current (centralized service from LL) single point of failure fails,.. the grid is offline/broken/interrupted severelyl..impacting ALL residents)

See it as the current ISP market.. in older days, everything was centralized.. meaning if a major isp had a problem, allmost the whole world (online that is), had a problem. Now everything is decentralized.. and the NET cannot be brought down anymore. it simply cannot.. there are too many isp's too many servers/routers/darkfiber/peers/transits etc etc that is simply impossible to bring down that easy.

Think about it, LL....
Better be safe than sorry, especially as SL is still hyped (a bit), don't loose the battle with other parties emerging... (sony's home, etc etc)

Stay future minded, let your dominant monopolistic arrogant feel go (no offence here, but it's common with monopolistic players in a certain segment.. success makes more hunger for success and money), and openup.. share.. commit.. you will be rewarded massivly, i promise you that.. (their are numerous projects to back that up)

8:09 am est, 5:30 pacific huh, so its something else that has logins disabled? Oh nevermind.

*crosses fingers that all will be smooth this weekend for events* Last weekend was an f'in nightmare.

The work they are doing this morning is an effort to get rid of the cause of that message, so it looks like this morning may be a really good time to get it over with =)

I like Chance was disabled well before 5:30AM. Not to mention being dropped out of game. Would be nice if its going to be 5AM to say 5AM. Thanks for doing this "if" it gets things stable though!

........as the devil drives you FJ. Good luck to you and the team.

eeek! a database migration..... rather you than me

Those in line won't be bumped is crap!! I was in the middle of hosting a shift and 15 of us where booted as soon as 5.30am SL time struck. We couldn't chat, IM or Local.

This is unfair when it's posted you can stay in world and too much of a coincident to not have been a known issue.

SL once again screws over it's customers!! I am getting fed up with this as i am sure many others!!

OMG , they kikc me out of SL

Why weren't there any detail of how this procedure is taken in the grid status website?
I read the news on RSS feed on the status site, and I thought I was to be kicked out from SL so I told others the same, said good bye and logged out.



NOW I CANNOT LOG BACK IN and KNOWING I COULD HAVE JUST STAYED THERE!!!


Please please please put full detail on the status websites, not the meaningless front blog site!