Source Of Server Trouble

Thank you for your feedback on my little story Madame

I'll just throw my opinion in the hat so it is known. I came into Second Life nearly two years ago and I instantly fell in love with it. I saw myself change with Second Life as much as the world around me changed. Some old friends stayed and are still here while others left and I meet new people and make new friends all the time. This is NOT a world I would give up easily because of a few problems. Linden Labs spent New Years fixing the server so we could have our celebrations and took off time with their friends and family to help us. They are very dedicated to their work and that's all we can ask for.

As for the story, I wrote it late in the night while I was watching the scene with the Architect in The Matrix and saw how Second Life is kind of like its own Matrix without the killer robots (except for a few of the Lindens ) and plugs in our heads. I hope you all got a kick out of it

I think frustration has taken it's toll on humour! and people to forget to take it at face value.

Derek, I loved your little short story so much I shared it with my non-sl friends. We all got a big kick out of it. SL is the closest someone can get to "a dream come true". Where else can one take the path not chosen the first time around and if you mess up take it again?

Thanks to this game, I'm slim, I'm beautiful, I have closet that puts the Hilton girls to shame, I have my own island and a husband that loves me!! Yay!

I have to admit I feel irritation at times, when SL is not performing as I want, but at the same time I don't complain - that's deconstructive. If I have a suggestion, I'll make it, that's constructive. When I see something funny I LMAO or ROFTL. Actually I don't really roll on the floor . When SL is down, I come here to the forums or, HANG ON TO YOUR PANTS, I actually work productively at my cubicle (shhhh!!)

Write more Derek! I love your mind

Dearest Gwyneth (and others),

You said: "Aww Vendaia, that's really Murphy's Law with a vengeance! I mean, you couldn't have a larger stroke of bad luck - it's almost impossible!"

A touch of sarcasm? Thanks for your foreswearing of ad hominem attacks.

I love SL, and I agree with your assessment of the time required to stabilize a large software project. (This is why banks still run and maintain legacy COBOL, perhaps?)

I was genuine in my appeal. I wish to find out if the part I own in my interaction with SL can be optimized to compensate for bandwidth or graphics processor shortcomings. These are things I can control. Too, I would hope to offer some meaningful feedback to LL on system performance issues. I am one of those with "programming experience" (Assembler and C, Microcontrollers).

I perceive that you seem to feel that constructive feedback is somehow an attack on LL, SL, or even yourself. If my perception is incorrect, please advise.

So far, I am still having problems with client freeze (Windows Task Manager gets it unstuck, but showing a 100% processor usage - a loop?), sim crashes (rescued by another longtime resident in the nick of time from one of these, advised that the sim was overloaded.), and the immobility problem (AVie does not respond to movement controls, but otherwise the system is functional.)

The reason these problems are so frustrating is that I get so deeply involved in interaction. I may be in the middle of a very intimate conversation, or, well, whatever, and BAM! - a freeze! Oooo... it's so frustrating!

Keep in mind that some folks may be turned off by a product that is a little shaky due to its vast complexity, and not have the understanding that you and I do of the difficulty of building a huge project. This hurts the growth and health of LL. And that prevents me from making too much of a financial investment in SL. We all have different levels of risk tolerance. Those should be respected.

I earnestly want SL, unique notion that it is to blossom. I hope that we all want that. For that to happen, we must make visible our difficulties, if the system is to improve. We should be able to do that without risk of being flamed, even slightly sunburned, in the process.

Hello all. I'm somewhat mollified that I never swung by here to find such a cool thread.

For those who have not seen my earlier posts, I am a UNIX SysAdmin for a business in the Financial sector. So I'm quite familiar with what SL has been going through, and its probably still not near enough to what I've gone through in the past. Still, my heart goes out and my hat's off to them - I'm enthralled that the concept works at all.

I agree with the general gist of the thread - there is a problem somewhere in the SL system as a whole, and it desperately needs to get fixed. I bring this up as I think the asset server or something talking to it is having the hiccups again, to the point where I cannot teleport out of my home island (where I was at the time of logout, and therefore quite stuck *shrug*).

What's more important - and I've brought this up elsewhere - is that there is a point where the non-SL software may need to be looked at as well. The Asset server, for example, could probably be better done as an Oracle or Sybase clustered database (and therefore get uptime of four or five 'nines' to use the terminology). But the licensensing and support agreements for both are expensive, and the SL code will probably need to have all the queries re-done as well.

So such a thing may happen, but as a very long-term plan. But it may well need to be done. There are just times when you'll eventually need to throw closed-source software (and in the process big amounts of RL money) at the problem. I have nothing against MySql, it's nice in its own way, but would personally prefer software in use that has had more problems thrown at it for heavy real-time processing (which is why I suggest Sybase or Oracle - both of which I have personal experience with).

The asset server isn't a database. I hope the nice person who agreed to talk to me doesn't mind this masked reprint (was a Linden):

Jsecure Hanks: oh the asset server is a chained replication system?
Jsecure Hanks: cool, that's a smart idea
***: It's not actually a database, but we use a similar method to replicate the assets.
Jsecure Hanks: it's not? How do you store all the data?
***: It's all done via http. PUT/GET.
Jsecure Hanks: So you have a mysql system, which talks via apache in HTTP PUT and GET?
***: No, it's jsut storing the assets in a hashed directory structure.
Jsecure Hanks: ok so how does one of those work, you have a file for each asset? (I'm learning a LOAD tonight
***: Yep.
Jsecure Hanks: it's quite fast really....
Jsecure Hanks: So every time you fetch an asset it skims a file for the details and sends that back via apache
***: It just sends the file. It's that simple. You want that texture? Ask the asset server for it, and it grabs it and sends it to you.


Think outside the box people, the Lindens did

Vendaia, there was absolutely no sarcasm intended! I really meant it. You were a victim of several things that went wrong at precisely the same time. This is rather unusual, and that's what I meant with my comments. I'm sorry, I didn't know anything about your own background. I didn't mean to sound condescending/sarcastic/whatever. Electronic communciation has its quirks for the lack of body language and I do apologise if you seemed to get the wrong impression.

Jsecure, now I really loved your post So that's the reason someone wrote in an earlier post: "LL does not use MySQL for the asset server". I was intrigued by that, because it did not make any sense to support two different databases (especially if both were open source!).

My reasoning at that time was, well, assets are just hashes, if they'd ask me to implement it, I'd just use the filesystem and no database at all. However, I shook my own head with a smile, and remembered that I'm used to work with crappy hardware and limited budgets - meaning I often need to resort to very crazy ideas to get results fast - but the Lindens have no such constraints and they probably have implemented something Way Cooler.

Also, the hashing-system-using-filesystem is hard to replicate, thus, it's a "hard to scale well solution". You either have to replicate the whole filesystem and keep it in sync... or change the clients to connect to different asset servers depending on the hash... in any case, it's hard to scale well. Worse than that, I wouldn't trust a Linux system to handle more than one thousand simultaneous connections to an Apache server, so there would really be a problem when the population in SL increased.

So I casually dismissed the idea, and thought that they were simply using something else

Well, unlike many of you, I now feel more in awe with the Linden team. Seems that they're not afraid to test "unconventional" ideas because they're so much simpler to implement and have much better performance In any case, yes, there are scalability problems with that approach, but I guess they fixed them. As I once wrote, I hope that when they finally release the code to the public as Open Source, I have enough time to see all the cute tricks they've done. It will certainly be a lesson

I must admit that I spent almost all my professional life searching for unconventional solutions with limited hardware, software, and a budget approaching zero. Sure, I've also "thrown hardware" into problems to fix them. But I still remember my disappointment when I once replaced a Linux mail server with 20.000 mailboxes with a Sun Enterprise machine. Sure, there was no question denying that it was much better. But certainly not "better" in the proportion of money spent Ah well. As all of you will agree, a better algorithm will beat faster hardware every time...

Trade-offs, trade-offs... Right now at this point in time, I'm not sure what really pays off anymore:

- an Oracle/Sybase solution, which will demand much more from your hardware, but that will work flawlessly for years without maintenance, but you should be prepared to pay for it anyway - you're going to need it if something goes really, really wrong.
- an MySQL system with zero costs, but which needs one or two DBAs + sysadmins to constantly look & tweak the system to keep it running. Your initial investment is zero, but running costs are probably higher than a simple "maintenance fee" to an Oracle solution.

What is best? I really think the answer depends so much on the corporate culture of the company buying the solution. You either trust your team, and go the way of MySQL. Or you trust nobody but brands, then Oracle is the way to go. As to squelching performance from either, the answer is simply - rely upon a good DBA. A badly deployed Oracle installation will always be worse than a badly deployed MySQL installation. You need to tweak either type to get decent results. As for stability and reliability... in my experience, MySQL works as well as the filesystem underneath it (while Oracle tends to be pretty stable even with a crappy filesystem below - for historical reasons, Oracle Corp. never trusted OS filesystems in the past, so they always worked around it). Put a Veritas filesystem beneath MySQL on a nifty RAID box, and you won't lose a bit of data with MySQL. Then again, for the same price tag, perhaps you could run Oracle on top of your FAT32 filesystem, and don't lose a bit of data that way, either. Again, trade-offs, trade-offs. I really don't know what the best solution is

Indeed there are problems with e-communications. And you have mended our small misunderstanding quite graciously, and I mean that sincerely.

Have any of y'all read "In The Beginning Was The Command Line" by Neal Stephenson?

I honestly wish that Windows would give me some sort of meaningful error message when SL, or any app for that matter, crashes.

(Please, please, please, all you fine Linux users, don't jump on me about being a Windows user. I beg of you!)

Had I some sort of message, then I could give better feedback to LL on issues.

As for the client freezes, Task Manager shows 100% CPU utilization, then the client returns from "Not Responding" to "Running". My guess is a loop that runs until my network is able to receive data. But the client has yet to un-freeze without my deploying Task Manager to check process status.

I have noticed that The Immobility Problem occurred yesterday after my Avie had been seated for a while, then stood up. She would not respond to movement commands for a while, then she started to do so. She was able to turn around, but spatial translation was disabled.

As for client crashes, I have no clue. No pattern has yet to emerge.