News.com article on SL's scalability problems

Thank you, Jarod, for a lucid explanation.

Now, as a transitional step to a peer-to-peer system of asset serving(?) which I remember being bandied about a ways back I also distinctly remember (I think) a Linden referring to having multiple asset servers. Is this reasonable, and given SL's existing architecture, is it even possible to make a transition to a peer2peer sytem?

I don't think Second Life, will ever have a true peer-to-peer architecture, at least not how I think of P2P. I think the best model we can hope for is something like a many(1)-to-bridge-to-many(2) design: many(1) being varying asset servers; the bridges being something like squids, a nodal system for maintaining some kind of visual continuity; and many(2) being the client software that users interact with.

I would welcome being proven wrong, but even Garry's Mod for Half-Life 2 requires a HL2 server running when multiple people play. In that instance, the HL2 server (I believe) ends up acting like a Squid, it maintains the world so long as it's running. Obviously there's no visible asset server in that model, but I believe HL2 allows you to save instances of a world. If I'm remembering correctly, at that point everyone playing becomes a potential asset server. They dump their version of the world into the Squid when they want to play...

That's kind of going out on a limb of a yarn, though.

It's very technologically possible to transition to the M2B2M design I described, but the real question is that of social possibility. Right now uploading textures is one of the strongest money sinks we have in Second Life. If anyone can host their own textures on their own asset server, there's no need to upload. (This may change with web on a prim, but for the moment it is a valid point.) Also, would many skin and clothing retailers be willing to sell the actual JPEG files of their textures to people, so customers could upload them onto their own asset server? Would those textures retailers be willing to host the textures? Could they afford the bandwidth fees? And the same goes for animation and sound sales.

Yes, it's very possible to do this. That's a lot of what they're changing on the back-end now. However, and this will be the telling point about whether Linden Lab really wants to help build "the metaverse" or if they're just hosting a world: are they willing to open up the channels to their Squids, will they let us host our own asset servers?

I hope so, but I think we've got a long road ahead as soon as land barons and texture sellers realize just how obsolete their business models are going to become.

Too bad this theory has already been tested with WoW and there were many top notch interfaces that were made.

Chaff tends to float to the bottom my friend.

I guess the question I would have is it just an asset server or is it something more akin to the likes of a NAS?

The main performance hinderence when your in a sim IS the asset server, you can see this clearly as your moving about and the client struggles to download all the crap which has to go from asset server to squid to sim....

Ok if this is the case why not give each server the basic asset search engine and then hook all of this up to a mounted partition on a NAS server. On an EMC each mount has the ability to deal with 250 connections at a time. If this was done each SIM would essentially act as its own asset server.... this could be made even faster by changing the way the asset database deals with items in each sim.

The best part of a NAS is its the very definition of scalability ; )

Sure, and that's why Linden Lab has been migrating away from a centralized architecture. That's what I meant you don't know what you're talking about.

People complain a lot about what LL has done or should do, but they have to realise that getting something like this off the ground is an enormous challenge and all the problems have to be solved in order and not right away.

Errr, sorry, wrong post.

I think we have to give Linden Lab more credit. They're going to solve the problems in order as it is appropiate. This is a tremendously difficult thing to get off the ground.

It could be an NAS. Pretty much anything that runs a web server, as I understand it.

I'll be interested to see how this changed (or doesn't change) when we see sims and servers run simultaneously on multi-core/multi-processor computers.

You'd have to ask them about that. It sounds like a good model to me.

Sure it is, and I agree we do have to give Linden Lab credit for creating a wonderful technology.

Yes, because as we know game engines are a really rare thing.

good caching would solve a lot of this, too.

i devised a caching system that relies on zones and version numbers. it goes something like this:

areas of some arbitrary size or shape in SL are a zone. for fun, let's use property lines as the zone bounderies.

The zone has a version number. that number starts out as 0 when the databae is first initialized and increments every time a major change happens to the zone. (more on that later).
When downloading data to render, the client cache looks at the version number of the zone. If the version number hasn't changed since last time, the client simply renders the cached data.

this leaves two loose ends:
1. What is considered a "major change"? Any change in the fixed objects (as opposed to dynamic objects) can trigger a major change. The change would not actually be finalized immediately, though. When a user creates a new object, it would start the change cycle. The zone would be considered "open" until the change cycle ends. the cycle would end when no changes have happened within a certain period of time (say, 1 hour). Also, only "authorized" objects would be considered permenant, and would trigger the cycle. Unauthorized objects would be transitory and would not be cached. Instead, they would be downloaded every time a user enters an area. Unathorized objects would be anything that would be eventually auto-returned.

2. what about dynamic objects (doors, windows, and anything that changes geometry or appearance via scripts)? Anything with physics and anything that has had its properties changed via a script (and still has that script in it) would be flagged as dynamic. Dynamic objects would be updated every time the viewer enters the area. An object would be flagged as static if the script is removed or disabled. Changes to dynamic objects would NOT affect version numbering. Instead, the server will simply send updates to the client, which will display the updates.

If the zone has changed in minor ways, the cache would be simply updated with the changes (CVS style). All that's necessary for this is to send out any object with a version number higher than the client's stored version (deleted objects would be logged and send out as deletions). If the changes are drastic (more than a certain percentage of items deleted), then the zone will be reset and the client will receive a new, complete copy of the zone. At any time, the client could request a zone update. Likewise, the zone owner could request a minor or major version update of the zone (such as before starting a major build). Also, if a user wanted to work "off line", they could simply suspend cache updates, and new people coming in to the zone would see the last version number. Only the owner would see new geometry. When the owner resumes updates, the system would work normally. This would allow people to have a "sand box" on their own property, and either commit or roll back changes when they're done fooling around. this would all work because the updates and changes would be happenening completely on the client side with cached data.

This also gives a good way to back up content: just let the client save a portion of the cache to a separate file on the hard drive. Then, if the user ever wanted to restore the saved data to a zone on the map, it would be a simple matter. this would also be handy for moves: you simply save your lot as-is, go to the new lot, and plop everything back down.

I have to agree with Anna Boobysocks that Jarod certainly seems to have no idea what he's talking about.

Okay well not really, I just wanted to add something to the thread and couldn't think of anything constructive to say.

So, working, output eminent, squids don't really care what server holds an asset they're retrieving to cache. Why then can we not save assets to our own systems? That puts the onus on me to be sure that all the assets my sim(s) are representing be available from my resources.

Oh I see what you mean. Linden Lab is currently affording us the illusion that they somehow protect our creator rights while helping us maintain our anonymity. It's a key component of the Wora'uld Deception. They're scaling for space, not traffic. The users are wholly inconsequential as long as people are paying the tier to keep the servers online so that the world itself grows. That's the LL definition of scalability ~ sims on the grid.

Okay well that makes a lot more sense than thinking that their definition of scability including usability by a growing number of accounts or an increasing number of avs in the same sim. The latter would require some sort of load balancing between the servers so that unused sims could support processing for crowded ones as Jarod discussed in this thread two years ago.

But LL haven't done that. Now I understand why. Hail and Pray Silence for the Central Grid supported by The Central Asset Server! May It's Sim and Account Numbers Increase Most Radically! Long Shall It's Usefullness Decline to the Glory of False Scalability! Praise and Adoration to the Seed of the Metaverse!