SimTorrent

I don't play SL as much as I used to, but Khamon and I discuss is a lot! Yesterday we were talking about the infrastructure of the sims. He explained to me that sim-lag was a bandwidth problem, and not a processing/memory issue. The lag was caused by the NIC not being able to output the data it needed fast enough.

Now, when I'm not playing Second Life, I tend to steal comic books from the Internet (until I can get to my local shop, then I pay for the physical book as liner for my birdcage -- I'm lying, I don't have a bird). Stealing comics requires a lot of bandwidth, as each book's scan usully goes for around 7-to-10mb, and when you group a lot together it can get up there. However, the central node for this theivery uses BitTorrent to distribute the load.

The central node offers a seed that downloaders latch on to, and as more people connect to the seed, the majority their download comes from the parts that other have already gotten. (It's considered good manners to upload as much as you download, so you often stay connected long after your download is finished, for those who don't know BT.) As Khamon and I were talking, it occurred to me this would be a great way to reduce lag in SL.

Since LL refuses to release their software, in what can only be viewed as a form of corporate suicide, so that we can setup out own grids, why not allow us to distribute the load another way. Have the sims recognize users who have been their for a while, recognize newly arrived people, and have the new people get their initial streams from other clients. You could even teir it, so that people with better connections could act as proxies for other people. Thus the more people who came to (and stayed) in a sim, the faster it would actually work.

LL seems to be dead-set against P2P applications, favoring a centralized-Skynet model of their operations... But it was an idea I had, and I thouht I'd share with the rest of the class.

The more people you pack in a sim, the faster it gets. Makes me wonder....

BitTorrent's model wouldn't be too usefull for SL since the data here is ever-changing. Also, all the sript execution and physics simulation take place on the sim CPUs. Everyone needs new data (a rezed prim, a moved avatar) at the same time, so sending data to just some of the players who then send it to more, would only introduce delays as the data is passed from hand to hand, live.

BT is great for static data where each client can ask all the other clients "do you have piece X?" and then get it, or "go fish" to the seed source.

Also, all the sript execution and physics simulation take place on the sim CPUs.

But each instance of a physics simulation or a scipt is only done once, the problem crops up when you start broadcasting the results to everyone. (Mind you, I'm looking at this strictly from angle that it's a banwidth issue, and there's memory/cycles to spare -- that could very well be a erroneous view of the situation.)

Everyone needs new data (a rezed prim, a moved avatar) at the same time, so sending data to just some of the players who then send it to more, would only introduce delays as the data is passed from hand to hand, live.

Aye, they do. However, fifty people asking the server for a new prim takes up a lot less bandwidth than fifty people asking the sim for every prim. The prims would need to be stamped with a time/date for every change, so that the server could compare who on the sim could/should broadcast.

Say I've been on a sim for five minutes and have everything cached, and someone new comes on the sim. I could offer them data for anything that had been there longer than I had, thus leaving the sim to only have to send new information. (Granted, physics and scripts would have to come from the server, which may blow this whole theory.)

Not everything in a sim is constantly dynamic, however everything in a sim has to be squirted out as you approach the sim... As I understand it, that's where the lag comes from. I'm not saying a complete-BT model is called for, just in part.

BT is great for static data where each client can ask all the other clients "do you have piece X?" and then get it, or "go fish" to the seed source.

Making it real-time was something that I figured would be the achilles heel in the plan. However, to play devil's advocate, couldn't it use the BT model in part? Maybe the full BT model isn't the way to go, but surely there is someway to distribute updates quickly.

If memory serves, this is pretty much how IRC works.

I'm not knocking the idea, it would be great if it was feasable. I don't think bandwidth at LL's end is the main bottleneck. They knew from the get-go that they would be doing some heavy streaming. Philip's background with Real is experience enough there. So I think their machines are on a pipe that more than large enough for SL as it is, and as it continues to grow. At some point, they plan to open new physical locations around the country to have even greater combined bandwidth.

I think in most cases, it's the end-user's "last mile" connection to their ISP that is the greatest speed sapper. Cable modems especially, and some ADSL lines. When the local bandwidth is shared, and not exclusive, your game playing experience can really take a hit if all your neighbors start downloading porn.

The bigest problems with live BT networks would be the unpredictability, duplicate packets, and overhead of timestamping every obejct. Someone's machine may hit a slowdown and their response to your request could be delayed, or they log off before fullfilling it. To try and overcome that you might have to ask multiple machines for the same data, but then the bandwidth on your end might become saturated (if it wasn't already). And accurate timestamps are rather large... Possbly adding 10-20% to an object description's size.

I'm pretty sure that any lack of speed in object streaming I see on my machine is due to my ISP. It seems more tied to my local internet health than to what's going on in SL. But if that weren't the case, I'd love to see any feature that improves speed.

Oh, and if data was transfered direct from player to player, you KNOW someone would come up with a hack that lets them modify that data to their own advantage... Even if it was encrypted. (Which would be even more overhead.)

Again, good idea, but I personally don't think it's viable.

oh i see. what you mean is just for new arrivals to be able to download static data from people who've been in the sim long enough to already have all that. i suppose you'd see varying results based on the type of sim you were in. some sims are chocked full of constantly updating items. nobody hosts events in them though. that would be foolish.

of course updates always need to come directly from the server. so once everybody settled in, there'd be no more bt going on. it would allow more people to participate in an event if everyone sat down and behaved so that they weren't getting constant updates from each other.

most events that lag aren't hounded by too many people they're ruined by a few people wearing primmy attachments, particle generators and scripted items. it's often more a matter of consideration and etiquette than software.

Sim lag isn't caused by bandwidth issues. The problem of objects not loading for a while is caused by low bandwidth... and as Tiger correctly points out, it's always low bandwidth between you and your ISP.

It's not necessarily a bad idea, but we just don't have fast enough connections yet for it to be practical.

Personally, I favour the creation of networked mini-cache servers that someone could set up on their LAN to supply local clients with textures, animations and sounds.

Again though, the cost/benefit ratio probably isn't ultimately worth it. The number of people who would use it is too low for it to really be worth doing.

so we're back to sim software itself needing to revamped in order to accomodate what's now considered a reasonable load. every sim will have either a club, a mall, or a texture laden particle gallery of light in it. and we still want to be able to host events for a hundred avs at a time.

it seems that people's hopes, plans and aspirations have far outstripped the ability of a sim server in it's current configuration. all those things we, including ll, dreamed of are not ever going to be feasible with the current software.

i suppose we really do need a feature freeze and a server rewrite. that means we'll have to live in the world without any updates for at least a year. this is one of those critical times when ll must make a decision that will make or break it's future. wonder how wide this window is?

Again though, you're confusing client-side rendering issues, caching issues, server performance and bandwidth. They're all related in the sense that they affect your overall experience, but they're not inherently connected. Any one performance bottleneck can be as great or greater than the rest.

Assuming you had a T3 and the "pefect" computer --basically the most expensive one you could buy... from about two years in the future-- very few of the performance problems you experience would continue to exist.

Rewriting the server is not going to improve the time it takes for a user on a slower connection to download textures. It's not going to affect the client's ability to draw particles or light sources. It'll improve physics and, should there be many scripted objects and avatars in a sim, allow the sim to handle somewhat better under load. That's it. Textures not downloading quickly, objects not loading, low frame rate when surrounded by dozens of avatars, that's all your computer's rendering capability or your network connection getting maxed out.

In my opinion, what the Lindens would best be advised to do to combat the myriad causes of lag would not be a feature freeze and months and months of work rebuilding something that already functions better than the rest of SL altogether, but by adding another feature. There's no real technological way for SL to do anything even remotely BSP-like well, despite the obvious benefits to framerate, so what I propose is this: allow us the means by which we can block off rendering, even downloading of content outside (and inside) a certain area. This would allow us to create isolated zones --rooms or buildings-- within which people could enjoy the same server as the rest of us, but without the bother of having to render anything outside it. Were the occlusion box dual-sided, it would allow the rest of the sim to continue unaffected by the dozens upon dozens of textures within it.

There's no real technological way for SL to do anything even remotely BSP-like well, despite the obvious benefits to framerate, so what I propose is this: allow us the means by which we can block off rendering, even downloading of content outside (and inside) a certain area.

What about a platform in the sky? If you get past the 256m maximum distance for loading prims, textures, etc. then wouldn't that be the same thing? (Actually, have the club owners ever tried this? Do we know if isolating 40+ people 300m above ground affects anything?)

I suggest that as a hack until they give us what you're suggestiong, that is.

i understand your point catherine. it distresses my that my 2M cable connection, geforce card and 512MB of RAM are not sufficient enough to get any better than 100 fps from most of the sims.

your feature suggestion is technically good; but it comes dangerously close to exposing the lie that computers are easy to use. it's taken us a long time, as an industry, to convince people of this so that they'll spend loads of money before realizing that they have no idea how to manage the equipment. don't be threatening the flow of money now.

basically, short of such an implementation, there's nothing to be done? we are just stuck with lousy performance because our client side equipment isn't up to snuff without spending several thousand dollars for equipment up front and several hundred a month for connectivity? that's too sad. what a crock.

p2p is not suitable for any dynamic data. It's bonuses lie in data that is not likely to change, eg textures, and for that LL has plenty of spare bandwidth.

Prims, and other objects which can be influenced by script or avatars, is instantly thrust out of the static catagory. Also, managing ala bittorrent the sheer number of clients, which portions they have, who they are, etc. would have a major cpu drain (more than what is saved by bandwidth gains).

BitTorrent is fast; granted, it's a great idea - but it's a solution to downloading fixed data. SL would not be suitable for this method, for the above stated reasons plus:

1) Insufficient demand on specific items to gain bonuses from using the tracker/node model. (your only downloading XYZ texture for a very brief period of time, plus you are only downloading the LOD version of that texture), most people wont be downloading XYZ texture at the same time, let alone the same LOD level.

2) Implementation time, changing this would require seriously modifiying the client protocols. Not about to happen soon.

3) CPU and Bandwidth losses due to coordination, would most likely outweigh any potential gains.

4) Speed/Response Times, a centralised download can begin at once, a distrobuted one needs to synchronise with each item in the swarm before commencing. For lots of small downloads, this isnt suitable.

-Adam

[/B]

Yes, actually that works rather nicely. It's why a lot of scripters and clothing designers have small studios and workshops up in the sky. For a public example, Briana Dawson's Galleria sim uses a two-level structure, where you teleport several hundred metres above the ground by default. The textures will still kick your computer's ass, but it's a lot better than it would be at ground level.

No, Khamon, I still don't think you're getting this. Your hardware has nothing at all to do with the Sim FPS. Your hardware only has to do with your system's FPS. The only way to get sim FPS above 100 is to be in a a relatively empty sim, or to vastly increase the hardware speed. As a rule, everything above 100 is functionally the same, anyway. If you're in a sim that's hovering around 10 FPS, then yeah, that's not so hot.

SL is not intended to be as fast as Quake. It's a fundamental difference between a proper BSP tree and just having a lot of objects. SL will ALWAYS be slower than a game with an engine like Quake's.

Now, that said, if you feel your CLIENT is performing unusually badly given your available hardware, by all means, post your specs, and I can offer some advice. Doubling your RAM might be a good first step.

so the answer is yes. improving the server code won't help. improving the server hardware won't help. improving my client won't help. improving the connectivity at any point won't help.

there is nothing to be done. we're just stuck with what we have. the only solution is to block off areas of a sim that let avs pass into and out of them, but don't "see" out or let the rest of the sim "see" in.

we might as well operate our own sim servers and just browse between them as we navigate websites now.

Getting a super-fast, dedicated, business-class DSL line WILL help with loading the grid into your house as fast as possible.

Beefing your computer with more RAM, faster CPU, and a better video card will increase your framerate.

Moving closer to Linden Lab's co-loc, where their servers are housed, will decrease latancy, making it more responsive when walking, flying or piloting a vehicle.

And <insert ridiculously over-the-top activity here> will <insert extreme uber-user result here>.

Also, try adding a "Type R" sticker to your computer, and putting a spoiler on your cablemodem. I hear it works wonders

I just want to drive down to the colo, set up my box, lay in front of my monitor, patch in, and surf the grid at 100BaseT... giga if I upgrade my NIC maybe then my objects wouldn't randomly twitch as UDP packets are resent out of order x.x

winner


i didn't want to read all this, i did however skim it and read possibly good posts.

everyone failed to mention what i'd like to mention how.

the bt structure.

The tracker would have to keep up with each user, and thus this uses bandwidth. A tracker holding everyone on SL wouldn't work, but it's possible per sim. However, you would have to download information on how to connect, and currently, bencoded (and thats not a typo) .torrent files are varying from 32 to 100 kb in size, much larger than the textures themselves.

a different type of p2p may work better, but seriously, you run into more problems. essentially something called 'connectable' in most bt sites. I'll explain.

Many many home users are NAT routed or have some sort of firewall between them and us. Let's just say NAT as it's easiest. a NAT router has to have a list of 'ports' to 'forward' to the appropriate machine (www.portforward.com can explain).

Now, most of the SL users are not too intelegent (just accept this). Most of them will not be able to figure out how to open their ports. Why, you may ask? Well, when you walk on into a sim, you connect to that sims 'tracker'. this tracker lists users who are able to send packets to you. now, what if the users around you were behind said nat router, and when you try to 'connect' to them, the router has no idea where to send the packet, so it's discarded. This is bad. However, the REAL bittorrent has a way around this. Every 20 or so mins (depends on tracker settings) you will ANNOUNCE to the tracker. When you do this it will send the list of ips, so that people who aren't "Connectable" can connect to those who are, and send/recieve packets.

However, if this number were decreased (so you could see the ground within a minute) then the bandwidth would increase between the users and the tracker, and you don't save any real bandwidth at all.


Hence, it won't work. Just won't. Period. I hope you understood, please ask for clarifications because this was written rather fast. Most info can be found at btfaq.com and wikipedia.org.