An idea to somewhat reduce lag in crowded areas

Hey everyone. Haven't been here in a while because of computer not being usable.

Ok, so it's obvious that because some areas have too many prims, there will be lag regardless of the graphics card due to the network streaming everything.

So here's my idea.... What if for areas with extremly high prim counts, users are given the option to download the area onto their hard drive. This way, the game can start loading the area off of the drive incase lag accurs This would make it easier on you network when playing the game.

Now I'm aware that sometimes things get updated and some times people are actually moving things around. But the world on the hard drive can be synced wtih the online version so that the offline one will update in real time as well... I know this may be harder done than said, but it's just a thought.

This may not be fore all areas, but may be only certain ones if the owner allows it. Maybe the user will have to pay a price. The user can't edit the world or anything, but it'll just make it easier on the computer and the net because nothing, or not as nearly as much will have to be streamed, therefore increasing framerate.

Well, taht's my idea. Don't tare it down too much.

I was just thinking about this the other day, as I'm an ex-Ultima Online player, and that's very close to how it works. The actual landscape (along with all items, but that's not the point here) is already on the hard drive and doesn't need to be streamed. When there's a change to the scenery, the player's topography is patched so things are the same for everyone.

There might have to be some limitations, such as changes to topography not being allowed for a given amount of time before this sort of download could be allowed - else players would be going through it all the time. Its the basis for a good idea, though.

Yeah, I know I've seen people bulding things rigth there on the spot. I mean, you can turn the "hard drive stream" or "stream from disk" off and just play from the net like normal, but yeah, there'd have to be some limits on how ofter the owner could move things around... the only places that probably couldn't stream from disk were sandboxes. But I mean like... there's this one club I go too a lot that looks really cool, but there's so much stuff in the area and in the sim in general that it just makes ai a bit laggy. If I could have the game pull the sim off my HDD instead, I'm sure that'd fix it.

It already exists... It's called the cache.

Most all of the unchanging data about the region you're in is stored in the cache in effort to save bandwith both server side and client side. While cache management is deeply flawed, it is functional and the majority of bandwith spent in a region that is cached is completely dependent on physics and downloading un-cached textures. (^_^)

A larger cache will arguably reduce network lag for someone who stays in a region for a long time, but can lead to local instability as the file size does become difficult for the client to manage without lagging a bit or corrupting data. (^_^)

Oop... My inner-geek seeps out again... (@_@) ...... (^_^)y

The cache system already does this to an extent, because it caches textures. I don't think it caches objects in the environment, but usually the textures are the bulk of the information.

It's not clear whether caching more types of information will help. My gut feel is that the cache itself needs to be sped up first.

It's my understanding that if the builder of a prim crowded sim would make as many of the prims phantom that it would greatly reduce lag because the server wouldn't have to make physics calculations and such. I'm not techy but this is what I've heard.

Physics information "objects" kinda has to stay server side because it's the server that calculates the physics to be sent to all clients at once. If objects were calculated client side, that calculation would still have to be pushed up and piped through the server to all the other clients within view. That would make for increased bandwith usage, not decreased. (^_^)y

I agree that caching is the solution here, and speeding up the existing cache could be a big win, as Kidd says. It could also be that a caching algorithm with user adjustments could be useful to many folks... something like: "For as long as I'm in this region, the stuff I see should be retained preferentially to other stuff" so that the next time you visit, more stuff might load faster. Indeed, if the cache were allowed to grow very large, with managed "stages" on disk and in RAM, it could be helpful to be able to "pre-heat" the RAM cache by pulling in a Region-specific block of disk-based cache.

There is a lot of talk about the cache going on the the SL OS community right now. Some f the notes are up at https://wiki.secondlife.com/wiki/SLDEV_Discussion_-_Texture_Cache_Plan_of_Attack

The big scary expensive part of rendering stuff is still the textures, particularly that it takes a lot of horsepower to unsquish those JPEG 2000 images (whether grabbed from cache or the wire).

Why then is it, that if I relog into an area I've JUST left, I hafta watch as textures and sclupties hafta slowly re-res? I have my cache set IIRC set to 'pproximent'ly 3/4 of the scale.


As far as I can tell.... The cache exists ONLY to cache PROBLEMS, and must be cleared in order to correct 'em!

You have two things going on there. First is that the stuff in the cache is still compressed, so you have to go through that expensive decompression all over again.

A second problem that isn't so apparent is that textures can look like they're all there when really they aren't quite finished. That's because they're using progressive JPEG. If the cache is missing even a little tiny bit off the end, currently it starts over.

I don't know absolutely, but I think in crowded places, more of your lag is coming from the avatars who go in and out, rather than the place itself.

Every avatar is bringing in huge texture files (skin and clothes), prim attachements (hair and jewelry), and many animation files (from AOs or dances). When it comes to the prim attachments, keep in mind that avatars have no limit to the amount of prims they can bring to an area, the way that land is limited to the amount of prims that can be rezzed on it.

Every time an avatar enters your area, or least within rendering range of your avatar, all of their textures and prims and animations are going to start loading on your computer.

But probably contributing more to your lag problems in crowded areas is not the time it takes to transfer others' textures and prims and animation to your computer, but the fact that they all get stuck in your cache. As avatars come, they dump a lot of things in your cache, and as avatars go, all that stuff remains in your cache. This goes on until eventually your cache becomes so bloated that it saps your computer's resources.

It's an interesting idea to be able to let a user choose to keep certain resources from often-visited areas into a permanent cache, and it would probably improve performance in those areas to a certain extent. But among all the problems that contribute to lag, it may not be a huge help.

Simply because too many people put too many 1024x1024 textures on too many faces of too many prims. By the time you set your cache high enough to store all that crud it's already unmanageable by the client. (=_=)y

Does rezzing really take that long for everyone?

Unless the sim is lagging or the grid is just acting up I'll have enough rezzed to see most everything within 10 seconds and be able to start walking (with the minimap for avies that haven't rezzed in yet) or camming, after 30 seconds everything's rezzed except for smaller prims, after a minute everything is really about fully rezzed.

Other avies are an exception and only usually seem to start rezzing in for me after all the prims and most of the textures that take up the most screen pixels.

If I'll be somewhere for an hour or even 30 minutes then waiting 1-2 minutes for a whole rez doesn't really seem like much of a problem. It's the performance *after* rezzing that would seem the most important .

(It's not really related to puter at all either, things don't rez any faster/slower than they did on my old puter)

I tend to spend most of my time in one sim so rezzing is generally fast - even with draw set at 256 and all the sliders up full whack. As I am currently on a 3g widget rather than broadband I woul have expected there to be some delay in loading.

The AV's that visit do sometimes take a while to emerge from their cloud of steam though so this is likely the hair and skins texturing

When I leave the sim to visit others It's definitely alot slower than it used to be on broadband which points at the cache issues scenario described. My solution to this is never to leave my sim unless absolutely essesntial

I think probably a lot of this issue is bandwidth. If I do a speed test on my internet connection, of course it's lightning fast to my local server, but if I test down to San Francisco or Dallas (isn't it Dallas?) where the LL servers are, I get a big drop in download speeds, by about half of what is available locally, with Dallas being lower than San Fran. If I TP to a sim that lives on a Dallas server, it's going to be slower to send me info than a comparable server in SF. Rezzing does seem to have slowed in general over the last two weeks for me, though.

If you plan to stay in one area for a length of time, something that helps is going into the debug settings, and increasing your cache size to 10 gb or more.

As Immy said though, once you leave that area, it is best to clear cache/relog to empty it out - I can usually go about 4 days at a 15 gb cache before I start to get items that won't texture properly.

Also, very very important to make sure you defrag your system nightly with a cache this large. When the cache gets large and fragmented, you will suffer from the turn/pause/crash bug more often.

One way of bringing the existing cache closer to the OPs request would be adding a TTL ("time to live" parameter to cache entries. The short and dirty would be to give all av-attachment-schtuff very short ttl's and any ground-based prims much higher ttl's. Down the road, it would be really cool if the land/prim owner could set the ttl parameter thus making "semi-permanent" structures with a very high ttl.

Another issue which has puzzled me alot, for a program which requires 1GB of RAM a 1.5GB max value for the hdd cache seems quite pointless. I find it difficult to imagine blowing thru 1.5GB worth of textures, but with the heterogeneous content creation, and if 1024^2 textures are as common as people insinuate...it could be done. Perhaps the slider should have a higher max setting. Folks with quad cores and sata2 raids could easily use it (tho not I, hence it's a user param).