The town, LINDENS please respond

Los Lindens,

As you are no doubt aware, we made a town out in Tehma. Right now traveling through the town is _very_ laggy all the time, with frequent client crashes and at least one sim crash per night for the past few nights. Also, we are often hitting up against the sim population limit.

We players have come up with some possible remedies for this, based on our own guesses and some informal discussions with liasons in-world, but without insider information it's hit-or-miss as to what will actually help.

So, we need your advice. What should we do to fix these issues? We've talked about moving the whole town, Simpson's style, to a 4-sim corner, and this would at least help with the population problem. What else?

Should we wait for the next version?

Can we get a beefed up server with a fatter network connection (if this would even help)?

Can the game system just not handle this kind of density and maybe we should just give up on the whole crazy idea?

The lag is caused by the combination of many avatars in an area of high object density.

Each avatar is getting information about many objects, which makes the server work hard. This scales linearly with the number of avatars.

The physics engine is also working hard because the avatars are having lots of potential collisions with nearby objects.

It sounds like the town residents have already taken some useful steps in the fight against lag by suggesting:

1. everybody bring their display distance down to the minimum (64 meters)

2. no particle system scripts in town


We will be trying to improve scalablility for the next release. So I don't think you should give up on the whole crazy idea. In fact, since people are chosing to hang out in the town despite it's lag, it indicates that it is an interesting aspect of SL that we will want to work hard to support.

In the meantime I can provide some advice for continuing the fight against lag, however some of these suggestions run counter to the "cool" aspect of content so the town residents will have to decide which ones to try:

A. Moving the town to a 4-corner will help a little bit for sim performance. The physics engine will benefit the most, the servers will still need to send info for objects in the sims next door, so there will be no benefit on that front.

B. Keeping the display-distance down to 64 does reduce the load on the servers.

C. Reduce the number of scripts. Lots of "active" scripts will contribute load on the sims down. By "active" I mean scripts that are constantly listening for chat, looping, or timing. This includes scripted attachments.

D. Texture animation scripts increase bandwidth more than they should. We've got some fixes for this that should be out in the next release.

E. Reduce the overall number of objects by linking them together. Theoretically this should help increase the speed at which the servers can send information to each user, as well as helping the physics engine sort through the objects. It will work best when the linked collections tend to make tight collections.

For example, suppose you made a tall tower. Rather than link the whole tower together, or even link one whole wall together, it would be better to link into heights that are about the same scale height as the width/depth of the building. That is... link the pieces into approximately cubic blocks rather than long thin blocks.

On the other hand, if you have a collection of primitives that already contains one long thin piece... if the other pieces of the object are more or less inside the sphere centered on the long-thin piece you might as well link it all together.

F. Don't move. If everyone in the town were to stop in their tracks... the load will be reduced on the server... unfortunately that isn't much fun.

G. Recycle textures. The more unique textures in an area, the more information that must be sent to every person who visits it.

H. Set your client caches very high. The more textures you can store on your client the less information the server has to send you the next time you visit the town, or even when you walk from one side of the town to the other.

J. Put a park in the middle of the town and make it not contain big linked objects. Also, make the park at least 3 times the scale-size of the largest objects (total linked scale) on the edge of it ==> people hanging out in the middle of the park will not be "potentially colliding" with the nearest large objects.

Finally, I should mention that we can't upgrade the server for that area because it is already the fastest variety of server we have. We will instead be working on optimizations and bug-fixes.

Finally again, the town is a great stress/test case for us to use in our efforts to make the eperience better, so keep up the good work.

Thanks for the detailed (!) reply Andrew.

A few questions:

What is the best way to write a very-low-load script? I had assumed that using an infrequent timer was best but your comments seem to contradict that. Should we be using llSleep instead? Also, does using llSetMinEventDelay() with some high value help?

There are two kinds of texture animation scripts, ones that change the texture, and ones that re-offset the same texture. Is the latter as bad as the former?

Regarding using a small number of textures, most of the buildings in the town are shops, with lots of items to sell, so my question is: how smart is the server when sending textures? If I have a bunch of objects with different textures that are in an avatar's field of view, but hidden behind something else, is the texture sent? What I'm asking is would hiding small objects in a display case help?

Does the proposed park have to be completely empty of objects to help? Could we put down paths and such? Are there some kinds of collisions that are much worse than others?

If the collision engine being overworked is part of the problem, does setting objects to STATUS_PHANTOM help? Given that this must be done with a (1 line) script, does it help more than the cost of running the script?

Setting objects to "phantom" will help the physics engine only in the case where the collision would otherwise occur. In other words, setting a bunch of objects on shelves as "phantom" will not help sim performance, unless compared to a sim where someone is trying to walk or otherwise collide with those objects. In short, don't bother setting phantom except for objects you want phantom for phantom purposes.

The one-line phantom script is a good example. That script will load the simulator only once, and then never again. Compare that to a script with a 10 second timer... that timer is checked at a frequency much higher than every 10 seconds just to see if it is time to run the timer callback.

Consider two scripts... one with a 0.5 second timer and another with a 100 second timer. If the timer callback is some simple operation, then these two scripts will have approximately the same impact on the server.

The simulator does not do blocking checks for objects behind other objects. It is currently too expensive to do server-side for the general case. So no, hiding objects behind a diplay case will not help.

The client will ask the the texure if it doesn't have it. So increasing the client cache will reduce the number of times the client must download the various textures that are needed.

The park doesn't have to be empty. I was merely mentioning that if the park is big enough, then people visiting the park will not be inside the large bounding boxes that surround each of the objects the will line the outer edge of the park. An empty park isn't interesting... but it will run faster than a park full of stuff... much like an empty sim isn't interesting, but an empty sim runs faster than one with a town in it.

Here's a suggestion that just occured to me. Perhaps the town should be organized into several "squares", plazas, and parks rather than streets. That is, line the buildings around several "open spaces" where the open spaces are slightly larger in scale than most of the buildings themselves. Let the streets be short connections between the open spaces. In any case, a good town needs a few "open spaces" for gatherings.

Some collisions ARE worse than others. Did I ever mention how concave, cut torii are tough on the physics engine? I'm sure I did somewhere.

The two types of texture aniamtions cause the same bandwidth... except in the case where nearby users don't have all of the textures downloaded yet, in which case the one with several different textures will slow the sim down more than the others. So, as new people show up at the town the animation that requres several textures will overall take more simulator CPU processes.

Here is a quick sketch for a "New Lindenburg" city plan ....

Is this kinda what you were talking about Andrew?

Key
Green = park
Gray = road
Yellow = plots for stores etc.

Im still trying to get my spot! Im always told to get lost for now and come back later.

Ama,
Not bad actually I like it. One issue I see though is there are 8 lots that have no access from the "street".

Here is a variation of your submission eliminating that issue. The legend is the same and the middle lot would be town hall.

I assume the one in the very center will be town hall, otherwise that is gonna be some very high-valued real-estate.

-Pat Murphy

Good point Misnomer. And I forgot that streets are 2 land blocks wide. I took the one you made Misnomer, adjusted the streets and expanded the inside corner lots to be 'primo' lots. I added alley ways so the park in the middle with the town hall could be bigger.