lag detection/avoidance tools, sim/region/estate custom prim/agent limits

Here is the proposal I drafted from comments gained from :
a topic on lag caused by clubs.

Please review it and offer any suggestions you have for improving it before I post it to the FVT.

----- 1: Intro

Q: What is this document?
A: This document is a summary of proposals designed to help ensure fairness and flexibility in sim resource allocation. This proposal aims to give estate owners and managers the ability to use sim settings to : set up parcels with custom prim limits, stop one laggy parcel from making the whole sim lag, stop one avatar who'se loaded down with scripted attachments or prims from making the sim lag, and various other things. It also suggests giving all residents new tools to find out where and why lag is happening, so they can do something about it.

Q: Where did this document come from?
A: Two people are responsible for it. Angel Fluffy and Effsey Nelson were both working on this problem for awhile, we just decided to team up more recently
Effsey Nelson's Quota proposal, as he writes it, is at : /13/c7/140089/1.html
Angel Fluffy's lag-buster proposal summary is at : /13/f2/139162/1.html#post1293268

Q: What do you propose?
A: Effsey and Angel have different ideas, and there are some points on which we differ.
This document represents Angel's ideas, however I suggest you read Effsey's too as there is a lot of good work there.



---- 2: The Proposals

Put simply, there are 5 questions :
A) What resources are we talking about?
B) What counts as over-use of these resources?
C) How can residents find out who is using too much resources?
D) How can estate owners/managers quota or reallocate these resources?
E) What else needs to be done?


A) What resources?
IMHO, the following crucial resources should be considered :
* Prim counts (on parcels, and on avatars)
* Script time (total per parcel, and total per avatar)
* Agent limit (per parcel)

B) What counts as over-use of these resources?
Put bluntly, over-use of a resource is using more than your fair share of the resource when the resource is in high demand.
If the resource is not scarce, then it makes sense to allow anyone to use as much of it as they want. For example in a sim with almost no scripts, it makes little sense to quota script usage, because it would slow down existing scripts which could be making use of that spare capacity.
On the other hand, if the resource is scarce, then each parcel (or possibly avatar, or group) should have a quota. Quotas for parcels would be based on what percentage of the sim they occupy (or perhaps, what percentage of the sim the owner of that parcel owns, to cover multiple parcels with the same owner). Quotas for avatars and groups could be set manually by the estate owner/managers.
Simply put, it is just as unfair for someone to be using more of the sim's script time than they have paid for as it is for someone to use more of the sim's prims than they have paid for. In both cases they are using resources that have been allocated to other people, which they have not paid for, and which it is not fair for them to use. In conditions of high sim load, parcels should have a quota proportional to how much land their owner owns in that sim. To do otherwise is to allow people who have paid less for land to 'borrow' resources from people who have paid more, and this is unfair.

C) How can we find out who is using too much resources?
Grumble Loudon has an excellent post on this topic, which you can read at : /13/44/133174/1.html#post1283958/13/44/133174/1.html#post1283958
Put simply, every resident should be able to access an info-panel which will give a 'rating' for each parcel on the sim in terms of things such as :
* Prim count per land SQM
* Prim limit per land SQM
* Agent count per land SQM
* Script ms per land SQM
* Script ms per avatar.
* Image MB per land SQM
* Image MB per avatar.
* Number of physical moving objects per land SQ
* Average prim updates per land SQ
* Average prim updates per avatar.
This would allow residents to see which parcels are sucking up the most sim resources proportional to their size.
Example: a 1000sqm parcel using 1ms of script time has the same script time / land sqm ratio as a 2000sqm parcel using 2ms of script time. They both have a ratio of 1/1.
If, however, a 1000sqm parcel tried to use 10ms of script time, that would result in a ration of 10/1. This would mean that this parcel could be easily identified as hogging the script time.


D) How can estate owners/managers use quotas to stop abuse?
Put simply, estate owners should be able to set limits on a per parcel, per-parcel-owner, or maybe even per-avatar basis for the above stats in their sim.
This would allow things like :
* changing a parcel's prim limit on an individual basis - ever wanted to tweak a parcel without affecting all parcels in the sim? This is your chance! This would result in new markets by truly decoupling prim count from parcel size.
* putting a limit on the maximum script time or number of active scripts per avatar to stop a single avatar with lots of scripts slowing the sim to a crawl.
* putting an agent limit on popular parcels to stop them making the whole sim inaccessible by filling up the sim's agent limit.
Remember, these quotas would only come in when the sim was under heavy load. Their job would be simply to ensure that when the sim was under heavy load, everyone gets the resources that they paid for.


E) What else needs to be done?
Firstly, 'region override' behaviours needs to be clearer.
Currently, they are inconsistent and at times confusing for residents. For example : if I disable push for the region, a "region override" shows in the about land / options dialog. Yet the same is not true if I disable terraforming, safe areas, fly or scripts. It should be. People need to be able to tell where the restriction is coming from, to avoid cries of "I've enabled terraform but it still doesn't work?!?!?".
Secondly, the behaviour of "run other scripts" and "create objects" needs to be fixed.
As I have posted about (at /139/b7/143714/1.html), disabling "run other scripts" as it currently stands is pretty futile. Not only does it seem to completely fail above 30m, but it doesn't seem to actually disable all scripts, either. As an estate owner with some very popular parcels on my estates, I would REALLY like the ability to have a *serious* "don't run other scripts" option. Such a serious option would mean that unless the attachment or prim was either owned by the parcel owner, or set to the parcel's group, or was being sat upon by its owner, it does not run. Anyone TPing in has all their scripts checked and disabled as needed. Same with anyone crossing the parcel border into the parcel. Sims have a maximum active script count of 6000 before they lag. There are scripted objects, like Souls of the Damned, which carry over 1024 active scripts in them (I hear). It seems pretty obvious we need a serious way to turn OFF scripted attachments on our land to avoid having things like this running all the time and lagging our sims to hell. I know Souls of the Damned is not a typical attachment, but when you have 30+ people on your sim, and many of them have lots of scripted attachments, it becomes really difficult to keep the sim low-lag without forcing anything popular to be built on the ground and forcing them to disable "run other scripts". We really need a better way of stopping these scripts eating the sim.
As to the "create objects" restriction... objects seem able to bypass it when their owner is in the sim - yet this fact is not documented readily, which lead to lots of confusing behaviour when I tried to put this in place on my parcel. People reported that turning it on broke things, but when I TP'ed in to check, it all worked It seems crazy to suggest "they all stop working when I leave the room, but start working again the moment I come back.... they're broken, I tell you!". But it seems this is exactly what is going on... please, at the very least, document this fact in the about land options page. Even better, come up with an implementation that means objects which are set to group CAN rez new objects, and so can avatars who are in the group. Please don't let it stay time-dependent, this is quite annoying and causes lots of confusion.

i think parcels should have a limit on the number of agents allowed in it based on the size of the parcel.

Issue:

If you divide up the 40 avs per sim into per parcel, a 512 sq. m. plot of land is allowed just over 1 avatar.

Issue:

If you allow a 16 sq. m. plot of land be equal to 1 avatar, a sim would thereby contain too many avatars when fully loaded (1/2 sim club, etc.)

well I don't mean to pick up a baseball bat and smash you down with it, but...


Mono is coming, it should cut all the script lag down by ton, more than 50-150 times faster.

Prims doesn't push Server's CPU down except its bandwidth/downloading rate when you first arrive to the sim or whatever come in your camera view. Don't get confused with Server lag and Client-side lag performance. Server doesn't have to "render" prim because... it's a server to give client-side where they are. Not rendering like your videocard does.

Otherwise, how can you really tell if all that 50 high polygon models, "Avatars", is lagging the server or your videocard?

Hit Ctrl+Shift+1 to see the actual status of the sim and your PC performance.
The very top one, FPS (frame per second) is actually your PC performance.
Look at Sim FPS to see what's the server rate (standard is 45). Also look at Agent Updates/Sec, that tell you how fast the data is sending between you and the server. (standard is around 20) If that went down lower, that means too many avatars are moving/walking/running around the sim. Thus lag.

Want to see more than just that? Hit Ctrl+Shift+9
It look something like this. http://vincent.blueforgefire.net/things/edgestats.jpg

Now here's the real deal, if you ask Linden Labs to make CPU resources "fair" by the size of the land... It will require MORE CPU to ensure that resources "fair" by land size. It's retarded. And they DID come up with something new for the sim/group related.
Set the land to no script, run group scripts only.

Let's me repeat that for the love of god...

Set the land to no script, run group scripts only.
Don't go around that and make excuse about how your "world" is too laggy.

In fact, I went over to The Edge sim. (see the link to the screenshot above) They had 51 people and running just fine with their SimFPS count, Time Dil, and Agent Update/Sec. The SimFPS was at 44. Standard is 45. But of course, my PC FPS drop down to 6 while all my setting was at the max, but they are running great! They are having a party!

The only problem I could see was the bandwidth, not the CPU. Linden Labs can't make every ISP faster.


Angel Fluffy... just admit that you don't like clubs. I don't like it either, but no need to go all demand-y at Linden Labs to kill the lag. They know about the lag, every single day.
Just let them work on something new to change that problems. I do trust Linden Labs plan to make Second Life better, why not you too? but in meantime, just be patient.

Seriously, be patient, I'm sure they had over 10,000 people bugging them about the lag problem every single day.



EDIT:
Real issue I could foresee will be physical engine weighting on server's CPU power when they jump onto Havok 2 or ODE.

I hope Mono is coming too. I'm just not sure when it's going to get here.

I know that having quotas may consume some CPU time, but I hope that there is a way of using quotas such that one laggy area can't lag down the rest of the sim.

Setting land not to run "Other" scripts is great - if your build is below about 50m. I've seen cases where the "no script" restriction simply fails to work at higher elevations. This means it is not a lot of help to all of us who have popular places in the sky.

I have nothing against clubs per se. I even go to them sometimes. The problem is more that, as things stand now, a single parcel that's popular can exaust the sim's resources and mean that either people can't get in the sim at all (full agent limit) or they can but it lags terribly.

I know this isn't a crucial issue. I know that for most people this is an annoyance. I'm saying that, as someone who runs sims and some popular places, it would help a lot of we had some kind of quota system.

Heck, I'd probably be happy if they fixed the "run other scripts" option so it worked up to an infinite height. That would solve most of the lag issue from avatars and thus allow a higher agent limit per sim too.
Solutions don't have to be big and complex. If you think the problem is small, or not important, then a simple, small solution can take care of it.
I'm not demanding that LL make this their #1 priority, I'm just trying to get it on the Linden radar. I may write very seriously, but I know that my ideas are very unlikely to be adopted as-is by LL - I know that, so I just try to get the issue on their radar. I realise that if they know Mono is coming soon, or they have some other magic bullet, that they will stick with removing the sad and annoying heigh limit on the "run other scripts" option. That's ok

That sounds like a huge amount of work for Linden Labs, and frankly there are many other pressing issues that require attention (issues that affect the entire grid, not just an individual sim).

I own a popular parcel of land, and regularly receive complaints from my neighbours regarding lag and performance issues. Heck, a lot of the time my parcel of land is blamed for problems that have nothing to do with our traffic volume. Trust me, if a grid wide issue pops up, I usually receive at least one or two terse IMs.

I think instead of spending huge amounts of money to come up with a bandaid solution, eliminate things that contribute to the lag. Camping is one issue that affects the sim avatar limit. Ban camping! Done. Next, what about avatars marked as away? Why prevent a sim owner from accessing their land because 5 or 10 visitors marked themselves away so they could watch Everybody Loves Raymond and eat a plate of Kraft Dinner? If a sim owner (or really, anyone) tries to access a parcel, why not bump an away avatar (in order, based on how long they've been away). Sweet. Next, how about performance issues? Linden Labs probably has server logs. They should set up a little batch utility that parses the logs and comes up with a report of sims which are consistently and constantly lagged, and work with the sim land owners in priority sequence (if owners themselves actually have the tools to help mitigate the issues), or they can upgrade the hardware of the server on an as-needed basis. Now, before you flame, they may already do this. I have no idea. I've only been playing SL since September, and started running my own place in October, so it's a lot to absorb in a short period of time.

At the end of the day I think LL needs to work on load balancing across multiple sims. I think the whole "per sim" architecture is the real issue, and anything else is just a quick and dirty bandaid solution. Most people can't afford private islands anymore, so capping traffic on a per parcel basis might leave popular parcel owners with no affordable, viable options. I have a feeling changes like this would shut down most of the places I visit, and definately the place I run. Some people might consider this a good thing though, but our 400+ members wouldn't.

I think another thing that needs to change is the idea that just because you pay for a parcel of land, means you're guaranteed access to it. If you have an expectation such as this, I think a private island is the way to go. I'm in the same boat as the other owners in the sim, so any issue that affects them affects me. Now, I've yet to run into significant lag (other than the regular grid-wide issues), or even the avatar limit, but then again everyone has different expectations!

Either way, I fully agree that something needs to be done. These issues affect the owner of a popular parcel just as much as everyone else. I'd sooner have fun and actually play the game rather than field complaints.

Just my 2 cents!