High script performance numbers

I have a question regarding my home sim of Hawkins. It has been having extremely high script performance ips numbers. Last night when I logged in, the numbers were fluctuating around 130,000 ips. They have been up around that number for more than a week now.
I know "typical" sim ips numbers are usually around 20-30,000 ips, so I'm wondering what would cause the numbers to be so high.

Active scripts are approx 3,400

The neighboring sims have normal ips numbers of approx 15-20,000 ips

Everything seems to work fine (besides my house being a little slow to load when I get there), but I'm just trying to understand what would cause this.

What causes high ips numbers. (I'm assuming ips stands for Instances Per Second?)

Is it certain scripts that are running very innefficiently? (and if so, how can I find them?)

I had this fairly recently. Turned out that the sim had someone running several temp on rezzers which were badly coded - can't remember the name I am at work - however this particular temp on rezzer is now covenant banned throughout all sims owned by the Estate Owner of the chain of islands.

Also, they were running bots. I hate camp bots. HATE them.

Actually, "ips" means "instructions per second"...the higher the better.

Unusually high ips can actually be indicative of a problem (ironically all scripts will run rather sluggish) with the sim.

I'd wondered about ips being instructions.
I've been somewhere where the dance routines were running rather fast ... more disconcerting than lag I can tell you!

Theres no campers in my sim. There are usually only a few people there at a time.

I originally suspected that a large mega prim build was causing the problems. When that went away, my ips numbers went down to normal, and then another mega prim build went up in its place a week later and my ips numbers shot up again. But now THAT mega prim build is gone also, and my ips numbers are STILL high....sooo, I'm stumped.


I'll look around for temp rezzers. Do you recall what they were rezzing?

Hmmm... well, just to confirm: yes, very high ips suggests one or more scripts gone haywire. (On the plus side: at least you know it's a Class 5 server--you'll never get numbers like that on a Class 4). There's really nothing about mega-prims that should inherently cause such numbers; the only excuse I could come up with for that would be if there were a script in a physical object getting bogus collision events from a non-phantom mega-prim, but that's really a stretch. There's also nothing inherently *script* laggy about temp-rezzers (and, if they're rezzing objects of reasonable primcount (and phantom, and non-scripted) they are very unlikely to have much other impact on sim performance, either. But they're scripted, so subject to lag-inducing bugs like any other script.

I guess I'd also suggest tracking statistics over time, to see if any fluctuations give a hint. Particularly, it would be good to know how the sim behaves when there are no avatars present. I'm thinking of AOs (always heavy-hitters, but some are *unbelievably* poorly scripted), and sensors as in security devices that might have too-frequent scans.

Oh, also: you'll never "feel" lag caused by script execution per se; scripts just get whatever time the sim has left over when it's done scheduling physics, image downloads, etc. But scripts can cause noticeable lag if they have side-effects in the sim (like doing physics, for example).

Ah my hero Qie ... this is great stuff, ty

I can't remember tbh. BUT Markubis, when the offending club was removed by the Estate owner the problems persisted as the scripts running wild were still running wild. Your sim needs rebooting.

thanks Qie
good information!

the mega prims being used were all non phantom and they had used around 50 of them.

do you know of any way to pinpoint the offending scripts (if there are any?)

I suspect that the sim might need rebooting also.

I seem to recall that there was a rolling restart back around the time that the script performance dropped down to normal.....then the second mega prim build went up and the numbers shot up again.

I've already submitted two tickets to LL but both times they came out and said everything seemed fine.

Well, if it's an estate, there are some tools--but, impoverished Mainlander here, I've no experience using the estate tools. On a Mainland sim, the process of isolating a script is pretty much trial-and-error--and pretty much limited to mod-perm prims one owns, for which you select them (one at a time, or in a group) and use Tools | Set Scripts to Not Running in Selection, and just watch the stats bar. (This is not without risk, though, because some scripts are not meant to be in the running state all the time, so flipping back to Set Scripts to Running in Selection is fraught with some degree of peril. For objects one isn't sure about, the save way is to do it one contained script at a time. If one can mod the prim, one can toggle Running for the script, even if the script itself isn't mod-perm.) One can use View | Beacons | Scripted Objects to help hunt for scripted objects, or a simple sensor script that does pretty much the same thing, locating the 16 nearest active scripted objects within 96 m.

One thing, though: since the numbers really do seem to be associated with the comings and (partially) goings of non-phantom mega-prims, I'd probably start with beacons for physical objects; even though you're not seeing physics lag, I'd be suspicious of a physical script getting spurious collisions from Havoc. Could be a sensor event, too, I guess, but even so, possibly something thinking about doing physics.

Sorry but I have to correct you on this - Scripts, particularly badly designed and written scripts - are the primary cause of the lag that you 'feel', and you certainly DO feel it. And no, it's not because of side effects, it's the actual scripts themselves.

If you check the 'Time' section of the stats for a typical sim you will find that 70-80% of the sim load is scripts - either on the sim or on the avatars in the sim. The sim does not give scripts the 'time left over' - it reacts to changes in load AFTER the fact.

Let me try to explain...

A sim has a maximum amount of time to process a single 'frame' (22.3mS I believe). However, the amount of work it has to do within that 'frame' varies constantly and unpredictably. So rather than using 'time left over' it completes the task, however long it takes. Then it checks how long it took and modifies the amount of time it allocates to script processing for the NEXT frame in the hope that it will be similar.

Every time it takes more than 22.3mS to process a frame, you get 'Time Dilation' (lag).

Max

I am suspicious that there may be a borked script running on my sim, but of course am not the estate owner so can't use the great tools in the estate menu. Are there tools/scripts available for purchase anywhere that can do the same things? List objects with top collisions, top scripts, etc.?

I'll be clearer about what I mean about "side-effects" below... I did use an example that suggested something misleadingly restrictive.But you'll see such numbers on a completely idle, isolated sim with no scripts on the sim or in any attachment. In fact, the higher the script time-slice number, the better the sim is performing, almost always, because it means there's not a lot of physics or network or agent load hitting the sim. (If accompanied by high ips, there's a script problem--but notice how rarely the ips number tracks the proportion of the frame that's devoted to "scripts."

And pure script instructions don't lag a sim, and do not run to task completion within a frame: write a script that calculates digits of pi in an infinite loop, and it won't lag the sim--it will execute in the script slice of the scheduler, each frame, forever, so it will take a share of resources from other scripts, but it won't make physics happen any slower, for example.

Nonetheless, scripts can put a load on a sim that you can feel, even if it shows up only in the script count--or nowhere at all! A good example of how hidden this can be in the statistics is something that checks too frequently whether an avatar is online; that script hits the presence server, which makes that service lag in all sims, but won't even show up in the script statistics. Such library functions I should have mentioned in the category of script-caused lag--though they're not the first thing one thinks of as "side effects." Another more sim-local example is the temp-rezzer run amok: It may lag the sim dramatically, while adding very little to the ips, because it's making the sim do all kinds of non-script work--usually represented as Network time, and maybe Images, as it pushes the newly-rezzed prim information to all nearby viewers (and if, god forbid, the rezzed objects are physical... well, you get the idea.) Vendors are another huge source of lag, even though the scripts themselves do almost nothing; almost all vendor lag comes from image downloads when the textures change; that's under script control, but isn't "script lag."

So... I dunno if this is helpful in understanding what might be wrong when reading sim statistics. But it's important that folks not look at the proportion of the frame devoted to scripts and think that a large percentage there means that scripts are busy in that sim.

Nika,

Not sure about your estate, but the estate owner for mine (or one of his staff) have always been ready to come run the estate tools on the sim I'm in and check for such things. Hopefully you haven't already imed the owner requesting this and been turned down, since it is in the estate owner's interests to keep the folks paying the tier happy.

Well, he looked at the overall sim performance and collision stats and said its fine -- and of course it was when he was looking! But the sim has weird "flashes" of bad performance, and I've seen ips's spiking to 75,000 (usually runs at about 50K), so thought that might be related. I spend time building there, so thot if I had a tool with better stats I could just catch it acting badly.

To check for rogue scripts on a private estate (island).

==> World ==> Region/Estate ==> Debug ==> Get Top Scripts ==> Time

most scripts on island record .001 to .005, a few up to .100 very few over this, if it exceeds .500 we usually disable and return to owner.

Dekka, it's grayed out if you're not the estate owner. Or at least mine is.

It would be nice to have a tool that you could go around and check individual scripts for their performance.
(turn on script beacons and fly around inspecting the scripts)

Sure, it might take awhile, but being its been 2 weeks now of this, I wouldn't mind spending some time.

This thread has some good info in it, and leads me to think of some more questions and comments.

I always have my stats bar running, and I keep an eye on time dilation, sim FPS and physics FPS. I also watch scripts IPS. If I notice things don't look 'right' I'll check the debug scripts and find out what scripts are taking the most time. There are a few scripts on my sim that are consistently in the 0.100 to 0.300 range.

What I find frustrating is that the debug scripts ignores scripts that are within attachments. I've had times with only 10 or 12 people in my sim, that the IPS goes over 100,000 and I've noticed scripts start behaving quite slowly, even if other things (like physics) are working fine. Then things return to 'normal' when a few folk leave. So I know someone's probably wearing something that's going haywire, but I can't tell who or what.

Perhaps the most frustrating thing though, is we have all these reporting tools and functions, and no clear guidelines as to what to look for, what is good, what is bad, and what remedies we can take.

I've learned some stuff by asking questions and just direct observation:
Time dilation should be at 1.00 or as close to it. Sim and Physics FPS should be at 45 or as close to it. Script IPS is a fuzzy one for me, on my sim it seems to average about 50,000 but it wanders up and down from there. I notice script performance issues when it's over 100,000. And I've noticed on other sims, significant performance issues when IPS is under 1,000. The time stats, I've read somewhere that if the total is over 20 then it's bad, and script time over 10 is bad.

In the debug menu, I don't know what to make of some of that. I've read differing opinions on what represents a threshold of bad, for time of individual scripts. Although that debug scripts does not tell you individual scripts, it tells you items which can contain many scripts.

And finally, I'm not really sure what is the right method for finding problem scripts when a script problem is suspected, and if you do find something that you think is problematic, what is the best (most effective, while being customer-service oriented) way of dealing with it. I.e. I can't just start returning people's objects, that will make my customers unhappy.

-Atashi

Sorry I should have pointed out that this is only possible for owners and managers

Lots of good info there Atashi, thanks!