Avatar lag needs to be looked at!

From this thread since it most likely won't get looked at and I want to bring attention to it.

Since SL 1.7, SL has been very slow when avatars first enter a scene (in your draw distance--or visibility). After thinking it was too many scripted listens, and talking with Vektor Linden, he mentioned to me back then that the problem was some kind of AGP pipeline issue, but I don't remember, or recorded, exactly what he said it was. Local lighting is barely standable unless your draw distance is at 64m and there are no prim-heavy (twisty hair/jewelry) avs in-range. Why, as soon as such an av comes in range (even if I'm not LOOKING at it), it tanks my framerate I just don't know--even after their textures have downloaded!

On further, recent investigation, when certain avatars, wearing lots of high-prim attachments (like jewelry) with lights (and local lighting on, obviously) and particles (i.e. bling) enter the scene (but not necessarily when directly looking at them), client framerate drops to around 4 FPS and lots of avatar joint/attachment object updates occur as per the debug fast timers. This has been happening ever since SL 1.7 and nothing (avatar attachment LOD, etc) has fixed it!

SL builds with relevant (at least I think so) updates:

1.7
* Particles behind your camera do not slow FPS more than those ahead
* Lighting on avatar seems to flicker at times when you are in the dark

1.7.1
* Improved updates for objects travelling out of one's view [this one is suspicious]

1.7.2
* The frame rate when the local lighting option is turned on has been improved [it has?!]

1.8.1
* Only the modified part of objects will be updated, as opposed to updating the whole object [obviously this got missed for avatar attachments]

1.8.2
* Avatar LOD system
** Improves viewer performance by selectively culling the display of attachments to agents a certain distance away from the person viewing them

1.8.3
* Resolved an issue where a updates to a complex object would sometimes cause linked prim to disappear

Come on, LL, investigate this issue already please! This issue REALLY needs to be fixed ASAP!

First, turn on local lighting. Then, set your draw distance to 64m (the lowest it can go via preferences). Visit the welcome area and experience the SEVERE avatar lag. Ask any lit-prim-attachment-heavy bling hog (LPAHBH) to join you in a secluded, empty area to test this issue.

I would think thats just plain old local lighting lag. They said a little while ago (if I remember correctly) that they were going to rework it.

Several people have commented that local lighting lag got worse after 1.7, should think a lot of based on computer specs.

No, Zap, when SL suddenly gets a LOT laggier after ONE (1) release, something's changed in the SOFTWARE, not the HARDWARE.

I noticed a severe slowdown in Local Lighting since 1.7. And some definite performance bugs like this. I'm looking forward to a future lighting model that makes better use of the graphics card tho!

Hopefully more will post their experiences as for the other things.

But, Torley, obviously something changed in SL 1.7 with the CURRENT lighting model so why is it so hard to figure out WHAT changed with SL's lighting between 1.6 and 1.7? Subsequent attempts to fix object updating and avatar attachment LOD have done NOTHING to fix this problem so rather than rework lighting COMPLETEY, how about LL just restore how SL 1.6's lighting worked NOW and THEN redo lighting? I just don't get it...but I wish LL would because this lag is RIDICULOUS!

Obviously, jumping up and down screaming your head off won't solve anything that the Lindens are already well aware off. Its like chopping off a chickens head. It knows that its head is no longer connected to the body, but it keeps running and running until all its blood lets, and someone picks it up to cook it in the oven. The good part about the headless chicken is that it no longer makes any noise after the head is chopped.

Um, I'm not screaming and/or jumping up and down, Cot. I use the capitals for EMPHASIS not SCREAMING/YELLING (a common misperception). NOW THIS IS SCREAMING! Note the use of capitals THROUGHOUT the sentence vs. on a single word/phrase/name/etc. Also note the exclamation point (!). Without it the all-caps sentence could still, technically, not be screaming.

I just can't believe LL is so inept as to not be able to fix this bug--especially when they caused it in the first place in SL 1.7...

As shown, the avatars causing the lag (especially Edera Kenzo's) have prim-heavy, lit attachments (jewlery) that also had "bling" particles. I forgot to turn on statistics but my FPS was around 4.

And another:


Lit prim-heavy attachments are definitely what's causing the full-object updates.

Also from your SL Answers question:

I've seen that with Local Lighting as well--exactly the same thing you're having. I bug-reported it earlier as well, and it's come up on several other occasions. Instead of muddle in the muck, Steve Linden spoke about a more elegant longterm solution:



Source

Torley, I KNOW what Steve Linden wrote but that does NOTHING to fix the CURRENT issue. How hard is it to simply go back to pre-1.7 and test this issue (since, obviously, LL can't seem to figure it out by looking at its current code--surely it keeps old code to compare when bug-fixing!)?

I would appreciate if Steve, Andrew, or whoever has more info on this issue, to PLEASE reply here and let us know what screwed up in SL 1.7 and why it can't be fixed before a completely new light model is implemented MONTHS (at least--mots likely longer) from now.

We are at this very moment working on a complete replacement for local lighting. It will be in one of the next few releases, so all I can suggest is to disable it for now and wait for something better that we can actually make fast. I know it is frustrating, but if we spend all of our time supporting broken features (and for a variety of reasons the existing local lighting model is very broken) we will never have time to implement new, better features.

Works for me. Designing a fully functional replacement rates higher on my charts than supporting an inefficient and buggy implementation. Of course I think what Eep is getting at is along the lines of, "it WAS working, why did you break it?" and by extension the line, "why make broken code, make working stuff instead" that's echoed by most of the community.

Well, we didn't *intend* to break it We have been making a number of architectural changes in the past six monts that sometimes have unanticipated consequences. Because less than 1% of the users have local lighting turned on (this was true even before it became more broken; it has always been slow on all but the very fastest machines) it has not received sufficient testing attention. The new implementation will run quickly on all but the slowest machines.

After a week of fine-tuning its brand new top-of-the-line machine, running Windows XP x64, and with two nVidia SLI cards working in tandem (2 GB of RAM and 512 MB of video RAM ), my friend and associate at RL work (which uses SL for several educational and cultural projects) wanted to create the ultimate SL powerhouse, to be able to demonstrate SL to customers/partners with 50 fps on drawing distances of 256 m and the remaining Preference settings at their maximum levels. No less should be expected from a highly fine-tuned machine that does amazing things like 300 fps (!) on some of the "game benchmarks" like 3DMark.

When launching SL, the best he got was... 4.7 fps.

You cannot imagine his disappointment. At that point, he was so totally frustrated, assuming that SLI technology was too "early" in its technology stage (although having been around for a year or so) to be able to use it on "serious" platforms for SL. Sure, most other games ran well, and with some tweaking, fine-tuning, new memories, better drivers, and all sorts of dirty tricks in the field, he managed, after a week or so, to get a cool-running machine with incredibly smooth rendering on almost all things he had loaded on his computer.

Except for Second Life. Which refused to spew out more than 4.5-4.7 fps, slightly more on totally empty sims, but not much more. Next to him were people using 3 and 4 year old computers, with cheap $60 video cards, happily with 15-17 fps on the exact spots.

After another session of deep frustration — he had some classes to teach SL over the weekend and wished to "show off" SL at its best — he went to home earlier, shaking his head, and lamenting the poor code of SL, that can't fully use such advanced technology like double cards and the latest drivers. He was already starting to think about alternative platforms, something that he always does when he gets a high frustration level with Second Life. You must understand that this guy is "selling" SL to all sorts of customers and partners for over a year now, and he needs to feel confident on technical issues related to Second Life. The usual concept of saying "oh, it works well if you have the right hardware" is a common "excuse" when people complain of a slow-running SL. This time, however, he did not have that excuse; he had the best machine his money could buy (for a limited budget), which was able to deal with highly complex 3D games of all kinds — just not SL. SL for him was fatally flawed.

Well, I hardly expect to understand 10% of the technical buzzwords he uses when explaining how 'video pipelines' work, but I've been a Mentor for over a year now, so when he left, I tried to go over his SL configuration. My first attempts of looking at the parameters did not improve anything. So I thought I should apply the same rules I use when dealing with newbies complaining about "lag": turn local lighting off, Disable far clip, reduce the drawing distance.

I unchecked the box for local lighting, and suddenly I couldn't believe my eyes: from 4.7 fps the machine jumped to 47 fps in the same place!

Having turned off local lighting since 1.7 on all computers I have ever put my hands on, I haven't ever seen if this feature worked or not; it never worked for me after 1.7, so it's easy to overlook it on other peoples' configuration. Well, happy with the results, I started to raise all values on Preferences (stopping short of putting drawing distance on 512 m — that, at least, will really slow down the machine and the compromise of having a drawing distance 160 m with 30+ fps *everywhere* but at The Edge was more than enough for me ).

This impressed me so much — I had absolutely no idea that local lighting was that badly broken! As a matter of fact, I watched suddenly something I had never seen on SL before. When turning up the bandwidth to 1000 Kbps, and since this awesome machine is able to load all textures so stupendously quickly, uncompress them blindingly fast in memory, and push them inside the vast amount of available memory on the graphics cards... suddenly, the whole world seemed to slow down and work in slow motion when I was flying around a sim!

What was happening? Turning on statistics, the FPS rate was still in the upper 30s, and everything was rendering so amazingly fast that I couldn't understand what was wrong. The sim was empty of people, low script usage, and while it certainly had tons of textures and prims, the machine dealt with them like a warmed-up knife cutting through butter.

Then I watched the time dilation: it was down to 0.52! Oh, so something was wrong with the server then; no problem, I could always go to the next one, even flying at "slow motion". What was weird was that every time I stopped, time dilation went up to 1.0. Fly around, and it dropped to 0.5 or thereabouts. Hmm.

Entering the next sim, the same thing happened. And on the next one. And all the neighbouring ones. Uh-oh. What was going on??

The answer seemed to be simple. This machine had such computing & processing power, it was squeezing the sim server dry — pulling megabytes and megabytes of texture data, dealing with them in nanoseconds, and asking the server for more, more, and more. Well, in that struggle, the poor server couldn't keep up with the demand — this was the strange situation of feeling that I had now a machine that was more powerful than the sim server itself! It processed data faster than the sim server could deliver — and this was causing time dilation. Wow. That was very cool!

And this from a machine that only a few hours ago coughed up just a handful of frames per second, painfully struggling to render them — just because it had local lighting on!

My suggestion/recommendation to Linden Lab: while you fix local lighting on a subsequent revision of the renderer — simply turn this feature off. In almost all decent computers, it comes on by default. It should not only NEVER be a default — it should be greyed out and not selectable at all! Remember your new residents. They come to SL with their top-of-the-line computers, fine-tuned to play WoW at 80 fps, just to encounter a 3D virtual world where they get 4 or 5 fps on a good day. But if they only could launch the SL client without this option on, they'd get 30-50 fps easily on their machines! (and we Mentors would be able to explain to them that the reason why they 'only' have 50-60% of the performance of other platforms is due to the streaming nature of SL).

So, please, Linden Lab, get rid of that option. Your own statistics tell the same: less than 1% use local lighting anyway, at least since 1.7. I loved it before 1.7, while it worked reasonably well. I almost don't know anyone who has turned it on (I get 0.2 fps when I use it...). It's a useless feature, and also an unused one. Get rid of it fast. Don't continue to persist in offering an option that, sincerely, does not work and causes excessive frustration!

I'm almost considering setting up a big panel on the Welcome Area saying: "NEW RESIDENTS! WELCOME TO SECOND LIFE! TURN LOCAL LIGHTING OFF NOW AND GET A 10x BOOST ON YOUR PERFORMANCE! THIS IS A FAST PLATFORM IF YOU TURN IT OFF! IT'S A BROKEN FEATURE FOR ALMOST A YEAR NOW, WE KNOW IT, AND LINDEN LAB IS WORKING ON IT AS YOU READ THIS!"

... but it would be so much simple to turn it off forever, and patiently wait for the upcoming changes on the renderer. LL did the same for the AGP "improvements", at least on the Mac version, a while ago (when it was broken for a few releases)...

.

Disabling local lighting by default is fine but removing it completely until LL implements a new light model is just stupid since it IS useful if not used in high-prim attachments (specifically jewelery).

1) I wasn't kidding, Champie Unless any other effect was present at the time, what I described was consistently observed across several different sims, all of which empty of avatars and with a high prim usage (> 10,000). When I've reduced that machine's bandwidth consumption to 500 Kbps, the time dilation drop effect did not appear any more. Alternative explanations of what I've observed (when flying, server time dilation drops; when stopping [ie. no need for loading further textures], server time dilation climbs back to 1.0) are welcome.

I'll try this out on different times of the day just to make sure it's fully reproducible; I might even be able to make a short movie to illustrate what happens I just need access to that machine again, hehe.

2) Eep, you seem to suggest that all local-lighting-enabled attachments should be removed, then? Or that local lighting does not take into account attachments for the time being, until the renderer gets further improvements on its code? Mind you, I have absolutely nothing against local lighting myself; as said, I loved it from SL 1.4 to 1.6, while the performance hit was less than 20% on the total FPS. I even attended some classes on lovely effects you could create to take advantage of local lighting. It was a beautiful world

Although it's not only attachments that are the culprits; all you need are a couple of "light-enabled" prims inside very complex structures.

Since I'm all for compromises, I'd be glad if this option is simply not turned on by default...

I was helping a friend of mine squeeze out some better frame rate performance on his Toshiba Laptop, and after reading the SL Wiki there was one bit of info that helped. On some systems using ATI video chipsets, the option under "Adv. Graphics" tab, removing the check mark for "Avatar vertex Program (Faster)" would actually run better if that was UNCHECKED. And it made the game playable for him, which was otherwise cumbersome with the 5 fps he seemed to getting with that option on.