The Lag Monster Myths

sigh. Valid concept Dnate. Totally incorrect application. Been there. Done that. Not gonna cover it again.

I don't agree with this at all. When LL releases Mono, HAVOK 2, and HTML, and other such features that have been stated, I think that SL will be the platform that is to be. How many millions have already been spent on R&D for SL? How much time? These are all things that cannot be easily done by another company. The current SL is the begining of things to come, it is like Windows 1.0, very basic. It will grow and mature as time goes on, but it will only get better. I have seen SL grow, improve, and all in all get better as the time has gone on. Give it 10 more years and then see what it has become.

That is what you said, I was just pointing out that you are wrong there. Things go wrong on servers, that is true, but if the problem is always happening in one sim, and not others, then the content is most likely the cause.

Let's see what happens then. I'm sure IBM had the same attitude when unknown Compaq came in and took over their PC market. Frankly, I seriously doubt SL will survive another 2 years, let alone 10.

Oh i agree i read every post in this thread and i see your point exactly. I too was miffed when i scripted a physics enalbled object my first time(which is definetly smaller in scope compared to your troubles) and was like "WTF why isnt it working wright I know my code is good!" After looking over my code for a couple of hours i came to this forum to get my answers and while trying to figure out what i was doing wrong i found out its a problem with the physics engine. As i kept on reading i found out that everything that attracted me here doesnt work right lol i guess thats life. I do still play because i still do have fun. Well if you can get LL to address the lag issues with a seek and destroy mentality, you will get the Gold star in my book. I wouldnt even mind if they shut everything down to locate the source and i believe many of the other SL'ers will agree.

Have you seen Compaq lately? I think they got eaten by HP, if I remember correctly. IBM is still a HUGE company making PC-related things. Plus part of the problem with IBM is they stopped innovating, which if LL stopped that to, I would say they are doomed too.

PS, stop with the ninja edits!

heh no LL doesn pay me for this stuff, i'm a 'feeb' or somethin heh

anyway basically what iv'e seen is that there are specific problems where some sims are being addressed to the wrong hardware aka three sims are being addressed to a two core hardware box and that does have a very specific identifyable lag footprint, and in cases where that has been made evident, LL has indeed taken the time to manually re-address the machines when it has been brought to their attention.

As to the specific issue of lag, in the 20-30 sims i have been brought out to diagnose 'lag' on for people, only once have i ever seen it be this addressing problem... *ALL* the other times it was just bad content or too much good content...

so when i say its content most of the time i mean that in >95% of the specific cases i have studied, it was indeed the user created content of the sim that was causing the performance issues, not a back-end LL problem.


Now if you want to say LL should be responsable for all user created content, well thats plain silly... you cannot blame microsoft for the fact someone uses visual basic to code a really crappy scanner program... you blame the people who coded a really crappy program.. because they COULD have done better, and specifically chose not to.

same with SL content... you CAN (for one recent example) make a keyboard that appears and dissapears as you type, and have it use *almost* no sim resources, just a low speed timer and some basic alpha set calls... at the same time, you can also use multiple high speed timers, and set alpha continuously, so rather than just setting when the avatar types, you continually set the object either clear or solid, depending on what it should be...

in this case, this specific object i am talking about literally lagged our entire simular almost to its knees and was caussing MASSIVE load to the clients, well over 500kbit just for that ONE object, continuously, forever

is that LL's fault?

*NO*

they provided the ways of doing it right, this person simply hadn't used them... so i fixed it for them, after seeing the problem, and thats one less really laggy object in world.

LL *IS* working on providing more and more from the backend side of things... the synchronous sim processes should really help combat time dilaton on heavily burdened sims... as was noted by the massive stress test yesterday... they are looking at a massive over-haul of the back-end of LSL as well, taking it to a more advanced open source system... but again this takes time, and dev costs... and that also does not 'exempt' people from the necessity of writing good code

just because you will have lets say 3x more overhead, does not mean you can just flood the sim with 3x as many toys and then whine about lag again..

you are (virtually) living within the SL world, and it is YOUR responsibility to understand the limitations of the world and learn how to live within them *NOW*, while of course always pushing ahead for progress...

where possible you can even help out BY finding things that users created that do not run well, and help them turn around an become more efficient..

when 'pose ball' furniture became in-vogue there were tens if not hundreds of scripts written in that style that were absolutely horrenduous, and there were also some really good ones..


now from the 'general' user in SL all tey know is they drop a couch and sit... what they don't know is that some of those things cause absolutely massive lag, while others cause literally none at all..


i do agree that we do need better tools to see which are which, and it should be more obvious to general users, because there is nothing that solves bad code quite as rapidly as attention... and this goes for end users at least as much as it goes for LL...


you want the easy way out wayfinder, you want someone to wave a magic wand and make it all better... well the wands are under construction, but in the meantime take some own damn responsibility for your area and work on helping it work within the CURRENT limitations... you'll be rewarded with a faster, healthier sim, *NOW*, not 6 months down the road..

This is pretty much exactly the advice i gave to you back when i actualy looked at yer sim and i'll give it again now.

Yer problem is not in the backend of SL, its with the fact you are unwilling to accept the current limitations on the front end... and while they are being slowly pushed back, you need to do your part as well, or you are going to be in for a long, frustrating wait.

Maybe I am looking at this in the wrong light. I think I may be misunderstanding you somewhat. Since this is a thread brought back from the dead, I know a lot has changed since you first posted this thread. Has the lag you have gotten any better over time? Does the lag come and go with any sort of pattern? When your island is laggy, is the entire grid laggy? Do others in the sim experiance the same lag as you at the same time? Are parts of the island more laggy then others? Does the lag go away if you reboot?

I am talking now. It is very possible that it is not just server side, but it is not just you either. It maybe some odd combination of the two. Lets try to get to the root of the issue, instead of trying to place blame. It could help the Lindens to try and pinpoint what part of the server code needs to be looked into more.

LOL sorry bout that. I tend to think on the run on these forums. Then have to go back and edit. I'll try to pull me thockth up.

Hey, I didn't say Compaq did EVERYTHING right. But you know... if I'm remembering correctly they were bought out by HP for how many billions of dollars. Man, I'da taken the money too. LOL

IBM completely left the PC market. They hung in there for a while, but every step they took, they failed to pay attention to their clients. They approached the PC market with a corporate mentality and failed to recognize what the client really wanted. IBM thought it knew better than their clients. After all, what could their clients possibly know about computers?

I'll tell you what their clients knew: they weren't tech saavy, they weren't computer saavy. They weren't programmers and didn't know squat. But they knew what they WANTED, and they held the checkbooks. Compaq and other PC manufacturers studied things not from the tech books, but from client-side. They beat IBM at its own game because they listened to the end client and provided what that client wanted, no matter what it was or how much it went against existing corporate philosphy. Compaq basically put themselves in their clients shoes and said, "If I was a small businessman or a home computer user, what would I want?"

That's why Compaq succeeded and IBM failed. Compaq listened to the ones who would be paying the bills.

I see your point. I don't feel that LL is at that point yet. I truely believe they are just overwhelmed with the amount of work that needs to be done. They are trying to please everyone, but most of the time it is user error, not code error.

Yes, no, sometimes, yes, sometimes, occasionally but not usually. LOL

We have conducted test after test, performed simulation after simulation, taken data readings, kept statistical records etc etc etc ad nauseum.

Experiences of sim lag were shared with other sims all over the grid (at times), were experienced locally (sometimes) and had no discernable pattern or cause. They'd just come out of nowhere, disappear just as quickly. They might last a few minutes, or a few weeks. Then suddenly vanish, as if they'd never existed.

We'd go from 600fps down to 25fps, from 1.3 RTms to 16 RTms for no reason whatsoever. We would check the sim parcel by parcel, prim by prim, using script searchers and eliminating every single unidentifiable object on the sim.

We were told by LL that it was the number of prims on the sim, the number of active scripts, the number of textures, the way we hold our tongues when we type at the keyboard. Nothing, ever, not once, pointed to sim content as the cause of lag. I approached this like I approached every other bug-finding project I've ever worked on, and my professional conclusion was that the sudden causes of lag and that the cause of general, extended lag that would suddenly vanish... was totally server side. Period.



(note: I don't think LL ever actually came out and said it was the way we hold our tongues. It just seemed to be implied.)

hmm do they charge money..Yes..they do so they are responsible for providing the consumer with assistance. Think about this: You pay your car insurance either monthly, biannually or annualy(same as SL Charge) . You get into a car accident. Its your fault. Now does your insurance company have the right to say "to bad its your fault". Think about it.

You know, I agree with you there Dnate.

I absolutely do not mean to sound anti-Linden Lab. I am anti-LL-attitude (ie, poor customer service and communications). I am anti-smokescreen-BS. I believe in being honest with clients, not giving them the mushroom treatment.

Yeah, there is an incredible amount of work required in running SL. One of the solutions to that is to stop coming out with new content until the old content is fixed. (Can you imagine some guy building a garage for his skeedo when his living room roof is full of leaks?).

I think there are a lot of basic, fundamental concepts in the SL system that are not being addressed. I think LL is greatly in need of a watchdog to bite them in the butt once in a while and bring them back to customer-service reality.

I'm definitely not anti-LL. I am pro-LL. If I were anti-LL... I'd leave and tell them to kiss me backside as I go out the door. But frankly, after the stuff we've put up with for the last 3 weeks... I'm starting to get to that point. I seriously regret agreeing to set up a 3rd sim on this monster.

You get insurance for that reason, incase you do reck it. If you buy a car from a dealer, and then wreck it, you will still have to pay your loan off. You can't goto the dealer and say, I need a new car because I broke the old one.

LOL.. well, that's a two edged sword. I've had 'em do that.
But yeah... you're right. The insurance company takes money to accept that responsibity-- they're required to assume that responsibility.

Actually, to fine tune the example: you BUY a car from a manufacturer. You get in an accident. Now the question: was the accident your fault, the other guy's fault.. or did the car itself fail?

When the car continually fails, and is found to fail for thousands of clients, they have a recall and fix it. And yup, that's the company's fault.

Now this has all sorts of permutations and combinations that can be debated. But bottom line is: if the company is going to manufacture a car, they have a certain amount of responsibility to make sure it works on an acceptable level.

One thing I wonder: when you buy a car from a company, do they have the right to give you HALF a car? I don't care how fast the car goes...

You know, I could have sworn Philip said something about no new features untill the bugs were fixed, right before they did streaming video....

I do agree that the customer-service needs help. I have been dealing with a issue with them for 5 months now, and they keep telling me the same thing over and over again. It seems that if it is not a simple "uninstall, upgrade drivers, reinstall" they don't know what to do.

Eltee, I think I've taken more responsibility for the way my sims operate than most other users on the grid. But there's only so much we can cut and still retain a semblance of operational capacity. I think one user described it well when he posted here the "Linden Lab Credo":

"An empty sim is an efficient sim." LOL

Has nothing to do with the reality of someone paying $195 a month for a service and expecting that service to work reasonably well. But there's only so much a sim owner can do when Linden Lab refuses to provide us with the basic tools we need to properly police our sim, isn't there?



No Eltee, my "problem" is that I'm not stupid enough to accept lame claims and excuses when something goes wrong. I'm not ignorant enough to let someone tell me it's my content at fault when all of our data analysis proves just the opposite. I'm a businessman and when someone makes an excuse or tells me something that is contrary to observable fact, I'm going to approach it from a business standpoint and if necessary, start busting responsible parties.

You claim you've "debugged" 25-30 sims and always found the problem to be their content. To be honest, if you "debugged" them like you debugged ours, I'm not surprised at that claim. You're a good scripter from what I've seen. But like many techs, you seem to focus on the technical rather than reasonably meeting end-user needs. I've worked all my life doing the later and I've found that when the techs are screaming-- the end user is usually happy. The idea Eltee, is to find out how to make the technology work the way the user works-- not the other way around.

And I am going to say one more time because you just don't seem to get it:

When content doesn't change and lag does... that's server side.

Maybe you need to print that out, blow it up, and hang it on the wall until it sinks in.

No personal offense intended. Just being humorously snide. LOL

To add to the above made point:

I visited another sim today. They were only using about 5500 prims. They had about 450 active scripts, most of them vendor oriented. Hardly anyone on the sim (I think 2 avs other than myself).

ElvenGlen on the other hand, right now has about 13,500 prims and 550 active scripts of a wide variety.

Our sim was running twice as fast as theirs.

Now, what's that again about content?

I think that is where you stray from what eltee, Lee, and I are saying. As soon as you put in a script that runs, your content is no longer static. I could have a script that runs fine in states A, B, C, and D, but if it ever gets to E it starts taking a lot of server resources. Most of the back-end issues are with content pulling more then it should with resources, causing sim wide issues. You, yourself, is not a static thing. Your eyes move, your hands move, you breathe, you are an active agent. You inventory streams to you, your attactments change UUIDs. So saying "nothing changed but the lag" is not true at all, unless nothing is moving, listening, sensing, or being looked at. SL is a dynamic world that is always changing.

Yes Dnate... and didja know that 2+2=4?

All this stuff Dnate is elementary. It's kindergarten computer. You think we're not aware of this? You think we haven't factored that into the math? chee guy, gimme a break.

We've done our homework. Come back when you have some real data dude.


[EDIT: Forgive me, continued frustration is taking its tole. Allow me to properly reword and then change MY attitude.

Yes Dnate, we're aware of this. We factored all that into our data analysis. This was considered and adjustments made to determine accurate results. Thanks for mentioning these things.





Even if they are...
SMACK!
NO! BAD WAY!

Sigh, you can't compare sims with diffent content. You don't know what the scripts on said island are doing. There could be a few that are really badly done, causing the entire sim to slow. You need two sims with the exact same things happening to both of them at the same time, and then you need to show one that is running slower then the other, that will PROVE that it is not content.

OR... on the other hand, you can take ONE sim and increase/decrease its content and take readings for accurate statistical analysis over a period of time... like we did!

wow... you mean... someone actually RESEARCHED this stuff???

I want to get to the bottom of it too. I just think you may be are jumping too fast to blame LL. It very well could be all server side. From what I have seen in the almost 2 years of making things in SL is that a lot of the issues with lag in sims is not due to back-end code, but mostly content, and then connections. The back-end needs work, but I am just not so sure I would jump right to blaming them. Is there a way I can see your data? I would love to look at it and try and pinpoint your issues.

if i 'want' to have 15,000 flashing red and blue cubes in my sim its not 'LL's fault' if that sim is laggy...

lets say you won't be happy until you can effectively fully re-create Doom3 in SL inside your sim... well thats not LL's problem

you are responsible for knowing the limits of what you are buying...

yes, yer paying $195 a month, no that does not mean you can rez 15,000 lag inducing objects and then whine you aren't getting your moneys worth

now i know thats not what you are doing, its an exageration, a caricature i am using to prove my point here..

you are getting your moneys worth for your sim... you want to do more with it than the current system just really happily supports, yer stressing it.. you yourself agreed you didn't know the impact of everything you had out, and had scripts/objects you didn't really have a way of quantifying the sim impact of...

what i am saying is that there are less laggy ways of doing alot of what you are doing, that would not be visible to a random user as any different, and that is where your responsibility begins..

find less laggy ways of doing what you want to do, because they are there, and yer overall attitude is 'fix it for me' and well thats quite honestly BS, of all the people i have worked with to find lag, i found none as overall disinclined to actually WORK on the problem than you with your sim... its nice you want to have all the toys in the world out for everyone to play with, its also wholly unrealistic, thats NOT what LL is 'selling' you for your 195 a month, plain and simple.

its yer responsibility tho, as an owner, given a template sim, blank as it were, to fill it in a way that best meets everyone's needs. You are going to have to make compormises to work within the current system's capacity, or *YOU* are going to lag your sim.. its honestly as simple as that.

as to yer 'data' its by in large bull

you don really know what yer lookin at, you give meaningless examples, and wave numbers around that don't really have any relaton to what the actual issue at hand is...

in fact some of yer examples go so far as to begin to actually disprove yer own case, but yer not realizin it..

'active' just means a script is doing something

'10 actives' can be more damaging than 1000, if they are a REALLY bad 10... so the number is more a metric to begin to guide you, than any formal 'performance' measure

static (unscripted) prims actually don't use any resources on the server's actual processing thread so whether you have 100 plywood cubes, or 15,000 only affects the framerate of the people looking at them, nothing more.

the *ONLY* contingent you have is that your 'hardware' is inferior and yet this has been proven to really not be the case... its that the stuff you have out, isn't running well... and thats as much yer responsability as LL's...

they *ARE* working on it, you aren't... now who is to blame for the fact that right now, this minute, its 'slow'?

Yeah, but that also leads to issues. What content are you placing out and taking a way? A script is not a script, and a prim is not a prim. If you take a way a physical scripted object and put 3 non-scripted prims out, you are adding more content, but it is not a fair comparison. Also, don't forget that the physical sim changes too over time, you will sometimes get bumped to a different address is one has an issue.