Severe Packet Loss After Patch

Hiya guys, I am having some issues with SL. Today, when I downloaded the 1.6.10 patch, my SL client slowed down severely and I can't move much or do anything. I realized that I had over 50% of my packets dropping on my for no reason after the patch. I am not behind a firewall, I reinstalled SL, I called my ISP and they said they didn't mod anything, and I have no idea whats wrong.

I did a trace route to the sim I was in, along to the data server of SL. The results is attached.

|------------------------------------------------------------------------------------------|
| WinMTR statistics |
| Host - % | Sent | Recv | Best | Avrg | Wrst | Last |
|------------------------------------------------|------|------|------|------|------|------|
| nrp12.macau.ctm.net - 0 | 977 | 977 | 0 | 11 | 71 | 20 |
| v601a.macau.ctm.net - 0 | 977 | 977 | 0 | 13 | 261 | 10 |
| gw2-192.macau.ctm.net - 0 | 977 | 977 | 0 | 15 | 220 | 10 |
| 158.205.250.13 - 0 | 977 | 977 | 70 | 76 | 471 | 70 |
| ge2-2-0.tokg4.idc.ad.jp - 0 | 976 | 976 | 70 | 85 | 281 | 180 |
| so7-1-0.laxm1.idc.ad.jp - 0 | 976 | 976 | 200 | 205 | 410 | 200 |
| 6-18.car2.LosAngeles1.Level3.net - 0 | 976 | 976 | 200 | 208 | 440 | 200 |
| ae-1-52.bbr2.LosAngeles1.Level3.net - 7 | 976 | 911 | 200 | 206 | 461 | 231 |
| as-0-0.mp2.SanFrancisco1.Level3.net - 0 | 976 | 976 | 180 | 201 | 450 | 191 |
| so-10-0.ipcolo1.SanFranciso1.Level3.net - 0 | 976 | 976 | 190 | 194 | 361 | 190 |
| gw-level3-sfo.internap.com - 0 | 976 | 976 | 180 | 191 | 420 | 190 |
| border1.ge1-1-bbnet1.sfo002.pnap.net - 0 | 976 | 976 | 180 | 199 | 471 | 190 |
| data.agni.lindenlab.com - 0 | 976 | 976 | 190 | 197 | 371 | 200 |
|________________________________________________|______|______|______|______|______|______|
WinMTR - 0.8. Copyleft @2000-2002 Vasile Laurentiu Stanimir ( [email]stanimir@cr.nivis.com[/email] )



My SL status;




Second Life 1.6.10 (3) Aug 8 2005 13:47:08

You are at 246003.6, 255377.9, 23.5 in South Paris located at sim685.agni.lindenlab.com (69.25.105.117:13002)

CPU: 0.13 micron Intel Pentium 4 (2405 Mhz)
Memory: 512 MB
OS Version: Microsoft Windows XP (Build 2600)
OpenGL Vendor: ATI Technologies Inc.
OpenGL Renderer: RADEON 9700 x86/SSE2
OpenGL Version: 2.0.5147 WinXP Release
Packets Lost: 11406/18449 (61.8%)
Viewer Digest: 1c5feebd-156c-d5d9-eb24-8b87d9f38b54





What is going on? The patch seems to involk a severe packet loss.

For some reason, updates sometimes invalidate your whole cache requiring 500MB or more to be reloaded. There have been some reports of it getting "confused". I recommend you try clearing your cache as the stats you posted look pretty good. Depending on what else you are running, 512MB RAM might be a little small but 1.6.10 shouldn't have pushed you over the edge (unless there was more to it than the release notes say).

I have had the same reaction to the 1.6.10 patch, and now cannot move about on my own in SL, and crash several times an hour trying to. (In my case, the symptoms seem to starte with the previous update, and just got a lot worse in the last.) Oddly, while the way it acts is just what you would expect fom severe packet loss, the SL reports 1.6% or less packet loss at all times... (I just spent two days working, with my providor's assistance, to eliminate all the sources of signal interuption we could find. I'm getting a very clean connection right now!)

One of the Lindens talked this over with me, and all we could come up with was "update my video drivers." The thing is, if my drivers weren't suitable, why did they work fine before the newer SL? In any case, I am wondering if any of you are using an HIS RADEON 9600 Pro, (128M), and if so, what driver version? (I have 6.14.10.6458 )

I am running win XP pro, with all the updates,
on a Pentium 4, hyperthreading CPU at 2.4Ghz.
Memory: 1Gig of DDR ram at 800 FSB.

The crash logger doesn't work either, btw, ever...

Because this is the standard response from tech support to get you off the phone, since they have no clue why it's not working. What, your system was stable before? Everything ran great before? No crashes before? No sluggishness before? You just upgraded and now it's not working? Ah, well, clearly your video drivers are outdated! Now serving number 8,257....

I've noticed too within the last update prior to this one that I started seeing what looked to be significant lag on the sims where I have land. Neither sim runs many scripts, so I was pretty amazed at the lack of responsiveness. A few times I had to log off because my AV would not be able to land on solid ground. I noticed this on 3 completely different PCs, that include a laptop with a P4, a desktop with an Athlon, and a desktop with a P3. Oh wait, I bet all the video drivers on all 3 machines must be faulty - stupid me!! *smacks head* And here I thought the SL patches might have been to blame. What was I thinking?!?!

You hit that one right on the head. Here's a true-life story...

Last April we set up ElvenGlen sim. Thing was running like greased lightning. Running 750 active scripts with an average Sim FPS of 950+++.

About the middle of April, was like someone flipped a switch. Sim FPS dropped to 150... 100... 49... and RunTask ms jumped to 4.5.. 5.9... 7.9...

We called a Linden (won't mention the name, but a well-known one) and the royal run-around began. Over the next several weeks/months, here are a few of the lines we were fed:

"You have an excessive number of active scripts". When we mentioned that NOTHING ON THE SIM HAD CHANGED OVER THE LAST 30 MINUTES, and that WE ARE THE ONLY AVS ON THE SIM... they still stuck to that line. TOO MANY ACTIVE SCRIPTS!.

So we cut scripts to the bone. Got down to 450. Guess what Linden Labs told us?
STILL TOO MANY ACTIVE SCRIPTS! That HAS to be what's causing you to lag.

Oh, and they added to it, "Your sim is lagging because of CONTENT". (Ignore the fact it was the SAME content we had when we were running 950FPS+).

So we started taking sim figure readings over a period of a couple of weeks and proved beyond the shadow of a doubt it was NOT content. We took those figures to the Forums. Not only did LL stick to their guns, LL groupies joined them. TOO MUCH CONTENT!

So we took further measurements and proved that content had nothing to do with it. How? We ADDED scripts to the sim. We ADDED content. And the sim actually ran FASTER than before. LOL.

So, point proven. Took that to the forums. I was even mean enough by this time to start intimating LL might not be on the up-and-up. Maybe they were hiding known problems from users... were they perhaps crowding bandwidth or running more than one sim on a server? NO, NEVER!

But Lindens did then come to our sim and started checking around.

OH, YOU HAVE MORE THAN 1,000 TEXTURES! THAT'S THE PROBLEM!

(That one made me laugh out loud).

You know the one thing they never mentioned?

SL data systems suck swamp water. SL internal routines are bug-ridden. We might be having some internal problems all right. Not ONCE did LL accept the remote possibility that their system just MIGHT be to blame.

I have more than 20 years experience ferreting out stuff like this; I'm a business computer consultant and have worked on some of the nastiest problems known to Murphy's Law. I knew from the start it wasn't content (if it was content, why did it go from greased lightning to snail in a New York minute?). LL had been informed of my background and that I was absolutely certain, based on our figures, the problems were server/program side. Think that phased them? Nah.

It's CONTENT!

A few weeks after this incident, a few announcments were made that LL had discovered severe data bottlenecks in the email and inventory systems (as is obvious, inventory bottlenecks still exist and the LL database system is destroying people's hard work). It was also discovered that at times SL bundled up more than one sim on a single server. (*gasp*!)

We lost more than 1000 prims today from a near sim destruction caused by a major SL bug that we reported to them 3 months ago. See /111/c7/57279/1.html

So yeah, you're right on the money in your appraisal of LL response to problems and expertly passing the buck back on to the user.

Wayfinder, I read your posting, and sadly this has become the norm for dealing with issues, not just in SL but many places. Logic and reason fly out the window, replaced by ego, laziness, cost cutting. and failure to listen. And the problem is not limited to the company. As you pointed out, the "LL groupies" are right there to admonish should you so much as utter a note of dissatisfaction - forget the fact you pay $200 a month (i.e., pay the salaries of the people who work here). How dare you so much as suggest there may be an issue - traitor!! *chuckles*

To the groupies, we must look like we hate SL. Wrong! We love SL! I spend hours online every day. Like you, I spend a fortune on it monthly with all the land I own. And that is what makes it so frustrating, because we do love it so much.

Also, like you, I have many many years of experience with computer systems, and what works and what doesn't work. When a system is relatively stable, new features are a wonderful thing. When a system is not stable, it's a recipe for disaster. And when management can't see the impending disaster, then they they ultimately doom the project.

Like I said, I have a large amount of land, and right now I am very concerned to build anything. I keep reading threads where people are losing prims right and left, and prims are moving, etc. I have lost objects myself too, and I have seen the lag appear in sims overnight. Why would I want to invest hours, days, maybe even weeks in a project that could be destroyed in the blink of an eye, because the maintainers of the system aren't interested in fixing the known core problems and respond to logical reasoning with scripted ignorance?

There you go, groupies - some fodder for you. Knock yourselves out.

Kavai, I am sorry I don't have an answer for you, as I know you must be very frustrated. I am hoping that maybe someone will start taking notice of these issues and really look into them, from the perspective that maybe, just maybe, there may be some bugs in SL causing this. But it requires people speaking up and putting pressure on Linden Labs to respond. I apologize if it looks like I am masking your issue. In fact, I am trying to get someone to look at these patches and realize that's where the problem most likely exists.

Yup, that hits the nail. If I didn't care about SL, I'd have done like thousands of others have done and just left the board and not looked back. Yesterday I spent TWELVE HOURS online. The majority of that was spent solving other people's problems, and at least 3 hours was spent trying to keep our sim from self-destructing due to that continuing SL deed to group bug. I care about SL. But like so many others, if LL keeps ignoring major issues like this, there will come a time when I cease to care.




And that's the deal. Like you, in this instance I have no answers for Kavai... just the reassurance that s/he's not alone in experiencing such problems. But you're absolutely right-- why should we continue to build and create when it is known that SL destroys builds, loses inventory items, has linking and disappearing prim problems, etc. Would I build a house in RL if I knew that large chunks of it will be missing next week? Nah.

SL is an extremely complex program. In ANY such program there are going to be bugs. I have no problem with that concept. I *do* have a problem with LL continuing to ignore serious, build/sim-destroying bugs. I do object to them not instituting badly needed functions such as officer-to-group communications (ie, a simple group newsletter function). I object to Linden Lab treating us like we're morons when we do find a serious bug and trying to pull the wool over our eyes by telling us it's our fault.

That's not what we're paying fees for. To set up and operate a sim requires $3,200. We have TWO sims; that's a $6,400 investment over the next year. For that kind of money, I durn well expect better customer support than we're getting, and I certainly do not expect our sim to tear itself apart just because we decide to deed it to our land group!

Problems such as Kavai is experiencing... the computer and everything running fine then suddenly it doesn't run (oh, it's your GRAPHICS drivers!)... sims running just fine then suddenly NOT running just fine... prims walking off of sims under their own volition and self-destructing... these things are not what we pay some of the highest prices on the net to experience. We do not pay Linden Labs to ignore major system bugs... nor to ignore the clients who pay for their lunch.

I have speculated that SL version 1 has exceded its design limits long ago and that they now have a system with emergent properties that literally no one understands. We've also been given some peeks at SL 2 by the development team which - as Fred Brooks will tell you - will likely be better as they've learned from the failures of SL 1. I've also conjectured that most of the development staff is busy on SL 2 which is why we are getting releases like 1.6.10 which (according to the release notes) probably had a constant changed or added. SL 1 appears to be in "let's keep it from dying until we're ready to launch SL 2". I really wish they just admit it as people will be much more forgiving of the "bleeding edge" failures of SL 1 if they had some reason to believe that things would improve and when.

And yes, months ago I implored the Lindens, personally, to be open with their customers. There are valid reasons for their choosing not to do so. However, as businesses are so fond of calling customers "partners" these days, I can think of few examples where the metaphor is more apt.

Disclaimer: I have no inside information on LL and have only been working on manifest conduct, public statements by LL, conjecture, and a little knowledge of software development cycles.