Slow Texture loading in recent viewers (1.21 and later)

There are times when even me wants to have Windows again hehe
I once compiled the viewer myself on Linux, but I was too lazy with this huge patchset you are handling every time. And the unpatched build did not have that advantage over the Linden Labs released binaries...

Thanks for your great support, Boy! Your builds are saving the virtual life of hundreds of users every day!

It's called re-implementing pseudo-TCP over UDP for bulk data transfer. They know about it and are working on fixing it. Patience, grasshopper.

I will try, master.

In fact, I know that, but I want to get a grip on every cause of the problem, try to build the hypothesis that covers these behaviours and try to fit every case into he hypothesis or extend the hypotesis if it doesn't fit. Up to now it seems to come down to these three problems for the users who described their problems.

I have repeated timout tests of the cache and it seems to be set at 30 seconds. That is with the Texture Console open, no textures downloaded inside the 30 seconds and reloading a set of textures on a set of prims on a HUD that is used to display groups of textures. All this done on a build platform 3000m up and the camera pointed at the zenith.

That's not all that bad. As others have also said, dropped packets aren't a good thing but the system is supposed to be able to recover from them. People start having real problems - and using SL over a wireless network is notorious for this - when the percentage of packets it has to redo starts getting large and the sim and your PC are always having to resend stuff.

The talk of UDP and TCP is really about which component makes sure that stuff shows up in the correct order and that it's all there. UDP is faster than TCP but it does not guarentee the order of delivery or that stuff will even make it to the other end - the sim and SL software on your PC have to do all that work and deal with anything that goes wrong. TCP isn't as fast but it does guarentee that things will show up in the correct order and it will automatically (down in the Windows operating system and on the sims OS) retransmit things if needed - the sim and SL on your PC don't have to worry about all that.

Anyway, your network looks pretty good - I can see Daniel drooling over your stats from here - although there might be some small stability issues. Nothing to get too worked up over, though. Whatever is causing the texture-loading problems for you is probably not network related.


They've been working on it for a couple years now. I remember Cory talking about that (and simcaps and such) around the time I joined. Not that I'm really complaining - I know there's a lot of work to be done..


Welcome to the dark side, Boy!

The first thing they did was to get caps working with the existing protocol... which led to the heterogenous grid and an end to restart wednesdays.

/me is still thanking them for that!! I find it hard to keep my mouth shut when people complain about the rolling restarts taking their sims down for 5-10 minutes..

All glory to the het grid!

A little add-on: Since nobody answered my question about what the debug setting "ImagePipelineUseHTTP" is doing, I assume, noone really knows.

So what about setting this to TRUE if this does any difference for all of you?

Infos on how to use the debug settings:
http://wiki.secondlife.com/wiki/User:Torley_Linden/Debug_Settings

I'd be glad if anybody could post their experiences with this. For myself, I can't see any difference for now, but didn't try thoroughly yet.

LOL.

They've been working on it for a COUPLE OF YEARS now?

LOL LOL LOL...

Lord... heads should be rolling...

(Well, come to think of it, I guess Cory's head did roll.. to an extent...)

Seriously, two years to work on correcting a transfer protocol? Oh wait... wait... I understand. Rather than making sure that textures and chat actually work correctly, they were working on Windlight, sculpties, flexis, local lighting, Dazzle, voice, movies... all those things that took priority over core foundation performance.

And we're supposed to not complain, and after two years, we're supposed to still be patient little sheep? We're just supposed to keep handing LL the big bucks and take whatever decisions they make without any say (or rights) whatsoever?

We can't post our opinions and feelings on the JIRA because of course, the JIRA is pure white-tile tech rather than a customer service vehicle... and those who DO state opinion (or heaven forbid, show a bit of ire) will be summarily banned. Is that what corporate policy is coming to now-- overt JIRA censorship?

And we customers have no excuse for attitudes and complaining? Or even more wisely, we're just supposed to "love it or leave it", scrap our considerable investments, hard work and community and leave SL?

Well in fact... customers are... to the tune of some 3,000 sims as of January 1... and that number increasing. The fallout by July 1 should be fairly alarming.

Sorry folks I tell it like it is. The "Dream" has been exchanged for profit schemes by a company that considers it proper to remove island metrics so people can't see the results of corporate decisions.

So folks can follow the corporate bandwagon all they want, but I'm with the ones who are crying out "The Emperor has no clothes!". LL is charging my group (and everyone else) through the nose for a platform that should have been stabilized two years ago... at least where aspects of texture and chat are concerned. So they can make all the excuses and blow all the smoke they want... thousands of users are no longer buying it. There is no excuse for taking TWO YEARS to replace a basic transfer protocol. (If that is even an accurate time frame. As far as I know... the UDP problem has been going on for at least 5 years now...)

LOL, Daniel must be referring to the three problems I stated (and he supposedly didn't read) a dozen messages back, yeah?

Master Argent, bop Grasshopper upside the head, will ya?

Hi Dytska. You're pretty spot-on; that's the <i>average</i> texture loading time published by Dan Linden himself.

The UDP protocol resets in increments of 10 seconds... then starts reloading the texture from the very beginning instead of from the point of the failed packet onward. As a result, textures will tend to load in increments of 10... 20... 30... 40 seconds or more. Approximately 85% (? I forget the exact figure) of specific texture loads require 30 seconds or longer.

It follows that if UDP were replaced with a protocol that resets immediately upon faulty packet and resends individual packets instead of the entire file, we could expect a significant improvement in texture performance.

Whether that will fix the cache problem and resultant DoS attack on the servers is doubtful... but LOL... it sure can't hurt.

There's no technical reason they can't continue the transfer from where they left off with UDP either. In fact I believe they used to do that.

That would surely be better. The only drawback (from what I've know of UDP) is that it takes 10 seconds for the protocol to reset. I'm not sure that's always the case (maybe it is; I honestly wasn't aware UPD could be set to packet mode) but 10 second recycle seems to fall in with the statistical data of textures taking 10/20/30/40 seconds (never 15, 23 or 37).

But still, it would likely be better than it is now; 10 seconds is better than 40+.

UDP is a connectionless protocol. They may have a 10 second watchdog timer in their code, but I don't see how it would cause a problem with restarting from the last successful packet.

Just points more and more to the concerns that the code was poorly written to begin with. I'd almost bet doughnuts at this point that SL is largely spaghetti code... which may be while the post-Cory programmers are having such a hard time with it. It's pretty apparent after the release of Client source code that it contained several megs of useless routines... routines that to their credit, either LL coders or third-party volunteers have been removing.

(Man, LL has it made. Release the source code and let paying customers work on updating it for free. LOL. Talk about buttering the bread on both sides. )

All they have to do to make connectionless UDP work for texture transfer is adopt the same sort of block X of Y header that we used for Zmodem all those years ago. When you lost a packet, did Zmodem go out and start over again? Or did it call to resend the lost packet?

Granted, I'm talking about a system for computers that would have at most 16 users logging into it. So holding a file on the server until all parts have been sent was childs play then - with 70K people transferring a hundred textures all at once any given second, the poor server would really be strained to pull this off. But if you can have something like Zmodem headers on the packets, you would not need to resend the entire thing.

Perhaps the server can load the texture on command and send it all, then reload the texture to resend only those packets not received. That would eliminate the need to hold so many textures in Ram on teh server and should be very easy to add to the communication code. If you get a request for texture, you send all the packets. If you get a Bad Packet from the viewer, you reload the texture and only send the packet called bad. We used to do this all the time back in the BBS days...

That's why they moved responsibility for this to the sim. So it only needs to deal with at most 40 users transferring those textures.
They do. They have described the protocol, and the protocol they described doesn't require resending the whole file. As I recall, they DO need to start over from the point of the lost packet, but that's not anywhere near the same as resending the whole file. IF it IS resending the whole file now, that's a bug in the implementation... not in the protocol.

I'm absolutely disgusted at the fact that someone in this thread decided to take up the Maurice Linden stance of... "It is your connection/ISP". Sadly, people actually went through the motion of jumping through the hoops to check it out. If other graphically intense games you play online, as well as surfing the web is working just fine for you, then why would you assume it is your connection? In addition, if it was your connection, wouldn't other things be affected in SL besides textures?

I will tell you that location doesn't matter so much. I experienced these issues when I lived in the U.S. and now I'm also experiencing them in Japan.

If I don't have the problem and somebody with a virtually identical PC always has the problem, isn't networking a reasonable thing to look at?

And doing a ping and tracert is hardly jumping through hoops. Or are you part of the crowd that thinks the whole debugging thing is a waste of time and that they should just fix it instead?

Networking is a very reasonable thing to look at, but not so reasonable when it is applied to every issue/everyone. In addition to it being the automatic response I get every time I contact support, It is the very first thing I verify is working properly whenever I'm having performance issues in SL. I'm certain there are a few people who do fall into the category of having a faulty network connection. But even if it is a reasonable thing to look at, I believe it has been ruled out as the root cause of this original JIRA Issue.

In some situations. But lately, it seems to me that the variablilty among folks running very similar hardware/OS has arisen more from the different situations they're exposed to in world than the slings and arrows of outrageous latency and packet loss.

In our RL home we have two intensive SL users on pretty much identical, rather high-end hardware. We share the same 15/15 fiber connection. Yet we're able to see divergent results more often than you'd think.

I think as time goes on and good connectivity becomes more and more prevalent, we'll see less and less "it's your connection" fingerpointing.

Hopefully less "it's your driver", too. The "palleted textrures" embroglio was a perfect example of this; when memory leaks kept clobbering me in the 1.20 days the suggestion was repeatedly mooted that my video driver was at fault and was perhaps downlevel. So when I pushed up to bleeding edge video drivers, and avatar textures turned to crap, it was "you're on beta drivers, they must be buggy".

Of course, we all know what the real story turned out to be on *that* score.

If you read the origins of the networking sub-thread here, it was:
person a: what speeds does your ISP say you get?
person b: that's a stupid question
person c: ok.. what's a good question then?
person b: ping and tracert

It also looks like it went to "your network looks ok to me" pretty quickly.

/me isn't sure why you're "absolutely disgusted" in the brief network digression..

Right now the viewer code used by that setting is disabled in the source, because the server-side part has yet to be completed.

One, two, skip a few!

I wish this worked, but I haven't noticed any differences in loading of moused ones and non-moused ones. Heck, I've seen distant textures rez long (sometimes minutes) before the near ones start to ungray.

In other words, I have to wonder if they fubar'd the loading priorities, instead of closest rezzing first.

/me wonders if that is related to the problems the RC is having figuring out what you've right-clicked on - something about raytracing being wrong and not being able to edit prims that intersect with an avatar?

If you move the camera around so that there aren't any avatars between you and a gray prim, does the texture load quicker if you mouse-over or right-click it?