Slow Texture loading in recent viewers (1.21 and later)

heh indeed I already thought of installing a local squid cache in transparent caching mode and force SL to use the proxy with some ipchains routing rules. But this would need the viewer to implement HTTP download of textures first...

BTW does anybody know what the ImagepiplineUseHTTP option does? It's existing for such a long time. In the past I thought it would do what we needed, downloading textures by HTTP, but after reading the JIRA this can't be the case, is it?

I'm talking about replacing the existing texture cache code completely. The UUID is already a random hash, so a 3-level squid-cache-style directory tree would be trivial to implement and blow away the 1GB cache limit.

Thankyou
I do get grey or invisible textures quite often, but they eventually load if I wait long enough. I can't say I have ever had textures that never load, they just take an awfully long time.

A simple method would be to hash a combination of the avatar name and the UUID to arrive at a unique filename to save the texture JPG2000 in the drive cache. That would make it fast to create the filename both for retrieval and original storage, and harder to snoop for textures in the cache as well. Not impossible to snoop, just harder. And if LL puts a wrapper around the JPG2000 before saving it to the drive, it's even harder to snoop since simply turning on thumbnails won't work.

user needs texture x
viewer sees that texture x is one it has on the drive with its own cache log
viewer generates filename by hashing av name and x together to create filename y
viewer pulls y and displays it.

user needs texture z
viewer cache log has no such texture
viewer gets texture z from asset
viewer generates filename y from av name and z together
viewer displays then saves texture as filename y
viewer updates its log that texture z is present

Sounds interesting! And you're right, we don't need a tough encryption here, just something hampering the normal curious user.
Users who _really_ want to snoop the textures always can, because the viewer is open source, so they can change the code to copy anything the viewer does render.

Not to mention GLIntercept. We have to face the facts - if someone wants a texture, they are going to get it. There's not a thing can be done to stop programs like GLintercept, so we may as well accept the world as it really is.

I agree that encrypting of data has merits, but there is a downside that it is wise to examine: How much time will such encryption add to the use of such data? Maybe it's so insignificant that such encryption is valid. But if it increases the access time by even 10%... that can be significant over 1,000 textures. Just a thought... especially since as stated, the average user doesn't know how to access texture cache anyway... and those who do after five years have probably already swiped the majority of textures on the grid.

(Or doesn't anyone wonder where those boxes of "150 Gillion Free Textures" came from?)

The new catch phrase "meta issue" has recently (as of this week) been applied to the texture loading problems-- an issue that is likely one of the most damaging ongoing problems to face the Second Life platform.

While fancy labels sounds all nice and catchy and complicated sounding... it's also something we've pretty much already knew-- months ago. Such a concept is nothing new.

The following things have already been discovered BY CUSTOMERS and have been realized for months (even years) now:

* UDP keeps dropping packets, resulting in total reloads of textures in 10 second increments-- resulting in 20, 30, 40 second and longer texture loads.

* The texture load algorithms are malfunctioning, causing the system to lose track of what textures have or haven't been loaded... and resulting in some textures NEVER loading.

* Evidence indicates these issues seems closely tied in with avatars remaining gray or "fuzzy clouds", new skins and clothes taking FOREVER to rez, as well as sculpties remaining spheres instead of rezzing.

* There is a major (and apparently relatively recent) texture cache snafu that is causing textures to load redundantly from the asset servers instead of from user cache. That redundant texture loading issue is resulting in two major problems:

1) Textures that were very recently viewed turning gray again and taking the same amount of time to load as the first instance (or even longer)
2) A delay-of-service attack on Linden Lab asset servers that is increasing with every new user that joins the system... and that rises in parallel with user concurrency (the more users online at the same time, the worse the problem becomes). This DoS attack is slowing down texture and asset retrieveal by exponential amounts.

So "meta issue"??? Of course it is. That doesn't mean it's not one great big related issue, and it doesn't mean people have to file a dozen different JIRAs to discuss these things. Why?

Because these issues all boil down to three areas that will likely fix the majority of problems no matter where they lie:

1. Trash-can UDP protocol and implement an expert-level, efficient texture transfer protocol, either HTTP or even a proprietary transfer system specially designed to transfer textures as quickly as conceivably possible. That will remove the issues of network glitches, dropped packets and high ping rates. Problems solved!

2. FIX the cache problem. Apparently cache used to work. Who broke what, and when? Fixing the cache problem will likely immediately stop the DoS attack. Problem solved!

3. FIND and FIX the graphics memory leak that has been plaguing and crashing users for YEARS now. Surely the company has testing procedures capable of finding a major leak that takes memory usage from 60 megs to 2 gigs in 15 minutes or less! (Then immediately drops back down when the screen is minimized for a few seconds.) FIND that leak... FIX the problem... the majority of user crashes end. Problem solved!

Those three items alone will fix the vast majority of the problems listed above and indeed, the vast majority of the MAJOR problems that currently plague Second Life.

"Meta issues"? Pure business suggestion to LL: stop asking for still more user input, stop asking for more user-provided data, stop asking users to file multiple and confusing JIRAs, stop making excuses and coming up with new catch-phrases... and focus on fixing the issues. In short: Get 'er done!

We're talking THREE problems... and there's not much doubt those problems are the source of the majority of serious SL performance issues. So what more "user input" or separate JIRA posting is really needed here?

Honestly, all that just makes people paranoid and makes us wonder who is hiding what and why. Seriously people... AS a professional coder with significant experience, I'm going on the record here: its' not all that complex. The entire issue has just been outlined in one forum post.

--

UUIDs are already generated randomly. There's no need to complicate the file name... and in fact the current cache uses the UUID as the file name.

Encrypting or obfuscating the contents of the file is a separate issue.

HTTP with streaming is already such a protocol, and HTTP-based based access is already in there.
The SL cache has never worked well. A simple squid-type cache is fine.

What I mention is that while cache has never worked well... it's only since mid last-year that the apparent DoS attack began (at least as far as we can tell) and such seemed to start with a server change that was apaprently made about the time of the appearance of Client 1.20.

I can't agree that HTTP-based access is already there... since that is supposedly what Dan Linden is working on at this time (knowing your expertise Argent, perhaps we're discussing two different things without realizing it?). At least that's what the claims are... that he's working on changing UDP to HTTP protocol. If that has already been done-- I see no statistical evidence of such. Texture loading has not improved... and surely not on the levels such a change of protocol should bring about.

Do we think VWR-8503 is cache related or are we just musing about ways to make the cache better?

/me agrees it needs some serious work - I'm just wondering if that's suspected of causing this particular problem..

Time i don't have much lately unfortunate...

Could someone check if the following Debug settings differ between 1.20, 1.21, 1.22RC, 1.22NP.

RenderBumpmapMinDistanceSquared
RenderAvatarMaxVisible
RenderDynamicLOD
RenderShaderLODThreshold
RenderUseCleverUI
RenderNameFadeDuration

Maybe some changes here could have an impact... ?

Huggles and kisses for whom who has had a chance to have a peek into this

We could be. I'm talking about the support for RESTful HTTP requests from SL. I'm not talking about viewer-side code that actually uses it. They have to keep BOTH working together in parallel, too, because they need to keep supporting the older clients.

What I'm talking about here is that there's no need to create a new protocol, that's all.

Yeah, we were talking about different HTTP areas... and I agree there's no need to create a new protocol (if) HTTP adapts well and properly to the SL platform.

(And as a side note... I personally think that Sindy asking a question that's already been completely answered is somewhat humorous... because if she hadn't copped an attitude and put me on her ignore list-- she'd already have the answer to her question in spades. Consequences of drama. Sigh.)

The answer to that question is (or is around) post #81 ("A Meta Issue?", which outlines the entire problem as closely as I can present it without having direct access to LL servers.

Second Life 1.22.5 (107013) Dec 31 2008 13:30:40 (Second Life Release Candidate)
Release Notes

You are at 236758.2, 292227.0, 494.1 in West Texas located at sim2782.agni.lindenlab.com (216.82.19.25:13002)
Second Life Server 1.24.10.106829
Release Notes

CPU: Intel Core 2 Series Processor (1866 MHz)
Memory: 2045 MB
OS Version: Microsoft Windows XP Service Pack 3 (Build 2600)
Graphics Card Vendor: NVIDIA Corporation
Graphics Card: GeForce 8600 GTS/PCI/SSE2

But I have problems no matter where I am in SL.


Eventually textures do rez. It just takes forever. And if I quit looking at them for longer than a minute they are gray again and need to rez all over again.

I do not know how to do that other test with the pinging ...if someone gave me instructions I could probably figure it out.

If you think something's funky with your network, you can do ping and tracert to see how your network is doing from you to someplace else..
- Log into SL
- Click the windows Start button then Run...
- Type 'cmd' (without the quotes) to open a command window
- Type "ping -t 216.82.19.25" if you're still in West Texas
- If you're in a different sim, you can do Help->About SL and get the IP address. See my earlier post with the a.b.c.d stuff
- Go get a drink or something.. Waste 5-10 minutes
- Press CTRL-C to stop ping
- Right-click and do Mark then drag your mouse over the last 4-5 lines of the ping output
- Press ENTER to copy the marked area to the copy buffer
- Post back here and do CTRL-V to paste the copy buffer here

Tracert is the same deal but you do "tracert 216.82.19.25" instead of ping -t.

Reply from 216.82.19.25: bytes=32 time=47ms TTL=51
Reply from 216.82.19.25: bytes=32 time=45ms TTL=51

Ping statistics for 216.82.19.25:
Packets: Sent = 1236, Received = 1211, Lost = 25 (2% loss),
Approximate round trip times in milli-seconds:
Minimum = 43ms, Maximum = 79ms, Average = 48ms

How is that? I forgot and went about 35 minutes

Except for the 25 lost packets, those are very good numbers. For the lost packets, it's not a good sign but isn't always a bad thing.

If, after being logged in for a while, you do Help->About SL and scroll down the top part, what's the last line say about packet loss?

Packets Lost: 925/147477 (0.6%)

That's an interesting difference. Sindy and me don*t have packet loss at all so far and have lesser problems.
Belle, please note, that does not mean it's your fault or your ISPs fault. Such a paket loss is normal, no need to worry on your side in general.
But if my hypothesis is right, that could mean: Whenever a packet is lost in the transfer of a texture, it might be that the loading of this texture stalls, either forever or for a too long time.

That's why right-clicking/hovering does not work for you: The texture already has enough priority to load, but the broken network implementation of the SL Viewer has problems with your little network jitter.

So I think we might have identified 3 main problems that are connected to the slow tex problem:
1: textures have wrong priority for loading
2: already little and normal packet loss makes texture loads stall or hang too long
3: Textures in the local cache get invalid too fast and cahce is too small because of wrong technique

Is this feasible?

The JIRA documents these problems very well. Just one lost packet (and this is quite common on networks), causes a texture load to stall until some other activity restarts the load. I've seen stalls running into several minutes, so it is not a bandwidth issue, but instead either a server side stall or the fragile UDP transport taking a long break.

Belle, I have to speak up on this. IMO you're being lead to jump through a bunch of silly hoops that all evidence indicated were not necessary in the first place. In most instances, the only time you need to perform such steps is if there is excessive and problematic issues with your overall internet access.

A packet loss of 25 out of some 1200+ packets is QUITE NORMAL and is experienced by vast numbers of users. Further, your average ping time is really fairly excellent. So I think you're being lead on a wild goose chase. I'd listen to Balpien; some packet loss is normal.

IF you were having excessive packet loss on a a regular basis, Second Life would be all but unusable to you... and perhaps unusuable period. You would likely also be experiencing severe problems with your other internet applications as well.

Minor packet loss such as you're experiencing is very common in all aspects of internet use. The two people focusing on local network packets is drawing attention away from the primary issues and focus here: that of severely malfuncitoning Second Life systems and coding. It has ALREADY BEEN ESTABLISHED that Linden Lab is having major protocol, redundancy and caching problems. Your local network issues are extremely unlikely to be causing you in particular severe texture access issues.

Again, if networking were the case, you probably wouldn't even be using Second Life out of sheer frustration. Just wanted to save you further time and effort from following these dead-end postulations presented by two people who aren't even following the entire thread.

As Balpien accurately stated, the only problems packet loss is likely to cause is forcing a UDP packet to reload-- and at 25 out of 1200+ packets, I assure you that while such might affect occasional texture issues, it does not indicate impacting of your overall texture experience. While it is obvious that those with NO packet loss (a very, very rare thing) are going to experience no UDP reloading... that's not going to prevent all the other issues that are going on Linden Lab side.

Best wishes to you. I'd stop fretting about so-called "network problems" being thrown at you. If you're able to use SL on a regular basis the problem is not your network stats... it's Linden Lab systems-- as Linden Lab itself is well aware.

I've made a new Windows build based on the latest SVN 1.22.6.107912. If someone wants to give it a shot. Link below.

Second Life Public Nightly 1.22.6.107493 Jan 9th, 2009
Partial Fix: VWR-8503[c]: Textures loading worse in 1.21/1.22
Fixed: Sculptie textures have too high a priority in 1.21, causing slow overall texture loading
Fixed: Eliminate redundant priority calculation
Fixed: Textures with no data (gray) are not prioritized properly
Fixed: Fetches with priority of 0 get stuck waiting for network data instead of terminating
Fixed: Negative priorities are not being handled correctly
Fixed: Modify a debug timer to measure texture fetch time instead of network download time

Yes exactly, that's what I wanted to make sure. This packet loss completely fine and fully normal and the ping times are just fabulous!

Normally every network should be able to handle this little normal packet loss without any measurable drawback. We have some serious bug in SL when it comes to handle this.