Lag still a major problem

Irk... have they considered whether the heat generated might cause spontaneous chain-reaction combustion of the atmosphere leading to global thermonuclear destruction?

LOL that what you call a "smokin" system? rofl

Not the top of the thread, the first few paragraphs in the last few messages were in a response to your complaint that Linden Labs (apparently) asked you to check your content when you complained about sim lag.

That doesn't mean that I'm saying that *all* lag is caused by customer content, I was just pointing out a situation where it might not be.

This is like your apparent confusion over what I said about databases. You pointed to a particular problem and said it was a database problem. I said that specific problem was probably not a database problem. Now, somehow, you think I said that there are no database problems.

I'm sorry, but it wasn't. When I referred to the SL client as a client the part of your message I was responding to didn't mention "client content", it just referred to "sims" and "clients" without qualification. I apologise for being confused by that, and I won't use this clearly ambiguous term in the future.
Indeed, and I took it that way... and my response was also intended as a humorous comment that had nothing to do with flaming each other. I'm sorry that you didn't take it that way.
I agree. In fact I've explicitly said as much. In this thread even.
I agree. All I'm saying is that when you ask them to fix the problem... talk about the problem. If you don't know whether it's a memory leak, a database problem, a protocol problem, or a resource allocation problem (and I've seen evidence for all of these) then it's not *useful* to say "there's a database problem in the inventory". And it's also not useful to respond to "that particular problem doesn't seem to be a database problem" as if I'd said "there are no database problems" or "Linden Labs doesn't need to fix anything".

The skydiving launcher on Avalon Lagoon, the big crater at the south end of LostFurest dAlliez, uses MAX_FLOAT push to (safely, and in a way that can't be abused by anyone who buys one) launch an avatar into the upper atmosphere for skydiving. It's one of the earlier objects I created in SL and nobody's ever complained about it.

The very same thing could be done with a landmark. It isn't necessary to use a 9 billion push to accomplish such things.

[edit]
Actually, I may be mistaken in this. I think a landmark has a maximum altitude of about 700+ meters. If someone wanted to travel to 1,500 or higher... landmark wouldn't work.

However, I've seen lots of devices that do work without pushing someone into the stratosphere. Flight tools are most often used... such as the X-carbon rod which will zoom someone to 1500m in nothing flat. Considering that heavy push is used for griefing far more than it's used for skydiving... it's likely a matter of the greater good. Unless of course, you've done skydiving lately from 240,000m.

However, I did myself think of something that moderately heavy push is used for: sim cannons. People enjoy climbing on a cannon and being shot across a sim. So there is some need for moderate-push. Which brings us back to the concept that in a non-battle-zone, users should be required to give permission before push is applied to their avatar. They have to give permission to apply an animation; same principle. That would be a very simple solution.... and would allow the skydiving launcher to continue blasting people into space.

Now, thinking back, I dunno how we got from system lag to griefing/push concepts, but... for what it's worth, there it is.

Was gonna just let this go, but since you decided to make this look like yet another instance of me failing to undrstand you... I think it's pretty clear from this quote...
...that you neither took this to be a joke nor did you reply as such. As for the rest of the claims in that lengthy message Argent... well, just about as factual.

I've said it before in other threads Argent-- there are others out here equally qualified in the areas of computer operation. We're not dunces that are unable to understand either you or the operation of Linden Lab and Second Life. Dispersions against people's intelligence and/or knowledge never comes across as friendly. It's far easier to keep it clean up front than to have to backtrack later.

The point of the launcher is not simply to put you at that altitude, it is to do it in a particular way, with glowing beams of light, electrical sparkles, and a burst of flame... without sitting on an object, attaching a flight script, and so on. There's a Cubey Terra pod next to it, and I also sell a flight attachment (open source, even).

You didn't say "there's no important use of mega push" or "there's no use of mega push that can't be worked around some other way", you said "there's no legitimate use of mega push". One single artifact that uses mega-push without griefing is enough to establish a legitimate use. There are others... the one I use for space exploration, for example, can't be used for griefing... and yet it launched me over two billion meters in the air back in 1.7! Beatfox Xevious has even more powerful space travel devices, and I'm sure we're notthe only ones.

And in any case, push is a non-issue now. Push has been comprehensively nerfed, almost everywhere you go you can see that little icon (what the hell is it supposed to represent) saying that push is disabled. I actually went and made a "push enabled sandbox" for people to use to play with push without having to own land because it's been so comprehensively disabled. At this point in time, there's no point making further restrictions on it.
I was up at 300,000,000 meters plus just last week.
Users can only be pushed by objects owned by the landowner or members of the landowning group, or in zones that permit push.

You already won this battle, push is effectively dead as a griefing tool. Why even bring it up any more?

In the world of electronics, the person that builds or repairs your computer or server is a tech. Writing scripts doesn't make one a technician.

It was an example Argent, to indicate the need for LL to put governors on certain things rather than complaining about "client content"-- and to focus on LL-side lag rather than blaming client content for what are obviously system-wide issues. And as seems to be a habit, that minor example was tunnel-focused focused on rather than the main point being recognized-- and you turned a molehill into a mountain. (As a note... it's been a while since I've seen LL make such a claim. It's now usually a handful of forum-dwellers that continue to make such claims despite all evidence to the contrary. It's almost like all they live for is to argue.)

I come to forums to exhange information, make suggestions and learn. I don't come here to hear people brag about how much they know, to hear them tell others how little they know, or to have someone try to regulate what I and others should or should not be posting.

I'm glad you enjoy freefalling from 300 million meters (that takes what... how many hours to reach ground? During which there is nothing on the screen but your av?). But do the very few people that enjoy such a thing balance out against the thousands of users who have been unwillingly pushed into the stratosphere by griefers during all that period of time?

Just because push has been somewhat taken care of for the last month or two does not alter the fact that for the past two years it was not taken care of. This being the case, it was a perfectly valid example of failure to regulate the system for the good of SL as a whole-- which was the primary point being made.

That's why.

The subject of this thread is LL-created lag. Not push scripting. Not grammar. Not customer-created lag. Not prims, scripting or anything of the other nit-picking things that no one else has brought up-- and that have only served to derail this discussion. If you want to discuss those things, I'd encourage finding a thread relating to that subject. This thread is about deep-core lag, server side, server programming, asset server, client software, etc. The subject of this thread is discussing what could be causing serious, grid-wide lag... and what LL might be able to do about it. So let's get back to the topic, eh?

Hmmm...

I assumed that Wayfarer was referring to the person's level of competance and their responsibility for the results of their work, rather than their assigned job.

OK.

It's still on topic.

If LL were to put enough governers on scripting that scripts couldn't produce lag, they would end up with an incredibly constrained scripting language. The people who were adamantly against Push wanted to strip it down so far that it wouldn't actually be useful for anything. The same objections were used against llRezObject, llGiveInventory, and they both got nerfed (though LL were convinced to roll back the restrictions on llGiveInventory), which broke a lot of content that wasn't causing a problem. In more recent extensions, LL has pre-nerfed the calls: for example, you don't automatically grant camera permissions to attachments, and llHTTPRequest has limits that make it useless for many purposes... forcing people to use existing (and laggier) protocols instead.

Indeed. Who brought the resident content issue up in this thread?

[more stuff about push skipped... as you say, it's irrelevant]

I'm trying to do that, mate.

Deep-core lag is caused by many things, and some of the causes would be relatively easy to fix.

For example, a sim can easily handle the 40 TCP connections that would be necessary to use TCP for chat and asset downloads... if they used HTTP for the downloads they could set up dedicated proxy servers for sims to hand TCP connections off to when they're overloaded. This would mean fewer download problems, and may help with your inventory issues if (as I postulated) they're partly due to packet loss.

Adding more asset servers is an obvious win.

Giving people the ability to use more local cache. If there's a scaling problem, then let the cache be per-sim, and allow the user to specify that they want to use N sim caches. That way when you teleport to a busy sim and back again the sim you got to wouldn't have flushed all the objects and textures out of your original cache. This would also improve the performance of sims, since they wouldn't have to serve as much content to the client, and reduce the load on their databases.

Again Argent, it was brought up as a basic example of blaming users rather than taking the steps to control that content and keep it in check in the first place. It was not presented as a major topic, nor was it an open invitation to go on yet another thread-derailing tangent about whether or not client content causes lag. All that does is frustrate others and draws away from the main issue. I mean, sorry, but in this thread alone you've tried to redefine the use of "tech", denied that "clients" refers to customers... etc etc. Let's stop the nitpicking and just get back to the issue. Pay attention to what people are saying... not how they say it.

There we go. Now we're back on track.

Yup. Many systems store a "last place visited" and "places often visited" cache on the local drive, which allows very fast loading. Then all the server system needs to do is check the validity of that cache against the present condition rather than going to all the hassle of reloading textures, etc. A computer can rip through 15,000 comparisons very quickly. The only things that would need reloading would be new items. Any item that is not gone or changed position could easily have the new parameters loaded without having to reload every single bit of information on every single item on the sim. Websites do that all the time. They probably do a bit of that already... but to what extent?

I know there's been a bit of hesitation about storing items on a client hard disc for fear of copying items... but data can be effectively encoded and decoded at will. I know the game Unreal uses a heavy cache system. I don't particularly like the way they do it; whenever one enters a new area, quite some time is spent downloading everything there at one time. Future visits however, load very quickly and instantaneously. Now SL has all kinds of issues different from Unreal... one of which is one sim connecting with up to eight others. So they obviously cannot do things the same way Unreal does it (in Unreal, each land is independent). But... it's very likely that the lag problems could be greatly reduced through a combination of protocol and cache systems. As to how exactly... well, that would be up to the coders.

So put simply (and I know some of these things mean a lot of work on LL's end)...

* Checking for memory leaks and stopping them up
* Checking the basic data layout to make sure it is set up and accessed correctly
* Re-examining information transfer protocols
* Reassessing client-side graphics rendering software (it really does seem extremely glutted) and making it work faster and smarter... and requiring less powerful graphics cards

... all of these things would be a bonus.

Truthfully, from what I've seen lately... the graphics issue is what I foresee as the next big deal. In the almost 2 years I've used Second Life I have seriously upgraded my graphics cards three times... only to have the system demand more and more of my cards to the point that I have to upgrade again. I honestly don't know how SL graphics are put together or how they work, but I do know that when they tax an Nvidia 6600 card (much less an ATI X800XL)... they're demanding far more than the system really requires. Put SL next to Unreal, Quake or just about any other gaming system out there (except for maybe There) and the graphics look relatively dinosaur. I'm not a graphics tech so I can't earnestly speak from anything but user experience... but just as I know that Windows XP is a real resource pig... it seems to me that SL requires far more client-side resources than it should.

Many other people-- very educated and in-the-know people-- have commented on these threads that SL needs a good streamlining. Code needs to be gone back over, line by line if necessary, so that clutter and flaky code can be replaced with faster and more efficient routines. Some have stated there are so many patches in SL that it stumbles all over itself all the time. A lot of treating symptoms with stop-gap fixes instead of diving into the problem itself. I have no way of knowing itself, since I've never seen a single line of code. But my third-eye gut-feeling tells me they're right on the button. I don't say this as an insult to Linden Lab at all, but as a statement of reality. Almost every program I've ever written, once it was done, benefitted by me going back through and double checking the code to see if it could be done more efficiently now that I have a better overall picture of the project.

Of course, such things take time and manpower. But frankly, the future existence of SL depends on it. They sure can't keep growing like they are and imagine the infrastructure is going to hold up.

Second Life does this as well. The problem isn't that they don't cache the data, or that they don't want to store the cached items on the client's hard disk, it's that they don't do a very good job of it. They only allow you 1GB of cache on disk, maximum, because their caching algorithm doesn't scale up very well... if they could handle 20 or 30 GB of cache they wouldn't need to create special-purpose caches like the per-sim caches I was suggesting, or the ones other people have brought up... they would just have 10 sims worth of data on disk and you really would only need to check the most recently changed data... teleporting between a handful of locations would, within a couple of cycles, be instant... no matter what your draw distance or how cluttered the sims were.

If you want to find someone who really knows about the cache and its limitations, you need to find Strife Onizuka. He's a maniac scripter, but he may know more about the internals of the SL client than anyone.

Hum... somebody with some time should put together a libSecondLife program, using SLProxy as the base, to augment the caching. It would actually be really easy to do, and would be a nice way to test out some theories.

Interesting stuff. Any idea how much space it takes to cache one sim? Just a shot in the dark: if it took 1 gig per sim, then just a local area could be 9 gigs. Or... rather than caching on a sim basis... it might cache on a "visible area visited" basis, so that no matter where you were you'd only need to cache the surrounding area within 128m or 256m eyesight. But even that... well, if you were standing on the corner of 4 sims, you could pretty much see all 4 sims.

Yeah, I'll buy that the cache doesn't work well. I also might even postulate, just for the sake of throwing it out there... that Windows itself might compete with SL for system overhead. I hate to say it, but Windows has become a real resource hog and a half and then some. The OS does a lot of things, but like much of Micro$oft software... does them poorly, inefficiently and prone to crashes. No telling how much faster SL would run under a better OS. I know I've had two Mac users tell me their graphics are not laggy. I wonder how SL runs under Linux?

Don't get me wrong. I'm not blaming Windows for the majority of SL lag. But I'm not counting it out as a significant factor, either.

I just noticed something from my own hard drive...

When I store a photo from SL, it's automatically stored in .bmp format. It takes over 2 megs of storage. That same photo can be stored in high-quality .jpg format at less than 100k. That's a major difference. So one has to wonder: why does SL store it in .bmp format?

If that thinking applies to the rest of the system... no wonder it lags.

Anyhow... here's why I discovered that.

It would be nice (if it doesn't already) for SL to store textures on the client-side disc. Most sims have under 2,000 total textures. Total disc storage: 200 megs per sim. And that's on heavily-textured sims. Most come in far less than 2000 textures. Would make sense to keep as many textures as possible stored on the client hard drive for fast and easy retrieval and take some of the stress off the servers. Textures can be discarded as new sims are visited.... say, it might store 20 sims worth of textures. For my daily travels, that would be plenty. LOL

Maybe that's already being done to an extent. Just throwing stuff out for consideration. Maybe someone at LL will read this thread and one or more of these thoughts from one of us will turn on a light. One can only hope.

The values will be all over the place. The biggest part of the cache is usually textures, and of course some sims are immensely more texture-heavy than others.

The kind of caching that I'm talking about really isn't "caching a sim", it's using the sim name or coordinates to label a cache directory, so you could have multiple cache directories created as you move around in the world. You wouldn't be caching the content of the sim, you'd be caching the content visible to you when you're 'close to' that sim. You wouldn't want to invalidate the cache as you crossed a region border, or cache the same texture twice on both sides of the border.

So if you teleport to Ahern, you'd have a cache called "Ahern" that would be used until you teleported again or until you were "far enough away" from Ahern that it wasn't useful as a tag any more. Say, you might fly to Abbotts and end up with a Goguen cache as well as an Ahern cache because you happened to be over Goguen when the client decided you were too far from Ahern.

As for Windows... I don't know. My Windows box is running Windows 2000, it ONLY runs games (and these days that means ONLY SL), and I've got 2GB RAM, and my pagefile is on a different drive than my cache. I do notice that on the Mac I sometimes get a message that it's validating the cache during downloading, and I don't recall seeing that on Windows.

Got me. Possibly so people could use downloaded textures as backdrops back when Windows only understood BMP? It doesn't use BMP internally or in the cache, it uses Jpeg 2000.
That's what most of the cache seems to be.