Welcome to the Second Life Forums Archive

These forums are CLOSED. Please visit the new forums HERE

Lag still a major problem

Argent Stonecutter
Emergency Mustelid
Join date: 20 Sep 2005
Posts: 20,263
08-20-2006 07:37
The biggest problem in SL, the cause of all the downloading-related problems (including things like ghosting) is because LL is using UDP to transfer *everything* from the server to the client. If a UDP packet is lost, it's lost, that's it. When the packet represented "avatar is HERE moving THIS direction" that's OK, because a second or less later there will be another one. That's why UDP is used for streaming data.

When you're downloading a megabyte of texture, though, and you lose packet 6617, you have to have some way of requesting that packet be resent, or else the texture will never load. When you're sending an important packet, like "Wayfinder Wishbringer just logged off", you need to know the client actually GOT the packet, or else the client will still show them sitting there hours after they left.

So they've added patch after patch to UDP. First they requested re-download of incomplete textures, then everything after the lost packet, then just the missing packets. They've added retries for important packets (and then people started saying things two or three times... for a while, anyway... I haven't had that happen in a while). They still have chat arriving out of order because the retries have no sequence information.

What they need to do is use TCP for batch and overall control (for things like logins and logouts and chat), and UDP for real-time events. That way the TCP retry and retransmission algorithms, which do a very good job, will just automatically make downloads go smoothly. It'll also mean that you won't have to sit there and tune yor bandwidth, because TCP has algorithms to automatically adjust to bandwidth. A huge number of the problems in SL would just vanish, like that.
Candide LeMay
Registered User
Join date: 30 Dec 2004
Posts: 538
08-20-2006 16:11
I have to agree with Argent. And things only get worse if you are on a distant network with many nodes between you and SL. It's almost 30 hops between my desktop and a sim and sometimes I can't even upload textures because of lost packets (the whole transfer will timeout and you get the popular "Asset request failed" popup)
_____________________
"If Mel Gibson and other cyberspace writers are right, one day the entire internet will be like Second Life." -- geldonyetich
Wayfinder Wishbringer
Elf Clan / ElvenMyst
Join date: 28 Oct 2004
Posts: 1,483
08-20-2006 18:19
From: Candide LeMay
I have to agree with Argent. And things only get worse if you are on a distant network with many nodes between you and SL. It's almost 30 hops between my desktop and a sim and sometimes I can't even upload textures because of lost packets (the whole transfer will timeout and you get the popular "Asset request failed" popup)


I agree that lost info is one of the problems with lag. And like Argent says, that needs to be fixed. However, serious lag happens even when dataflow is just fine. So as stated, there are several causes of lag. The dropped data packet is likely a very significant issue. But I don't think it's the cause of an entire sim lagging to mush.
_____________________
Visit ElvenMyst, home of Elf Clan, one of Second Life's oldest and most popular fantasy groups. Visit Dwagonville, home of the Dwagons, our highly detailed Star Trek exhibit, the Warhammer 40k Arena, the Elf Clan Museum and of course, the Elf Clan Fantasy Market. We welcome all visitors. : )
Wayfinder Wishbringer
Elf Clan / ElvenMyst
Join date: 28 Oct 2004
Posts: 1,483
08-20-2006 18:23
From: Peter Schmo
This really should be their top priority.


I agree with that 550% (maybe more. LOL). All the new toys in the world do little good if one can't move.
_____________________
Visit ElvenMyst, home of Elf Clan, one of Second Life's oldest and most popular fantasy groups. Visit Dwagonville, home of the Dwagons, our highly detailed Star Trek exhibit, the Warhammer 40k Arena, the Elf Clan Museum and of course, the Elf Clan Fantasy Market. We welcome all visitors. : )
grumble Loudon
A Little bit a lion
Join date: 30 Nov 2005
Posts: 612
08-21-2006 12:52
From: Argent Stonecutter
{snip}What they need to do is use TCP for batch and overall control (for things like logins and logouts and chat), and UDP for real-time events. That way the TCP retry and retransmission algorithms, which do a very good job, will just automatically make downloads go smoothly. It'll also mean that you won't have to sit there and tune yor bandwidth, because TCP has algorithms to automatically adjust to bandwidth. A huge number of the problems in SL would just vanish, like that.


I disagree. I have looked into these types of situations and the TCP over UDP is the best option until TCP/IP version 6 is fully implemented.

Only the new TCP/IP version has any useful priority settings and most of the Nat firewalls and ISP's do not support the new priority settings.

This means that the only way to reliably send both time critical data and bulk transfers is to implement your own TCP system on top of UDP packets. There is nothing inherently bad when doing this provided it is fully implemented with packet numbers, channel numbers, and retry retransmission algorithms.

Imagine this scenario, your PC opens 100 TCP channels and starts to download data and then a UDP packet from SL is sent to you. Your local ISP connection is much slower than the Internet backbone and your ISP buffers the data in the order it was received. The result is that the UDP packet will be delayed for several seconds!

Another way to look at it is that TCP uses UDP packets! It just changes their type code so that they are labeled as TCP data. Now if there are TCP/IP re-buffers in the middle or if UDP packets bypass the FIFO buffers at all the routers then separate TCP channels would be useful. But right now we have bandwidth limitations on the SL side and on the client side.
Edit: typo
Argent Stonecutter
Emergency Mustelid
Join date: 20 Sep 2005
Posts: 20,263
08-22-2006 08:02
From: Wayfinder Wishbringer
I agree that lost info is one of the problems with lag. And like Argent says, that needs to be fixed. However, serious lag happens even when dataflow is just fine. So as stated, there are several causes of lag. The dropped data packet is likely a very significant issue. But I don't think it's the cause of an entire sim lagging to mush.
There's lots of things that can cause the performance problems people lump together as "lag": scripts, physics, and so on can lag a sim. Complex objects and content can lag the client. Network problems can produce effects that are superficially similar to "sim lag" or "client lag", as well as breaking downloads. It's important to get familiar with the symptoms of the different kinds of performance problems in Second Life and I really need to quit referring to them all as "lag" myself.
Wayfinder Wishbringer
Elf Clan / ElvenMyst
Join date: 28 Oct 2004
Posts: 1,483
08-22-2006 08:45
:eek:
From: Argent Stonecutter
There's lots of things that can cause the performance problems people lump together as "lag": scripts, physics, and so on can lag a sim. Complex objects and content can lag the client. Network problems can produce effects that are superficially similar to "sim lag" or "client lag", as well as breaking downloads. It's important to get familiar with the symptoms of the different kinds of performance problems in Second Life and I really need to quit referring to them all as "lag" myself.


I think you make some valid points. However, I think "lag" is a valid term, as it generically refers to anything that slows down the client experience of SL.

I've told this to computer companies every since I've been in the business. The majority of end users don't care about TCP/IP vs UDP, packets, database structure, etc. All they care about is being able to walk across a sim without feeling like they're tredging through syrup. They don't care about ISAM vs Sequential or Random files... they just want to access their inventory as fast as possible.

Tech is the job of the technicians. It's a job that can either be done well or poorly. When a client comes to LL and says, "Man, my sim is running really really slow"... the last thing any company would want to do is tell that client "oh, it's your content"... because that content is what the client is paying for... and paying heavily. First year operational costs of a sim is US$3,500+. At that level of cost, there's no room for lame excuses.

Now yes, client content can cause serious problems if a user has no idea what he's doing. But many private sim owners DO know what they're doing... and their sims still lag. At that point, the issue isn't client content. So then it's up to the techs to try and trace down what's wrong.

As you correctly state Argent (and I fully agree, more people should be aware of this), there are many things that cause lag. It can be caused by the internet itself and the actual location of the user along the internet pipeline. But... that's not what's causing "lag" right now, on SL, for the majority of the users.

I'm aware that Linden Lab is working hard trying to trace down some major issues. We've all seen very honest announcements from them that they are aware the whole grid is lagging and asking for patience while they fix it. I have absolutely no problem with that; nothing runs smoothly all the time and I can sure cut LL some slack... especially when they ask for patience. They deserve that consideration.

But what really needs to be addressed, found out, discovered, tackled and eliminated... are the deep-core programming issues that cause serious, everyday lag problems like the Ghosting Monster used to cause ghosting before they tracked it down and killed it.

So when I try to search my inventory and EVERY TIME it lags me to a standstill (and sometimes even freezes me and forces relog)... that is not pipelines, protocol, internet, or user content... that is deep-core programming and data access. And that needs to be fixed... because inventory access is the core of SL functionality. Inventory, no matter how involved and complex.... is a database, pure and simple. Database structure and access is Programming 101. Of all the things on SL, accessing inventory should be the most deadbolt, concrete foundation thing on the system. If one can't get a basic database to work... that's just sad.

So when that's not working... and all it is is a database... what are we to say when we're walking across a sim and suddenly can hardly move... then 30 seconds later we're moving again? Are we to believe that's client content? Nah. When we're flying along and suddenly rubber-band to where we were 10 seconds ago... is that client content? No. These are programming and dataflow issues... and they need corrected. Agreed, it might not be easy to correct them... but, that's why LL is paid the big bucks. Last time I checked, I wasn't paying $15 a month for my sim. ;)

Let's see.. 1,000+ sims, $195 a month, $7-$9 a month in premiums for every single person that owns a piece of land on the grid, LL fees from L$ sales, etc etc. It adds up, eh? I don't expect perfection by any means. But I do expect to be able to access inventory, and I expect to be able to walk across a sim without being rubber banded back to where I was 50m ago... on an everyday basis.

Why do I keep harping on this? Because it affects SL's bottomline operation, and in the past, LL has shown themselves attentive to squeaky wheels. I want SL to be the best it can be, and that isn't accomplished by users sitting back on their heels and wishing. Unless users let LL know what they think-- LL won't know. So we talk about lag issues and user experience and debunk the bogus claims... and that helps LL track down things (or in some cases, even might force them to pay a little more attention to areas they've ignored) and eventually... the Lag Monster will be as dead as the Ghost Monster. :D
_____________________
Visit ElvenMyst, home of Elf Clan, one of Second Life's oldest and most popular fantasy groups. Visit Dwagonville, home of the Dwagons, our highly detailed Star Trek exhibit, the Warhammer 40k Arena, the Elf Clan Museum and of course, the Elf Clan Fantasy Market. We welcome all visitors. : )
Argent Stonecutter
Emergency Mustelid
Join date: 20 Sep 2005
Posts: 20,263
08-22-2006 14:30
From: Wayfinder Wishbringer
Tech is the job of the technicians. It's a job that can either be done well or poorly. When a client comes to LL and says, "Man, my sim is running really really slow"... the last thing any company would want to do is tell that client "oh, it's your content"... because that content is what the client is paying for... and paying heavily. First year operational costs of a sim is US$3,500+. At that level of cost, there's no room for lame excuses.
If the client is creating the content, then that client is a Tech.

If that client isn't creating the content, then the people who are creating it are Techs.

They have to be able to tell whether the problem is in Linden Labs' court, or theirs.

The fact that they're renting paying $3,500 a year in rent is irrelevant. When we're trying to solve a problem in our product, we make damn sure it's not a bug in our own code before we go to HP complaining about the compiler bug on the $100,000 computer we bought.

This isn't about "giving LL slack". I don't give them much slack, myself. But I do try and figure out where the problem is and why it's happening first.

From: someone
So when I try to search my inventory and EVERY TIME it lags me to a standstill (and sometimes even freezes me and forces relog)... that is not pipelines, protocol, internet, or user content... that is deep-core programming and data access.
I'm pretty sure that is protocol. Your client isn't making MySQL requests to the asset server to get that information. No, they're sending you that inventory information using the same hopeless pseudo-TCP-on-top-of-UDP protocol that they use for everything else. So your client is getting an incomplete, out of order, and endlessly deferred stream. That is a deep-core programming problem. But it's nothing to do with "databases".

From: someone
So when that's not working... and all it is is a database... what are we to say when we're walking across a sim and suddenly can hardly move... then 30 seconds later we're moving again? Are we to believe that's client content? Nah. When we're flying along and suddenly rubber-band to where we were 10 seconds ago... is that client content?
The former may be client content (physics or scripts loading down the server), or it may be database problems. I'm pretty sure that LL doesn't have half the asset servers they need for the load they're under, for example. The latter is almost certainly a communication failure... rubber banding is caused by the client not getting position updates until its estimate of your position is too far from your actual position.

From: someone
Why do I keep harping on this? Because it affects SL's bottomline operation, and in the past, LL has shown themselves attentive to squeaky wheels.
Harp all you want, but try to identify the real causes of the problems you see so you can harp effectively, and harp on problems that they can fix, so you can harp effectively. Pointing to obvious communication failures and calling them "database problems" isn't debunking anything.
Wayfinder Wishbringer
Elf Clan / ElvenMyst
Join date: 28 Oct 2004
Posts: 1,483
08-22-2006 16:15
From: Argent Stonecutter
If the client is creating the content, then that client is a Tech.
If that client isn't creating the content, then the people who are creating it are Techs.
LOL. I think that about 99.9% of the people on SL would question that statement. Are you postulating that someone who creates a build, designs a garment, or owns a piece of land is a TECH? LOL Argent, I don't know what you're drinking, but... don't light a match.

From: someone
They have to be able to tell whether the problem is in Linden Labs' court, or theirs.
No... that's Linden Lab's job. Debugging the system is not the duty of the customer.

From: someone
The fact that they're renting paying $3,500 a year in rent is irrelevant.
It's quite relevant. If I take my car to a mechanic and pay him $50-$75 an hour, I expect HIM to fix the problem, not tell me that I need to know how the engine works for him to do so.

From: someone
That is a deep-core programming problem. But it's nothing to do with "databases".
Maybe. But how do you know for certain? I don't know how the inventory database at LL is written. Do you? ;) All I know is that my inventory lags to a standstill and often forces relog.

When the system had a problem with the Ghost Monster, no one blamed the clients. Same for lag... especially when the client has done everything possible to reduce such. The major lag problems are apparently on LL's side. It needs fixed. No matter what a Client does on his end... server/programming issues are STILL going to cause lag. It's just that simple. Once LL fixes the problem on their end, then the customers will have a chance at fighting lag on the content end.

From: someone
I'm pretty sure that LL doesn't have half the asset servers they need for the load they're under, for example. The latter is almost certainly a communication failure... rubber banding is caused by the client not getting position updates until its estimate of your position is too far from your actual position.
Yeah, I agree there. At least, in theory. Hard to tell of course... but I've long believed that asset servers are not set up to withstand the load that SL puts on the system. This is evident from the fact that the more people on the system, the worse it lags. We want to see system-wide lag? Come online any Saturday night around 7pm.

From: someone
Harp all you want, but try to identify the real causes of the problems you see so you can harp effectively, and harp on problems that they can fix, so you can harp effectively. Pointing to obvious communication failures and calling them "database problems" isn't debunking anything.
Argent, I think I've been very factual in my presentations. I realize you don't personally agree, but what else is new? Frankly, I don't think you've considered all the facts or evidence. You're off on another tech-tangent and tunnel-focused... and it's preventing examination of the overall issue.

I have stated there may be database issues in SL. You state it's protocol issues. You may be right. I may be right. We may both be right. What proof do you have that there are absolutely no database problems in Second Life? When was the last time you had a good look at SL coding? What absolute proof do you have that Linden Lab inventory data handling routines are up to snuff? So how can you sit there and state this is not a database issue... especially in light of evidence?

Techs often get so wrapped up in tech that they fail to speak or even think in common "English" and common "people". When I talk about database... I mean everything involved in it. That includes basic storage of data, program access of data, retrieval, transmission, receipt.. the whole 9 yards. You can techspeak all you like, but tell me this: Why did inventory work like greased lightning before the search function was added... and why does it work like cold syrup now? Why is it that during an inventory search, the first several thousand inventory items goes smoothly and quickly, as it should... but then slows down to a crawl as it nears the end?

That isn't packet Argent. That indicates database, sort and search issues. Such symptoms are unlikely to indicate protocol issues or transmission problems.

It is not my job to figure out for Linden Lab why the inventory system is not working properly. As a paying customer, it is my right to let them know when its' not working. It's my right to tell them the symptoms. It's even my right to suggest (not knowing the internal workings of LL and SL) at what might be the problem. Whether I'm on the button or not is irrelevant. As a customer not having access to LL code or facilities, I'm not expected to be right on the button. That's what LL trouble-shooters are for. What is relevant is that inventory does not work right and that affects LL's customer relations. If inventory failure is caused by database handling routines, packet loss, transmission protocols or someone spilling coffee on a keyboard every other day... the cause needs to be hunted down and fixed.

And that isn't my job. It's not your job. That's Linden Lab's. They're providing a service. We're paying for it. Making the service work is their duty, not ours.
_____________________
Visit ElvenMyst, home of Elf Clan, one of Second Life's oldest and most popular fantasy groups. Visit Dwagonville, home of the Dwagons, our highly detailed Star Trek exhibit, the Warhammer 40k Arena, the Elf Clan Museum and of course, the Elf Clan Fantasy Market. We welcome all visitors. : )
grumble Loudon
A Little bit a lion
Join date: 30 Nov 2005
Posts: 612
08-22-2006 21:35
From: Argent Stonecutter
I'm pretty sure that is protocol. Your client isn't making MySQL requests to the asset server to get that information. No, they're sending you that inventory information using the same hopeless pseudo-TCP-on-top-of-UDP protocol that they use for everything else. So your client is getting an incomplete, out of order, and endlessly deferred stream. That is a deep-core programming problem. But it's nothing to do with "databases".


LL has in another post indicated that the sim code has a memory leak related to the sim cache system ! This means that the more data you download the more memory it has to swap to disk and eventually it slows down.

How can it be a communications issue when the sim itself slows down and the sim framerate and time dilation drops to very low values. LL is monitoring sims and restarting them on a constant basis now and I am sure they are focused on fixing it.

Both TCP/IP and UDP/IP protocols use IP(Internet protocol) as the base layer. UDP is just an empty layer and passes the data on. This means that the concept of using TCP over UDP is fine and is used in a lot of other applications.

1. They have access to tools that we don't so we must let them work on the problem.
2. We need to provide them simple factual statements as to what we see.
3. Arguing about the merits of different architecture is not going to help them fix this bug.

Edit: typo
Wayfinder Wishbringer
Elf Clan / ElvenMyst
Join date: 28 Oct 2004
Posts: 1,483
08-22-2006 22:14
From: grumble Loudon
LL has in another post indicated that the sim code has a memory leak related to the sim cache system ! This means that the more data you download the more memory it has to swap to disk and eventually it slows down.

How can it be a communications issue when the sim itself slows down and the sim framerate and time dilation drops to very low values. LL is monitoring sims and restarting them on a constant basis now and I am sure they are focused on fixing it.

Both TCP/IP and UDP/IP protocols use IP(Internet protocol) as the base layer. UDP is just an empty layer and passes the data on. This means that the concept of using TCP over UDP is fine and is used in a lot of other applications.



Thanks for the info Grumble. That verifies and explains a lot of things. I wasn't sure how SL cache was set up, but the visible issues point to what you've pointed out here.

The memory leak you mention... from what I understand that affects individual sims... but since it affects all sims equally, it would have the affect of making it appear a grid problem? Does that summarize it properly or is there a bit more to it?

From what you describe, I agree... that would be a major factor in sim lag, at least. I know just tonight ElvenMyst was lagging like a fiend. I reset the sim... and it ran fine the rest of the evening. But up to that time, almost everyone who visited the sim commented that "SL is laggy all over tonight". So it's a pretty severe problem. Once that leak is plugged, we might get a better handle on what other issues might be involved. :)
_____________________
Visit ElvenMyst, home of Elf Clan, one of Second Life's oldest and most popular fantasy groups. Visit Dwagonville, home of the Dwagons, our highly detailed Star Trek exhibit, the Warhammer 40k Arena, the Elf Clan Museum and of course, the Elf Clan Fantasy Market. We welcome all visitors. : )
Argent Stonecutter
Emergency Mustelid
Join date: 20 Sep 2005
Posts: 20,263
08-23-2006 08:30
From: Wayfinder Wishbringer
LOL. I think that about 99.9% of the people on SL would question that statement. Are you postulating that someone who creates a build, designs a garment, or owns a piece of land is a TECH?
The person who wrote the scripts are techs. If you're building with your own scripts, then you're the tech. If you're building with someone else's scripts, they're the techs.
From: someone
Debugging the system is not the duty of the customer.
Debugging scripts is the scripter's job.

From: someone
But how do you know for certain? I don't know how the inventory database at LL is written. Do you?
I know quite a lot about it, yes, since it's using MySQL. That's got its own problems, but even if the inventory database was perfect in every way the network protocol would absolutely be capable of causing the problems we see with inventory, and would almost certainly continue to do so.
From: someone
When the system had a problem with the Ghost Monster, no one blamed the clients.
They should have. A ghost is an object that the client believes is there, and the server believes is not there.
From: someone
Same for lag... especially when the client has done everything possible to reduce such.
Um "the client" in this client-server system is the application "second life" you're running on your computer. Not the customer.
From: someone
Yeah, I agree there. At least, in theory. Hard to tell of course...
When they lost one asset server and that took the whole thing down, that's proof enough for me.
From: someone
Argent, I think I've been very factual in my presentations. I realize you don't personally agree, but what else is new?
Um, I agree with a lot of your comments. This has nothing to do with my "agreement" with your position, it has to do with your pointing your finger at the wrong problem.

If you go to the mechanic and insist that he fix your radio because your car stalls out when you play the radio, and he's wasting his time on a "tech tangent" talking about the alternator, and he's "tunnel focussed" on it... hes not going to be impressed.
From: someone
I have stated there may be database issues in SL. You state it's protocol issues.
I have not said that there aren't database issues in SL. There definitely are. The group limitation is a perfect example of a database issue. The asset server problems and asset lag are another. The problem is that many of the particular problems you're pointing to as database issues aren't evidence of database issues. Some of them are clearly protocol issues.

From: someone
What proof do you have that there are absolutely no database problems in Second Life?
Now it's my turn to ask you to stay away from open flames. I have never said there are no database issues in SL. I have never said anything even vaguely like that.

From: someone
When I talk about database... I mean everything involved in it. That includes basic storage of data, program access of data, retrieval, transmission, receipt.. the whole 9 yards.
Do you call electrical problems in your car "radio issues"?

From: someone
You can techspeak all you like, but tell me this: Why did inventory work like greased lightning before the search function was added... and why does it work like cold syrup now?
That's a client issue. The inventory management application is in the client, search is done in the client. There are a number of possible reasons for the problems you see, including the search function requiring that the client get more data from the server, that the client's internal data structures were modified for search in a way that makes them less efficient, that another change made at the same time caused a problem in search. But it's *not* an indication of a problem in the database.

From: someone
It is not my job to figure out for Linden Lab why the inventory system is not working properly. As a paying customer, it is my right to let them know when its' not working. It's my right to tell them the symptoms. It's even my right to suggest (not knowing the internal workings of LL and SL) at what might be the problem.
Great. And it's my right to suggest that, as someone who knows a bit about cars, that maybe the problem in your radio is caused by a bad alternator, and you're likely to get your problem fixed quicker by getting a battery check at Sears than by getting a new radio.

You're not expected to be right on the button, but what I don't get is why you're so upset about someone pointing to the button that might actually work?

From: someone
And that isn't my job. It's not your job. That's Linden Lab's. They're providing a service. We're paying for it. Making the service work is their duty, not ours.
Absolutely. You have no *duty* to listen to my suggestions as to how you might accomplish your goals better. But it'd be nice if you'd try it.
Argent Stonecutter
Emergency Mustelid
Join date: 20 Sep 2005
Posts: 20,263
08-23-2006 08:42
From: grumble Loudon
Both TCP/IP and UDP/IP protocols use IP(Internet protocol) as the base layer. UDP is just an empty layer and passes the data on.
Er, no, that's not actually the case. TCP and UDP are peers in the network stack. They both implement multiplexing (ports) on top of the IP layer. TCP also implements reliable transmission, UDP implements content checksums (IP only implements header checksums). TCP is oriented towards bulk data ransfers and non-real-time control. UDP is oriented towards streaming data and real-time control. SL uses bulk data transfers (content like textures and prims), non-real-time controls (chat, inventory, editing), and real-time controls (physics, movement, navigation).
From: someone
This means that the concept of using TCP over UDP is fine and is used in a lot of other applications.
Encapsulating IP in UDP packets to tunnel IP (including encapsulated TCP and UDP) is common. Re-implementing TCP-like reliable transport using UDP is (thankfully) rare. It's rare because it's unnecessary (TCP exists) and because it's hard to do anywhere near as good a job as TCP.
Wayfinder Wishbringer
Elf Clan / ElvenMyst
Join date: 28 Oct 2004
Posts: 1,483
08-23-2006 10:36
From: Argent Stonecutter
The person who wrote the scripts are techs. If you're building with your own scripts, then you're the tech. If you're building with someone else's scripts, they're the techs. Debugging scripts is the scripter's job.
This thread has nothing to do with writing scripts or debugging such. It has to do with LL software/hardware created lag.

In the "Why is this even important?" department: I have to disagree with your definition of "tech". A coder (scripter) is not a "tech" in the RL computer world. He/she can be... but it's not a given. A programmer and a "tech" are two different things. As with many fields, they can crossover into the other field. A programmer who writes system programs can be considered a "tech". A programmer/scripter who writes applications is not. That's just general field definitions.

Client content and poorly written customer scripts are irrelevant to the theme of this thread. We know that poorly written scripts and badly designed builds can lag sims. That is not the issue of discussion. The issue of discussion are the things that lag sims anyway, despite all that users do to combat lag.

From: Wayfinder
When the system had a problem with the Ghost Monster, no one blamed the clients.

From: Argent
They should have. A ghost is an object that the client believes is there, and the server believes is not there. ...Um "the client" in this client-server system is the application "second life" you're running on your computer. Not the customer.
Now see Argent, this is exactly what I mean when I talk about being so tech that you forget how to speak plain english. It is obvious from the context of my messages that the CLIENTS I'm speaking about are Linden Lab CUSTOMERS... not client-side software. I did not say "client software" or "client side software"... I said "clients". The term "client" existed long before computers were even a twinkle in someone's imagination. When people talk about CLIENT CONTENT... they are obviously not talking about a software program written by Linden Lab. Man, you need to get away from your computer and talk with real people once in a while. Seriously guy, you need to shift gears and get out of technobabble mode. ;) I don't mean that as a flame, I mean that as a reality check. When you tell us that "clients" does not refer to customers, you're in waaay too deep.

From: someone
I have not said that there aren't database issues in SL. There definitely are. The group limitation is a perfect example of a database issue.
First, you certainly have said there aren't database issues. In fact, you derrided me for even hinting at such when in your book the problem is protocol/transmission rather than database. In fact, you accused me of barking up the wrong tree... ie, "not harping effectively"... and you used the illustration of telling a mechanic that an engine problem is radio oriented (which by the way, many a radio electric problem have shorted out an engine, eh? So don't be so quick to discount possibilities. I made part of my living on troubleshooting unusual problems). At least, that's how it came across to me-- you poo-pooed the idea of deep-core database issues and proposed it was all in transmission protocol and lost packets.

I don't believe the group limitation is a database issue. It's a design and conceptualization problem... ie, the designers not forseeing the extensive need for groups. They still haven't forseen it, although they've been told. I told them directly, months ago, that 25 groups will absolutely not be enough. So did others. We recommended a minimum of 200 groups, or even dropping the static limit and making groups dynamic. They decided against that, for whatever reason. Within a week they're going to find people filling all 25 groups and begging for more. As with so many things, the new group "expansion" is too little too late... and far less than needed to meet the demands/needs of the customers. That's just poor. It's an example of yet another "halfway" measure that is insufficient to meet customer requirements-- and will likely need to be revised yet again. That's a total waste of programming time and effort. While I appreciate 25 rather than 15 groups, it's like giving a starving man a teaspoon of rice.

A database problem is when a designed database does not work properly. Ie, sort routines malfunciton, data is stored incorrectly, data is not accessible, etc. That's far different than insufficient conceptual design.

From: Wayfinder
What proof do you have that there are absolutely no database problems in Second Life?
From: Argent
Now it's my turn to ask you to stay away from open flames. I have never said there are no database issues in SL. I have never said anything even vaguely like that.
Argent, asking you a legitimate question in direct reply to a statement you made is not "flaming". The quote from you was:

From: someone
Harp all you want, but try to identify the real causes of the problems you see so you can harp effectively, and harp on problems that they can fix, so you can harp effectively. Pointing to obvious communication failures and calling them "database problems" isn't debunking anything.
This was in relation to a prior quote from you:
From: someone
So your client is getting an incomplete, out of order, and endlessly deferred stream. That is a deep-core programming problem. But it's nothing to do with "databases".
Now Argent, maybe what's needed is for you to step back and realize that there are other people on SL who know what they're talking about, who have been professionals in this business just as long (or longer) than you have... and cut everyone some slack.

This is a simple discussion about system lag. That lag is OBVIOUSLY LL-side and not client (ie customer) side. The point is that LL needs to dig in and find out what is causing that problem. Whether that cause is database oriented or protocol/transmission oriented or hardware failures or whatever... it is not the duty of the clients (ie customers) to figure this out. That job falls squarely in the lap of Linden Lab. All we can do is report to them what we observe. There's no way we have of determining the exact cause, because we have no way to access Linden Lab code, equipment or networking systems.

From: someone
You have no *duty* to listen to my suggestions as to how you might accomplish your goals better. But it'd be nice if you'd try it.
From what I've seen Argent, I don't really belive your suggestions are all that valid. And believe me, I've read every word. You may have a couple of points that need checked... but they tend to get lost in you telling other folks they don't know what they're talking about... and in comments like this one. ;)

We've discussed this in other threads.. and I've even sent you IMs. Just discuss the issue without making derrogatory comments about other people's knowledge, abilities or motives. No one wants to hear someone spout about how they know so much more than anyone else and how we don't know what we're talking about. All we need is the information. :rolleyes:
_____________________
Visit ElvenMyst, home of Elf Clan, one of Second Life's oldest and most popular fantasy groups. Visit Dwagonville, home of the Dwagons, our highly detailed Star Trek exhibit, the Warhammer 40k Arena, the Elf Clan Museum and of course, the Elf Clan Fantasy Market. We welcome all visitors. : )
Argent Stonecutter
Emergency Mustelid
Join date: 20 Sep 2005
Posts: 20,263
08-23-2006 12:15
From: Wayfinder Wishbringer
This thread has nothing to do with writing scripts or debugging such. It has to do with LL software/hardware created lag.
If you follow back to what brought up this comment in the first place, it was about whether lag caused by resident-created content was the responsibility of Linden Labs or not, when LL asked you to check your content...

From: someone
A coder (scripter) is not a "tech" in the RL computer world.
If he's doing it for money (even Lindens) he better be. It doesn't matter whether he's writing system code, application code, web code, LSL, Java, or Javascript.

From: someone
The issue of discussion are the things that lag sims anyway, despite all that users do to combat lag.
You brought up LL's suggestion that the lag might be caused by resident-created content? All I was saying is that... it might be, they can't prevent that kind of lag without restricting scripting beyond the point where I'd find it interesting to play (and I suspect you might too, given the number of 'potentially laggy' calls there are in LSL)... and it's not LL's job to debug resident scripts.

From: someone
It is obvious from the context of my messages that the CLIENTS I'm speaking about are Linden Lab CUSTOMERS...
No, it wasn't obvious. I've never run into anyone refer to residents as "clients", and when you're using the term "client" in conjunction with the term "server" in a client-server system...


From: someone
First, you certainly have said there aren't database issues.
Quote please. Exact words.
From: someone
I don't believe the group limitation is a database issue.
Linden Labs has stated that it's a database issue.
From: someone
A database problem is when a designed database does not work properly. Ie, sort routines malfunciton, data is stored incorrectly, data is not accessible, etc. That's far different than insufficient conceptual design.
And here you are accusing me of getting too technical. Database design, implementation, and software are all "database issues", both to technical and non-technical people alike.

From: someone
Argent, asking you a legitimate question in direct reply to a statement you made is not "flaming".
That was a reference to your comment (only one message back!) that went something like "I don't know what you're drinking but stay away from open flames".

If a light-hearted response to your comment about my writing messages while drunk is "derogatory", but your own comment isn't, well... whatever.
Kristy Cordeaux
Registered User
Join date: 13 May 2006
Posts: 94
08-23-2006 12:31
Yes, lag is a problem (as always)

Waiting to see how many posts we get from people bragging about their rigs now.......
_____________________
eMachines T5010, but modified to: 2.30Ghz AMD Athlon 64x2 Dual Core processors, NVIDIA GeForce 6150 (128 MB). 250GB HD.
Dr Drebin
Registered User
Join date: 19 Mar 2006
Posts: 66
08-23-2006 13:48
I have never understood why they don't assign servers to agents rather than sims.

I look at the map and see dozens of unoccupied sims which equates to untapped processing power.

Then I look at a crowded sim which is laggy because 1/4 of a server is trying to deliver content to more than 10 agents.
Shjak Monde
Registered User
Join date: 10 Feb 2004
Posts: 111
08-23-2006 13:54
Wayfinder has Much Passion and Wayfinder is exactly correct also... perhaps that may be where the passion comes from.
Derogatory Statement is not nessasary just to blow smoke and dance around the subject.
The Problem IS on LL side and Needs it fixed Badly.

LL problem number 1:
Clients ( SL Customers ) will never build things perfectly lag free... they will always over build and over script... this is a fact and the LL knows this... work with it... and there are ways to work with it in the data retrieval area.

LL problem number 2:
Stop over loading servers with totaly usless Info that NO BODY WANTS.. this is one of the major lag creaters and LL is at fault here...

LL problem number 3:
The Dynamic Cacheing system is a total bust... wash the slate and go back to the Static Caching System... and don't throw that stuff about SL being to dynamic to use a Static caching System.. that Dog don't hunt and is totaly boggus...LL is just trying to reinvent the wheel and are just refuseing to give up that lost cause.

All in All I stand with Wayfinder... I also am getting a bit tired of the danceing around the subject and acting as if they do not know what we are talking about and spliting hairs and trying to redirect the subject from what we are talking about.

FIX THE LAG!!
Shjak Monde
Argent Stonecutter
Emergency Mustelid
Join date: 20 Sep 2005
Posts: 20,263
08-23-2006 14:16
From: someone
Derogatory Statement is not nessasary
I agree. I wasn't posting drunk, and Wayfinder shouldn't have said that I was.
Wayfinder Wishbringer
Elf Clan / ElvenMyst
Join date: 28 Oct 2004
Posts: 1,483
08-23-2006 15:00
From: Argent Stonecutter
If you follow back to what brought up this comment in the first place, it was about whether lag caused by resident-created content was the responsibility of Linden Labs or not, when LL asked you to check your content...
Not to my understanding, it isn't. (And you're probably going to see this answer repeated here). I don't see anything in the OP about resident content being responsible for anything. ;)

From: someone
You brought up LL's suggestion that the lag might be caused by resident-created content? All I was saying is that... it might be, they can't prevent that kind of lag without restricting scripting beyond the point where I'd find it interesting to play (and I suspect you might too, given the number of 'potentially laggy' calls there are in LSL)... and it's not LL's job to debug resident scripts.
I think it's been directly stated that the kind of lag being discussed is not caused by customer content. And that was the whole point of the initial statement in the first place... is people continuing to insist it is... when it isn't. Lag that comes out of nowhere, lag that lags the entire grid, lag that involves inventory and texture access... is not sim content. It's coding and (possibly) transmission problems. Which is the whole purpose of this thread.

From: someone
No, it wasn't obvious. I've never run into anyone refer to residents as "clients"
Like I said, ya need to get out more often. ;) Argent, really, I don't live my life by "The Argent Computer Glossary" (no offense intended at all. Just a statement of fact). If you have never in your life heard Linden Lab customers called "clients" then you're reading different forums than I read. Like numerous other words in the English language, "client" has more than one meaning. The meaning is indicated by the context. The context in these messages was clear. To my knowledge he term "client content" in no situation refers to Linden Lab software code. While it can refer to scripting by customers in-world, it is never used to refer exclusively to scripting... but to scripts, builds, textures, music, the whole works. So again, I think the previous references have been pretty clear to the majority of the people reading this thread. I seriously doubt many would misunderstand the use of the term "Linden Lab Clients" as refering to software rather than customers. Again I'll say this: it's a matter of common everyday English vs technospeak. People can insist on technospeak to the point of being a pain... but it doesn't change how the average person talks.

From: someone
Quote please. Exact words.
If you will note... I did.

From: someone
That was a reference to your comment (only one message back!) that went something like "I don't know what you're drinking but stay away from open flames".
The exact quote Argent, was: "LOL Argent, I don't know what you're drinking, but... don't light a match." Just about anyone reading that would understand it was a humorous comment and had nothing to do with you or me flaming each other. C'mon man, lighten up. Just stick to the facts. This isn't a contest of who is smarter than who, who knows more or who has the most experience. It never has been, in any thread.

The very first message of this thread presented the problem of the test grid lagging like a fiend and once again stating that LL needs to attack the Lag Monster just like they attacked the Ghost Monster a year ago. That has nothing to do with client content. It has nothing to do with client computer systems. It has to do with deep-core, system-originated lag. That might be SL server software, asset servers, client-side SL software, glitchy database coding, transfer protocol, worn out equipment, mice chewing on the wires, sleepy technicians, power outages, repeated lightning strikes, volcanic eruptions... whatever. But whatever it is... Linden Lab needs to track it down, identify it, and rather than just patching symptoms... fix the core problem(s).

You postlate the problem is protocol related. I can get on that wagon. Others postulate it's memory leak. I can go with that too. I've stated they seem to have database handling issues. I think that's a valid issue as well, considering the evidence. I fully believe the problem is multi-level rather than one single core issue. But if it's 1 problem, 3 problems, 5 problems, LL needs to put all due attention into finding the sources and fixing them... because lag is right now arguably the #1 problem on Second Life.
_____________________
Visit ElvenMyst, home of Elf Clan, one of Second Life's oldest and most popular fantasy groups. Visit Dwagonville, home of the Dwagons, our highly detailed Star Trek exhibit, the Warhammer 40k Arena, the Elf Clan Museum and of course, the Elf Clan Fantasy Market. We welcome all visitors. : )
Wayfinder Wishbringer
Elf Clan / ElvenMyst
Join date: 28 Oct 2004
Posts: 1,483
08-23-2006 15:17
From: Argent Stonecutter
I agree. I wasn't posting drunk, and Wayfinder shouldn't have said that I was.
I don't think anyone took it seriously. But if any did-- and I realize you did-- it was a joke. ;) No offense intended.
_____________________
Visit ElvenMyst, home of Elf Clan, one of Second Life's oldest and most popular fantasy groups. Visit Dwagonville, home of the Dwagons, our highly detailed Star Trek exhibit, the Warhammer 40k Arena, the Elf Clan Museum and of course, the Elf Clan Fantasy Market. We welcome all visitors. : )
DigiKatt Shaw
Registered User
Join date: 16 May 2005
Posts: 108
08-23-2006 15:27
I won't pretend to know much at all about the technical aspects of this thread. I took two years of programming in college, thought it was boring, switched to business management and promptly forgot most of what I learned. However, I do know that the lag is KILLING me in sl. When you complain about it you're told to update your drivers. lol. give me a break! When i started in sl I was on my pc with a 128 meg geforce4 graphics card using 1gig of ram and a p4 2.0. I was on 512 connection. There was lag but it wasn't all the time. Only during heavy sim traffic. I then went to a 1 meg connection and upgraded to 2 megs of ram in my pc. but unfortunately that was when the lag issues started gaining momentum. I moved (was using the 1 meg wireless *only 50 yards from the tower. Its on our property) Now I'm living in seattle. Have cable, using a 6 meg connection with comcast. Upgraded my video card to 256 meg. Lag is worse... so the fiance bought me a new lappy last week. Its an acer aspire with all the bells and whistles. intel core duo 1.66ghz, bluetooth, fast system. 1 gig ddr2 ram. ati mobility radeon x1600 graphics... fast huh? not in sl. i notice very little difference in the speed of sl. or maybe that's not correct. i think that the lag has gotten even worse in the past two weeks or so. i go into my shop and i can't move. the min i appear in sl i have red bars all the way to the top for packet loss and bandwidth spikes. i try to use my camera to see something and i freeze up. two days ago i uploaded a texture for some hair. it was only 256x256. i finally gave up after more than an hour waiting for it to rez. I relogged a few times. nothing worked. finally the next day i logged on and after about half an hour it finally rezzed for me. my shop is losing traffic and business because of the lag. (my sim *milyang) seems to have more problems than neighboring sims for some reason. or maybe its just my imagination. All I know is that sl is my only source of income right now and I can't afford this lag to destroy my business. Something needs to be done to fix this problem. I'm at the point where I'm logging in and just leaving my av there to help with the traffic. some people come anyway. repeat customers mostly. but i've had a lot of them say they won't be back because of the lag in my shop. and neighbors complain they are having the same issues. they reset the sim. it doesn't help. or maybe it will but it doesn't last more than a few mins. they said the scripts are fine and whatever else is fine. they have to obvious reasons for the problems experienced. I've been having hair that i've retextured and retinted and put into boxes and it resets to previous textures and tints once inside the boxes. I've got boots I made in different colors and I've fixed the dang boxes of boots FOUR TIMES NOW. Just had to do it again the other day. Cuz I tint them and they reset again once i boxed them. grrrrr... so yes. something needs to be done, and soon. because why would people want to come into sl to freeze up and not be able to do something?

Digi
Wayfinder Wishbringer
Elf Clan / ElvenMyst
Join date: 28 Oct 2004
Posts: 1,483
08-23-2006 15:30
From: Shjak Monde
The Problem IS on LL side and Needs it fixed Badly.

LL problem number 1:
Clients ( SL Customers ) will never build things perfectly lag free... they will always over build and over script... this is a fact and the LL knows this... work with it... and there are ways to work with it in the data retrieval area.

FIX THE LAG!!
Shjak Monde
Thanks for jumping in Shjak. You mentioned something others have brought out in other threads and I'm glad you said it here.

Many have stated this all over these forums: LL created the environment. They can't blame the customer for using it. LL has the ability to establish control factors.

a) If they want to prevent physics collision... have items turn phantom the moment they touch another object and then turn back when they're no longer doing so. (I know, other things involved, but ya get the basic idea).

b) If they want to prevent griefer weapons, limit push power (I mean really and seriously, name me one single legitimate item on SL than requires a 9 billion push factor).

c) If they want to prevent caging, make bullets totally useless outside of battle zones.

d) Limit the number of scripts per parcel/sim just like they limit the number of prims.

e) Limit the number of prims and scripts on an avatar just like they limit prims/scripts on land. I mean really, there is a limit to how much everyone on a sim should suffer just because one person wants to wear unacceptably complex hoochie hair or 200-prim wings.

f) If too many textures on a sim lags a sim, then limit the number of textures allowed per plot of land. (Frankly, I think the "too many textures" thing is mostly bogus. Multiple textures may take a bit longer to load as a whole, but once the textures are loaded.. they view just like any other pixel).

These things can all be programmed into the SL system... and are not.

Since they know customers will push the limits of the system... set the system to expect this and deal with it. However, all of the above said, I do believe that if they track down the core issues and fix them... the above may not be necessary. But how many times have we entered a market sim and hardly been able to walk because it contains 12,000 scripts? Gimme a break. Does anyone realize that 25 scripts per 1024m is only 1,600 scripts? 50 scripts is 3,200. Who in the world needs more than that on every 1024m parcel on a sim? So perhaps the system should be designed to work effectively with X number of scripts per 1024m... or limited to not allow any more. If a landowner exceeds a script limit... they simply cannot add more scripts. Maybe they could even have a "script overhead" limit... if your scripts demand more than X amount of processor time, they take low priority over other scripts.

One thing LL was asked to do llong ago, is allow land owners to track how many scripts are on a PARCEL, just like they track prims. That would allow sim owners to be able to track down who is abusing script usage, what scripts are being used, what their overhead demands are, etc. If they expect customers to control scripts, we need the tools to do so.

So glad you mentioned this. It does no good to give customers power then blame them for using it. ;)

Now, that said, I would like to say something really postive about Linden Lab-- things that we requested long ago, and are now realities:

* Joint Estate tools
* Expanded number of groups (although prediction and long ago predicted: 25 will quickly be not enough) :D
* Group announcement and notecard ability (and nice implementation too... pretty easy to understand and use)
* The Ghost Monster is dead
* Teleportation no longer knocks people offline several times a day (for most folk, anyway)
* Searchable inventory (although it needs to be implemented far better so that it doesn't lag or knock people offline)

... and a multitude of other things that don't come to mind right now. So they do listen. Thus the reason for posts like this-- because Linden Lab obviously is trying. And I will be the first to step up and acknowledge that. So continuing to discuss issues that are still problems will likely mean that LL may dig in and fix this one too, and eventually the Lag monster will be dead-- to the benefit of everyone. :)
_____________________
Visit ElvenMyst, home of Elf Clan, one of Second Life's oldest and most popular fantasy groups. Visit Dwagonville, home of the Dwagons, our highly detailed Star Trek exhibit, the Warhammer 40k Arena, the Elf Clan Museum and of course, the Elf Clan Fantasy Market. We welcome all visitors. : )
Wayfinder Wishbringer
Elf Clan / ElvenMyst
Join date: 28 Oct 2004
Posts: 1,483
08-23-2006 15:50
From: Kristy Cordeaux
Yes, lag is a problem (as always). Waiting to see how many posts we get from people bragging about their rigs now.......
ME! ME! LOL.

To echo DigiKatt's statement: when I first entered SL, I had an Athlon 2800, 128m ram with 60 gig hard drive and Nvidia 5500 graphics card. Yeah, SL lagged... but only after about 25 avatars entered a sim. We crashed when we teleported and Ghosting was an issue, but inventory worked like greased lightning. Lag has always been an issue... but not cold Karo lag with 3 avatars on a sim. For the most part, my system worked very well... and with the exception of local lighting and flexi prims, the graphics worked pretty much as well as they do now.

NOW... I have a duo-core, 1 gig ram, 500 gig HD and ATI X800XL graphics (ie, it rocks)... and I lag daily to a standstill... often requiring relog. There is regular system-wide lag. My group members constantly report to me lagging all over the grid, not just at our very-well-organized sim (about 8000 prims and script-controlled). I regularly have to reset the sim, after which it works fine (indicating as someone pointed out... a cache/memory leak or pileup or something similar).

Before the ATI 800... I had upgraded to an Nvidia 6600... and it worked ok... but SL demanded every ounce of that card. My question and a question repeated by very informed people is: why? There have been lengthy debates on that one, but the general conclusion is that SL is being a resource hog and a half and really needs cleaned up and code/graphics simplified (not reduced...simplified). There really is no reason any decent graphics card (even the Intel system) should not play SL just fine. (at least, that's what I've been told by the techs). Any card that can play Unreal or Quake without a glitch should be able to handle SL graphics. That isn't the case. So that could use some attention. (not that I'm trying to start a big debate on that one. Just making an observation that's been made by a lot of users lately...). I have friends that I no longer see on SL because the system out-hogged their relatively decent computer.

K, that's my system brag. :D Everyone else's turn. I know someone out there can outspec me right away. (I would just LOVE to have an Nvidia dual 7900....but man, I don't understand why a graphics card should cost as much as an entire computer...).
_____________________
Visit ElvenMyst, home of Elf Clan, one of Second Life's oldest and most popular fantasy groups. Visit Dwagonville, home of the Dwagons, our highly detailed Star Trek exhibit, the Warhammer 40k Arena, the Elf Clan Museum and of course, the Elf Clan Fantasy Market. We welcome all visitors. : )
DigiKatt Shaw
Registered User
Join date: 16 May 2005
Posts: 108
08-23-2006 16:05
K, that's my system brag. :D Everyone else's turn. I know someone out there can outspec me right away. (I would just LOVE to have an Nvidia dual 7900....but man, I don't understand why a graphics card should cost as much as an entire computer...



did you read the maximum pc mag that has the dream machines in them each issue? the lastest one has quad video lol... the whole system is 10k and some of the stuff isn't out yet. the whole thing is fast fast fast!
1 2 3