Dear Lindens: Voice does TOO affect performance.

Well, just disabling voice won't cut it, I've learned.

You have to GO BACK TO THE PREVIOUS VERSION.

coco

Whatever happened to a network being only as fast as the slowest machine connected to it?

What difference does it make whether voice is running on separate servers if the clients using it are so bogged down processing at the cpu level that they can't parallel task anymore and send requests to other active processes? You know, processes like other SL packet transactions that are running on the main SL servers?

I may be completely off-base here, but:

1) Client A sends an avie into that really popular dress shop to buy a dress. Client A is already so bogged down processing packets to render the graphics and normal SL sound that it barely can get the job done.

2) At the same time, Client A's user begins trying to process full duplex voice streams from the perv standing next to her. As she rifles off her snappy reply, the CPU bottleneck gets worse at Client A.

3) Client A then sends a purchase request, involving a vendor script, to the server. It is severely hampered in its ability to handshake during the request and as such, takes forever to get through the process.

4) The transaction is hanging due to cpu lock at the client and the server leaves the thread open trying to finish. The "retry" transaction option vaults itself into Urban Legend - like levels of infamy.

When this happens enough times, across God knows how many sims, the servers will slow down. This is my theory (which may be completely wrong as my focus was never load balancing). Of course, with the amount of scripts that will hang transactions at the client for everything from blinging bracelets to AOs to parcel camera security systems, there are tons of malformed packets traveling from overworked clients into the system. Packet collisions and resent packets increase traffic and soon everyone feels it.

Are my drugs better than I thought, or do I make sense?

Hmm Kascha, it probably is happening but i'm not sure how big an effect that would be. Idle threads ... not sure they will bind the servers up that much. And, I'm running on a teeny laptop that is always at 100% CPU utilization in SL and have never noticed performance impact when running Skype or Yahoo voice as well. There should be nothing worse about SL voice, from the client's perspective.

But I won't discount it. It seems to me likely that some second-order unforeseen consequence of voice could be the source of the last two days' "colo" issues. And that is indeed what you describe.

But isn't it true that the built in SL voice system is spatial - trying to render the voice in 3D according to where the source of sound is? Wouldn't that make this just another form of the EAX or Hardware 3D Sound bottleneck that has caused the makers of first person shooters to recommend that slower machines go with either Dolby or software-based sound to prevent lagging and lockups during deathmatches and whatnot?

I mean Skype and other messengers are not rendering the sound - they're just playing it back. And it's the client side processing equipment that has to distribute that sound according to location - not the server. So with the inline sound option you are now asking a machine with 100% CPU utilization from rendering 3D imagery to now also render 3D sound streaming full duplex. I mean I dunno. And it's not so much idle threads I'm talking about as it is the number of malformed packets that must be discarded and resent due to the client side overload. A dirty loud network is effectively just as bad as an underpowered one.

Hmmm well I wish we were on the phone, I think I could learn something here. /me knows nothing about audio rendering.

However, from what you say I suspect that many game solutions are peer-to-peer in nature? Diamondware (the toolkit SL is using) is client server ... take a look at the architecture summary and you tell me whether you think it would have performance issues client-side. Looks to me like mxlib is where all the actual rendering takes place, and that is in the mixer, which HAS to be server-side, yes?

http://www.dw.com/about_technology_voip_buildingblocks.php

<-----Part of this group

Ya I can see what you're saying here. After reading that thingy though they don't make it really clear which portions of the system are placed where. Of particular interest are the Telephony Sound Toolkit and MixLib components, the first of which handles microphone to speaker interaction and the second which handles the 3D positioning data. From what is described, it is not clear which of these applications is actually handled by the server component and which is embedded in slsound.exe or whatever the sound client's name is.

Be that as it may, what the whitepaper does not elaborate on is the fact that there is no path from the microphone to the application, or from the application to the speakers, that does not go through the sound card and its driver. Spatial sound capability or lack thereof does not depend on the server side components. If the sound card on the client cannot reproduce it the sound will be 2D no matter what the server has done. I believe that both of the aforementioned components are located in the client side component to this system. Data is sent from the server to this component including a set of xyz coordinates detailing the locations of source and target. The component calculates relative position, volume and whatnot and sends that info to the card driver sitting between it and the sound hardware. It is then up to that driver to transfer whatever levels of what go into channels on the client. The same thing happens in reverse to go from client side microphone to server.

A CPU bottleneck on the client screws up that whole process, because if there is lag during transmission of the data from application layer to hardware layer and back again, you fall father and farther behind real time operation. What results is sound jumps where sound is not transmitted because of buffer overflow and is then rammed down the pipe later in an attempt to synch with real time. The bigger the bottleneck, the worse this gets until the application becomes unusable. The way of preventing this recommended by most streaming developers is to reduce the bitrate of transmission or to increase buffer size. The JitPP component appears to be used to handle that, and could conceivably reside on both sides of the transaction in one form or another. Either way, the problem is too much data coming down the pipe too fast.

Again, I could be horribly wrong. But I don't think so. If I am right, then people on both sides of this argument are correct. The data handling failures are in fact client side. Their cause is server side to the extent that an application requiring a large amount of processing priority and buffer space has just been added to the mix and is sending truckloads of additional data to the client sound component causing the CPU to overload further. The collective effect is like an auction caller waiting for a winning bid from a roomful of bidders with ADD and stuttering problems.

after reading all this , (which wasnt the easiest thing for me)

If sounds like what you are saying is Voice:

Requires SOME more server power
Requires SOME more network thoughput
Requires SOME more Client power

Resulting in general slowdowns for everyone, that will be worse for some than others.

@Colette: yup

@Kascha: Interesting. If you're right, then I see potential for rapid death spirals on the client. Before, at interrupt level you had only the graphics driver as a major hog. Now, you have the sound driver in contention, resulting in much worse cache coherency. Which is death to Intel processors.

Maybe particularly noticable on dual cores, if they previously shook down so that the graphics processing ended up on one core, and application space on the other. Throwing a CPU-intensive sound rendering driver component into the mix would majorly mess up such a balancing act. But I'm speculating about that, I'll stop now!

Tell me, might the CPU utilization be improved if you configured your sound for mono? Then, although the sound would obviously still go through the driver and the jitter buffer would still consume memory, a semi-intelligent driver might skip rendering? Your sound would no longer be spatial, but at least you'd hear something and crash less!

And to all of you, I present the beautiful Colette - breaker-downer of all things technobabble! Hehe

Yes that's pretty much my theory. That no matter what happens there is always a portion of the processing that must be handled at the client and if the server has to keep on waiting for bottleneck clients then QoS (or quality of service) goes out the window on both ends. The variance is greater or less depending on how much of a bottleneck the client is experiencing, so for some clients, Control-Alt-Delete may even be impossible, but not for others. This is a real problem. There are no shortage of internet deathmatch or network deathmatch gamers who can relate at least one story where an entire session was dragged down by the one or two client machines that had to keep on asking the hosting server to send it custom map, texture, or sound data that the other machines pulled down completely in one pass.

Unfortunately the "what to do about it" portion is not as simple as more more more -

/me sings "How d'ya like it how d'ya like it"

Sorry about that. Since my theory revolves around client bottlenecking it is actually a problem of *too much* data. Increasing the pipe to the client will make the problem worse. I imagine the result would be not unlike increasing a record to 78 speed from 33, mainlining heroin and then expecting the lyrics to be even easier to understand.

Actually I apologize. That analogy at the end was terrible - horrible in fact. I have failed my saving throw vs. Lunacy

Analogy 2.0 -> Increasing the bandwidth from the server to the client and throughput with it would be like :

Trying to solve a problem where you can't get an ocean worth of water through a garden hose aimed at a pinhole by switching out the garden hose with a fire hose.

Yay! Thank God for that +3 to Wisdom I got for being a three headed alien

This is the logic behind disabling EAX and/or hardware 3D audio to increase FPS performance. The client no longer has to calculate for the z axis number one, and also no longer has to keep track of things like decay or sound-object collision relating to who can hear what where. Anybody not separated from the source of a sound by a wall will hear it and will hear it through all available speakers.

Or something like that

Actually though, the problem with earlier systems and graphic interaction was not unlike what is happening here. Applications were still ending up CPU bound because the graphics subsystem was not capable of pulling its own load, leading to the paging out of graphics tasks to system memory to compensate. The industry response was to introduce AGP to widen the pipeline between the OS and the GPU and to vastly increase GPU memory and parallel processing ability, taking the video processing chores back from the CPU. Now the cards are so fast that the CPU is now the bottleneck because GPUs are rendering triangles so fast and in such great amounts that the CPU cannot catalog them fast enough and cannot sync the non-graphical stuff up with the rendering. All roads lead to Rome I guess.

Either that, or I'm on the best drugs EVER!

Maybe what to do about it is scream and yell at the voice companies that sold them this junk ...


and change voice to a 1 on 1 and group chat system more like skype and team speak.


Just a pure guess of course ..

Well see though

Though I find it Ironic that the paranoid anti voice crowd said two things would happen.

One group said there would be Social Implications that would be negative - oh gee.

One group said there would be performance Issues that would be negative - oh gee.


Maybe they will have some big fix for the performance stuff.

Can hope.

IF all that is true, is it any surprise? Everything they have farmed out from CS to billing and whatever else, seems to have gone to the lowest bidder. You get what you pay for.

good point.

Awww geez. I just got an Armageddon flashback. LL has become NASA and SL has become the Space Shuttle...

Does that make Uncle Phil Ben Affleck?

Certainly makes his techie-design department the WMD used to take out the asteroid, doesn't it?

((Note: I view SL as the asteroid, moving toward Earth-as-conventional-internet...though some days, I wonder whether Linden Lab as asteroid, descending inevitably on Second Life is more apropos))

Hehe - to me SL appears to be the shuttle that didn't make it

It doesn't matter aparentl how old or new your machine is. Mine is fairly new. It was upgraded a few months ago to better suit my gaming needs. I have horrid packet loss. My bandwith goes to the crapper as well. I had numbers ranging from 7 - 60 this evening before I said screw it, I have a book somewhere collecting dust!

reposting my 2 cents here!
With all the problems since voice (searches not working, tp's not working or intermittently working) besides hurting everyone who wants to shop or runs a business... it is the SL client taking over the CPU usage at over double what it was before the addition of voice. To find that out... hit control + alt + delete - go to performance tab... and see the % CPU usage. There is NO NEED FOR VOICE if it is affecting searches/tp's etc. until they (LL) finds a way that it does not take over your DOUBLE CPU usage. SL had enough problems with the game (for years) and they don't fix things they ADD. Also on a private sim every login you do on the sim you own, voice shows enabled even though on region/estate tab it is off. To fix, turn on and off again to disallow it on your sim. However it will not change CPU usage.

IT IS ABOUT TIME SL FIXED THINGS! I personally have lost a ton of sales as has my mall due to searches and tp's not working when CPU is maxed out. It is hurting people's ability to shop, manuever in the game, teleport, search etc. And when anyone lands on island it spikes. BEFORE THE VOICE it was NOT using 1/3 the CPU it is now. I run 3 GB of RAM and am having big troubles when I use my computer as I am sure thousands and thousands of users are, especially older or lower end computers, let alone laptop users.

IT IS NOT FAIR to players who are here to work/create and enjoy the game. Since VOICE the problems have become UNBEARABLE and ONGOING and EXPENSIVE. Disk cleaning, cache cleaning, rebooting etc etc etc over and over again to NO AVAIL.

PLEASE FIX THE GAME. GET RID OF VOICE till it can be done without taking over your computer. There is Skype, Yahoo, MSN etc. NONE OF WHICH pull CPU like SL now does with VOICE.

I AM FED UP!
I have noticed that with all the screwups that happen with the game LL never loses its ability to take my tier fees on time! I know that a problem such as that would be fixed ASAP. I don't personally care if IBM and DELL and other corporations want VOICE. The game has been and is created and supported by individual users! And as a longtime user.... I JUST WANT THE GAME TO WORK!

VOICE IS A CPU HOG which no one wanted in the first place! There are too many other voice options that work fine.

SUGGESTION: 2 SL CLIENTS.. One for voice users, ONE for those that want a good smooth running game (after they fix all the other problems that been ongoing for years)!

QUIT trying to dump the operations of this game on users! EXTERNALIZE VOICE!!!

I don't see how the voice feature can affect packet loss, but recently my packet loss bar's been in the red- and the latest spate of problems coincides with that.

I'm on a pretty nice, up to date computer (dual core, geforce 8, 2 gigs of RAM, all that fun stuff) and I've noticed that voice has caused SL to run horribly, sometimes. It will run at about a frame a second until I disable and re-enable voice. This only happens sometimes, and it's that easy to fix, but it's still annoying. I just thought I'd bring that up.

But anyway, for those of you who don't use voice and complain about it anyway, just don't enable it. I don't think the voice client runs until you do enable it. My firewall only asks to allow it after I enable it, and it seems to be an extra program running at the same time as SL.

I keep telling you - just disabling it DOES NOT FIX IT.

You have to GO BACK TO THE PREVIOUS VERSION.

And even then, the previous version will not be as good as it once was. Packet loss, etc.

Now, I don't know why it is all this way, and I also don't care.

I don't care how they did it, but they ruined SL.

coco

Not only does the previous version work better with less packet loss in SL, the web browser works way better with SL minimised. That new viewer is a resource pig.
Voice is a pig. I have it shut off on my land and estate and the damn icon still shows on the top of the page. It is a broken POS.

We need an open source viewer with all the updates and none of that broken voice crap.
And I like the friends button on the bottom with the older viewer too.