Tactical solution to audio+video problem

Many people have suggested that mplayer be used to handle all audio and video traffic for the Linux SL client. This makes a lot of sense, since it is an extremely capable multimedia player, very flexible, widely used, and very efficient too. The problem for LL (as some have suggested) is that there may be some risk attached to actually specifying mplayer as an official requirement for running the client, given the putrid lawyer-ridden society in which we live.

The answer is simple: don't specify which application should handle the audio or video at all. Just send the multimedia stream(s) to named pipes or to specified TCP ports on localhost, and let the users set up any application(s) they like to read the streaming traffic which appears there. The FOSS community is pretty good at documenting solutions using FAQs and HOWTOs and wikis, so I expect that most Linux users would find this "sort it out yourselves" solution perfectly acceptable.

Clearly this would be trivial to do for audio, because all audio streams can be mixed together and sent to a single socket. It wouldn't even need mplayer, ie. a simpler application like mpg321 (an improved version of mpg123) or xmms would suffice. Furthermore, limiting the "public audio API" that LL needs to worry about to just a single socket would greatly reduce the support burden for LL, because low-level Linux sound is somewhat messy and not everyone runs ALSA: eg. OSS is still around, and some sites use esound or NAS, and the higher-end folks use Jack for low-latency sound and very high flexibility. All of these issues are avoided by sending the audio stream to a socket and leaving the rest to the user.

Video is more problematic because it usually requires overlays to place the video frame in the right place in the 3D world and with correct transformation and clipping, but the interface to the overlay handler can still be a simple socket interface. Furthermore, some types of video content actually benefit from display in a separate multimedia window, and that would be very simple to implement rapidly using the stream-to-socket approach.

What I'm proposing then is the engineer's favourite solution: decouple!

A simple socket is a remarkably good decoupling point, and if LL can avoid specifying which applications the user should attach on the other side of the interface, this could have significant corporate advantages.

There's already some effort going on about playing sound asynchronously from the SL client. We can't say how far it has come yet, because it involved some deeper intrusion into the SL protocol. To my understanding there have been some security issues with that, which are being looked into now.

While there are definate advantages to the approach you suggest, and technically its a great idea. I don't know if it would be an acceptable fix for LL or the non-techy user base. "Its a game and I have to what with what? Every other game on Linux manages to provide sound!"

You can't have company policy along the lines of "No, we don't support sound on Linux due to patent/licencing/legal/hula-hoop restrictions. *COUGH*mplayer*COUGH*", so whatever the final fix is, it will have to be something LL can stand by.

Most users, even the Linux ones don't read these forums, they will just end up wandering around in world with no sound and not knowing why. This can only have a negative impact of their SL experience.

The same goes for video. The problem needs to be fixed at its source, not looking around for bandaids and expecting the users to 'know' how to apply them.

I absolutely agree that the final product needs audio and video (if at all possible) integration into the SL client itself. But like with some other shortcomings, I'd love to have a workaround available for the time being, until LL gets around improving the Linux client. To be honest, it's not hard at all to implement audio playback, so I expect it to be one of the first things that get fixed. However, we don't know when this will happen, so the Linux community is bound to get "creative" in the quest for sound

How about outsourcing the Linux development to some of the residents (Linux Linden Labs, LLL)? I promise you instant improvement in the Linux SL experience

Oh me too! I'm just a little hesitant about advocating creative temporary hacks, as once in place they tend to become permanent.

I don't think that would fly. Audio needs very low latency and excellent host synchronization to work correctly - look at the hoops we had to jump through to make it work on Wine (time critical threads etc).

The right way for LL to solve this is just to bundle GStreamer or Helix, both of which can play almost all formats modulo the Sorensen codecs, and then forbid/phase out their use.

Yes, that could work, Mack. While I haven't used it personally, Gstreamer has a good reputation and is very capable.

However, just to correct one point you made, there is no latency or sync problem in the slightest in Linux audio if you use the Jack Audio Connection Kit. It's very much a pro' sound fabric, with real-time scheduling and very low latency, used by the Linux musician's community for handling multiple diverse sound sources and sinks like synths and sequencers and sampling etc --- and they have very demanding standards. What's more, it's available in all distros and for all architectures, and most of the major multimedia players can hook into it.

That said, I haven't heard of any significant latency issue even using plain ALSA as an output device for ordinary media players. Of course you'll get delay if you use pre-buffering to overcome network lag, but that's the case with all players on all systems.