In-World Live Music: Question about synch

Hi,

I'm looking for insight into how well live musicians might be able to synchronize playing virtual instruments in Second Life. By "playing virtual instruments" I mean scripted objects that produce in-world audio when activated by a musician, such as a virtual piano, or guitar. I am not talking about audio media played via an external URL.

Imagine this scenario: 3 musicians are in different parts of the U.S., but are all logged into Second Life. One clicks on a virtual guitar to play a chord. One clicks on a virtual bass to play notes. One plays a virtual drum kit. Each musician has professional-level timing in real life, and is familiar with playing their instrument in Second Life. In this scenario, what are the approximate chances of all three musicians being able to synchronize a rehearsed performance of Blue Suede Shoes played on virtual instruments? What technological challenges might impede the performance, and how might they be overcome?

Any virtual musicians want to take a stab at educating me on this?

Zero, the basic problem as you know would be latency. It's even now, pretty much impossible to bring musicians together in "real" real time over the Internet because of the network delays and latency in the PC bus itself. The more performers you add, the worse it would get. Add also the lag within Second Life and you'll get a pretty insurmountable problem, even if using scripted inworld instruments.

One way of overcoming it is paradoxically to actually add more latency. Software can delay the transmittal of the fellow musicians' performance so that the extra latency allows everyone to catch up synchronously with the performance. It sounds mindboggling but it seems to work. NinJam is an example of mediatory software that does it:

http://www.masternewmedia.org/music_collaboration/music_jamming/online_music_collaboration_software_20050715.htm

I guess the issue in SL would be to take this paradigm and script inworld instruments that exploit it in some way. I'm just an "end user" so I have no idea if that would be doable.

This used to be something I thought about day and night. It's such a distant topic to me now, but from a performance standpoint, if you have an elementary master clock that syncs to bars and quantizes--rounds them down to the nearest unit--like beats, quarter-notes, and so on, at least everything will be locked to the groove. There are a variety of software and hardware "grooveboxes" that in fact do this, so for example, you can push a button to trigger a riff early, and it'll just get played when the next bar comes along. It can be pretty forgiving, so even someone with an awful set of timing can stay locked in.

This, in a rudimentary way, can actually be done in Second Life, and was done long ago on a soundmaking machine called the Mixmaster. See Andrew Linden's comment on my blog. Currently, Resi Octal Khan has turned part of his online home into a studio which is very easy-to-use, and nicely arranged:


One of the most effective things I found to do in music is have a tight, very mechanical underpinning foundation of loops (heavily quantized, in other words), with free-'n'-wild improvisation sprayed overtop. Best of both worlds, as I like to clichedly say.

Kei, Torlely, thanks for the thoughtful replies -- you have confirmed what I suspected. I like Torley's idea -- something like it was used by Star Wars Galaxies, where player musicians riffed synchronously through quantization. Riffs were all preset though, so synching everyone up in that situation is easier than an open-ended virtual band such as is possible in Second Life.

You should take a look at the Elven Drums. They play in sync and are quite fun. I don't think they are exactly what you are thinking of, but perhaps the core of what you want to do is there. I wish I could remember the name of Robbie's store. But I know they are also on display at Nocturnal Threads (and he has a vendor there). Grab a couple of friends, head over there and enjoy a nice drum circle.

Astrin Few and Flaming Moe did a simultaneous performance each from their respective locations last year. It is not easy. I'd ask them about it.

I just tried out the Elven Drums, which both look good and play well. There is a synch system in place that can be toggled on or off. This system works well for pre-defined loops, but wouldn't facilitate the example scenario -- EDIT: Actually, it would in a way, just not exactly how I described it.

I'll try and hunt down Astrin Few and Flaming Moe to see how they did it. Hamlet tells me "Flaming Moe plays sax from the East Coast while Astrin Few plays guitar from the Midwest, while their engineer mixes their instruments and vocals in Canada." The thid-party pre-mix seems to indicate the two are not playing directly in Second Life, but are using media streams instead.

That's correct. I mentioned them as a source because they are pretty familar with the methods and obstacles which live musicians face.

Do go to the source, but failing that, my guess is they had to have been playing to a click track and not hearing each other. Even telephony specifications allow for 50ms latency which is fine for conversation but is a disaster for syncro-jam. At 120 beats per minute, you have 2 quarter notes per second and internet latency of 100ms would put you 20% or 40% off each other.

Talking into an delay box and hearing yourself 1/10th of a second later makes speech really hard - music even more so.

added: Club DJs are skilled in slight tempo tweaks in beat matching songs for switching. I would not be surprised if the engineer for that gig was doing something similar.

Thanks so much for the link, most interesting!

I'll tell you what; if someone found a way of inputting midi data in SL I'd be their friend for life.