audio particle visualizer

I saw one other post related to this awhile back.... I wanted to see if anyone had any interesting ideas.

I was talking with a DJ friend and I thought it odd that there were no tv screens in clubs with visualizers to complement the music. Having it sync'd to the music would be event cooler, then you could have SL VJs.

Unfortunately with bandwidth issues and the limitations of LSL, this all seems like a huge pipe dream.

Unless...

Well, one thought I had was using the in-world particle system to create visualizers. Ok, that's doable. However, you can't really get any data on the music you are playing to effect any kind of change in the particle system. That's a problem.

Solution. Perform a spectrum analysis of your music track outside of SL and encode the data into a format that can either be pasted into an SL notecard or transmitted in-world via http. That might work, though in some cases it might necessity a bit of preparation and planning on the DJs part. Either way, it seems that processing this information outside SL and them transporting it inside in an usable format is the way to go.

So, that might work, but there is another problem. I don't believe you can trust the streamed music to be in sync with everyone, so it becomes next to impossible to sync the particle system in a way that every client could see. Hmm... that's where I'm stuck, I don't have any thoughts on getting around that.

I'm muling over some ideas about how I could "fake it", but it needs more thought. I don't want it to be totally fake...

Has anyone else pondered this topic? I'd like to hear some thoughts and ideas from others.

This is indeed a nice idea - but I'm afraid, you can't really sync the music to some inworld effects.

When a user starts to play the stream, the client will buffer a certain amount of data before it starts the playback. The amount of data buffered mostly depends on the available bandwith. The streaming server, on the other side, buffers data as well until it starts streaming it out to the clients. These two can vary from a couple of seconds up to like half a minute.

With these two totally unpredictable «lags», you will never be able to tell when a stream starts to play on a clients computer. You could try to start the particles manually and «hope» they're somewhat in sync but as particles are rendered on the clients computer as well (afaik), it will again be unpredictable when the particles start on the other client's computers...

You could use a live stream rather than a link to a static file, and have the server coordinate the time it plays the music. Then you could use the current wallclock time or datestamp to determine what particles to emit. Could be an interesting project....

I agree with Hewee...you could use a live stream.

just out of curiosity: how do you determine the buffertime on the clientside? When someone has a slower internet-connection, the amount of data buffered before the stream is played back will be higher than when someone has a faster connection.

Maybe you don't have to go that far either...

Music and random visualization will sync itself. Put on "dark side of the moon" and watch "wizard of oz". These two random media's will appear to sync themselves and appear to have been made for each other.

Here is my thought: Make a particle system that produces random patterns and color changes. visualizations that have the look of "beats" and rythms, fade outs etc. Let these effects occur randomly and I would think when coupled with music you will get the effect you desire.

Cheewha

Depends on what you have been smoking too...

But joking aside, yes.

We (the 'Cybernetic Art Research Project') have a very large venue which we call "Living Architecture"; this has moving walls, floors, scenery, particles, lighting, textures etc. All "randomly" set off by controllers (or can be a HUD)

The word "randomly" is in inverted commas, because the timing of the changes can be performed at certain common bpm.

The overall effect is remarkable, seeming to have been taylored-made for each performance, regardless of the artist playing the music or the genre.

Allow the human brain to "find" the appropiate pattern, which it will inevitably do whether such a pattern was designed or not.

For "The Wall" we used an entirely different solution. Live music was streamed into SL and a number of time-line controllers activated events within the venue. For example, when 'Mother' was playing a controller would activate the mother puppet at a precise moment of the song, and then switch off the puppet at another point in the timeline

This works well. However, it requires many hours of meticulous study and adjustment to get it just right.

I suspect this solution would be unsuitable for a DJ, in that all spontaneity of performance is lost.

Thanks for all the input... I was thinking, if I could just sync the visualizer to a specific BPM then that might be doable. The DJ would have to manually adjust the visualizer system to a song, but finding out the BPM of a song is not a difficult task. If you needed to, you could always "wing it". *shrug*