Proposal: 840 - Improved music & audio capabilities in SL

After 8 months in SL I have come full-circle back to my original reasons for joining: my never-ending search for musicians to jam/play/record music with. The music and audio limitations are so severe that I started Proposal 840 as a remedy. The text of the proposal follows:

Second Life caters admirably to programmers and graphics artists yet has a void of non-implementation as regards music and audio. The 10-second wav clip limit is a handicap beyond frustration. It actually impedes any realistic sound, music and audio implementation in SL. To rectify this dilemma I propose the following cluster of features:

1) The ability to upload & play/trigger keymapped multisamples in Akai/Halion/Kontakt/Sample-Tank/Emu/Gigasampler format. A possible limit could be placed on the maximum size of multisamples sets.

2) Basic editing functions for keymapped multisamples. Sophisitcated editing would not be required as there is a wide choice of off-line editors already on the market.

3) Full MIDI implemetation to allow for workable playing/usage of uploaded multisamples. MIDI can also be used to control non-musical devices such as lighting controllers and other 'club' devices.

4) Multi-way audio conferencing so as to allow musicians in different real-world locales to interact in real-time thru the Second Life medium. This would greatly enhance the live music possibilities in Second Life.

I would ask anyone who is a musician or has any involvement with music to vote for this proposition with as many votes as they can spare.

Seconded. I also believe the ability to change the pitch/playback rate of a sound would be very useful as well.

SL tends to be awfully quiet, that's true. I would be happy to have at least the ability to upload and play mod/xm/it files (which should be trivial to implement for LL since the library they use for sound output already has mod playback).

mod files? As in 'Mod-trackers'? I've heard of them.... crude step-sequencers with embedded 8-bit sound samples. I've seen a few few Mod-tracker sequencers written for PC's even tho they're largely redundant.... obviously a bit of retro-chic going on there.

I like the idea of the midi features.

This would be especially good if the files could be used for controlling things as well as for playing sounds. If it could be used to send out messages to linked objects as well, then it could be used as a means for animating prims.

I would like to add to your list the feature of being able to play part of a sounds file. WIth clever manipulation of the textures, you can show part of an image, but for sounds it is all or nothing.

This could be connected to the midi feature such that you could upload a small sound file with words or others sounds and then the midi could be used to play back part of it such as lyrics to a mixed poiece of music and/or special sound effects.

The "play within one sound file" feature would keep people from having to upload large numbers of smaller sound files and attempting to splice them together (Which SL doesn't do very well, I believe, by design to prevent copyright infringement and huge resource issues).

I'd be happy to be able to play any interested parties music that i had created elsewhere, whether asa MIDI file or an MP3.

The radio stations seem to be a very blunt tool. With streaming content that's particularly inappropriate to the environments.

Jam, MIDI files tend to have 16 discrete channels embedded in them and these tracks point to a selection of up to 16 different sounds usually taken from a waveset library as defined by the General MIDI standard or the XG standard for Yamaha soundcards / instruments / devices. MIDI data can and does control a wide variety of devices other than musical instruments.... all that's required is the ability to interpret the incoming messages.

MIDI files do not contain embbed sounds or wavesamples. This was more common with the old 'mod' files which were common prior to the establishment of the General MIDI standard by Roland Corp. While MIDI is not limited to General MIDI, sounds are never embedded in the MIDI file, but are generated by software and/or hardware synthesizers / samplers / drum-machines which are controlled by the MIDI file. MIDI files can, and often do, contain patch information for specific synthesizers to enable them to create sounds not part of the General MIDI set, but this requires that the listener have a specific make & model of synthesizer.

As for playing specific parts of a sound.. I shall take it that you mean sound effects or environmental / ambient sound, that's another reason why I included keymapped multisamples in my proposal. Keymapped multisamples aren't always $1,000 velocity layer-switched saxophone / violin / grand-piano / 't3h s00p4h l00p4hz' wavesets It is quite a straightforward matter to take a single sample and map a variety of different loops taken from that sample using different start and end points. You would assign each loop to a different key number and 'play' them that way. Similarly, lyrics for a song could be broken up into phrases, keymapped, assigned to MIDI channel NN and triggered from a MIDI file.

If the Second Life audio engine were to contain a generic sample player then all you'd need to do would be to prepare your wavesamples, upload them and then embed the information instructing the sample player to load XYZ 'instrument' on channel NN. If the SL audio engine were to handle MIDI files with embedded audio samples you would actually be creating a new format similar to the old 'mod' file format. Not a bad idea in itself except that it would require all midi player devices to have sample load & playback facilities as well as rendering the previous generations of MIDI synthesizers / players obsolete.

I think the implememtation of MIDI is a great idea, but I'd settle for longer .wav files and "push" and "pull" audio streams.

You are never going to "Jam" with another musician through the internet until you have fiber optic or the equivelent from end to end, with massive switches to handle the data. A 50ms delay is far to much, much less 200-300 which is about as good as you can possible do with dedicated embedded real time audio systems. You need real time, no deleay, for live collaboration, unless you're just layering tracks, which wouldn't be true collaboration to me.

I'd rather see the developement time go into .mp3 or .wav support, though I suspect LL will never do that because they don't want to get targeted by the file sharing nazi's.

The streaming audio feature is severely lacking IMHO until it supports both push and pull type streams.

Some number of years ago the musician Thomas Dolby was trying to get an internet live jam thingy going. This thread reminded me of it so I went to see what became of it and found that they are now producing ringtones for cell phones. Yup, internet time latency will kill collaborative jams, but revelations like that will really harsh your karma.

Look up drum circles to see what can successfully be done within SL. Ace Cassidy pioneered the SL synchronization tech (I think) but there is some other group that was carrying the torch recently. There was a more ambitious tonal use of the platform tried by (I think) Same Song a year or so back but the platform wasn't up to it. Neither Ace or Same are playing SL now so far as I know.

Erik Pasternak, I am aware of the limitations as regards realtime audio networking vis-a-vis live music. However Steinberg did offer the Res Rocket online jamming service back in the days of dial-up internet bundled in with the retail versions of Cubase. MIDI and keymapped multisamples shouldn't be all that demanding as MIDI itself is a very 'light' datastream. And the multisamples would typically not be uploaded during the performance, but at some time prior to the event / performance / recording / jam.

Churning out ringtones for a living? Yikes, that's a fate worse than death, Introvert Petunia. I have come across the drum circle at Elven Glen. It's regularly used tho more as a social hub than as a musical event.

Thank you everyone for your replies, comments and suggestions. I have a sinking feeling that Linden Labs will not budge on the music / audio implmentation for SL. So I'll bide my time until SL's successor comes on stream.

Alazarin, your sig is wrong in my case. I do know you
I've also had that sinking feeling for some time. btw - there are already a couple similar proposals on the feature voting tool. One for MIDI and one for realtime voice conf. Unfortunately, the latency thing is a real killer for real jamming (at least w/ non-midi) - I do see that MIDI /might/ be useful in some circumstances, as I think we discussed, and if LL would simply add a default soundfont w/ the GM spec instruments in it, it might be usable for realtime performance - it's hard to say w/o experimenting, which only LL can do. I think voice chat is likely going to happen at some point, but probably using a codec not suited for music (like ventrillo or teamspeak use).

I'm also quite frustrated at the 10 sec limit for audio (and the .wav req.) it's ridiculous that graphic artists can upload about any kind of image they want, and are trusted that the DMCA will be honored, but musicians/sound designers have to jump through huge flaming hoops to be able to try and monetize their creations - that don't really work well. The people who do not respect copyright do not see this as a real barrier either, as I've seen several instances of "jukebox" singles of current pop hits - made up of crudely strung together 10 sec samples (well, 9.99 sec grr) and currently the way the spacial audio works you just about have to be on top of a sound to hear it (if it's not clipping).

I've seen the same files, and agree that there really is no way to stop thos that wish to use other people's intellectual property for commercial gain, but LL is going out of their way to make sure the file sharing Nazi's can never accuse them of providing a file sharing platform.

Big mistake IMHO. File sharing is one of the # 1 things people do with their time in cyberspace. Porn is the other. LL seems to have given us all the tools necessary to capitalize on THAT now haven't they. This is about law suits. If LL wants SL to be the next "internet", then they need to get serious about digital multimedia support.

LL needs to work something out with the big 5 record labels, kind of like Itunes did, or Napster after the lawsuits. Pay the ASCAP and BMI royalty fees and let people REALLY be entertained in SL.

Get the big 5 involved now, and license the technology to make and distribute music to them and let them charge me to use it, I don't care, just provide it so I can put my own creations, unlicensed, independent music out there for other SL'ers to enjoy if they so choose with out having to pay huge streaming rental fees on top of my tier fees.

I assume the big 5 will sit on thier hands like they did when MP3's came out, by the time someone has implemented a viable way to distribute music in 2L and is making a mint off it, they will come along and sue them and steal their idea again just like they did with Napster.

I still say ISP's should be the one's paying BMI and ASCAP fees so we're free to listen to whatever we want for free and digital multimedia can reach it's full potential, but that's a whole other discussion.

If I remember this correctly, from the talks with older residents, the 10s limit on audio sample wasn't always there. But people would upload whole songs, and even though it's compressed to ogg vorbis, you still have several mb per song. And the asset server wasn't happy. And the Lindens weren't happy with that, so they introduced the sample limit. And the residents weren't/aren't happy.

I still think mod/xm files would work ok, they are relatively small and sound relatively good. Certainly better than what the default midi synth on most computers sounds like.

This is so, so, so out of date that it brought a tear to my eye. 16 and even 24 bit sample capabilities have been in the mod format decendents for almost... 15 years or so now? In fact, most of the newer consumer and professional grade music creation programs such as Fruity Loops, Reason, and even Sony's own Acid Pro are all basically fancy mod trackers (although the resemblence can be hard to realize at first). The main difference being that mod formats (MOD, S3M, XM, IT, etc.) are designed to be played in real-time, while the newer trackers favor audio rendering to a file (wav, mp3, ogg) for later playback, because of massive audio post-processing.

Module formats gradually faded out of the computer enthusiast public eye when MP3 came along (module formats were originally seen as a great way to share music, because people didn't HAVE broadband, and couldn't send huge gigabyte wav files), and making an MP3 took 1000% less expertise then making a module. Sure, there were some tracking enthusiast groups, and some have stuck around even until present day (modarchive.com being a huge example). But something you may not realize though is that until the last year or so, almost ALL music created for video game consoles is sequenced module music. Every Final Fantasy game made so far has used sequenced module music, and they are generally regarded as having some of the best soundtracks in the industry. Even today, a great deal of the games produced today used sequenced music instead of mp3/ogg.

The mod format is alive and well, just in a number of different incarnations.

And that was your random post of the day.

Thanks for that, Zod. I completely missed the 'mod' phenomenon. I was never much of a gamer and went from using hardware sequencers such as the Roland MC-202 to using an XRI Micon step sequencer with the Sinclair Spectrum to MasterTracks Pro on the Atari ST to Cubase VST / SX on the PC. I only really came across references to the mod format long after I migrated to using Cubase, a program that essentially turns your computer into an all-singing all-dancing portastudio.

One of the reasons it would be nice is because most off-the-shelf audio engines (modplug, fmod, audiere) support mod/s3m/xm/it etc. out of the box with maybe about 2 seconds of additional code, and the typical mod size ranges from 100K to 1MB (usually less)... that might be a reasonable size to allow uploading, will still allowing inclined designers to make music. It's very, very likely that SL is using one of these engines (probably FMOD?), so implementing it would take very little time. SL... is quiet. Unless you happen to like Club977 radio. :/ ALL THE TIME.

AFAIK, SL does use the FMOD engine. I came to this conclusion cos the Shoutcast server that I use to broadcast & promote my recordings keep a log of the IP addresses & applications that it streams to..... and FMOD comes up on the list very frequently. I've been able to correlate the start & stop times with people Ive been chatting with in SL, so it's a pretty good bet that SL uses FMOD.

fmod.dll 3.7.4.0 to be precise.