What are the current limitations of SL?

Hi, i apologize in advance if this in the wrong section. I'm trying find out information about the current limitations of SL such as custom content creation, custom character/avatar developement and character/avatar animation. A link to the relevant documentation will be highly appreciated.

The kind of questions i need to find answers to are:

What restrictions/limitations are there in loading detailed motion captured data (such as head scans and facial animation), if so what workarounds could there be?

Would streaming real-time motion capture be feasible?

Is there a way to incorporate hand animation?

What limitations are there on texture mapping if any?

This is related to a live concert in SL but at the highest level of detail and flexibility as possible.

I can't help you, but darned if you don't have my curiosity piqued.

As long as the output of said capture is a BVH of the right model (avalable in the downloads secion of the SL website) then it'll work just fine. Just keep in mind that anything higher "resolution" than 30 FPS is likely to be lost--most people are running under 20 fps these days.
(I am unsure if animations are played at 30 fps (regardless of created fps, such that anything created at 60 fps plays "half as fast", or their created fps. All frame-dropped/interpretted to the client's overall fps.



Probably not. Streaming media currently is audio (only) or video (with sound).



I believe that the fingers of the avatar are in a fixed position.



Um....depends on the object. Flat faces get a planear mapping (per face), sphere use spherical, cylindars cylindrical, etc. I think there are a few more options other than those, but I haven't messed around with it.



Best bet is to stream a video of things, model anything that doesn't change (use as many prims as possible for more detail, but don't worry about things such as rivets and the like--that can be done via texture and no one would be able to tell the difference).

Motion capture the band and upload that as animations for a non-live performance afterwards.

It is possible to stream 'live capture media' into SL IF:
You have access to the right equipment such as streaming servers and broadcasting servers/software Unicast/Multicast etc, coupled with deep pockets. However, the term 'live' isn't as real as it appears, there will be a minimum 10-15 second delay between what is captured and what can be rendered ingame.

Does it work for animations though? I've never seen a way to stream anims...

Our company has the technology for real-time motion capture so if it is indeed possible to setup a server for streaming in second life we would be highly interested to know more information about it, please could you (or anyone) elaborate and/or provide me a link to any documentation. Thanks in advance.



I meant in terms of resolution of textures.



Could you elaborate please? i'm not sure i understood you correctly.

@All, ideally we would like to import head/body scans + high-res textures with motion captured animation (possibly in (almost) real-time) applied. Instead of trying to make avatars of the band look like them by modelling in SL. However it looks as though one cannot use imported meshes for avatars or is this incorrect?

if it isn't possible to use head scans for avatars this isn't to much of an issue.

Thanks to all for the replies, please keep them coming, and again links to what technical limitations of second life are would be highly appreciated since i can't find much information on the topic

I don't think you could stream motion capture data into SL.

You can upload motion capture data, but you have to upload it a single "animation" that the avatar can run later. Moreover it costs L$10 every time you do it.

In theory there is no limit, but as the size increases download time rises. 1024x1024 is the largest texture in general use.



Spherical mapping is how you'd apply a flat picture of the earth to a globe

http://www.cgl.uwaterloo.ca/~b8chan/cs488/texture.png
Left to right:
spherical, cylindrical, cubic, planear.



Correct. The avatar mesh is the avatar mesh. You can hide it distort it and make it do funny things, but you can't replace it.

If you are looking to interlude head shots of the band lip syncing to the song, try CrazyTalk, but you would have to make those movie head shots ahead of time, save the movie snippets and interlude them real time, which at best, would be virtually impossible to get the timing right. CrazyTalk does a great job of creating the illusion your avatar is actually singing, although, this is not in realtime with streaming live.

You apply a 2d pic of your head onto the base mesh, link the moving elements of the facial features (mouth, eyes, etc) and load just the voice element of the song, and the programme syncs movement with the track.

Potentially, the 'easiest' way to do this, would be to shoot a whole movie of the band in SL, use CrazyTalk to create closeup shots of members 'singing' using CrazyTalk, edit those into the movie sequence, and show the finished movie on a screen behind the band, when they sing their songs in front of the live audience, at a later date.

We at WBA already have experimented with this, not with 'live bands' but with a fashion show, using our scritped catwalk system. This syncronises the models on the catwalk, so we can better predict movements and timings. Providing a model doesnt crash, it gives the illusion of a 'live' show being filmed. Of course we can broadcast this movie across SL at the same time, for all those that can't get into the sim to watch the show, using the Silver Stream Network system.

I'll just drop in my ramblings incase they're of some use.

It would be possible to build an avatar using prims (primitives, the building blocks of Second Life) which could be controlled using scripts (LSL, Linden Scripting Language, can control position and rotation of the prims(along with a lot lot more)). If these scripts were in communication with your motion capture system (you have a few options on how to communicate but I expect that the fastest method would be a client hack to "chat spam" values into SL via a dummy agent. Failing that you have E-Mail, HTTP-Request and XML-RPC, though each of these have time constraints) then the prims could follow the movements of your RL band/artists.

There would be quite a few technical limits to this. Animation of the prims from motion capture data alone would be quite a project. Another project would be modelling an avatar (not an agent's representation but a prim-model), perhaps with sculpties to get areas of detail, is no small feat. Also you'll need to overcome the communication issues I mentioned above. Finally, you'd probably want to animate (or feed live) the faces of the artists, this could be done with the parcel media but on-the-fly UV mapping and having a dedicated camera trained on each artist etc etc would be a hell of a lot of work.

I'm sure there are other pieces to this puzzle I've overlooked. As you can see it's one hell of a project and would take a dedicated group quite a while to put in place. This is only one (crazy) option, you'll find that there are a lot more available to you but this is all I could come up with to answer your requirement of using motion capture.

Another is the inherint slowness of faking a bone hierarchy in SL. Someone did a demo, and the sign on a "bendable pole" updates its position about once every second or two.

Carefully done with more than one script per "bone" you could get slightly faster movements. You could take as much of the hard work away from the scripts as possible (bump it all on the computer(s) doing the motion capture / communicating). I recon with that approach your main time limit would be communication.