Making virtual more real with new webcam technology

Hello everyone; I'm new to SL & have been lurking in the forums... I wasn't planning on posting for awhile, but I just saw a demonstration of new webcam technology that I humbly think could be tied into SL very effectively.

Before I go on, I am *NOT* endorsing any product whatsoever; I am merely pointing out cool new technology.

Here's the link demonstration: http://i-am-bored.com/bored_link.cfm?link_id=17226

Watch the part where the cam tracks the guy's face and animates the avatar's expression according to his real-life expressions. Wouldn't it be great if we could tie such a cam's ability to "communicate" facial expressions to SL avatars? And if we added voice chat to that, walking around in first-person view in SL would be very much like in real life... we can see the person's REAL expression while talking in voice instead of text.

I hope all the above babbling makes some sense; I imagine one big obstacle could be propriety cam software, but I don't see that as insurmountable if a company producing video cams with this technology was approached in the proper manner.

Anyways, it's great to be here in Second Life, and I hope I'm not out of line with this suggestion.

It makes me wonder if you can stream video using the current system via scripts.
I know you can attach avi videos as animated textures.
If there was, then you could, stream your webcam chat, with the system your talking about. That would be fun. However, I doubt it's possible at this point or you would be constantly seeing pervs genitals in real-time mapped all over the place.

I also wouldn't mind seeing an addition that uses audio recognition to apply mouth shapes automatically to match the phonems. There's alot of software out there that does this already. So any pre-recorded or streaming voice data could be applied through this system to make your avitar look like it is talking.

No, and I think I am going to do what it takes to see this technology through into Second Life. Lately I've been taking the charge of pushing my Second Life experience to a level I like; insane. Although, I am working on a major project, I am going to find this camera and deconstruct an idea to bring this technology into Second Life.

Although the system, I would gather, must all be rendered in the Logitech software package, I've got a friend who works for the manufacturing devision, so I might be able to see where they got the technology from.

I'm really excited to see this happen, and I would love to use my skills to help this along!

.fox

Well, I'm glad if this idea does become an exciting part of Second Life, as long as nobody uses their cam to animate huge genitals around me. Er, near me.

Another downside, besides the in-your-face real-time (um, male) genitals thing, that I can think of, is that most people in SL like the other-worldly separation. People seem to avoid RW pictures of people, as it breaks the illusion of the avatar's character and doesn't balance well with the look and feel of most sims. Those who roll-play, particularly, might not like it, unless they use it to give themselves silly costumes on camera. Many other's just like to remain anonymous. However, I think that many would find it fun and useful, if the option was there, and those that don't like it don't have to use it. The people who use it for tele-conferencing might enjoy the option. You could have a tv for a head that shows your real-face as you talk, that's assuming you could stream audio as well as video. This all would likely cause serious lag, and may not even be feasible. The video texture maps work because they get pre-downloaded and cached for smooth playback (I would imagine). Just trying to be realistic about our expectations, and the exceptance of this addition. The "new level" of interaction that SL has opened up to us presents many creative possiblities. It seems to be more than just a "sim" or "role-play" to many; it's about pushing the envelope. Many of us see this system as the catylst for pushing the entire web interactivity into 3 dimensions as well as the many other creative communication possiblities it offers. Such as the combined Real-World/Virtual World Overlay idea presented here. /13/a7/102758/1.html
I for one, think it's very cool that you want to pursue this thing Fox. Good Luck! and thanks for the post Buck.

There's rather strong polarization between those who see us building the precursor to a new electronic existence, and those who think of it as no more than another place onto which to project the old physical world. There is no way of reconciling these two opposite extremes since they exclude each other, so in due course we'll have entirely separate virtual worlds reflecting the desires of the different communities.

At present though, in this single uniform world of Second Life, coexistence is forced upon us, so the simple answer is: Sure, why not, import your physical world goodness or badness, and let's see what happens. Those who don't like it will move out of your area, or abandon SL altogether if they think it is lacking the virtuality which they prize.

These are experimental days. Do it, and we'll see how it works out.

Hey, Fox. Perhaps one way around the lag and to promote confidential interaction would be to allow only groups of individuals to enter webcam chat. This way it does not lag down the whole sim , only those entering streaming video chat mode. The processing happens directly between their systems.

Going back to "Snow Crash", people whose avatars were simply direct projections of themself in the metaverse were considered cheap and tacky.

I think I'd feel that way, unless they really went overboard with some kind of "werebot" shape, like a dressmaker's dummy with a TV for the head that had their face projected on it, something creative like that...

This is a cool idea...
Controlling virtual characters: The AAM Mickey - uses opengl
Using the AAM-API to convert speech to face movements
http://www2.imm.dtu.dk/~aam/

There seems to be a ton of information online....(google "facial tracking"

http://synapse.vit.iit.nrc.ca/doc/facetracking.html
http://students.washington.edu/xdsong/index_files/Courses/Computer%20Vision/Project%204/Experiments.htm
http://students.washington.edu/xdsong/index_files/Courses/Computer%20Vision/Project%204/index.htm

Face-tracking as an augmented input in video games: enhancing presence, role-playing and control
http://portal.acm.org/citation.cfm?id=1124772.1124936&coll=ACM&dl=GUIDE&type=series&idx=1124772&part=Proceedings&WantType=Proceedings&title=Conference%20on%20Human%20Factors%20in%20Computing%20Systems
.
.
.

Yes...some of us come to SL to escape our first ones...First Life and Second Life are to remain (as) seperate (as possible).

Just my two cents. I know I would never use this feature, the boys already can't handle my av, I'd hate to see what they'd do when I attach a real face to that...

Here's an open source program that does something similar, although a bit more primitive.

http://www.fuzzgun.btinternet.co.uk/rodney/humanbody.htm