Prop: 203 - Support local devices beyond mouse and keyboard

One stumbling block to really expanding the types of games we develop in SL is the limited kinds of input SL accepts from the user's computer. Prop #203 would help in that regard, by supporting the ability to integrate other forms of input into SL beyond usual mouse and keyboard, e.g. steering wheels, foot pedals, game console USB swords, MIDI input devices, VR gloves, and my own personal favorite, biometric devices such as skin conductivity detectors, heartrate detectors and EEG input for a truly immersive "metaverse" experience.

Essentially we'd need a client-side API and SDK to allow us to write drivers to process the data from these devices and pass it in using a format that can be captured by LSL scripts and used to drive scripted reactions in-world such as movement or other effects. XML-RPC can be used, but is horribly slow (about one packet per 2-4 seconds), not nearly fast enough to drive a game. The URL posted above has a good description of what would be required. Another forum thread discussing this is here.

This would also tie in to proposal #37, expand llTakeControls, and would include proposal #81, the delightfully creative (now abandoned) idea to add excercise bike input to SL as a movement mechanism. However, if the decision is made to XML-ify the client or to add a generic client-side API, this would be subsumed in that proposal. Alternatively, if the latency on inbound XML-RPC could be greatly reduced, we could work with that interface.

I figured people interested in SL games might be interested in knowing about this proposal.

neko

A better SL protocol to support open control data would be Open Sound Control:
/13/1d/36159/1.html

Control Protocol

--------------------------------------------------------------------------------

I'd like to broaden this discussion out a bit and focus on an SL implementation of a flexible control/data protocol that could really enable all sorts of interesting interfaces to SL.

Open Sound Control is a good example of one.
http://cnmat.berkeley.edu/OpenSoundControl/

"OpenSound Control ("OSC" is a protocol for communication among computers, sound synthesizers, and other multimedia devices that is optimized for modern networking technology"

It's basically a simple message format using either stream or packet based transport (TCP or UDP) that can be configured for whatever the device or application need.



I'd be interested to hear your thoughts.


Best,
komuso

See also: /165/10/45385/1.html

Right, Komuso. As you and I have discussed earlier, OSC is already used for things like VR gloves with other games and development environments, so it could work really well for this proposal. In the proposal itself I haven't tried to specify how the functionality would be provided, only asked for the functionality we're looking for. OSC sounds pretty good to me.

neko

As we talked about, I've done some prototypes using the Essential Reality 6dof P5 glove to control both games and new musical instruments, as have many other developers/musicians.

I think it's important to at least point LL towards some possible open standards type solutions that have already consided the whole interaction possibility space, rather than just generic feature requests.

Control interfaces are a hot topic, and combined with the move towards the wireless interaction environment we are starting to see all sorts of innovations on the game/application control interface front.

Prop 203 is up to 69 votes now, which is three times what it had last week. However, we've got a ways to go to get it up into the range of likely candidates for a near future release. If you're interested in being able to support joystick use for those Atari type games being discussed in the other thread, vote for Prop 203!

neko