Hi all, Gee! This is quite a thread! Thanks for all the suggestions, ideas and passion. As always, this community shows it's full of talented (and opiniated) people :)
Since I'll be the guy tagged with driving/mentoring the intern on this project on a day to day basis, I'm taking on summarizing and answering some of the questions that surfaced in that thread (vaguely in chronological order): - Why OpenCV? As pointed out by folks, it may look like overkill for the goals currently stated but those are goals *a minima*. We sure hope to expand from this first experimentation since, it is indeed an experimental project. Jan Ciger confirmed though that, if the goal is to capture a wide set of face and body elements, OpenCV is a good starting point. - Emulate mouse/joystick driver? Several folks mentioned implicitly the parallel between capturing elements and emulating yet another kind of input device. The problem with mice or joysticks is that they have only one or few effectors and that those do not map on avatars controls. With a camera, we certainly want more than few effectors. Eventually, we'll rapidly grow to the point we want VR like richness for that input. VRPN in that respect is an interesting idea (thanks Jan again). Besides, we're not currently trying to "drive" the interaction (i.e. building prims or triggering camera movements and avatar navigation) but rather *add* to the currently rather canned (and unused) avatar animations (aka gestures) so to see how much of the RL emotional state we can share with other residents. IOW, we want to improve and add to the current experience, not replace or use as an alternative to keyboard, mouse, etc... - Gaze tracking: Often mentioned so that's certainly a misnomer in our case: we'd like to track "what the user is looking at" rather than "what the mouse is pointing to" (as it is today) but not to control the interaction as is the case with gaze tracking as understood in the field. Just share eye attention between residents. It'll be perfectly inconsequential then if the gaze of the user change to look outside the scene (or the screen), we'd just switch back to the current attitude and animation - VRwear (mentioned by Aimee): In my former incarnation, I shared a panel on Orange Island with those guys discussing avatar animation and puppeteering. Their video is great but, last I tried (more than 6 months ago) I couldn't get the demo to work at all (other experiences?). Note that apparently they've been modifying the viewer directly which is not a bad idea if you want to get results fast and put something in the hands of users (that's what I did with HandsFree3D). Developing a protocol (using RPC or UDP or the already existing VRPN) is more open and would actually allow those guys to focus on the image analysis and extraction rather than spending time tracking the various viewer releases. Although we would provide a first widely available incarnation of this, the architecture shouldn't prevent others (including commercial firms) to come up with their own wares (hardware and software) promising a better experience (may be specialized on some aspects: face, body, emotion, whatever...). There are a lot of players out there (GestureTek, Softkinetic, Mamoca to name a few) not to mention again the ones pointed to in that thread already. - Puppeteering: Since it's mentioned in the thread, just reminding people that I've been interested in that since some time (http://www.handsfree3d.com/ ) and provided a patch to fix the crashing puppeteering branch (VWR-7703). Let's just say that full body puppeteering is great, feasible in SL, but there are a lot of non-trivial problems, far trickier than driving a bunch of effectors for a game. Rigging (i.e. the mapping of user's movement to an avatar that is geometrically different) in particular is a pretty big (and pretty interesting) problem which is very much SL specific. - What are "gestures"? We were really thinking about "animations" ("gestures" are they are called in SL), i.e. triggering those without requiring the user to enter keyboard actions. This indeed would not require any server change as pointed out by Melinda. Sorry for the confusion Jan. Stopping there for now but we'll sure be coming with more info in the coming days. Cheers, - Merov _______________________________________________ Policies and (un)subscribe information available here: http://wiki.secondlife.com/wiki/SLDev Please read the policies before posting to keep unmoderated posting privileges
