The Kinect Module

The Kinect Module Interface, here you can see some of the hitbokes around the tracked person, the boxes are set to light a different colour according to the limb which triggers them, along with sending a custom OSC string to the desired IP address

This module allows a performer to trigger cues based off Microsoft Xbox’s Skeletal tracking system. This, when combined with the unity engine and the OSC backtalk module can be used to fire present OSC strings when the performer enters a virtual hitbox.

There are two types of hitbox currently, firstly there are static hitboxes. These are in a set point in the virtual environment and are triggered when the actor touches them, an example of this would be one on the floor triggered when an actor steps on it. The actor would be able to walk around it on all sides, but the string would only be activated when they step on it.

The second type is mobile hitboxes. These are attached to the actor, and move with them, meaning that wherever they are in the virtual environment the action is the same. For example, if the desired gesture is moving their right hand above their head, it does not matter where they are in the cameras field of vision, or if they are sitting or standing. The trigger for the cue is the same regardless.

The hitboxes can be made to be sensitive to what part of the body is triggering them. And the same hitbox can be used to trigger different things according to what is triggering it. Currently the system can only track one person at a time.