This document was written in early 2011 for our animation and gameplay teams to do better visually integrated Kinect mechanics in Kinect Star Wars, Kinect Pixar Rush, and Kinectimals. I was an animation, graphics and Kinect interaction engineer on the NUI Publishing team at Microsoft Studios which produced these titles. ''Lewey Geselowitz, 2015

Kinect Augmentation Principals and Practices

Key techniques and technical notes for interactive character augmentation, adapted from Disney's Principals of Animation (described in The Illusion of Life), with examples and explanatory comments.

Essential Points

' Anticipation ' Ensure purposeful posture via constant anticipation, positioning the body to assist augmentation.

' Exaggeration ' Never 'drag' the avatar, all augmentations must exaggerate, adding extension and/or stress of some kind.

' Slow-In & Slow-Out ' Animate the rest of the body, and then the snap, to express weight through timing.

Definitions and Practices''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''

' Anticipation

' Augmented posture should visually hint, emphasize and make clear the movements which can be expected from that position. Every pose should be telling something, Anticipation is what it's telling.

' A sword held up and to the right in a ready position, should result in the avatar's shoulders slightly coiling backwards as though preparing to strike. Essentially fore-shadowing the movement, conveying the wind-up, and providing a better position from which to augment the swing.

' Leap, dash and step should be anticipated by subtly compressing the weighted leg, raising the free leg, pulling the torso over the weighted leg, and slightly extending the opposite arm; decided based on when balance is unevenly distributed between the feet.

' Note that these adjustments should not begin simultaneously, but should rather be started as offset sequences in the same order and timing as felt in the body (see Overlapping Motion).

' Exaggeration

' Believable representation of someone's own body requires visual exaggeration to match the intensity of direct personal perception.

' Reinterpretation of the user's hand position should result in an avatar hand which is both slightly more extended, and an arm posed with slightly more stress, such as elbow tilted higher than the users.

' Lack of exaggeration will be experienced as 'dragging' the avatar, while correct exaggeration will be received as 'responsive' and 'empowering' to the user, keeping their focus on the more active game, rather than on their own movements having to pull the game into shape.

' All mapped poses should exhibit this exaggeration behavior singularly, as well as when combined through weighted blending. This requires a consistent method of exaggeration for each blend set to avoid unnatural combinations.

' Note that Anticipation, Staging and other techniques applied to the rest of body can be used to achieve exaggeration in addition to linear extension.

' Follow Through & Overlapping Motion

' 'The body itself does not move all at once, but instead it stretches, catches up, twists, turns and contracts as the forms work against each other.'

' Swinging hard to the left from the right side should be seen sequentially as the hips rotating, the left shoulder pulling back, the right shoulder driving forward, the hand traveling left, the spine twisting and folding over, with the torso forming an obvious rotation about the axis of the weighted leg.

' Reference animation should be authored with either instantaneous exertion, or with offset event markers, so that the game can procedurally apply the delayed and prolonged movements. These procedural offsets should match the users input to when their exertion changes, and should provide continuity with the previous state and any connected masses otherwise.

' Follow-through is largely achieved through complimentary anticipation poses on either side of a large movement. When the right sword hand is held to the left of the body, the right shoulders should coil up and back, with the left hip edged forward, anticipating the swing to the right. In this way, a swing from the right ending at the left will automatically result in a more drastic body twist even if the motion itself is not directly augmented.

' Slow In & Slow Out

' Mass has a tendency to resist (build-up) and retain (follow-through) motion.

' During a powerful right hand sword swing to the left, the sword hand should be delayed behind the user's hand position, while the right hip twists into the swing providing continuous feedback to the user, distracting them from the sword which is building up, and then snapping into place as the motion ends.

' Providing continuous overlapping feedback in the rest of the body is critical to augmenting the timing on weighted elements.

' Staging

' 'Staging is the presentation of an idea so that it is completely and unmistakably clear.'

' Simplicity of posture and movement can be achieved by aligning limbs and movements to shared axes (lines/planes of action), which lends general consistency and harmony to the composition, removing irrelevancies and aligning the whole body with the message of the movement.

' During a left arm push, the diagonal length of the avatar's body should orient into a near straight line in screen space from the foot being pushed off from. If pushing from the right, the left leg should be folded to compliment the axial line, with the right arm forming either parallel or perpendicular positions with respect to the diagonal axis. If pushing from the left leg, the axis is along the left flank of the body.

' Note that these aren't perfect alignments, but rather target constraints such that the whole body visually moves towards their alignment. Even if they are not reached, the combined effect of motion toward a harmonic goal will lend clarity to the users' experience of the movement.

' Straight Ahead & Pose-to-Pose

' Animation is composed as both high level 'pose-to-pose sequences' across long time intervals (to ensure rhythmic and contextual consistency), and 'straight ahead' physical simulations (to ensure weighty frame-to-frame consistency).

' During a sword swing, posed key frames are required to ensure a powerful composition of the strike and that the general direction of the follow through is achieved, however a simulation should be used to blend back into the players control ensuring a realistically non-linear and unforced transition.

The other principals of animation, along with the finer points of the above, apply directly to augmentation and are expressed much more clearly by the masters of the art such as Richard Williams (of Warner Bros. fame) in 'The Animator's Survival Kit', they are:

' Arcs

' Timing

' Secondary Action

' Squash and Stretch

' Appeal

' Solid Drawing

- Lewey Geselowitz

NUI Publishing