automatic animation tagging
python / hydra / motionbuilder / deep learning
This system consists of two components: the core Hydra app that computes tags for the motion file inputs, and a MotionBuilder plug-in that visualizes the computed tags.
The Hydra app takes BVH input and outputs frame-level (e.g., gaze direction) and sequence-level (e.g., action type, emotion) tags. The tags are calculated using a mix of algorithmic and deep learning techniques.
gaze direction and dominant hand visualizations (3d view).
The MotionBuilder plug-in allows the user to load in the BVH, then visualize the tags calculated for it in 3D view as well as in a timeline view. For example, calculated gaze direction is visualized as a lookAt in 3D space, with a camera attached to the character's head to confirm the calculated values. Foot contact and trajectories are respectively visualized as color-coded markers (like footprints) and splines in 3D space. Foot contacts are also visualized as color-coded time spans in an interactive timeline editor that allows users to view, edit, load, and save labels.
foot contact and trajectory visualizations (3d and timeline view).
animation toolkit
python / motionbuilder
This system provides a CLI interface for users to allows users to retarget animations and convert between file types (smpl/fbx/bvh), in batches.
non-humanoid animation generation
python / blender / reinforcement learning
This Blender plug-in takes keyframes as input and populates Blender with generated physics-based trajectories that adhere to the given frames. On the back end, the trajectories are simulated then scored based on their adherence to the keyframe input. The top scores are returned to the user.
wildlife motion matching data pipeline
python / houdini / motionbuilder
I designed and developed a system that takes sparse wildlife locomotion animation files (i.e., if mocap is not available and/or handmade animations are insufficient) as input to output an animation database suitable for motion matching.
This multi-stage process involves cleaning, stitching, augmenting, validating using Python, Houdini, and MotionBuilder.