Hyperspectral Inverse Skinning

Songrun Liu, Jianchao Tan, Zhigang Deng, Yotam Gingold
Computer Graphics Forum (CGF), to appear.

Paper: PDF (15 MB)

Coming soon: Code, Videos, Data

Transformation matrices as points in R^(3*4*poses). Given several poses (e.g., animation frames) of a linear blend skinned model, each bone’s transformation matrices across all poses are vertices of a simplex in R^(3*4*poses). Any vertex’s transformation is the weighted sum of the bone transformations. The weights wi of the LBS rig are the barycentric coordinates with respect to the simplex.

Abstract:

In example-based inverse linear blend skinning (LBS), a collection of poses (e.g., animation frames) are given, and the goal is finding skinning weights and transformation matrices that closely reproduce the input. These poses may come from physical simulation, direct mesh editing, motion capture, or another deformation rig. We provide a re-formulation of inverse skinning as a problem in high-dimensional Euclidean space. The transformation matrices applied to a vertex across all poses can be thought of as a point in high dimensions. We cast the inverse LBS problem as one of finding a tight-fitting simplex around these points (a well-studied problem in hyperspectral imaging). Although we do not observe transformation matrices directly, the 3D position of a vertex across all of its poses defines an affine subspace, or flat. We solve a “closest flat” optimization problem to find points on these flats, and then compute a minimum-volume enclosing simplex whose vertices are the transformation matrices and whose barycentric coordinates are the skinning weights. We are able to create LBS rigs with state-of-the-art reconstruction error, and state-of-the-art compression ratios for mesh animation sequences. Our solution does not consider weight sparsity or the rigidity of recovered transformations. We include observations and insights into the closest flat problem. Its ideal solution, and optimal LBS reconstruction error, remain an open problem.

BibTeX (or see the Computer Graphics Forum page):

@article{Liu:2020:HIS,
 author    = {Liu, Songrun and Tan, Jianchao and Deng, Zhigang and Gingold, Yotam},
 title     = {Hyperspectral Inverse Skinning},
 journal   = {Computer Graphics Forum (CGF)},
 volume    = {to appear},
 year      = {2020},
 doi       = {10.1111/cgf.13903},
 keywords  = {linear blend skinning, deformation, animation, affine geometry, hyperspectral unmixing}
}

Funding: This work was supported in part by the United States National Science Foundation (IIS-1453018 and IIS-1524782), a Google research award, and a gift from Adobe Systems Inc.