Eurographics 2021: [YouTube], [Keynote (70 MB)], [PDF (20 MB)], [PDF with notes (20 MB)]:

In example-based inverse linear blend skinning (LBS), a collection of poses (e.g., animation frames) are given, and the goal is finding skinning weights and transformation matrices that closely reproduce the input. These poses may come from physical simulation, direct mesh editing, motion capture, or another deformation rig. We provide a re-formulation of inverse skinning as a problem in high-dimensional Euclidean space. The transformation matrices applied to a vertex across all poses can be thought of as a point in high dimensions. We cast the inverse LBS problem as one of finding a tight-fitting simplex around these points (a well-studied problem in hyperspectral imaging). Although we do not observe transformation matrices directly, the 3D position of a vertex across all of its poses defines an affine subspace, or flat. We solve a “closest flat” optimization problem to find points on these flats, and then compute a minimum-volume enclosing simplex whose vertices are the transformation matrices and whose barycentric coordinates are the skinning weights. We are able to create LBS rigs with state-of-the-art reconstruction error, and state-of-the-art compression ratios for mesh animation sequences. Our solution does not consider weight sparsity or the rigidity of recovered transformations. We include observations and insights into the closest flat problem. Its ideal solution, and optimal LBS reconstruction error, remain an open problem.

Much of the data can be found in the GitHub repository. Some data in convenient formats are snapshotted here.

- Ground truth input meshes animations (mostly from previous work) [442 MB zip]
- Ground truth linear blend skinning rigs for our new ground truth examples [26 MB zip]
- Output OBJs recovered by our approach [355 MB zip]
- Subset of output OBJs recovered by Le and Deng [2012] [65 MB zip]. (All output is in the GitHub repository in their TXT format.)
- Output OBJs recovered by Luo et al. [2019] [602 MB zip]

These videos show our algorithm's reconstructed output next to the input and Le and Deng [2012]'s SSDR technique (without rigidity constraints). We render with flat shading to accurately portray surface quality.

These videos all visualize our bi-quadratic flat optimization on the cylinder example, which has a known ground truth solution. (See Figure 10 in the paper.) There are four bones, each the vertex of a tetrahedron, so the desired handle flat is 3D. Each point is shown in the visualization is the closest point on the handle flat to each vertex's flat. Because the orientation of the visualized 3D space is arbitrary, (obtained via PCA from the ambient 48-dimensional space), we apply a procrustes transformation between adjacent iterations.

Two videos show optimization from a random initial guess. This optimization never achieves the known ground truth and never stops changing, even after 10451 iterations. One video shows all 10451 iterations at 50x real-time. The other video shows the beginning iterations at a slower speed (4x real-time).

- biquadratic optimization from random initial guess (10451 iterations at 50x speed).m4v
- biquadratic optimization from random initial guess (beginning iterations at 4x speed).m4v

One video shows our optimization from our proposed initial guess (the 50th percentile case in Section 4.1, Table 2). It quickly converges to the ground truth within 20 iterations.

Another video shows our optimization from an alternate initial guess (the 10th percentile case in Section 4.1, Table 2). It eventually reaches the ground truth after ~300 iterations.

A video of flat-flat optimization iterations for the scenario in 3D in which the given and unknown flats are lines (1D). There is a known zero-energy solution. The optimization algorithm is the explicit p,B parameterization, optimized using the Hessian-based trust region solver constraining B to lie on the Grassmann manifold. The optimization quickly reaches the known solution.

@article{Liu:2020:HIS, author = {Liu, Songrun and Tan, Jianchao and Deng, Zhigang and Gingold, Yotam}, title = {Hyperspectral Inverse Skinning}, journal = {Computer Graphics Forum (CGF)}, volume = {39}, number = {6}, pages = {49--65}, year = {2020}, doi = {10.1111/cgf.13903}, keywords = {linear blend skinning, deformation, animation, affine geometry, hyperspectral unmixing} }

This work was supported in part by the United States National Science Foundation (IIS-1453018 and IIS-1524782), a Google research award, and a gift from Adobe Systems Inc.