Paper: PDF, 600dpi images (20 MB)
Code: GitHub
Supplementary material: User Study statistics (32 KB PDF)
Frames of a recolored video and slices of its polyhedral palette.
Abstract:
Color correction and color grading are important steps in film production. Recent palette-based approaches to image recoloring have shown that a small set of representative colors provide an intuitive set of handles for color adjustment. However, a single, static palette cannot represent the time-varying colors in a video. We introduce a spatial-temporal geometry-based approach to video recoloring. Specifically, its core is a 4D skew polytope with a few vertices that approximately encloses the video pixels in color and time, which implicitly defines time-varying palettes through slicing of the 4D skew polytope at specific time values. Our geometric palette is compact, descriptive, and provides a correspondence between colors throughout the video, including topological changes when colors merge or split. Experiments show that our method produces natural, artifact-free recoloring.
Video (5 minutes, contains audio) MP4 (62 MB):
Presentation (4-minutes) Recording (MP4, 28 MB) | PowerPoint (96 MB):
Presentation (17-minutes) Recording (MP4, 85 MB) | PowerPoint (85 MB):
BibTeX (approximate):
@article{Du:2021:VRS, author = {Du, Zheng-Jun and Lei, Kai-Xiang and Xu, Kun and Tan, Jianchao and Gingold, Yotam}, title = {Video Recoloring via Spatial-Temporal Geometric Palettes}, journal = {ACM Transactions on Graphics (TOG)}, volume = {40}, number = {4}, year = {2021}, month = aug, keywords = {Palette, Color, Grading, Painting, Image, Video, Spatial-temporal, Recoloring, Layer, RGB} }
This work was supported by the National Natural Science Foundation of China (Project Numbers: 61822204, 61521002, 61863031), the United States National Science Foundation (IIS-1453018), and a gift from Adobe Systems Inc.