Decomposing Time-Lapse Paintings into Layers

Paper: PDF (28 MB) | PDF 600dpi (5 MB)

Presentation: Keynote 6 (142M) | PDF (53M) | M4V (166M)

Videos and other supplementary materials: Zip (130 MB)

Code: GitHub

Time lapse video © Marcello Barenghi.

Abstract:

The creation of a painting, in the physical world or digitally, is a process that occurs over time. Later strokes cover earlier strokes, and strokes painted at a similar time are likely to be part of the same object. In the final painting, this temporal history is lost, and a static arrangement of color is all that remains. The rich literature for interacting with image editing history cannot be used. To enable these interactions, we present a set of techniques to decompose a time lapse video of a painting (defined generally to include pencils, markers, etc.) into a sequence of translucent “stroke” images. We present translucency-maximizing solutions for recovering physical (Kubelka and Munk layering) or digital (Porter and Duff “over” blending operation) paint parameters from before/after image pairs. We also present a pipeline for processing real-world videos of paintings capable of handling long-term occlusions, such as the painter’s hand and its shadow, color shifts, and noise.

BibTeX (or see the ACM Digital Library entry):

@article{Tan:2015:DTL,
 author    = {Tan, Jianchao and Dvoro\v{z}\v{n}\'{a}k, Marek and S\'{y}kora, Daniel and Gingold, Yotam},
 title     = {Decomposing Time-Lapse Paintings into Layers},
 journal   = {ACM Transactions on Graphics (TOG)},
 volume    = {34},
 number    = {4},
 month     = jul,
 year      = {2015},
 articleno = {61},
 pages     = {61:1--61:10},
 numpages  = {10},
 url       = {http://doi.acm.org/10.1145/2766960},
 doi       = {10.1145/2766960},
 publisher = {ACM Press},
 address   = {New York, NY, USA}
}

Copyrighted artwork: Marcello Barenghi (yellowapple, rose, eye, egg, cube, cola, candy), Matyáš Veselý (graffiti), Will Kemp (lemon), Dani Jones (scrooge), and Sycra Yasin (woman).

Funding: Authors Tan and Gingold were supported in part by the United States National Science Foundation (IIS-1451198 and IIS-1453018) and a Google research award. Authors Dvorožňák and Sýkora were supported by the Technology Agency of the Czech Republic under the research program TE01020415 (V3C – Visual Computing Competence Center), by the Czech Science Foundation under research program P202/12/2413 (OPALIS), and by the Grant Agency of the Czech Technical University in Prague, grant No. SGS13/214/OHK3/3T/13 (Research of Progressive Computer Graphics Methods).