Decomposing Images into Layers via RGB-space Geometry

Jianchao Tan, Jyh-Ming Lien, Yotam Gingold
ACM Transactions on Graphics (TOG), November 2016.

Paper: PDF (83 MB) | PDF (9 MB)

  • Section 3.1 Simplifying the convex hull: Rather than updating our mesh data structure when collapsing the edge, we recompute the convex hull of the mesh's vertices without the two edge endpoints and with the collapsed edge vertex. This ensures that the simplified mesh remains convex.

Presentation: Keynote (146M) | PDF (42M)

Interactive recoloring GUI, layer decompositions, and additional examples: Zip (76 MB)
Additional interactive recoloring GUI examples: Zip (281 MB)
Code: GitHub

Interactive RGB-space visualization: in-browser demo (drag-and-drop any image)

An image; its pixels and simplified convex hull in RGB-space; and its layer decomposition. Artwork © Karl Northfell.


In digital image editing software, layers organize images. However, layers are often not explicitly represented in the final image, and may never have existed for a physical painting or a photograph. We propose a technique to decompose an image into layers. In our decomposition, each layer represents a single-color coat of paint applied with varying opacity. Our decomposition is based on the image’s RGB-space geometry. In RGB-space, the linear nature of the standard Porter-Duff “over” pixel compositing operation implies a geometric structure. The vertices of the convex hull of image pixels in RGB-space correspond to a palette of paint colors. These colors may be hidden and inaccessible to algorithms based on clustering visible colors. For our layer decomposition, users choose the palette size (degree of simplification to perform on the convex hull), as well as a layer order for the paint colors (vertices). We then solve a constrained optimization problem to find translucent, spatially coherent opacity for each layer, such that the composition of the layers reproduces the original image. We demonstrate the utility of the resulting decompositions for recoloring and object insertion. Our layers can be interpreted as generalized barycentric coordinates; we compare to these and other recoloring approaches.

BibTeX (or see the ACM Digital Library entry):

 author    = {Tan, Jianchao and Lien, Jyh-Ming and Gingold, Yotam},
 title     = {Decomposing Images into Layers via {RGB}-space Geometry},
 journal   = {ACM Transactions on Graphics (TOG)},
 volume    = {36},
 number    = {1},
 month     = nov,
 year      = {2016},
 articleno = {7},
 numpages  = {14},
 pages     = {7:1--7:14},
 issn      = {0730-0301},
 url       = {},
 doi       = {10.1145/2988229},
 acmid     = {2988229},
 publisher = {ACM},
 address   = {New York, NY, USA}

Copyrighted artwork:

Funding: This work was supported by the United States National Science Foundation (IIS-1451198, IIS-1453018, IIS-096053, CNS-1205260, EFRI-1240459, and AFOSR FA9550-12-1-0238) and a Google research award.