CraGL: Computational Reality, Creativity and Graphics Lab

Deep Sketch Vectorization via Implicit Surface Extraction

Chuan Yan, Yong Li, Deepali Aneja, Matthew Fisher, Edgar Simo-Serra, Yotam Gingold
ACM SIGGRAPH North America 2024 (Journal Papers). Also in ACM Transactions on Graphics (TOG) 43(4).

Paper

Poster

Supplementary Material

Code

From left to right: A line drawing labeled A of a girl holding an umbrella over a grey background. The image is labeled Raster line drawing. An arrow labeled Neural inference. An image labeled B of the same girl with overlaid dots and small highlighted regions. The image is labeled Dual contouring, Key points, Under-sampling map. Inset is a small image of three bumpy lines meeting in a highlighted region with blue dot in the center. Inset is also an arrow labeled Automatic refinemenet pointing to a small image of the same lines with the bumps removed. An arrow labeled post-processing. An drawing labeled C of the same girl holding an umbrella. This time, the background is white and the lines in the drawing are each a different color. An image labeled D of the same girl holding an umbrella. This time each part of the drawing is filled with a different solid color. The image is labeled Downstream applications (e.g., flatting). We propose a fast and accurate vectorization method that can turn clean raster line drawings with complex topology (a) into high quality vector graphics (c). Our results are sufficiently detailed for automatic downstream processing (e.g., [Yin et al. 2022] (d)). Neural networks predict dual contouring inputs, key point locations, and under-sampled regions (b). The key points and under-sampled regions are used to automatically refine the initial vectorization produced by dual contouring, correcting incorrect topology (b, top). Our method is also capable of handling high valence star-junctions. Girl image © David Revoy CC-BY-4.0.

Abstract

We introduce an algorithm for sketch vectorization with state-of-the-art accuracy and capable of handling complex sketches. We approach sketch vectorization as a surface extraction task from an unsigned distance field, which is implemented using a two-stage neural network and a dual contouring domain post processing algorithm. The first stage consists of extracting unsigned distance fields from an input raster image. The second stage consists of an improved neural dual contouring network more robust to noisy input and more sensitive to line geometry. To address the issue of under-sampling inherent in grid-based surface extraction approaches, we explicitly predict undersampling and keypoint maps. These are used in our post-processing algorithm to resolve sharp features and multi-way junctions. The keypoint and undersampling maps are naturally controllable, which we demonstrate in an interactive topology refinement interface. Our proposed approach produces far more accurate vectorizations on complex input than previous approaches with efficient running time.

Fast Forward (20 seconds)

Download as:

Presentation (10 minutes)

Download as:

Interactive Editing demonstration (2:26 minutes)

Download as:

BibTeX

(Publisher's Page) 

@article{Yan:2024:DSV,
 author    = {Yan, Chuan and Li, Yong and Aneja, Deepali and Fisher, Matthew and Simo-Serra, Edgar and Gingold, Yotam},
 title     = {Deep Sketch Vectorization via Implicit Surface Extraction},
 journal   = {ACM Transactions on Graphics (TOG)},
 volume    = {43},
 number    = {4},
 year      = {2024},
 month     = aug,
 keywords  = {vectorization, raster, sketch, drawing},
 doi       = {10.1145/3658197}
}