A Benchmark for Rough Sketch Cleanup
Art_freeform_PB_08
Author: Cartoon Animation by Preston Blair (1994). Mission Viejo, CA: Walter Foster Publishing (aka Preston Blair) Licence: Explicit permissionMessiness: 0.3163
Ambiguity (Chamfer): 0.02257
Shading: Yes
Scaffold: No
Texture Strokes: No
Background: Clean
Best automatic results (chamfer distance)
Column visibility:
Best automatic results
| Original | Ground Truth | Real-Time Inking | Mastering Sketching | Topology Driven | Poly Vector | Poly Vector → Stroke Aggregator | Topology Driven → Stroke Aggregator | Delaunay Triangulation | Fidelity Simplicity | ||
|---|---|---|---|---|---|---|---|---|---|---|---|
|  |  |  |  distance: 0.0011 |  distance: 0.00117 |  distance: 0.00133 |  distance: 0.00138 |  distance: 0.00141 |  distance: 0.00149 |  distance: 0.00245 |  distance: 0.00294 |
Rough Sketch
Column visibility:
| original | thresholded | all layers | shape | lighting | scaffold | shading |
|---|---|---|---|---|---|---|
|  |  |  |  |
Ground Truth
Column visibility:
| artist | all layers | shape | scaffold | shading | color region | text | extra |
|---|---|---|---|---|---|---|---|
| Branislav Mirkovic |  | |  | ||||
| Diego Barrionuevo |  | |  |  | |||
| Liliya Larsen |  | |  |  |
Automatic results, best result per algorithm
Select metric
Column visibility:
Display labels:
Chamfer
| algorithm | distance | result | running time |
|---|---|---|---|
| Delaunay Triangulation | 0.0024453972769047584 | distance: 0.00245variant: vectorized (shape strokes) 1000px | 8.0 |
| Mastering Sketching | 0.0011670886948269743 | distance: 0.00117variant: thresholded 500px | 5.0 |
| Poly Vector → Stroke Aggregator | 0.0014072085480974815 | distance: 0.00141variant: vectorized (shape strokes) 1000pxalg. parameter: noisy | 450.0 |
| Poly Vector | 0.001375372317067343 | distance: 0.00138variant: vectorized (shape strokes) 1000px | 266.0 |
| Stroke Aggregator | nan | ||
| Topology Driven → Stroke Aggregator | 0.0014875088823474991 | distance: 0.00149variant: vectorized (shape strokes) 1000px | 1544.0 |
| Topology Driven | 0.0013252035223418732 | distance: 0.00133variant: vectorized (shape strokes) 1000px | 29.0 |
| Fidelity Simplicity | 0.002937647635867779 | distance: 0.00294variant: vectorized (shape strokes) 1000pxalg. parameter: 0.25 | 171.0 |
| Real-Time Inking | 0.001099181817288776 | distance: 0.0011variant: thresholded 1000px | 3.0 |
F12/1000
| algorithm | distance | result | running time |
|---|---|---|---|
| Fidelity Simplicity | 0.4212157301186078 |  distance: 0.42122variant: vectorized (all layers) 500pxalg. parameter: 0.75 | 182.0 |
| Poly Vector → Stroke Aggregator | 0.5583075513838714 |  distance: 0.55831variant: vectorized (all layers) 500pxalg. parameter: noisy | 81.0 |
| Poly Vector | 0.5801559767738208 |  distance: 0.58016variant: vectorized (all layers) 500px | 283.0 |
| Stroke Aggregator | nan | ||
| Topology Driven | 0.5693487477813802 |  distance: 0.56935variant: vectorized (all layers) 500px | 50.0 |
| Delaunay Triangulation | 0.290702151636046 |  distance: 0.2907variant: vectorized (all layers) 500px | 4.0 |
| Mastering Sketching | 0.6460230577697276 |  distance: 0.64602variant: vectorized (all layers) 500px | 5.0 |
| Real-Time Inking | 0.7377035240245421 |  distance: 0.7377variant: vectorized (all layers) 500px | 2.0 |
| Topology Driven → Stroke Aggregator | 0.7889633940527181 | distance: 0.78896variant: vectorized (shape strokes) 1000px | 1544.0 |
F14/1000
| algorithm | distance | result | running time |
|---|---|---|---|
| Poly Vector → Stroke Aggregator | 0.669144836999165 |  distance: 0.66914variant: vectorized (all layers) 500px | 83.0 |
| Poly Vector | 0.679846300782125 | distance: 0.67985variant: vectorized (all layers) 500px | 283.0 |
| Stroke Aggregator | nan | ||
| Fidelity Simplicity | 0.5565659009344128 |  distance: 0.55657variant: thresholded 500pxalg. parameter: 0.75 | 168.0 |
| Real-Time Inking | 0.8493497015613516 | distance: 0.84935variant: vectorized (all layers) 500px | 2.0 |
| Topology Driven | 0.6885165025721625 | distance: 0.68852variant: vectorized (all layers) 500px | 50.0 |
| Delaunay Triangulation | 0.4533403296205309 | distance: 0.45334variant: vectorized (all layers) 500px | 4.0 |
| Mastering Sketching | 0.7668112798775989 | distance: 0.76681variant: vectorized (all layers) 500px | 5.0 |
| Topology Driven → Stroke Aggregator | 0.9035610108166304 | distance: 0.90356variant: vectorized (shape strokes) 1000px | 1544.0 |
F16/1000
| algorithm | distance | result | running time |
|---|---|---|---|
| Poly Vector → Stroke Aggregator | 0.7528517393156942 | distance: 0.75285variant: vectorized (all layers) 500pxalg. parameter: noisy | 81.0 |
| Poly Vector | 0.7624877770520216 |  distance: 0.76249variant: vectorized (all layers) 500pxalg. parameter: noisy | 279.0 |
| Stroke Aggregator | nan | ||
| Topology Driven | 0.779928584733803 | distance: 0.77993variant: vectorized (all layers) 500px | 50.0 |
| Fidelity Simplicity | 0.6407658572961484 |  distance: 0.64077variant: thresholded 500pxalg. parameter: 0.25 | 142.0 |
| Real-Time Inking | 0.9095977414877032 | distance: 0.9096variant: vectorized (all layers) 500px | 2.0 |
| Topology Driven → Stroke Aggregator | 0.9524598262463416 | distance: 0.95246variant: vectorized (shape strokes) 1000px | 1544.0 |
| Delaunay Triangulation | 0.6167099079468547 | distance: 0.61671variant: vectorized (all layers) 500px | 4.0 |
| Mastering Sketching | 0.8373384839964896 | distance: 0.83734variant: vectorized (all layers) 500px | 5.0 |
F18/1000
| algorithm | distance | result | running time |
|---|---|---|---|
| Stroke Aggregator | nan | ||
| Delaunay Triangulation | 0.7076845869065505 | distance: 0.70768variant: vectorized (all layers) 500px | 4.0 |
| Poly Vector → Stroke Aggregator | 0.8057002057012845 | distance: 0.8057variant: vectorized (all layers) 500pxalg. parameter: noisy | 81.0 |
| Poly Vector | 0.8134960450014009 | distance: 0.8135variant: vectorized (all layers) 500px | 283.0 |
| Real-Time Inking | 0.9360120618443936 | distance: 0.93601variant: vectorized (all layers) 500px | 2.0 |
| Topology Driven → Stroke Aggregator | 0.9729573884889414 | distance: 0.97296variant: vectorized (shape strokes) 1000px | 1544.0 |
| Fidelity Simplicity | 0.6907804772242095 |  distance: 0.69078variant: thresholded 500pxalg. parameter: 0.5 | 151.0 |
| Mastering Sketching | 0.8765337412073092 | distance: 0.87653variant: vectorized (all layers) 500px | 5.0 |
| Topology Driven | 0.8348164747004632 | distance: 0.83482variant: vectorized (all layers) 500px | 50.0 |
F110/1000
| algorithm | distance | result | running time |
|---|---|---|---|
| Poly Vector → Stroke Aggregator | 0.8515902692236998 | distance: 0.85159variant: vectorized (all layers) 500px | 83.0 |
| Poly Vector | 0.8549589215117015 | distance: 0.85496variant: vectorized (all layers) 500px | 283.0 |
| Stroke Aggregator | nan | ||
| Topology Driven | 0.883056580158367 | distance: 0.88306variant: vectorized (all layers) 500px | 50.0 |
| Delaunay Triangulation | 0.7798070376823003 | distance: 0.77981variant: vectorized (all layers) 500px | 4.0 |
| Fidelity Simplicity | 0.7408889142390971 |  distance: 0.74089variant: thresholded 500pxalg. parameter: 0.6 | 139.0 |
| Real-Time Inking | 0.956669993352194 | distance: 0.95667variant: vectorized (all layers) 500px | 2.0 |
| Topology Driven → Stroke Aggregator | 0.9862607675445368 | distance: 0.98626variant: vectorized (shape strokes) 1000px | 1544.0 |
| Mastering Sketching | 0.9127921013638116 | distance: 0.91279variant: vectorized (all layers) 500px | 5.0 |
Hausdorff
| algorithm | distance | result | running time |
|---|---|---|---|
| Stroke Aggregator | nan | ||
| Mastering Sketching | 0.2466576538085937 | distance: 0.24666variant: thresholded 500px | 5.0 |
| Real-Time Inking | 0.021540658950805664 |  distance: 0.02154variant: thresholded 500px | 2.0 |
| Topology Driven → Stroke Aggregator | 0.03046309280395508 | distance: 0.03046variant: vectorized (shape strokes) 1000px | 1544.0 |
| Topology Driven | 0.026000000000000002 |  distance: 0.026variant: thresholded 500px | 22.0 |
| Delaunay Triangulation | 0.03036445236206055 | distance: 0.03036variant: vectorized (shape strokes) 1000px | 8.0 |
| Fidelity Simplicity | 0.048662101745605466 |  distance: 0.04866variant: vectorized (shape strokes) 500pxalg. parameter: 0.3 | 150.0 |
| Poly Vector → Stroke Aggregator | 0.023323806762695312 |  distance: 0.02332variant: thresholded 500pxalg. parameter: noisy | 73.0 |
| Poly Vector | 0.023323806762695312 |  distance: 0.02332variant: thresholded 500pxalg. parameter: noisy | 290.0 |
Automatic results for each input variant
Column visibility:
| variant | Topology Driven → Stroke Aggregator | Real-Time Inking | Mastering Sketching | Fidelity Simplicity 0.3 | Fidelity Simplicity 0.25 | Fidelity Simplicity 0.5 | Fidelity Simplicity 0.75 | Fidelity Simplicity 0.6 | Topology Driven | Poly Vector noisy | Poly Vector | Delaunay Triangulation | Stroke Aggregator | Poly Vector → Stroke Aggregator noisy | Poly Vector → Stroke Aggregator |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| original |  |
 |
 |
 |
 |
 |
 |
||||||||
| thresholded |  |
 |
 |
||||||||||||
| thresholded 1000px | |
 |
 |
 |
 |
 |
 |
 |
 |
 |
|||||
| thresholded 500px | |
|
 |
|
|
|
|
|
|
 |
 |
|
 |
||
| vectorized (all layers) 1000px |  |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
|||
| vectorized (all layers) 500px | |
|
 |
 |
 |
|
 |
|
|
|
|
|
|
||
| vectorized (shape strokes) 1000px | |
 |
 |
|
 |
 |
 |
|
 |
|
|
|
 |
||
| vectorized (shape strokes) 500px |  |
 |
|
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |