A Benchmark for Rough Sketch Cleanup
Ind_product_SP_01
Author: Stacey Pietri Author Homepage: https://openlab.citytech.cuny.edu/groups/stacey-pietris-eportfolio/ Licence: CC-BY-NC-SA-3.0Messiness: 0.43628
Ambiguity (Chamfer): 0.05326
Shading: Yes
Scaffold: Yes
Texture Strokes: No
Background: Paper
Best automatic results (chamfer distance)
Column visibility:
Best automatic results
| Original | Ground Truth | Real-Time Inking | Poly Vector | Mastering Sketching | Topology Driven | Poly Vector → Stroke Aggregator | Fidelity Simplicity | Stroke Aggregator | Topology Driven → Stroke Aggregator | Delaunay Triangulation | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|  |  |  |  distance: 0.00089 |  distance: 0.00092 |  distance: 0.00103 |  distance: 0.00109 |  distance: 0.00119 |  distance: 0.00143 |  distance: 0.00163 |  distance: 0.00179 |  distance: 0.0027 |
Rough Sketch
Column visibility:
| original | thresholded | all layers | shape | lighting | scaffold | shading |
|---|---|---|---|---|---|---|
|  |  |  |  |  |
Ground Truth
Column visibility:
| artist | all layers | shape | scaffold | shading | color region | text | extra |
|---|---|---|---|---|---|---|---|
| Branislav Mirkovic |  | |  |  | |||
| Ge Jin |  | |  |  | |||
| Santiago Rial |  | |  |
Automatic results, best result per algorithm
Select metric
Column visibility:
Display labels:
Chamfer
| algorithm | distance | result | running time |
|---|---|---|---|
| Delaunay Triangulation | 0.002696532213868387 | distance: 0.0027variant: vectorized (shape strokes) 1000px | 4.0 |
| Fidelity Simplicity | 0.0014279472684370132 | distance: 0.00143variant: vectorized (shape strokes) 1000pxalg. parameter: 0.3 | 121.0 |
| Mastering Sketching | 0.0010304867644884378 | distance: 0.00103variant: vectorized (shape strokes) 500px | 5.0 |
| Poly Vector → Stroke Aggregator | 0.0011866494006329256 | distance: 0.00119variant: vectorized (shape strokes) 1000pxalg. parameter: noisy | 225.0 |
| Poly Vector | 0.0009232287996440448 | distance: 0.00092variant: vectorized (shape strokes) 1000pxalg. parameter: noisy | 142.0 |
| Real-Time Inking | 0.0008907189363574524 | distance: 0.00089variant: vectorized (shape strokes) 1000px | 2.0 |
| Stroke Aggregator | 0.0016321607010445108 | distance: 0.00163variant: vectorized (shape strokes) | 372.0 |
| Topology Driven → Stroke Aggregator | 0.0017905003514393086 | distance: 0.00179variant: vectorized (shape strokes) 500px | 95.0 |
| Topology Driven | 0.0010854939628460064 | distance: 0.00109variant: vectorized (shape strokes) 1000px | 16.0 |
F12/1000
| algorithm | distance | result | running time |
|---|---|---|---|
| Fidelity Simplicity | 0.13081431262479307 |  distance: 0.13081variant: vectorized (all layers) 500pxalg. parameter: 0.25 | 91.0 |
| Mastering Sketching | 0.4368490758645814 |  distance: 0.43685variant: vectorized (all layers) 500px | 7.0 |
| Poly Vector → Stroke Aggregator | 0.4329451234859032 |  distance: 0.43295variant: vectorized (all layers) 500pxalg. parameter: noisy | 92.0 |
| Poly Vector | 0.4299204551527192 |  distance: 0.42992variant: vectorized (all layers) 1000px | 603.0 |
| Real-Time Inking | 0.4231377276028465 |  distance: 0.42314variant: vectorized (all layers) 500px | 2.0 |
| Stroke Aggregator | 0.6325412043535706 | distance: 0.63254variant: vectorized (shape strokes) | 372.0 |
| Topology Driven → Stroke Aggregator | 0.3895461618208468 |  distance: 0.38955variant: vectorized (all layers) 500px | 1787.0 |
| Topology Driven | 0.44740449396623405 |  distance: 0.4474variant: vectorized (all layers) 500px | 18.0 |
| Delaunay Triangulation | 0.1369739592958218 |  distance: 0.13697variant: vectorized (all layers) 1000px | 16.0 |
F14/1000
| algorithm | distance | result | running time |
|---|---|---|---|
| Fidelity Simplicity | 0.20542057546780326 | distance: 0.20542variant: vectorized (all layers) 500pxalg. parameter: 0.25 | 91.0 |
| Mastering Sketching | 0.5372989511033411 | distance: 0.5373variant: vectorized (all layers) 500px | 7.0 |
| Poly Vector → Stroke Aggregator | 0.5426247124098702 | distance: 0.54262variant: vectorized (all layers) 500pxalg. parameter: noisy | 92.0 |
| Poly Vector | 0.5373223437802538 |  distance: 0.53732variant: vectorized (all layers) 1000pxalg. parameter: noisy | 599.0 |
| Real-Time Inking | 0.5017975048389086 | distance: 0.5018variant: vectorized (all layers) 500px | 2.0 |
| Stroke Aggregator | 0.7925863221976585 | distance: 0.79259variant: vectorized (shape strokes) | 372.0 |
| Topology Driven → Stroke Aggregator | 0.50094853123075 | distance: 0.50095variant: vectorized (all layers) 500px | 1787.0 |
| Topology Driven | 0.5500841832115249 | distance: 0.55008variant: vectorized (all layers) 500px | 18.0 |
| Delaunay Triangulation | 0.22216818979494285 | distance: 0.22217variant: vectorized (all layers) 1000px | 16.0 |
F16/1000
| algorithm | distance | result | running time |
|---|---|---|---|
| Delaunay Triangulation | 0.3040144767524675 | distance: 0.30401variant: vectorized (all layers) 1000px | 16.0 |
| Fidelity Simplicity | 0.2769976996204379 | distance: 0.277variant: vectorized (all layers) 500pxalg. parameter: 0.25 | 91.0 |
| Mastering Sketching | 0.5896247559633012 | distance: 0.58962variant: vectorized (all layers) 500px | 7.0 |
| Poly Vector → Stroke Aggregator | 0.6340605001623616 | distance: 0.63406variant: vectorized (all layers) 500pxalg. parameter: noisy | 92.0 |
| Poly Vector | 0.5939065461477885 | distance: 0.59391variant: vectorized (all layers) 1000px | 603.0 |
| Real-Time Inking | 0.5551458603031986 | distance: 0.55515variant: vectorized (all layers) 500px | 2.0 |
| Stroke Aggregator | 0.9068991790565176 | distance: 0.9069variant: vectorized (shape strokes) | 372.0 |
| Topology Driven → Stroke Aggregator | 0.574753079613145 | distance: 0.57475variant: vectorized (all layers) 500px | 1787.0 |
| Topology Driven | 0.6122175929726328 | distance: 0.61222variant: vectorized (all layers) 500px | 18.0 |
F18/1000
| algorithm | distance | result | running time |
|---|---|---|---|
| Delaunay Triangulation | 0.3521773052566785 | distance: 0.35218variant: vectorized (all layers) 1000px | 16.0 |
| Fidelity Simplicity | 0.3290929616097492 | distance: 0.32909variant: vectorized (all layers) 500pxalg. parameter: 0.25 | 91.0 |
| Mastering Sketching | 0.6189115232593494 | distance: 0.61891variant: vectorized (all layers) 500px | 7.0 |
| Poly Vector → Stroke Aggregator | 0.6788563068823614 | distance: 0.67886variant: vectorized (all layers) 500pxalg. parameter: noisy | 92.0 |
| Poly Vector | 0.6276629554271335 | distance: 0.62766variant: vectorized (all layers) 1000pxalg. parameter: noisy | 599.0 |
| Real-Time Inking | 0.5873749948211444 | distance: 0.58737variant: vectorized (all layers) 500px | 2.0 |
| Stroke Aggregator | 0.951067737329426 | distance: 0.95107variant: vectorized (shape strokes) | 372.0 |
| Topology Driven → Stroke Aggregator | 0.6158262123490036 | distance: 0.61583variant: vectorized (all layers) 500px | 1787.0 |
| Topology Driven | 0.6483993271957551 | distance: 0.6484variant: vectorized (all layers) 500px | 18.0 |
F110/1000
| algorithm | distance | result | running time |
|---|---|---|---|
| Delaunay Triangulation | 0.39207476546264586 | distance: 0.39207variant: vectorized (all layers) 1000px | 16.0 |
| Fidelity Simplicity | 0.37899428970083054 | distance: 0.37899variant: vectorized (all layers) 500pxalg. parameter: 0.25 | 91.0 |
| Mastering Sketching | 0.6428848854232022 | distance: 0.64288variant: vectorized (all layers) 500px | 7.0 |
| Poly Vector → Stroke Aggregator | 0.7134086360895474 | distance: 0.71341variant: vectorized (all layers) 500pxalg. parameter: noisy | 92.0 |
| Poly Vector | 0.6528520272219677 | distance: 0.65285variant: vectorized (all layers) 1000px | 603.0 |
| Real-Time Inking | 0.6137193761623035 | distance: 0.61372variant: vectorized (all layers) 500px | 2.0 |
| Stroke Aggregator | 0.9771121378459744 | distance: 0.97711variant: vectorized (shape strokes) | 372.0 |
| Topology Driven → Stroke Aggregator | 0.6469616889735887 | distance: 0.64696variant: vectorized (all layers) 500px | 1787.0 |
| Topology Driven | 0.6793773379992759 | distance: 0.67938variant: vectorized (all layers) 500px | 18.0 |
Hausdorff
| algorithm | distance | result | running time |
|---|---|---|---|
| Mastering Sketching | 0.29606756591796873 |  distance: 0.29607variant: thresholded 500px | 5.0 |
| Poly Vector → Stroke Aggregator | 0.016552946090698243 | distance: 0.01655variant: vectorized (shape strokes) 1000pxalg. parameter: noisy | 225.0 |
| Poly Vector | 0.017 | distance: 0.017variant: vectorized (shape strokes) 1000pxalg. parameter: noisy | 142.0 |
| Topology Driven → Stroke Aggregator | 0.015231546401977537 | distance: 0.01523variant: vectorized (shape strokes) 500px | 95.0 |
| Fidelity Simplicity | 0.011704699516296388 | distance: 0.0117variant: vectorized (shape strokes) 1000pxalg. parameter: 0.3 | 121.0 |
| Real-Time Inking | 0.012 |  distance: 0.012variant: vectorized (shape strokes) 500px | 2.0 |
| Delaunay Triangulation | 0.09097252655029296 | distance: 0.09097variant: vectorized (shape strokes) 1000px | 4.0 |
| Stroke Aggregator | 0.01715938409169515 | distance: 0.01716variant: vectorized (shape strokes) | 372.0 |
| Topology Driven | 0.01529705810546875 | distance: 0.0153variant: vectorized (shape strokes) 1000px | 16.0 |
Automatic results for each input variant
Column visibility:
| variant | Topology Driven → Stroke Aggregator | Real-Time Inking | Mastering Sketching | Fidelity Simplicity 0.3 | Fidelity Simplicity 0.25 | Fidelity Simplicity 0.5 | Fidelity Simplicity 0.75 | Fidelity Simplicity 0.6 | Topology Driven | Poly Vector noisy | Poly Vector | Delaunay Triangulation | Stroke Aggregator | Poly Vector → Stroke Aggregator noisy | Poly Vector → Stroke Aggregator |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| original |  |
||||||||||||||
| thresholded |  |
 |
 |
 |
 |
 |
 |
 |
 |
 |
|||||
| thresholded 1000px |  |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
|||
| thresholded 500px |  |
|
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
||
| vectorized (all layers) 1000px |  |
 |
 |
 |
 |
 |
 |
|
|
|
|||||
| vectorized (all layers) 500px | |
|
|
 |
|
 |
 |
|
 |
 |
 |
|
 |
||
| vectorized (shape strokes) 1000px | |
|
 |
 |
 |
 |
|
|
 |
|
|
||||
| vectorized (shape strokes) 500px | |
|
|
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
|
| vectorized (shape strokes) | |