A Benchmark for Rough Sketch Cleanup
Art_freeform_AG_02
Author: Anton Gulic Author Homepage: https://www.upwork.com/freelancers/~0168eeeb4816b24940 Licence: CC-BY-4.0Messiness: 0.76711
Ambiguity (Chamfer): 0.20347
Shading: Yes
Scaffold: Yes
Texture Strokes: Yes
Background: Clean
Best automatic results (chamfer distance)
Column visibility:
Best automatic results
| Original | Ground Truth | Poly Vector | Topology Driven | Stroke Aggregator | Mastering Sketching | Poly Vector → Stroke Aggregator | Real-Time Inking | Topology Driven → Stroke Aggregator | Delaunay Triangulation | Fidelity Simplicity | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|  |  |  |  distance: 0.00319 |  distance: 0.00321 |  distance: 0.00337 |  distance: 0.00338 |  distance: 0.00342 |  distance: 0.00386 |  distance: 0.00419 |  distance: 0.007 |  distance: 0.01016 |
Rough Sketch
Column visibility:
| original | thresholded | all layers | shape | lighting | scaffold | shading |
|---|---|---|---|---|---|---|
|  |  |  |  |  |
Ground Truth
Column visibility:
| artist | all layers | shape | scaffold | shading | color region | text | extra |
|---|---|---|---|---|---|---|---|
| Branislav Mirkovic |  | |  |  | |||
| Diego Barrionuevo |  | |  |  |  | ||
| Liliya Larsen |  | |  |  |
Automatic results, best result per algorithm
Select metric
Column visibility:
Display labels:
Chamfer
| algorithm | distance | result | running time |
|---|---|---|---|
| Delaunay Triangulation | 0.006998472975950172 | distance: 0.007variant: vectorized (shape strokes) 500px | 2.0 |
| Fidelity Simplicity | 0.010156895153252295 | distance: 0.01016variant: vectorized (all layers) 1000pxalg. parameter: 0.3 | 174.0 |
| Mastering Sketching | 0.0033809535566456193 | distance: 0.00338variant: vectorized (shape strokes) 500px | 5.0 |
| Poly Vector → Stroke Aggregator | 0.0034249228158957537 | distance: 0.00342variant: vectorized (shape strokes) 1000pxalg. parameter: noisy | 557.0 |
| Poly Vector | 0.003187756084303511 | distance: 0.00319variant: vectorized (shape strokes) 1000pxalg. parameter: noisy | 220.0 |
| Real-Time Inking | 0.0038632420573008436 | distance: 0.00386variant: vectorized (shape strokes) 1000px | 2.0 |
| Stroke Aggregator | 0.0033689270161024053 | distance: 0.00337variant: vectorized (shape strokes) | 481.0 |
| Topology Driven → Stroke Aggregator | 0.0041893658930442345 | distance: 0.00419variant: vectorized (shape strokes) 500px | 297.0 |
| Topology Driven | 0.0032117364255673485 | distance: 0.00321variant: vectorized (shape strokes) 1000px | 24.0 |
F12/1000
| algorithm | distance | result | running time |
|---|---|---|---|
| Fidelity Simplicity | 0.06710126204701422 |  distance: 0.0671variant: vectorized (shape strokes) 1000pxalg. parameter: 0.75 | 127.0 |
| Mastering Sketching | 0.3080403711736674 |  distance: 0.30804variant: vectorized (all layers) 500px | 5.0 |
| Poly Vector → Stroke Aggregator | 0.4064660008445298 |  distance: 0.40647variant: vectorized (all layers) 500pxalg. parameter: noisy | 85.0 |
| Poly Vector | 0.3027986293635849 |  distance: 0.3028variant: vectorized (all layers) 1000px | 750.0 |
| Real-Time Inking | 0.3509147764578988 |  distance: 0.35091variant: vectorized (all layers) 1000px | 3.0 |
| Stroke Aggregator | 0.5611807972030565 | distance: 0.56118variant: vectorized (shape strokes) | 481.0 |
| Delaunay Triangulation | 0.17413624888423332 |  distance: 0.17414variant: vectorized (all layers) 500px | 8.0 |
| Topology Driven → Stroke Aggregator | 0.029419046733575288 |  distance: 0.02942variant: original | 6.0 |
| Topology Driven | 0.031845396970961545 |  distance: 0.03185variant: original | 113.0 |
F14/1000
| algorithm | distance | result | running time |
|---|---|---|---|
| Fidelity Simplicity | 0.10292195533571698 |  distance: 0.10292variant: vectorized (shape strokes) 1000pxalg. parameter: 0.3 | 105.0 |
| Mastering Sketching | 0.4352166645949853 | distance: 0.43522variant: vectorized (all layers) 500px | 5.0 |
| Poly Vector → Stroke Aggregator | 0.5577576486674825 | distance: 0.55776variant: vectorized (all layers) 500pxalg. parameter: noisy | 85.0 |
| Poly Vector | 0.4469809177274435 |  distance: 0.44698variant: vectorized (all layers) 1000pxalg. parameter: noisy | 794.0 |
| Real-Time Inking | 0.5075229048843568 | distance: 0.50752variant: vectorized (all layers) 1000px | 3.0 |
| Stroke Aggregator | 0.7324121289562148 | distance: 0.73241variant: vectorized (shape strokes) | 481.0 |
| Delaunay Triangulation | 0.2777017704408804 | distance: 0.2777variant: vectorized (all layers) 500px | 8.0 |
| Topology Driven → Stroke Aggregator | 0.03675021931841782 | distance: 0.03675variant: original | 6.0 |
| Topology Driven | 0.03793833434180503 | distance: 0.03794variant: original | 113.0 |
F16/1000
| algorithm | distance | result | running time |
|---|---|---|---|
| Fidelity Simplicity | 0.11805644423841125 |  distance: 0.11806variant: vectorized (shape strokes) 1000pxalg. parameter: 0.5 | 107.0 |
| Mastering Sketching | 0.5294983041881444 | distance: 0.5295variant: vectorized (all layers) 500px | 5.0 |
| Poly Vector → Stroke Aggregator | 0.6609081954685738 | distance: 0.66091variant: vectorized (all layers) 500pxalg. parameter: noisy | 85.0 |
| Poly Vector | 0.5473298518349317 | distance: 0.54733variant: vectorized (all layers) 1000pxalg. parameter: noisy | 794.0 |
| Delaunay Triangulation | 0.3943285682438834 | distance: 0.39433variant: vectorized (all layers) 500px | 8.0 |
| Real-Time Inking | 0.6052540241960845 | distance: 0.60525variant: vectorized (all layers) 1000px | 3.0 |
| Stroke Aggregator | 0.8040746800262899 | distance: 0.80407variant: vectorized (shape strokes) | 481.0 |
| Topology Driven → Stroke Aggregator | 0.04128670618988557 | distance: 0.04129variant: original | 6.0 |
| Topology Driven | 0.04220078597226922 | distance: 0.0422variant: original | 113.0 |
F18/1000
| algorithm | distance | result | running time |
|---|---|---|---|
| Fidelity Simplicity | 0.12790790342701072 | distance: 0.12791variant: vectorized (shape strokes) 1000pxalg. parameter: 0.75 | 127.0 |
| Poly Vector → Stroke Aggregator | 0.7120086067111803 | distance: 0.71201variant: vectorized (all layers) 500pxalg. parameter: noisy | 85.0 |
| Poly Vector | 0.614635439894893 | distance: 0.61464variant: vectorized (all layers) 1000px | 750.0 |
| Delaunay Triangulation | 0.4793985947277374 | distance: 0.4794variant: vectorized (all layers) 500px | 8.0 |
| Mastering Sketching | 0.5897260824162583 | distance: 0.58973variant: vectorized (all layers) 500px | 5.0 |
| Real-Time Inking | 0.6603537303505114 | distance: 0.66035variant: vectorized (all layers) 1000px | 3.0 |
| Stroke Aggregator | 0.8308639904633367 | distance: 0.83086variant: vectorized (shape strokes) | 481.0 |
| Topology Driven → Stroke Aggregator | 0.044469827005261635 | distance: 0.04447variant: original | 6.0 |
| Topology Driven | 0.04529112886443492 | distance: 0.04529variant: original | 113.0 |
F110/1000
| algorithm | distance | result | running time |
|---|---|---|---|
| Fidelity Simplicity | 0.1369101404977352 | distance: 0.13691variant: vectorized (shape strokes) 1000pxalg. parameter: 0.75 | 127.0 |
| Poly Vector → Stroke Aggregator | 0.7529735115570123 | distance: 0.75297variant: vectorized (all layers) 500pxalg. parameter: noisy | 85.0 |
| Delaunay Triangulation | 0.5552970942226785 | distance: 0.5553variant: vectorized (all layers) 500px | 8.0 |
| Mastering Sketching | 0.6397691175331215 | distance: 0.63977variant: vectorized (all layers) 500px | 5.0 |
| Poly Vector | 0.6688826090984897 | distance: 0.66888variant: vectorized (all layers) 1000pxalg. parameter: noisy | 794.0 |
| Real-Time Inking | 0.6963328227972218 |  distance: 0.69633variant: vectorized (all layers) 500px | 2.0 |
| Stroke Aggregator | 0.849345933415995 | distance: 0.84935variant: vectorized (shape strokes) | 481.0 |
| Topology Driven → Stroke Aggregator | 0.04766652492931134 | distance: 0.04767variant: original | 6.0 |
| Topology Driven | 0.048483541130767166 | distance: 0.04848variant: original | 113.0 |
Hausdorff
| algorithm | distance | result | running time |
|---|---|---|---|
| Delaunay Triangulation | 0.08004998779296875 |  distance: 0.08005variant: thresholded 500px | 12.0 |
| Fidelity Simplicity | 0.09126335144042967 |  distance: 0.09126variant: thresholded 1000pxalg. parameter: 0.25 | 158.0 |
| Poly Vector → Stroke Aggregator | 0.06596968841552735 |  distance: 0.06597variant: vectorized (shape strokes) 500px | 22.0 |
| Poly Vector | 0.06596968841552735 |  distance: 0.06597variant: vectorized (shape strokes) 500pxalg. parameter: noisy | 113.0 |
| Topology Driven → Stroke Aggregator | 0.084 |  distance: 0.084variant: thresholded 500px | 129.0 |
| Topology Driven | 0.08459314727783203 |  distance: 0.08459variant: vectorized (shape strokes) 500px | 7.0 |
| Mastering Sketching | 0.28527178955078125 |  distance: 0.28527variant: thresholded 500px | 5.0 |
| Real-Time Inking | 0.0839344991048177 |  distance: 0.08393variant: original | 4.0 |
| Stroke Aggregator | 0.08186710103352865 | distance: 0.08187variant: vectorized (shape strokes) | 481.0 |
Automatic results for each input variant
Column visibility:
| variant | Topology Driven → Stroke Aggregator | Real-Time Inking | Mastering Sketching | Fidelity Simplicity 0.3 | Fidelity Simplicity 0.25 | Fidelity Simplicity 0.5 | Fidelity Simplicity 0.75 | Fidelity Simplicity 0.6 | Topology Driven | Poly Vector noisy | Poly Vector | Delaunay Triangulation | Stroke Aggregator | Poly Vector → Stroke Aggregator noisy | Poly Vector → Stroke Aggregator |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| original | |
|
 |
 |
 |
 |
 |
|
|||||||
| thresholded |  |
 |
 |
 |
 |
 |
|||||||||
| thresholded 1000px |  |
 |
 |
|
 |
 |
 |
 |
 |
||||||
| thresholded 500px | |
 |
|
 |
 |
 |
 |
 |
 |
 |
 |
|
|||
| vectorized (all layers) 1000px | |
|
 |
 |
 |
 |
 |
|
|
 |
|||||
| vectorized (all layers) 500px | |
|
 |
 |
 |
 |
 |
 |
 |
 |
|
|
 |
||
| vectorized (shape strokes) 1000px | |
|
 |
|
|
 |
|
|
 |
 |
|
 |
|||
| vectorized (shape strokes) 500px | |
 |
|
 |
 |
 |
 |
 |
|
|
 |
|
|
||
| vectorized (shape strokes) | |