Having implemented structure preserving stippling [1] and SLIC [2], I can now explore using the two techniques together.

My goal is to represent fine details and high contrast textures with stipples, while colouring featureless areas smoothly, such as with the average internal colours of the corresponding superpixels.

I also want to avoid stippling dark regions, which otherwise waste the "stipple budget", and avoid stippling smoothly-shaded light regions, which otherwise appear freckled.

I can use superpixels for two purposes:

• To divide the image into regions for the purpose of making per-region decisions of how to render the local area
• As primitives in the rendering process

Filtering superpixels

In order to use superpixels to choose where to apply various stylization techniques, I need to determine what measurements to consider as part of the decisions.

I first tried superpixel size, reasoning that superpixels with higher-frequency textures would be smaller on average.

The original image (my own photograph), is as follows:

The next image has two parts:

• The right half shows which superpixels "pass" (black), meaning that they are large, and should not be rendered using stipples (assuming that size correlates with texture).
• The left half shows superpixel sizes in shades of grey, where larger superpixels are shaded more lightly.

I used Otsu's method [3] to generate the thresholded right side of the image from the superpixel size data.

Clearly, this is a poor superpixel selection criteria for this image, because areas with the same level of texture are often different sizes, due to the underlying geometry of the objects in the image.

However, selecting superpixels that have lower standard deviations of pixel lightnesses produces more appealing results:

Here, a larger proportion of superpixels with a given texture end up on the same side of the threshold.

Basic overlays of stipples and superpixels

The image on the left depicts SLIC superpixels coloured with their average colours, whereas the image on the right has stipples overlayed on top. Note that stipples were computed from the original image, not the SLIC superpixellation of the image.

The image on the right is more interesting than a plain black and white stippling result, because of the background colours. The result is too dark, though, because the stippling method is designed for a white background. The darkness is easy to correct, but it should be possible to produce an even better result by making the stipples and superpixels interact. As such, the overlay of stipples and superpixels serves as a benchmark against which future work will be assessed.

Stipples and smooth shading

Presently, I am experimenting with placing stipples only on certain superpixels. Doing so introduces another challenge: How do I produce a result that has perceptual unity?

Superpixel borders may or may not correspond to strong edges in the image. For borders that do not correspond to strong edges, I need some way to blend stipples with superpixel edges to avoid introducing a (false) strong edge.

In the image below, I have stippled only superpixels with low lightness channel (from the CIE L*a*b* colour space) standard deviations. I used the following mechanisms to improve the blending between regions:

• Within a smoothly-shaded superpixel, I shade each pixel such that its lightness is a weighted average of the average lightness within the superpixel and its original lightness. The weight of a pixel's original lightness decreases exponentially away from the border of the superpixel. (Distance to the border is determined using breadth-first search.) This shading reduces detail within superpixels, while preventing (sharp) edges from being introduced between superpixels.
• Within a stippled superpixel, stipples closer to the border of a smoothly-shaded superpixel are given lightnesses that more closely match the lightnesses of the pixels in the original image. Lightnesses are computed using the same weighted averaging approach as for the smooth shading.
• Within a stippled superpixel, pixels closer to the border are given lightness values closer to their original lightnesses, whereas those further from the border are increasingly white.

References

1. H. Li and D. Mould. "Structure-preserving Stippling by Priority-based Error Diffusion," in Proc. Graphics Interface, 2011, pp. 127 - 134. (http://gigl.scs.carleton.ca/node/86)
2. R. Achanta et. al. "SLIC superpixels compared to state-of-the-art superpixel methods." IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 11, pp. 2274-2281, Nov. 2012. (Authors' website for SLIC: http://ivrg.epfl.ch/research/superpixels)
3. A. Greensted. "Otsu Thresholding." Internet: http://www.labbookpages.co.uk/software/imgProc/otsuThreshold.html, June 17, 2010 (Accessed Oct. 25, 2016).