Graphics, Imaging, and Games Lab

As is evident in my last post, colour images are far more attractive in general than greyscale images.

Consequently, I adjusted the process for blending stipples with superpixels to incorporate colour channels as well as the CIE L*a*b* lightness channel.

Playground Image

The following panel shows the original image on the left (my own photograph) and the result of stippling the full image.

When combining the SLIC segmentation with stipples, I tried two different levels of exponential decay: 0.9 and 0.99. These values are the ratios by which the weight of a pixel's own colour changes with each increment of distance from the superpixel border. The other component in the weighted average is:

• The mean superpixel colour, for the colour of the rendered superpixels that do not have stipples on top
• The average of black and the pixel's colour, for stipple colours. In this case, the borders used for distance calculations are those of the superpixels without stipples, not those of the current superpixel.
• The average of white and the pixel's colour, for the colour determining the "error" of not placing a stipple. As in the previous case, the borders used for distance calculations are those of the superpixels without stipples, not those of the current superpixel.

The average of the lightness used for a stipple and the lightness used for the absence of a stipple is the threshold lightness determining whether or not a stipple will be placed.

The background behind stippled regions is produced as follows:

• The a* and b* colour channels are calculated using the same process for determining the colours in superpixels which are not stippled
• The lightness channel contains the lightness value used for the "error" of not placing a stipple

Blending results

For a ratio of 0.9, with and without stipples:

For a ratio of 0.99, with and without stipples:

With the higher ratio, there is much less abstraction of superpixel details, although the blending between stippled regions and smoothly-shaded regions without stipples is much smoother. Perhaps I should consider using separate exponential decay ratios for the background colour transitions between superpixels, and the lightness values used to control stipple placement. Such a separation would allow for both high abstraction, and for smooth blending between stippled regions and their surroundings. However, I think the lower level of abstraction (ratio 0.99) within superpixels is nicer in general, because the ratio of 0.9 results in noticeable noisy borders between smooth superpixel interiors.

Unfortunately, the current (crude) method for choosing which regions receive stipples, a purely local decision based on lightness channel standard deviation within each superpixel, performs poorly on this image. There should not be any stipples in the sky.

Alternate stipple to smooth shading transition strategy

I tried a second way of transitioning between superpixels with and without stipples: Rather than have a smooth change of lightnesses values for the background and the absence-of-stipple error calculation, I jump immediately to the steady-state lightness. (The steady-state lightness is the average of the pixel's lightness and white.)

The result is an increase in stipple density towards the borders of superpixels without stipples. In some cases, the results are more pleasing.

For a ratio of 0.9, with and without stipples:

For a ratio of 0.99, with and without stipples:

Courtyard Image

Another of my own photographs (first image), with results for a ratio of 0.9 (second image and third image). The second image uses a smooth transition approach (low stipple density at region borders), whereas the third image uses the sharp transition approach (high stipple density at region borders).

I find the overall stippling and superpixel rendering result more attractive on this image as opposed to on the playground image, because the areas which receive stipples are largely dark to begin with.

For reference, the background behind the stipples (for the sharp transition approach) is as follows: