Graphics, Imaging, and Games Lab

Method

The cross-filtered images shown in this post result from the following process:

1. SLIC (Simple Linear Iterative Clustering) is used to obtain a segmentation of the source image
2. A new image is generated where each pixel is assigned the average colour of all pixels in the corresponding segment
3. Cross-filtering is performed using the cumulative range geodesic filter:
   • Filtering masks for each pixel are generated based on pixel distances in the source image.
   • Pixel colours in the output image are produced by averaging over the colours in the image from Step 2 that correspond to the coordinates of the pixels in the filtering masks.

Cross-Filtering Results

Source Image

Segmented Image with Average Colours (regionSize = 19, regularizer = 0.01)

Cross-Filtering of Segmented Image (regionSize = 19, regularizer = 0.01)

Mask size of 64 pixels, gamma parameter value of zero

Mask size of 256 pixels, gamma parameter value of zero

While not shown here, other segmented images were produced at different values of regionSize and images were also filtered with a mask size of 1024 pixels. As is evident in the above series of images, when the size of the segments is larger than the filtering mask size, unwanted edges remain in the output image following cross-filtering.

Increasing the SLIC region size resulted in an image with lower contrast and desaturated colours, where small features faded into their surroundings.

Comparison of Cross-Filtering with Normal Filtering

I was interested in exploring how the change in the image caused by varying the SLIC region size compared with the change caused by varying the filtering mask size. Using MATLAB to calculate the lengths of colour vector differences in RGB space, I determined that, for the three mask sizes used in cross-filtering (64, 256, and 1024 pixels), the most similar cross-filtered images to basic filtering results with those mask sizes (no cross-filtering) were those corresponding to the lowest regionSize value. This is to be expected, as increasing the SLIC region size causes the image with segmented colours to deviate farther from the source image.

I subsequently filtered the source image at a mask size of 4096 pixels and determined which of the images that I had obtained earlier was most similar to this filtering result. Consider the following images below and guess which of the last two images is most similar to the first image:

Basic filtering at a mask size of 4096 pixels and gamma parameter value of zero

Cross-filtering with a filtering mask size of 1024 pixels and gamma parameter value of zero (regionSize = 19, regularizer = 0.01)

Basic filtering at a mask size of 1024 pixels and gamma parameter value of zero

Interestingly, the average lengths of the RGB-space colour differences are 7.49 and 8.10 between the first and second and between the first and third images above, respectively. (The maximum RGB-space colour difference lengths are 100.1 and 94.4, respectively.) Therefore, it seems that an increase in SLIC region size produces an effect similar to increasing the filtering mask size.

Why is this important? The time taken to filter an image at a larger mask size is, for a sufficiently large change in mask size, considerably higher than the time taken to segment the image using SLIC and then cross-filter at the original mask size. Cross-filtering on images with pixel colours averaged over SLIC segments could be a way to achieve desired abstraction effects in much less time than with basic cumulative range geodesic filtering.

(Image source: Bernard Llanos)