Graphics, Imaging, and Games Lab

Method

Implementing some of the ideas discussed in my previous post (http://gigl.scs.carleton.ca/node/545), I created a version of the leaky integration cumulative range geodesic filter which provides great flexibility for adaptive mask termination. The filter program uses the following algorithm for adaptive mask termination:

• Given a maximum possible mask size, the filter builds a mask for each source pixel using the same process as the basic leaky integration filter.

• During mask generation, the filter keeps track of the maximum distance value from among all mask pixels.

• Before the filter averages mask pixels to determine the output colour value for the source pixel, a sub-routine scans through the mask pixels, in the order in which they were added to the mask. Upon finding a pixel with a distance greater than or equal to a given fraction (passed in as the first element of a filtering parameter called 'ratio') of the maximum distance for the mask, the sub-routine halts and returns the index of this pixel in the list.

• The filter was passed a lookup array, which it uses to map the index at which the threshold value of distance was passed to a mask size to use for colour averaging.

• Upon finding the mask size required for colour averaging, call it 'M', for convenience, the filter computes the output colour of the source pixel by averaging the first M pixels added to the mask during the earlier mask building process.

  • In a variation of this step, which I called "excise mode", the filter only includes pixels in the average if they have distance values which are greater than or equal to a threshold value. (For more flexibility, I chose to make 'ratio' a two-element array, where the first element is used to determine the averaging mask size, while the second element is used for selecting pixels to be included in the average, when in excise mode.) Therefore, the number of pixels in the average is the minimum of M and the number of pixels in the mask that exceed the threshold distance.

Applications

The filter that I described above can be used to smooth an image in a manner which is sensitive to region size. The mask size at which a filtering mask leaves a similar-coloured region and enters the background will be the point at which the distances of pixels within the mask exceed the threshold distance value, depending on the values of the 'alpha' and 'ratio' parameters. Consequently, it is possible to detect the size of a pixel's neighbourhood of similar pixels and make corresponding adjustments to the number of pixels included in the average when determining the given pixel's output colour. There are a few key applications of this technique:

• Prevent over-smoothing pixels in large regions with little texture, by reducing the number of pixels included in colour averages
• Eliminate outliers and tiny, contrasting features in the image, by including a large number of mask pixels in colour averages
• Avoid contaminating distinctly-coloured areas of the image with the colours of their surroundings, by ignoring mask pixels that belong to the background

Region Protection Results

In order to implement the third application listed above, I passed a lookup array to the filter which would assign a colour averaging mask size of 1024 to pixels whose masks crossed distance thresholds of 0.25 times the maximum distance at sizes less or equal to than a given "outlier region size". In contrast, pixels whose masks crossed the distance threshold beyond the outlier region size would be assigned a colour averaging mask size of 1 pixel.

For non-outlier pixels (region sizes above the specified outlier region sizes), I averaged colours without excise mode. For outlier pixels (region sizes equal to or less than the specified outlier region sizes), I tried averaging colours with and without using excise mode. For excise mode, I set a distance threshold fraction of 0.25.

All of the images in this section were produced with 'gamma' and 'alpha' values of zero.

Source Image

Image Credit: Rob Knight (via Flickr), http://www.flickr.com/photos/rknight/

Outlier size of 8, averaging in non-excise mode

Outlier size of 8, averaging in excise mode

Outlier size of 256, averaging in non-excise mode

Outlier size of 256, averaging in excise mode

Discussion

The images shown above demonstrate that, for this particular source image, the adaptive mask termination filter is very effective at distinguishing foreground colours from background colours. Regions which are identified as at or below the cutoff size are strongly faded into the background, while larger regions are not contaminated by the background colour. Perhaps on images where the contrast between foreground and background colours is more muted, the adaptive mask termination filter will not be quite as effective.

The comparison between excise mode and non-excise mode is somewhat complicated. On one hand, excise mode improves the extent to which regions identified as outliers fade into the background. On the other hand, excise mode causes some artifacts to appear in the image. Firstly, regions filled with the background colour, but surrounded by a thin border of a different colour, will acquire the colour of the border. Secondly, pixels along the edges of outlier regions are sometimes poorly smoothed by the filter. Without excise mode, this issue is not apparent, because the interior of a region retains much of its original colour. However, in excise mode, edge pixels contrast with interior pixels, so they become more visible. Consider the example given below:

Another disadvantage of the adaptive mask termination filter is its inability to choose the correct background into which small features should be faded. In this image, the debris near the ocean acquires the colour of the ocean, regardless of whether or not excise mode is used.

Lastly, setting a large maximum outlier feature size can potentially leave many noticeable untextured areas in the image. Filling in outlier features with the texture of their surroundings is something that probably cannot be achieved by tweaking the parameters of the filter, so I will simply avoid large cutoff feature sizes for now.

Results for Texture Equalization

In this context, I refer to texture equalization as the effort to leave a constant amount of texture throughout the image. In other words, the intention is to implement the first two application points discussed in the Method section above. For this purpose, I used colour averaging mask sizes that were exponentially-decaying functions of distance threshold crossing points, such as shown in the plot below:

Source Image

Comparison of Adaptive Mask Termination with Normal Leaky Integration Filtering

The above series of images was produced with 'alpha' and 'gamma' parameter values of zero. The central image was produced without excise mode, using a distance threshold fraction of 0.4.

Discussion

In general, it is very hard to find a suitable constant mask size at which to filter an image in order to compare the result with the one from filtering using adaptive mask termination. In the above montage of images, I find that the adaptive mask termination result most strongly resembles the small constant mask size filtering result, except for some very subtle fading of small details.

Further experimentation could be used to better characterize the use of the adaptive mask termination filter for texture equalization. However, from the results obtained for the two source images shown in this post, it seems be quite capable of fulfilling the applications described in the Method section above.

Portrait Image Source: Page 2 of Geodesic Image and Video Editing, by Criminisi et al. (ACM Transactions on Graphics, Vol. 29, No. 5, Article 134, Publication date: October 2010.)