Graphics, Imaging, and Games Lab

In my last post (http://gigl.scs.carleton.ca/node/536), I mentioned I would analyze the plane-fitting filters from a past post (http://gigl.scs.carleton.ca/node/522) with respect to the technique I recently developed for analyzing noise pixel smoothing (see http://gigl.scs.carleton.ca/node/535). The results are provided below:

I decided to start exploring how to filter out noise pixels from an image by implementing an idea that Dr. Mould had brought up with earlier. Dr. Mould suggested that outliers could potentially be identified by the number of times that they are added to filtering masks for the surrounding pixels, relative to other pixels in the image. In order to use this measurement, I created a two-pass filter, in which the entire image is first processed at a given mask size and the number of times that each pixel is included in a filtering mask is recorded. Next, the image is filtered in a way which makes use of the data obtained during the first pass, using the same or a different mask size.

Motivation

In the near future, I plan to develop a number of variations on the cumulative range geodesic filter, with the intention of building a filter having the following properties:

• Noise pixels in the image (outliers) are strongly smoothed towards neighbouring colours, and neighbouring colours are not contaminated by noise colours. To be specific, noise pixels are pixels whose colours are very different from the colours of their surroundings. Noise pixels do not contribute to the desired texture of the image. The current filter can fail to remove noise for a few reasons:

The histogram-based brightness adjustments that I used earlier (http://gigl.scs.carleton.ca/node/532 and http://gigl.scs.carleton.ca/node/530) were limited by the range of brightness values in each SLIC segment of the image. I decided to try using a brightness mapping function instead, which takes the average brightness value of an SLIC segment and outputs a new brightness value that does not necessarily exist within the segment. Since I want to increase the tonal contrast of the image, I selected a sigmoidal brightness transformation function, a few variations of which are shown below:

Check the previous blog post for more information: Spacing Control for Coordinated Particle System (1)

We have managed changing the spacing for each pair of particles, now we try to make it change slowly and smoothly.

Instead of making each pair of particles adjust to the new spacing immediately when particles enter a new area with different spacing, we give these particles an "adjusting" time.

Each particle will remember the old spacing that they were on. When particle A enters a new area with spacing 30 from the old area which has a spacing as 10, it will first compare its own old record with the current spacing, if the record is smaller, make it somewhat larger, and vice versa. In this case, A's old record will be added by 0.001. When it comes to compare the distance of A and its neighbor B with the spacing, to determine if there is a need of birth or death, it will compare the distance AB with the average of A and B's records. 

In this case, if AB > BIRTH_OFFSET * (A_record + B_record) / 2, then generates a new particle between A and B; if AB < DEATH_OFFSET * (A_record + B_record) / 2, then kill B;

By this way, AB will not just compare with the new current spacing, so that when the old spacing and the new one are too different, particles will not have a burst of birth or death suddently .

See the images below showing the effect of this feature.

Method

The method used to obtain colour data for averaging in this post is almost the same as that described in Part 4 (http://gigl.scs.carleton.ca/node/530), except that pixels in an SLIC segment are not assigned the same brightness value. Instead, pixels have their lightness values added to a value equal to the average lightness value of the segment subtracted from the n^th or (100-n)^th-percentile lightness value of the segment. The result becomes the new lightness value of each pixel. (In this post, I used the average brightness of the image to determine whether to use the n^th or the (100-n)^th-percentile to alter the brightness values of each segment, for reasons discussed earlier at http://gigl.scs.carleton.ca/node/529)

Having spent some time exploring the use of colour averaging within SLIC-generated segments, I am now investigating what happens if I apply colour offsets to the pixels within segments. As a preliminary test, I generated images where each SLIC segment was assigned a random colour vector that was added to all of the segment's pixels (in the RGB colour space). Colour components were then clipped to the range of 0-255. The components of the colour offset vectors were each normally-distributed around zero, and had the same standard deviations.

I experimented with another variation on the crossfiltering approach described at http://gigl.scs.carleton.ca/node/528. As before, pixels in an SLIC segment are assigned the average hue of the segment. However, the lightness component of the colour assigned to the segment is set equal to a given percentile of the lightness values of the pixels in the segment. (All lightness values were measured in the CIE L*a*b* colour space.)

The intention of this approach is to enhance the lightness contrast of the image. To achieve this goal, I set the new colour's lightness value equal to the n^th percentile of segment lightness values if the average lightness of the segment is above the average lightness of the image. Conversely, the average lightness of the segment is below the average lightness of the image, I set the new colour's lightness equal to the (100-n)^th percentile of segment lightness values.

Method

The images shown in this section are produced by a cross-filtering method similar to the one which I used in my last post (http://gigl.scs.carleton.ca/node/528), except that the colours assigned to the SLIC segments are weighted averages of the segment pixels. The weight of each pixel is the absolute value of the difference between its lightness component (in the CIE L*a*b* colour space) and the average lightness value of the entire image.

Method

The cross-filtered images shown in this post result from the following process:

1. SLIC (Simple Linear Iterative Clustering) is used to obtain a segmentation of the source image
2. A new image is generated where each pixel is assigned the average colour of all pixels in the corresponding segment
3. Cross-filtering is performed using the cumulative range geodesic filter:
   • Filtering masks for each pixel are generated based on pixel distances in the source image.
   • Pixel colours in the output image are produced by averaging over the colours in the image from Step 2 that correspond to the coordinates of the pixels in the filtering masks.