Graphics, Imaging, and Games Lab

The histogram-based brightness adjustments that I used earlier (http://gigl.scs.carleton.ca/node/532 and http://gigl.scs.carleton.ca/node/530) were limited by the range of brightness values in each SLIC segment of the image. I decided to try using a brightness mapping function instead, which takes the average brightness value of an SLIC segment and outputs a new brightness value that does not necessarily exist within the segment. Since I want to increase the tonal contrast of the image, I selected a sigmoidal brightness transformation function, a few variations of which are shown below:

Method

The method used to obtain colour data for averaging in this post is almost the same as that described in Part 4 (http://gigl.scs.carleton.ca/node/530), except that pixels in an SLIC segment are not assigned the same brightness value. Instead, pixels have their lightness values added to a value equal to the average lightness value of the segment subtracted from the n^th or (100-n)^th-percentile lightness value of the segment. The result becomes the new lightness value of each pixel. (In this post, I used the average brightness of the image to determine whether to use the n^th or the (100-n)^th-percentile to alter the brightness values of each segment, for reasons discussed earlier at http://gigl.scs.carleton.ca/node/529)

Having spent some time exploring the use of colour averaging within SLIC-generated segments, I am now investigating what happens if I apply colour offsets to the pixels within segments. As a preliminary test, I generated images where each SLIC segment was assigned a random colour vector that was added to all of the segment's pixels (in the RGB colour space). Colour components were then clipped to the range of 0-255. The components of the colour offset vectors were each normally-distributed around zero, and had the same standard deviations.

I experimented with another variation on the crossfiltering approach described at http://gigl.scs.carleton.ca/node/528. As before, pixels in an SLIC segment are assigned the average hue of the segment. However, the lightness component of the colour assigned to the segment is set equal to a given percentile of the lightness values of the pixels in the segment. (All lightness values were measured in the CIE L*a*b* colour space.)

The intention of this approach is to enhance the lightness contrast of the image. To achieve this goal, I set the new colour's lightness value equal to the n^th percentile of segment lightness values if the average lightness of the segment is above the average lightness of the image. Conversely, the average lightness of the segment is below the average lightness of the image, I set the new colour's lightness equal to the (100-n)^th percentile of segment lightness values.

Method

The images shown in this section are produced by a cross-filtering method similar to the one which I used in my last post (http://gigl.scs.carleton.ca/node/528), except that the colours assigned to the SLIC segments are weighted averages of the segment pixels. The weight of each pixel is the absolute value of the difference between its lightness component (in the CIE L*a*b* colour space) and the average lightness value of the entire image.

Method

The cross-filtered images shown in this post result from the following process:

1. SLIC (Simple Linear Iterative Clustering) is used to obtain a segmentation of the source image
2. A new image is generated where each pixel is assigned the average colour of all pixels in the corresponding segment
3. Cross-filtering is performed using the cumulative range geodesic filter:
   • Filtering masks for each pixel are generated based on pixel distances in the source image.
   • Pixel colours in the output image are produced by averaging over the colours in the image from Step 2 that correspond to the coordinates of the pixels in the filtering masks.

Introduction

Simple Linear Iterative Clustering is a recently-developed algorithm for segmenting images into regions of similar pixels.

A simple experiment wherein the first image below was used to generate filtering masks, but the colour data used to determine new pixel values was obtained from images similar to the second image below:

Background

Parallel computation is a possible way to increase the speed of the cumulative range geodesic filter. However, calculating the filtered value of many pixels simultaneously means that memory needs to be allocated for the working data of multiple pixels at once, potentially increasing memory usage significantly.

In order to find an upper bound on the memory needed to filter a single pixel, it is necessary to examine the number of neighbouring pixels involved in the filtering process.

Inspiration and Background

Prasun Choudhury and Jack Tumblin used a first-order approximation to the colour of a source pixel when determining the degree of similarity between the source pixel and its neighbours. Their results are described in the following article: The Trilateral Filter for High Contrast Images and Meshes, Eurographics Symposium on Rendering 2003, pages. 1–-11 (edited by Per Christensen and Daniel Cohen-Or).

The overall idea is to describe the neighbourhood of the source pixel with a plane respecting the average colour gradient in the area. We hope this will allow for smoothing which follows gradients in the image more closely than if we simply used averages of pixel values to describe regions.