Graphics, Imaging, and Games Lab

Following-up on the idea mentioned in a previous post (http://gigl.scs.carleton.ca/node/536), I am experimenting with mask inclusion counts as a method for recolouring images.

Procedure

The results below were produced using the following procedure:

1. Process the source image by building a filtering mask for each pixel. In this post, all results were produced with a "gamma" parameter value of zero.
2. Count the number of times that each pixel in the source image is included in a filtering mask. I will call pixels with lower counts "outliers" and pixels with higher counts "non-outliers".
3. Linearly scale the counts generated in Step 2 to values between 0 and 1.
4. Average the source image with a new colour, using the scaled counts of mask inclusions as a weighting for either the source image or the new colour.

I tried two variations of recolouring: In the first variation, non-outliers were recoloured more strongly than outliers, while in the second variation, I inverted the weighting system (i.e. 'weight' is replaced with '1-weight'), such that outliers were recoloured more strongly.

Results

Source Image

Visualizations of Mask Inclusion Counts

The following images illustrate the weights used during recolouration, where brighter tones are assigned to pixels with higher mask inclusion counts. Mask inclusion counts have been scaled linearly to brightness values in the full range of 0-255 (black to white).

Mask Size 64

Mask Size 1024

Recoloured images produced by weighted averages of RGB colours

Mask size 64, Recolouring non-outliers

Mask size 1024, Recolouring non-outliers

Mask size 64, Recolouring outliers

Mask size 1024, Recolouring outliers

Recoloured images produced by hue-component averaging

The images in the previous section are quite striking (garish) because of the strong colour contrast, but I find the loss of detail in strongly-recoloured areas undesirable. I decided to try converting the image into the hue, saturation, value colour space (using MATLAB's image processing toolbox), and then recolouring by only averaging the hue channel of the image with a new hue. The results of this process are shown below:

Mask size 64, Recolouring non-outliers

Mask size 1024, Recolouring non-outliers

Mask size 64, Recolouring outliers

Mask size 1024, Recolouring outliers

Discussion

The overall effect of the images presented above is, in my opinion, mostly unappealing, with the exception of the RGB-space averaging result, at a mask size of 64, where outliers were strongly recoloured. I like this particular image because the edges of features in the image are recoloured, causing a glow effect. It is likely that carefully selecting the filtering mask size, and perhaps using averaging weights in the range from zero to a value less than one, will result in a pleasing image. For this particular image, recolouring outliers has a nicer effect than recolouring non-outliers.

As for averaging only in the hue channel, it seems that this may not be a way to improve the recolouration process. I find that the images produced by hue-channel averaging look like pseudocolour images representing infrared radiation, for example. The colours in the images appear too vivid, and the gradations of colours are far too extreme to appear realistic. Furthermore, unnatural shadows and highlights are present in the image, corresponding to different hues, although this could perhaps be resolved by using a colour space in which brightness and hue are better decoupled.

(Image source: Bernard Llanos)