Blog

Blog from December, 2016

If you have a single image or an image stream and would like to apply to each image a mask generated from the image itself, you could use the mask filter.

The mask filter allows you to run a pipeline of almost any filters available in cv-cat on an image to generate the mask and then apply it to the original image.

The benefits of the mask filter are more obvious when it is used on an image stream rather than a single image, since it allows to quickly prototype and deploy an image processing pipeline with fairly complex filtering and masking.

The example below demonstrates how to (very crudely) extract the vegetation from an image. To try it, right-click on the original image and save it on your computer as rippa.png.

> # mask image
> cv-cat --file rippa.png "mask=linear-combination:-r+2g-b|threshold:50|convert-to:ub;encode=png" --output no-header > masked.png
> # view
> eog masked.png
> # or for a quick view run
> cv-cat --file rippa.png "mask=linear-combination:-r+2g-b|threshold:50|convert-to:ub;view;null" --stay

In the cv-cat command above the mask filter is given as its parameter the pipeline: linear-combination:-r+2g-b|threshold:50|convert-to:ub. (The mask, which is very crude, says: take pixels with lots of green and not so much of red and blue; the mask must have CV_8U depth, while the output of linear-combination is always in floats, thus, the explicit convert-to operation.) cv-cat runs this pipeline on rippa.png and applies the result to rippa.png itself.

The syntax of the mask pipeline is the same as for normal cv-cat filter pipelines, except for the mask filter pipeline the separator between the filters is '|' and equal sign is ':'. (To improve the syntax, in future, we may implement separator escaping.)

I.e. if instead of applying the mask, you just want to save it in a file, you could run:

> cv-cat --file rippa.png "linear-combination=-r+2g-b;threshold=50;convert-to=ub;encode=png" --output no-header > mask.png
> eog mask.png 

You may want to apply a constant pre-computed mask to images. The example below will produce the same result as the first example.

> cv-cat --file rippa.png "mask=load:mask.png;view;null" --stay

A more elaborate example of applying filters to masked images

Masking can be the first step during feature extraction process. The next example demonstrates how a vegetation index can be applied on top of the mask operation.

We start with the following rainforest photo:

Now apply a mask and then, in the same line, apply a filter emulating one of the possible vegetation index filters (note: this is only a crude approximation used in this demo as the image lacks near-infrared data that are typically used by realistic vegetation indexes).

"Vegetation index"
> cv-cat --file rainforest.jpg --output=no-header "mask=linear-combination:-r+2g-b|threshold:50|convert-to:ub;ratio=r/g;convert-to=ub,256;encode=png" > filtered-rainforest.png
> eog filtered-rainforest.png

The ratio=r/g operation is the in-place "vegetation index" filter. Its output has floating-point precision with the expected range of output values around 1; therefore, we use convert-to before writing an output file.

You also could visualise intermediate results on the fly:

> cv-cat --file rainforest.jpg "view;mask=linear-combination:-r+2g-b|threshold:50|convert-to:ub;view;ratio=r/g;view;null" --stay

 

The black areas of the output image are not green as defined by the mask. The subsequent ratio filter distinguishes between the shades of green.

In a more general case, the ratio filter may use multiple channels, like in the (contrived) ratio=(100 + 2g + b)/(1.5*r + g + a). Note that r, g, b and a here are short-hands for channels 0, 1, 2 and 3 of the input image, respectively. The data in those channels do not have to correspond to red, green, blue or alpha and can be arbitrary false colours. According to the ratio syntax multiple terms shall be surrounded in brackets, constants can be integer or floating point values, and multiplication signs are optional.