Suppose, you have two point clouds cloud 1 and cloud 2. Suppose, for each point P from cloud 1 you would like to get all the points from cloud 2 that are not farther then a given radius from P

Then, you could use points-join --all

> cat cloud-1.csv | points-join cloud-2.csv --radius 1.5 --all

Now, you also can specify variable radius for points in cloud 1. (E.g. your radius may vary depending on your point cloud density or structure, as it happened in our use case.)

Then you could run:

> cat cloud-1.csv | points-join --fields x,y,z,radius cloud-2.csv --radius 1.5 --all

(Note that, as a limitation, the point-specific radius should not exceed --radius value.)

Sometimes, when you run some slow processing of a point cloud and output the result in a file, you may want to monitor the progress. Then, the following trick may help you:

> some_slow_processing_script > points.csv &
> ( i=0; while true; do cat points.csv | csv-paste value=$i -; (( ++i )); done; sleep 30 ) | view-points --fields block,,,,x,y,z

Suppose, you need to go through a dataset to pick images for your classification training data.

cv-cat view can be used for basic browsing, selecting images, and assigning a numeric label to them:

cat images.bin | cv-cat "view=0,,png"

The command above will show the image and wait a key press:

Press whitespace to save the file as <timestamp>.png, e.g. 20170101T123456.222222.png

Press numerical keys 0-9: save the file as <timestamp>.<num>.png, e.g. if you press 5: 20170101T123456.222222.5.png

Press <Esc> to exit.

Press any other key to show the next frame without saving.


view parameters have the following meaning:

The first parameter is the wait in milliseconds for key press, 0 to wait indefinitely.

The second parameter is the window title (irrelevant for labelling).

The third parameter is the image extension e.g. png, jpg ...; default ppm.


A few new features have been added to cv-cat accumulate filter.

Before, it accumulated images as sliding window of a given size. Now, you could also ask for fixed layout of the accumulated image. It sounds confusing, but try to run the following commands (press any key to move to the next image)

> # make sense of the input
> ( yes 255 | csv-to-bin ub ) | cv-cat --input 'no-header;rows=64;cols=64;type=ub' 'count;view=0;null'
> # accumulate as sliding window of size 4
> ( yes 255 | csv-to-bin ub ) | cv-cat --input 'no-header;rows=64;cols=64;type=ub' 'count;accumulate=4;view=0;null'
> # accumulate as sliding window of size 4 in reverse order
> ( yes 255 | csv-to-bin ub ) | cv-cat --input 'no-header;rows=64;cols=64;type=ub' 'count;accumulate=4,,reverse;view=0;null'
> # accumulate images in fixed order
> ( yes 255 | csv-to-bin ub ) | cv-cat --input 'no-header;rows=64;cols=64;type=ub' 'count;accumulate=4,fixed;view=0;null'
> # accumulate images in fixed order (reverse)
> ( yes 255 | csv-to-bin ub ) | cv-cat --input 'no-header;rows=64;cols=64;type=ub' 'count;accumulate=4,fixed,reverse;view=0;null'

For example, if you want to create an image from fixed number of tiles, you could run something like this:

> ( yes 255 | csv-to-bin ub ) | cv-cat --fps 1 --input 'no-header;rows=64;cols=64;type=ub' 'count;accumulate=4,fixed;untile=2,2;view=0;null'

Say, you process images, but would like to view them in the middle of your pipeline in a different way (e.g. increase their brightness, resize, etc).

Now, you can do it with cv-cat tee. For example:

> # make a test image
> ( echo 20170101T000000,64,64,0 | csv-to-bin t,3ui; yes 255 | head -n $(( 64 * 64 )) | csv-to-bin ub ) > image.bin
> # observe that the images viewed in tee are passed unmodified down the main pipeline for further processing
> for i in {1..100}; do cat image.bin; done | cv-cat --fps 1 "count;tee=invert|view;resize=2" | cv-cat "view;null"

You could specify (almost) any pipeline in your tee filter, but viewing and, perhaps, saving intermediate images in files seem so far the main use cases.

Recently, we found that cv-cat view stopped working properly, when used several times in the same cv-cat call.

Something like

> cat images.bin | cv-cat "view;invert;view;null"

would either crash or behave in undefined way. All our debugging has pointed to some sort of race condition in the underlying cv::imshow() call or deeper in X-windows-related stuff, thus, at the moment, it seems to be out of our control.

Use the following instead:

> cat images.bin | cv-cat "view;invert" | cv-cat "view;null"

cv-cat is now able to perform pixel clustering by color using the k-means algorithm.

for example:

> cv-cat --file rippa.png "convert-to=f,0.0039;kmeans=4;view;null" --stay

input image:

output image (4 clusters):

A new convenience utility ros-from-csv is now available in snark. It reads CSV records and converts them into ROS messages with the usual conveniences of csv streams (customised fields, binary format, stream buffering/flushing, etc).

Disclaimer: ros-from-csv is a python application and therefore may not perform well streams that require high bandwidth or low latency.

You could try it out, using the ROS tutorial Understanding Topics (

Run ROS Tutorial nodes:

> # in a new shell
> roscore
> # in a new shell
> rosrun turtlesim turtle_teleop_key


Send your own messages on the topic, using ros-from-csv:

> echo 1,2,3,4,5,6 | ros-from-csv /turtle1/cmd_vel

Or do a dry run:

> echo 1,2,3,4,5,6 | ros-from-csv /turtle1/cmd_vel --dry
  x: 1.0
  y: 2.0
  z: 3.0
  x: 4.0
  y: 5.0
  z: 6.0

You also can explicitly specify message type:

> # dry run
> echo 1,2,3 | ros-from-csv --type geometry_msgs.msg.Point --dry
x: 1.0
y: 2.0
z: 3.0
> # send to a topic
> echo 1,2,3 | ros-from-csv --type geometry_msgs.msg.Point some-topic

A new convenience utility ros-to-csv is now available in snark. It allows to output as CSV the ROS messages from rosbags or from topics published online.

You could try it out, using the ROS tutorial Understanding Topics (

Run ROS Tutorial nodes:

> # in a new shell
> roscore
> # in a new shell
> rosrun turtlesim turtlesim_node
> # in a new shell
> rosrun turtlesim turtle_teleop_key

Run ros-to-csv; then In the shell where you run turtle_teleop_key, press arrow keys to observe something like:

> # in a new shell
> ros-to-csv /turtle1/cmd_vel --verbose
ros-to-csv: listening to topic '/turtle1/cmd_vel'...

If you log some data in a rosbag:

> # in a new shell
> rosbag record /turtle1/cmd_vel

You could convert it to csv with a command like:

> ros-to-csv /turtle1/cmd_vel --bag 2017-11-06-14-43-34.bag

Sometimes, you have a large file or input stream that is mostly sorted, which you would like to fully sort (e.g. in ascending order).

More formally, suppose, you know that for any record Rn in your stream and any records Rm such that m - n > N, Rn < Rm, where N is constant.

Now, you can sort such a stream, using csv-sort, --sliding-window=<N>:


> ( echo 3echo 1; echo 2; echo 5echo 4 ) | csv-sort --sliding-window 3 --fields a
> ( echo 4echo 5echo 2echo 1echo 3 ) | csv-sort --sliding-window 3 --fields a --reverse

As usual, you can sort by multiple key fields (e.g. csv-sort --sliding-window=10 --fields=a,b,c), sort block by block (e.g. csv-sort --sliding-window=10 --fields=t,block), etc.

Sometimes, you have a large file or input stream that is mostly sorted by some fields with just a few records out of order now and then. You may not care about those few outliers, all you want is most of your data sorted.

Now, you can discard the records out of order, using csv-sort, e.g:

> ( echo 0; echo 1; echo 2; echo 1; echo 3 ) | csv-sort --discard-out-of-order --fields a
> ( echo 3; echo 2; echo 1; echo 2; echo 0 ) | csv-sort --discard-out-of-order --fields a --reverse

As usual, you can sort by multiple key fields (e.g. csv-sort --discard-out-of-order --fields=a,b,c), sort block by block (e.g. csv-sort --discard-out-of-order --fields=t,block), etc.

The ratio and linear-combination operations of cv-cat have been extended to support assignment to multiple channels. Previously, these operations would take up to 4 input channels (symbolically always named r, g, b, and a, regardless of the actual contents of the data) and produce a single-channel, grey-scale output. Now you can assign up to four channels:

ratio syntax
... | cv-cat "ratio=(r-b)/(r+b),(r-g)/(r+g),r+b,r+g"

The right-hand side of the ratio / linear combination operations contains comma-separated expressions defining each of the output channels through the input channels. The number of output channels is the number of comma-separated fields, it may differ from the number of input channels. As a shortcut, an empty field, such as in

ratio syntax shortcut
... | cv-cat "ratio=,r+g+b,"

is interpreted as channel pass-through. In the example above the output has three channels, with channels 0 and 2 assigned verbatim to the input channels 0 and 2 (r and b, symbolically), and the channel 1 (symbolic g) assigned to the sum of all three channels.

As yet another shortcut, cv-cat provides a shuffle operation that re-arranges the input channels without changing their values:

shuffle syntax
... | cv-cat "shuffle=b,g,r,r"

In this case, the order of the first 3 channels is reversed, while the former channel r is also duplicated into channel 3 (alpha). Internally, shuffling is implemented as a restricted case of linear combination, and therefore, other usual rules apply: the number of output channels is up to 4, it does not depend on the number of input channels, and an empty field in the right-hand side is interpreted as channel pass-through.

When using view-points, there often is a need to quickly visualise or hide several point clouds or other graphic primitives.

Now, you can group data in view-points, using groups key word. A source can be assigned to one or more groups by using the groups arguments. Basic usage is:

view-points "...;groups=g1,g2"

For example if we have two graphs as follows:

$ cat <<EOF > edges01.csv

$ cat <<EOF > nodes01.csv

$ cat <<EOF > edges02.csv

$ cat <<EOF > nodes02.csv

We can separate the graphs as well as group together nodes and edges of different graphs as follows:

$ view-points "nodes01.csv;fields=x,y,z,label;colour=yellow;weight=5;groups=graph01,nodes,all" \
	"edges01.csv;fields=first/x,first/y,first/z,second/x,second/y,second/z;shape=line;colour=yellow;shape=line;groups=graph01,edges,all" \
	"nodes02.csv;fields=x,y,z,label;colour=green;weight=5;groups=graph02,nodes,all" \

Try to switch on/off checkboxes for various groups (e.g. "graph01", "nodes", etc) and observe the effect.

A quick note on new operations in cv-calc utility. Time does not permit to present proper examples, but hopefully, cv-calc --help would be sufficient to give you an idea.

cv-calc grep

Output only those input images that conform a certain condition. Currently, only min/max number or ratio of non-zero pixels is supported, but the condition can be any set of filters applied to the input image (see cv-cat --help --verbose for the list of the filters available).

Example: Output only images that have at least 60% of pixels darker than a given threshold:

> cat images.bin | cv-calc grep --filters="convert-to=f,0.0039;invert;threshold=0.555" --non-zero=ratio,0.6

cv-calc stride

Stride to the input image with a given kernel (just like a convolution stride), output resulting images.

cv-calc thin

Thin the image stream by a given rate or a desired frames-per-second number.

csv-shape is a new utility for various operations on reshaping csv data.

For now, only one operation is implemented: concatenate:

Concatenate by Grouping Input Records

> ( echo 1,a; echo 2,b; echo 3,c; echo 4,d; ) | csv-shape concatenate -n 2

Note: For ascii text inputs the records do not have to be regular or even have the same number of fields.

Concatenate by Sliding Window


> ( echo 1,a; echo 2,b; echo 3,c; echo 4,d; ) | csv-shape concatenate -n 2 --sliding-window


> ( echo 1,a; echo 2,b; echo 3,c; echo 4,d; ) | csv-to-bin ui,c | csv-shape concatenate -n 2 --sliding-window --binary ui,c | csv-from-bin ui,c,ui,c