Blog from April, 2018

Sometimes, one may need to repeat the same record, just as linux yes does. The problem with yes is that you cannot tell it to repeat at a given time interval.

Now, csv-repeat --ignore-eof can do it for you, which is useful for example, if you need to quickly fudge a sort of heartbeat stream, a simulated data stream, or alike:

> echo hello | csv-repeat --period 0.1 --ignore-eof | head -n5
hello
hello
hello
hello
hello
> echo hello | csv-repeat --period 0.1 --ignore-eof | csv-time-stamp | head -n5
20180420T034741.498771,hello
20180420T034741.600020,hello
20180420T034741.700202,hello
20180420T034741.800367,hello
20180420T034741.900539,hello

Binary mode is supported as usual.

points-calc nearest-(min/max) and percentile operations search within a given radius around each input point. This can take a lot of time for large amount of input data.

One way to speed things up is to, instead of finding the nearest min to each point in a given radius, find the minimum for the 27 voxels in the neighbourhood of the voxel containing the point. That computed value is assigned to each point in that voxel.

This optimization is used when points-calc nearest-(min/max) or points-calc percentile is given --fast command line argument. For example

> points-calc nearest-min --full --fast --fields x,y,scalar --radius 1
> points-calc percentile --percentile=0.03 --fast --fields x,y,scalar --radius 1

On large point cloud, like that of rose street (http://perception.acfr.usyd.edu.au/data/samples/riegl/rose.st/rose.st.*.csv.gz ), optimized operations were found to be 20 times faster for extremums and more than 100 time faster for percentiles.