Skip to end of metadata
Go to start of metadata

An application publishes data, and one or more clients are listening. How do you solve these issues?

  1. The application publishes infrequently, but a client would like the data more often;
  2. The application is waiting for more input, but a client (which perhaps has just connected and thus might have missed the previous output) would like to know the last output line published.

Enter csv-repeat.

csv-repeat will pass stdin to stdout, repeating the last record after a period of inactivity.

For example:

{ echo -e "1\n2\n3"; sleep 10; } | csv-repeat --timeout=3 --period=1

It might be useful to know if the data is from the original application, or being repeated. csv-repeat can decorate the output with additional fields:

{ echo -e "1\n2\n3"; sleep 10; } | csv-repeat --timeout=3 --period=1 --append=repeating

Or perhaps you'd like the lines to be timestamped also:

{ echo -e "1\n2\n3"; sleep 10; } | csv-repeat --timeout=3 --period=1 --append=repeating,time

And of course csv-repeat supports binary data with --binary=<format>.

csv-repeat can also act as a watchdog on a data stream. In the following example, it will exit, if the input stream fails to send an update on time:

> csv-repeat --timeout 3
csv-repeat: input data timed out

You could use it to raise an alarm without disconnecting from the data producer. E.g. if you want to exit after 3 timeouts, you could write:

> for i in {1..3} ; do csv-repeat --timeout 1 ; echo TIMED OUT >&2 ; done
csv-repeat: input data timed out
TIMED OUT
csv-repeat: input data timed out
TIMED OUT
csv-repeat: input data timed out
TIMED OUT
  • No labels