At that data-rate with those kind of intervals, I would pre-process the data before storing to disk, as mentioned before.
My approach is that I store the channel to individual binary files, not directly but via a buffer.
In one file the data is stored, in the other the timestamps, so you basically have an X-Y save.
Steps:
- compare signal values; if data is changed? add to the buffer datastring and to the timestring.
- If storage interval is reached, copy the buffers to the binary files
- If a certain time-frame is reached, say 1 hour, I would store to new files and auto-archive the older data.
In that way you get more but smaller files and only store relevant data, not values that do not change.
Typical applications are low-noise signals only gradually changing or only at small time-intervals, or digital I/O values.
On the decoding side, it takes additional effort.