Logging 4 Channels to TDMS - am I losing data?

[cpipkin]

Hello,

I'm using the 9229 and borrowed a community example to log 4 voltage inputs to TDMS. I need to output RMS voltage on channels 1&2 and the wavform on channels 3&4.

Essentially what I've done is averaged channels 3&4 using the same # of samples that the RMS is averaged, that way I'm able to make sure they are time synchronized (see code attached). Another advantage doing averaging the samples is that I am reducing the amount of data to analyze later.

When i compare the TDMS read results to the # of samples in the TDMS file there seems to be a discrepancy in time. I tried to add a time stamp to the logged TDMS file but couldn't get it to work. besides using time stamps, Is there an easy way to confirm that I am saving all of the data I am capturing? Eventually i will be logging data at 30min-40min intervals so I want to make sure that i'm not losing data.

 

TDMS_Logging_Simple_4chan.vi

[hooovahh]

As far as the code goes I think it is pretty simple, and looks right.  You have continuous sampling which is good, multiple finite reads is something I wouldn't recommend here.  I might suggest a few improvements to have better performance, not that what you are doing is wrong.  But I notice you are reading 28 samples at a time, and your DAQ session is setup to sample at 1612.9 Hz.  This means your loop rate will be no less than 17ms per iteration.  

This seems relatively fast.  Imagine if updating the graphs, and redrawing them, and logging the samples to disk takes 20ms what will happen?  Well samples won't be lost, right away it will just lag behind, and the data you log will be more and more from the past, until your DAQ read buffer fills up and then you'll get an error and stop.  Also even if you don't full the buffer and error out, clicking Stop will stop and the data in the buffer that you haven't read out yet won't be logged either.  

To help with this you can choose to not update the UI with every iteration, but instead just update it a couple of times a second which will probably be enough for most users.  Also instead of logging each sample as they come in, you can build a buffer of samples, and then when you get so many, log them at once.  The overhead of logging single point data to a TDMS can be larger than the payload of data to log.  This may mean a few changes with shift registers to keep track of the data in your own buffer, and changing the charts to graphs.

Of course a more simple solution if you can get away with it is to take more samples at a time, or making a seperate loop where you log, and update the UI.

[cpipkin]

Hooovahh, thanks for the insight. I am actually measuring wire that is moving through a laser micrometer at 14 inches per second. The reason i'm reading (and averaging) 28 samples at a time is so I can measure the wire every 0.25 inches.

I like the idea of building a buffer of samples and then logging that buffer at once. Would this mean a for loop within the while loop i already have? I can't quite picture how this would be done.

Thanks!

[hooovahh]

Sure thing, attached is a somewhat crude approach that isn't easily scaleable but I think it is pretty simple.  You have 4 shift registers for the data to log, and 4 more for the data in the graph instead of chart which has the benefit of having it's own buffer, but you have less control over it, like when it gets updated.

If you have more than 30 samples of data log it, this should be roughly once every 500ms.  This can be increased or decreased as needed.  After getting 30 new sample the data is appended to the end of the graph data, but if more than 1000 samples are in a graph, only the last 1000 are kept and shown.  Also the graphs will only be updated at most every 250ms.

Then after it stops executing the samples in the buffer that haven't been logged yet, are logged.  This will be samples between 0 and 30 that haven't been logged because there wasn't more than 30 at a time yet.

I also added a feedback node to measure the loop rate and put it on an indicator and chart.  Removing this will likely cause better loop rates, but if I were you I'd be curious what rates you get, and maybe remove this code once you are sure it is working well enough.

TDMS_Logging_Simple_4chan hooovahh edit.vi

[cpipkin]

@hooovahh so far this is working beautifully with simulated data. I've increased the buffer size to 280 (10x the loop) just to try and reduce system resources... it doesn't seem to be a problem now, but might as well play it safe!

I'm going to test it tomorrow with real data and will let you know how it goes.