Syncronization of three data sources

[Anders Björk]

Dear all

Have any of you written any code and could share some hints how to syncronize three different data sources, two of the can be made fairly in sync from the start (two daqmx units), but the third is a serial unit a fast GPS? The problem is the GPS sometimes lags behind some sec and therefore sometimes give two or more sets of position data (of course they are well time stamped as the data from the daqmx units).

I actually have a fourth data source that uses seriel interface and give data only every minute but the with 15/32 sec resolution, but for the moment I will use the supplied instrument software.

Sorry for asking a somewhat trivia question, but I stuck after thinking and analysis the problem and now I have to wrap-up things :thumbdown:

Best regards

Anders

[Mellroth]

QUOTE(Anders Björk @ Nov 11 2007, 02:26 PM)

...but the third is a serial unit a fast GPS? The problem is the GPS sometimes lags behind some sec and therefore sometimes give two or more sets of position data (of course they are well time stamped as the data from the daqmx units).

Hi Anders,

have you tried to use a timed loop for the GPS data, using the DAQmx sample clock as the timing source?

/J

[Anders Björk]

QUOTE(JFM @ Nov 11 2007, 03:34 PM)

Hi Anders,

have you tried to use a timed loop for the GPS data, using the DAQmx sample clock as the timing source?

/J

Thanks J, I will look into that I might not have discribed the problem the GPS gives ten values each second and I need to do down sampling to one second. Maybe I have done a problem of something that isnt, I only need to pick a message that is within that second, or the same fraction.

BR

Anders

[LAVA 1.0 Content]

QUOTE(Anders Björk @ Nov 11 2007, 10:14 AM)

Thanks J, I will look into that I might not have discribed the problem the GPS gives ten values each second and I need to do down sampling to one second. Maybe I have done a problem of something that isnt, I only need to pick a message that is within that second, or the same fraction.

BR

Anders

It sounds like you could use an http://forums.ni.com/ni/board/message?board.id=170&view=by_date_descending&message.id=240328#M240328' target="_blank">Action Engine to do the syncronization. The AE could have a SR to hold the most recent set of readings across the board. AS each of the sub-systems receive their measurements, they use an appropriate action to update the fields held in the SR. When the consumers of this data need to know the current values they read from the AE.

Ben

[Yuri33]

I have programmed large projects where I needed to synchronize DAQmx data with serial port data. The key to performing this task is to make sure you are doing buffered recording. This is easily accomplished by using queues.

What I do is create multiple parallel threads (while loops) in a producer/consumer architecture. You create one loop which will read the DAQmx data and queue it (1 queue element per sample), and another loop which will read the GPS data and queue that as well. Finally, a third loop monitors the 2 data queues and only dequeues a block of samples from both queues when enough data exists. In other words, let's say that the DAQmx data is producing 2 channels of data at 100 samples/s in a pretty steady fashion. In contrast, let's say your GPS data is producing 4 channels of data at 1 sample/s, but the data only comes in intermitantly. Both of the producer loops will queue up the data as soon as it comes in (the DAQmx loop at a steady rate of 100 queue elements per second and the GPS loop at an unsteady rate of 1 queue per second). You then have the consumer loop monitor the number of elements in each queue until an appropriate block of data is available (for example, 400 elements in the DAQmx queue and 4 elements in the GPS queue). When the conditions for a block have been met, your consumer queue can dequeue the appropriate block size, resample the data if needed, and display\record the data as needed.

Therefore, the only problem is ensuring that the two data streams are synchronized. Becuase you can't really "start" or "stop" the data that streams in from the serial port, you have to create an artificial "software start" for the serial data. I do this by initiating a flush of the serial port right before I start collecting data. That way, you know that all the data that streams in is from after the flush event. We know that all the DAQmx stuff can be set to start at the same time (e.g., RTSI start triggers, etc.). So the best we can do at synchronizing the serial data with the DAQmx data is to flush the serial port, and then immediately set the start trigger of the DAQmx tasks. After that, the queued producer/consumer model described above ensures that the data is appropriately recorded.

[george seifert]

So is passing the data between loops via a queue better than passing the values by reference? Passing by reference seems more straightforward, but I guess I wouldn't be surprised if it's less efficient.

George

[Yuri33]

QUOTE(george seifert @ Nov 12 2007, 02:08 PM)

So is passing the data between loops via a queue better than passing the values by reference? Passing by reference seems more straightforward, but I guess I wouldn't be surprised if it's less efficient.

Yes, passing values by queue is waaaaaaaaay better than by reference. Passing values by reference always causes a context switch to the UI thread, which is the slowest thread there is. Additionally, passing values by reference does not easily allow for buffered data transfers like a queue does. In fact, using references can cause race conditions, which are Very Very Bad .

The only time I use references to pass values are if I'm "mimicing" what a user might do (in which case you are already at "UI speed"), or if I need to change a singular value (like a single numeric or boolean) in a non-deterministic manner. Even in these two cases, I can usually find a better way to perform the task (user events, functional globals, etc.).

[george seifert]

QUOTE(Yuri33 @ Nov 12 2007, 02:51 PM)

Yes, passing values by queue is waaaaaaaaay better than by reference.

I'm glad I kept up with this thread now. One quick follow up. One of the values I pass by reference updates a graph on my front panel. In that case is it still better to pass the value in a queue (there are 4 other values that need to be passed so I'll still look at settting up the queue)?

George