Jump to content
Sign in to follow this  
Thang Nguyen

Read data at a specific time from TDMS

Recommended Posts

Hi,

I am working on a Testing project. The project will store data from the test in the TDMS file. What I need to do next is reading a sub-set of data from a start time to an end time which are specificed by user? How should I do this? I cannot find any TDMS VI provide the time selection.

Thank you for your time,

Thang Nguyen

Share this post


Link to post
Share on other sites

QUOTE(Thang Nguyen @ Jun 13 2007, 06:37 PM)

Hi,

I am working on a Testing project. The project will store data from the test in the TDMS file. What I need to do next is reading a sub-set of data from a start time to an end time which are specificed by user? How should I do this? I cannot find any TDMS VI provide the time selection.

Thank you for your time,

Thang Nguyen

TDMS VIs won't directly give you a time-based subset of your data, but all the information is there to do this with a little work yourself. Here are two approaches, the easy but inefficient approach and the more correct method (both methods assume you wrote your data to the TDMS file as waveforms):

Easy but Inefficient:

Read the whole channel from file and use the Waveform Subset VI to exctract the desired portion. Bad for very large channels.

More Correct Method:

If you write a waveform to a TDMS file, the underlying data is really stored as an array with a special channel property called wf_increment to store the dt value. You can read this property for your channel and get the double-precision dt value. The property for the t0 value I believe is called wf_offset (view the TDMS channel in the TDMS Viewer to get the exact names). Then you just calculate based on the t0 and dt the offset in the channel array where the desired waveform subset begins, and how long the subset will be. Then use the TDMS Read inputs Count and Offset to specify how many datapoints to read out. Then build an waveform from this array with the new t0 and output it.

Share this post


Link to post
Share on other sites

QUOTE(ragglefrock @ Jun 13 2007, 10:52 PM)

More Correct Method:

If you write a waveform to a TDMS file, the underlying data is really stored as an array with a special channel property called wf_increment to store the dt value. You can read this property for your channel and get the double-precision dt value. The property for the t0 value I believe is called wf_offset (view the TDMS channel in the TDMS Viewer to get the exact names). Then you just calculate based on the t0 and dt the offset in the channel array where the desired waveform subset begins, and how long the subset will be. Then use the TDMS Read inputs Count and Offset to specify how many datapoints to read out. Then build an waveform from this array with the new t0 and output it.

Thank you so much for your answer. The only think is although I log two channels with different rate, but the wf_increment the same and they have the value 1. And this is the dt of the waveform, not the dt of the logging rate to the tdms. I am thinking another solution to get the right value. Maybe I will read all of the channel, and divide this number for the number of data I have in the array. But this sounds stupid. Is there any better solution?

Thank you again,

Thang Nguyen

Share this post


Link to post
Share on other sites

QUOTE(Thang Nguyen @ Jun 14 2007, 07:22 PM)

Thank you so much for your answer. The only think make me confuse is although I log two channels with different rate, but the wf_increment the same and they have the value 1. Could you tell me why?

Thank you again,

Thang Nguyen

Most likely you are storing arrays and not waveforms, sadly.

Ton

Share this post


Link to post
Share on other sites

QUOTE(tcplomp @ Jun 14 2007, 12:59 PM)

Most likely you are storing arrays and not waveforms, sadly.

Ton

Nope, I store waveforms. I can see the waveform in the TDMS viewer :)

Share this post


Link to post
Share on other sites

QUOTE(Thang Nguyen @ Jun 15 2007, 01:19 AM)

Nope, I store waveforms. I can see the waveform in the TDMS viewer :)

Can you post a short TDMS?

And maybe the code of the saving?

Ton

Share this post


Link to post
Share on other sites

QUOTE(tcplomp @ Jun 14 2007, 11:19 PM)

Can you post a short TDMS?

And maybe the code of the saving?

Ton

I attach here the picture of VI I used to write to TDMS.

A TDMS example. But currently I don't have any input, these values just noise.

In TDMS, I have two groups of value. One is high frequency, another is low frequency. They have different number of value 1/3. Have the same wf_start_offset, but they have the same wf_increment. So they have different end time store of data.

Please modify the name of the tdms file. I don't understand why I cannot upload it.

Share this post


Link to post
Share on other sites

QUOTE(Thang Nguyen @ Jun 15 2007, 06:04 PM)

Please modify the name of the tdms file. I don't understand why I cannot upload it.

I think Michael limited that to a few extensions.

If I look at your file with DIADem I see the same as you... Can you verify that LabVIEW stores the data correct in the array.

Ton

Share this post


Link to post
Share on other sites

Yeah, they are stored in the right order. And I store data at different rate. You can see the number of data in high frequency is larger the number of data in low frequency. All of them start and stop at the same time.

Share this post


Link to post
Share on other sites

QUOTE(Thang Nguyen @ Jun 18 2007, 12:23 PM)

Yeah, they are stored in the right order. And I store data at different rate. You can see the number of data in high frequency is larger the number of data in low frequency. All of them start and stop at the same time.

Looking at the file with the TDMS Viewer, what you have is:

  • different channel lengths (high freq channels have 618 values, low freq channels have 224)
  • same dT (1.00 for all channels)
  • varying starting times (T0) for every channel

It looks like you are using waveforms to save single values. In that case, I'm not sure that DAQmx or other functions that put out waveforms will set dT correctly, because there is no second value to reference to. If you save a series of single values to a waveform channel, you need to be really sure that they are equally sampled. If you're not sure, you should rather split up the waveform data type and store the timestamps and the data values to different channels (e.g. one timestamp and one double channel).

Saving single values to TDMS like this is also not a very efficient thing to do. It is a lot more efficient to gather a bunch of values and write them as a larger array. You can have the TDMS API do that for you by setting the channel property "NI_MinimumBufferSize" to the number of values that you wish to buffer. In your case, good values might be 1000 or 10000.

Hope that helps,

Herbert

Share this post


Link to post
Share on other sites

QUOTE(Herbert @ Jun 18 2007, 01:58 PM)

Looking at the file with the TDMS Viewer, what you have is:
  • different channel lengths (high freq channels have 618 values, low freq channels have 224)
  • same dT (1.00 for all channels)
  • varying starting times (T0) for every channel

It looks like you are using waveforms to save single values. In that case, I'm not sure that DAQmx or other functions that put out waveforms will set dT correctly, because there is no second value to reference to. If you save a series of single values to a waveform channel, you need to be really sure that they are equally sampled. If you're not sure, you should rather split up the waveform data type and store the timestamps and the data values to different channels (e.g. one timestamp and one double channel).

Saving single values to TDMS like this is also not a very efficient thing to do. It is a lot more efficient to gather a bunch of values and write them as a larger array. You can have the TDMS API do that for you by setting the channel property "NI_MinimumBufferSize" to the number of values that you wish to buffer. In your case, good values might be 1000 or 10000.

Hope that helps,

Herbert

In my program I put store VI into the loop. It scans through the array name of channel, then, read and store the correspondent data into the file in that sequence. That why those channel don't have the same start time. But the diffference is just some msec.

I have two questions for you:

A) Is the wf_increment depends on the dt? If that is the case, I understand why the data are stored with the same wf_increment.

B) Do you mean that I should gather all of the Y value of each channel into an array, then put everything into the TDMS file? Could you tell me more detail how to store everything at once into the TDMS file?

Thank you,

Thang Nguyen

Share this post


Link to post
Share on other sites

Thang,

A) The idea here is that users should never have to touch properties like wf_increment or even know about them. We use the wf_xxx properties to store things that are embedded in LabVIEW data types (e.g. T0 and dT are embedded in the waveform data type). If you use waveforms correctly, all of these properties should be written and read without you doing something special. That of course only works if the waveforms have the correct values in them. Since you are asking - here are the important ones:

  • T0 is saved to wf_start_time (timestamp).
  • dT is saved to wf_increment (double).
  • If your data is not time-domain, wf_start_time will still be set, but your X0 value goes into wf_start_offset (double). This will happen for example with frequency-domain data or histogram results.
  • If you exchange data with DIAdem, you need to set the wf_samples property to something other than 0 (we ususally set it to the number of values in the incoming waveform, so in your file, it is 1). DIAdem will use this property to determine whether a channel is a waveform or not.

B) That's exactly right. The only thing you need to do is set the property NI_MinimumBufferSize (integer) for each of your data channels to 1000 or 10000 or something. The TDMS API does the buffering automatically (requires LV 8.2.1). This is not crucial to the functionality of your application, but it will speed up writing and reading quite a bit.

Unrelated) I see from the flags on your account that you're from Vietnam. I just came back from 2 weeks of vacation, visiting friends in Vietnam. They took me on a roundtrip through the country, including Hanoi, Ha Long, Nha Trang and Saigon. Best vacation I had in a long time. I'm addicted to Caffee Suo Da now :thumbup:

Herbert

Share this post


Link to post
Share on other sites

Herbert,

Thank you so much for your meanful information.

I want to confirm this information:

Is it always true that storing the Y value of waveform and seperate it with the timestamp is more efficient than storing the waveform?

Based on the property of TDMS channel (wf_increment, wf_start_time), I think TDMS naturally used to store waveform.

But in this case, it is easier for me to read data out, if I seperate the data and timestamp.

Love to hear about this:

QUOTE

Unrelated) I see from the flags on your account that you're from Vietnam. I just came back from 2 weeks of vacation, visiting friends in Vietnam. They took me on a roundtrip through the country, including Hanoi, Ha Long, Nha Trang and Saigon. Best vacation I had in a long time. I'm addicted to Caffee Suo Da now :thumbup:

I have never come to Caffee Suoi Da, actually, there are a lot of coffee in Vietnam ( a lot ...). I lived in Saigon. :). Do you feel hot there?

Thank you again,

Thang Nguyen

Share this post


Link to post
Share on other sites

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Sign in to follow this  

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.