Jump to content

Why TDMS?


Recommended Posts

Hello people
i saw many comment regarding TDMS. i have a problem regarding this. 
I am working on my final year project.
I was advised to use TDMS icons to make the database. But i don't understand how
to start and make it even though i undergo a lot of reading online and search for information. 
 

 

My Title

Lab Equipment Monitoring and Lab Component Sorting System using QR Codes

Synopsis

The system developed will keep track of all the part/components/devices and lab and will assist both students and staff in locating/ borrowing parts/components/ devices.

Key Elements:

- Design the interface for Lab equipment monitoring and borrowing system using QR code

- Design and develop the sorting system for the lab component for experiments

 

 

 

 

Software Control System

 

  A full featured database system is implemented which is more appropriate to monitor and keep track by sticking QR code on each equipment in the laboratory. It is a processing system where all the information such as maintenance records, quantity of components, equipment cost, product suppliers and all specifications about the equipment are recorded. A monitoring system helps to have quick data entry as well as decrease the workload in the laboratory. In addition it allows the user to store and retrieve component in a more structured way. In this proposed system, LabVIEW or excel software will be used to develop the software control system.

 

 

When borrowing equipment, the ID card of the student will be scanned. Hence his Name, TP Number will be stored in the system. The borrowed apparatus will be scanned along to store the model type, name of the apparatus and its location saved in the database under the name of the student. Specifically, the time and date of the borrowed item are added which will help to maintain the history. Similarly when the student is returning it, the date, and time is updated as well as the borrowed apparatus will be replaced to its original place. This is done on a weekly basis so that the person can easily check the system as well as all the equipment in the laboratory.

The working principle of the database is such that when the QR code on the apparatus is scanned, the decoded result should match with database. Therefore it is easier to know its location and specifications. As such the graphical user interface helps the user to find and search for equipment they need for their experiment.

Link to comment

Thank you both of you sirs Crossrulz and stobber.

Actually my supervisor told me no need to use sql server. This is the whole concept of my project.

If any1 have some sample. please send me.

 

 

Hello people
i saw many comment regarding TDMS. i have a problem regarding this. 
I am working on my final year project.
I was advised to use TDMS icons to make the database. But i don't understand how
to start and make it even though i undergo a lot of reading online and search for information. 
 

 

My Title

Lab Equipment Monitoring and Lab Component Sorting System using QR Codes

Synopsis

The system developed will keep track of all the part/components/devices and lab and will assist both students and staff in locating/ borrowing parts/components/ devices.

Key Elements:

- Design the interface for Lab equipment monitoring and borrowing system using QR code

- Design and develop the sorting system for the lab component for experiments

 

 

 

 

Software Control System

 

  A full featured database system is implemented which is more appropriate to monitor and keep track by sticking QR code on each equipment in the laboratory. It is a processing system where all the information such as maintenance records, quantity of components, equipment cost, product suppliers and all specifications about the equipment are recorded. A monitoring system helps to have quick data entry as well as decrease the workload in the laboratory. In addition it allows the user to store and retrieve component in a more structured way. In this proposed system, LabVIEW or excel software will be used to develop the software control system.

 

 

When borrowing equipment, the ID card of the student will be scanned. Hence his Name, TP Number will be stored in the system. The borrowed apparatus will be scanned along to store the model type, name of the apparatus and its location saved in the database under the name of the student. Specifically, the time and date of the borrowed item are added which will help to maintain the history. Similarly when the student is returning it, the date, and time is updated as well as the borrowed apparatus will be replaced to its original place. This is done on a weekly basis so that the person can easily check the system as well as all the equipment in the laboratory.

The working principle of the database is such that when the QR code on the apparatus is scanned, the decoded result should match with database. Therefore it is easier to know its location and specifications. As such the graphical user interface helps the user to find and search for equipment they need for their experiment.



I am very sorry. My internt is very slow. I didnt know i get all this response.

I dont get any update in my hotmail. 

Thank you very much all of u who helped me.

 

Actually i can conclude that its better to use TDMS right. but what do u mean by the actual database.

My supervisor told me not to use the SQL server. Make somthing easy concerning labview itself.:(

Link to comment
My supervisor told me not to use the SQL server. Make somthing easy concerning labview itself. :(

 

  1. SQLite is a serverless DBMS. You just work with a file directly. It's ridiculously easy.
  2. "LabVIEW itself", by which I assume your supervisor means the native APIs, doesn't have a good solution for this. "LabVIEW itself" will in fact make this harder. Your supervisor needs to let you use an appropriate tool for the job.

  • Like 1
Link to comment
Actually i can conclude that its better to use TDMS right. 

 

Wrong. Your specification actually states:

 

A full featured database system is implemented

 

TDMS is not a "fully featured" database system.

 

My supervisor told me not to use the SQL server. Make somthing easy concerning labview itself.:(

 

None of this is "easy" in LabVIEW.

 

1. LabVIEW doesn't support QR codes. There are some threads on this forum for some 3rd party solutions

2. You will not find an "off the shelf" solution as this is a specific requirement that many would outsource - you will have to write it..

3. If you have the Database Connectivity Toolkit - use that (you probably do as you are a student).

4. If you don't have the Database Connectivity Toolkit - use SQLite.

5. You will have to learn SQL.

  • Like 2
Link to comment
  • 4 months later...

Hi Sorry to necro this thread.

Direct DMA streaming from channels to disk including channel scaling and settings (my understanding is this happens without CPU intervention).

 

Is this quote from @Hoovahh correct? Specifically the part about not requiring CPU intervention?

 

If that is true, does it mean that when running parallel processes, it's better to let them maintain their own .tdms files, rather than message the data to a central "File IO" process which is maintaining a number of independent .tdms files? If it helps make the context clearer, I'm saving data from 3 different sources at different rates into 3 different .tdms files on a solid state drive.

Link to comment

Hi Alex,

 

I suspect it's not quite true but there are a couple of shortcuts.

 

As long as you are reading data into your application there is CPU intervention then. There are a couple of techniques with TDMS where this doesn't happen, I don't think it is direct DMA but it is lower level:

  1. Using the DAQmx logging the data is logged in the DAQmx driver layer rather than your application, it is also logged as raw data with scaling information making it very fast.
  2. You can read a DVR from FPGA targets instead of data and there are corresponding write functions for TDMS which can mean better performance.

One thing I would say for your situation either way is that TDMS files don't do great with being streamed at different rates, it keeps rewriting the header portion causing the file to be fragmented which means worse write performance and file sizes increase rapidly. You may be worth keeping them as different files anyway.

 

Cheers,

James

Link to comment

Hi James,

 

I was imagining that perhaps a performance boost could be obtained by keeping it as 3 different files, but making the TDMS files the responsibility of the process driving each relevant hardware component, rather than having a process (or actor) which functions purely as a File IO engine.

 

For the most part, I don't imagine there is much to be gained. In either case, a data copy must be made as the data is simultanesouly being displayed to the user (at reduced rate) and saved to disk (at full rate). If there is a difference, I'd be surprised and would attribute it to some low-level functionality like the ones you mentioned.

 

Incidentally, I've never heard of being able to read a DVR from an FPGA target. Do you have a link to a white paper or similar where I could read about how this works? I don't understand how one passes DVR's of data from FPGA to host? Or am I misunderstanding the level at which the DVR is created? Is it instead created at the point the host first receives the data (a DMA fifo perhaps)?

 

Many thanks for your help.

 

Cheers,

Alex

Link to comment

Hi Alex,

 

SUre it was a new feature in 2012 I think. The DVR is created in the RIO driver and is really intended to be fired straight into the TDMS function. The help page is at http://zone.ni.com/reference/en-XX/help/371599J-01/lvfpgahost/fpga_method_fifo_acqread/ but can't find much else. There is an example of it under Hardware Input Output>>FlexRIO>>High Throughput>>High Throughput Streaming.lvproj

 

One thing I have never worked out is that everything references it as an "external" DVR but I never found any documentation about what this differentiation means. The one important comment is that it must be deleted before the driver can acquire more data, it must be this.

 

Cheers,

James

  • Like 1
Link to comment

Very neat I didn't know this feature existed.

 

Back to the TDMS question.  I'd recommend you stick with the three separate logs at different rates because of the mentioned issue with huge log files, and the TDMS header being written all the time.  The good news is that TDMS files can be combined easily using a command line "copy /b" command.  So you can log to 3 separate files, each in their own group, then combine them into one file with 3 groups.

Link to comment
  • 2 weeks later...

A couple of follow up questions on the nature of TDMS files.

 

1) Do TDMS Files need to have their pointer maintained? As in, update the reference to the file after a write? (In a similar fashion to reading/writing files with low level vis).

 

2) If they don't need to have the pointer maintained (opened in append mode or something), how expensive is an open operation assuming the file already exists? In other words, can I get away with calling open every time I have data to stream?

 

3) Finally, if the answer to question 2 is yes you can get away with it, does the same hold true for the special Asynchronous versions which must be configured on open?

Link to comment
A couple of follow up questions on the nature of TDMS files.

 

1) Do TDMS Files need to have their pointer maintained? As in, update the reference to the file after a write? (In a similar fashion to reading/writing files with low level vis).

 

2) If they don't need to have the pointer maintained (opened in append mode or something), how expensive is an open operation assuming the file already exists? In other words, can I get away with calling open every time I have data to stream?

 

3) Finally, if the answer to question 2 is yes you can get away with it, does the same hold true for the special Asynchronous versions which must be configured on open?

After a Read, Write, Get Property, Set Property, or List Contents I know that the reference is not changed.  Meaning the TDMS reference going in, is the same as the one going out and there is no need to update the reference.  I think that is what you mean by is the pointer maintained.

 

An open operation does take a long time on large files, and is likely very hard drive intensive.  Like most file I/O it is best to open it once, at the start of your application, perform all the read and writes with the reference, and then close once.  I wouldn't recommend doing an open, then operation, then close, every time you have any data you want to read or write.

 

I've never used the Asynchronous versions but I would assume they work the same way where the TDMS reference doesn't change between operations.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.