Why TDMS?

[Zahrah Chaumun]

Hello people
i saw many comment regarding TDMS. i have a problem regarding this. 
I am working on my final year project.
I was advised to use TDMS icons to make the database. But i don't understand how
to start and make it even though i undergo a lot of reading online and search for information. 
 

 

My Title

Lab Equipment Monitoring and Lab Component Sorting System using QR Codes

Synopsis

The system developed will keep track of all the part/components/devices and lab and will assist both students and staff in locating/ borrowing parts/components/ devices.

Key Elements:

- Design the interface for Lab equipment monitoring and borrowing system using QR code

- Design and develop the sorting system for the lab component for experiments

 

 

 

 

Software Control System

 

  A full featured database system is implemented which is more appropriate to monitor and keep track by sticking QR code on each equipment in the laboratory. It is a processing system where all the information such as maintenance records, quantity of components, equipment cost, product suppliers and all specifications about the equipment are recorded. A monitoring system helps to have quick data entry as well as decrease the workload in the laboratory. In addition it allows the user to store and retrieve component in a more structured way. In this proposed system, LabVIEW or excel software will be used to develop the software control system.

 

 

When borrowing equipment, the ID card of the student will be scanned. Hence his Name, TP Number will be stored in the system. The borrowed apparatus will be scanned along to store the model type, name of the apparatus and its location saved in the database under the name of the student. Specifically, the time and date of the borrowed item are added which will help to maintain the history. Similarly when the student is returning it, the date, and time is updated as well as the borrowed apparatus will be replaced to its original place. This is done on a weekly basis so that the person can easily check the system as well as all the equipment in the laboratory.

The working principle of the database is such that when the QR code on the apparatus is scanned, the decoded result should match with database. Therefore it is easier to know its location and specifications. As such the graphical user interface helps the user to find and search for equipment they need for their experiment.

[crossrulz]

Yeah, that is definitely a database situation.  TDMS is not good for this.  Look to the SQLite solution you were given before.

[Zahrah Chaumun]

Thank you both of you sirs Crossrulz and stobber.

Actually my supervisor told me no need to use sql server. This is the whole concept of my project.

If any1 have some sample. please send me.

 

 

Hello people
i saw many comment regarding TDMS. i have a problem regarding this. 
I am working on my final year project.
I was advised to use TDMS icons to make the database. But i don't understand how
to start and make it even though i undergo a lot of reading online and search for information. 
 

 

My Title

Lab Equipment Monitoring and Lab Component Sorting System using QR Codes

Synopsis

The system developed will keep track of all the part/components/devices and lab and will assist both students and staff in locating/ borrowing parts/components/ devices.

Key Elements:

- Design the interface for Lab equipment monitoring and borrowing system using QR code

- Design and develop the sorting system for the lab component for experiments

 

 

 

 

Software Control System

 

  A full featured database system is implemented which is more appropriate to monitor and keep track by sticking QR code on each equipment in the laboratory. It is a processing system where all the information such as maintenance records, quantity of components, equipment cost, product suppliers and all specifications about the equipment are recorded. A monitoring system helps to have quick data entry as well as decrease the workload in the laboratory. In addition it allows the user to store and retrieve component in a more structured way. In this proposed system, LabVIEW or excel software will be used to develop the software control system.

 

 

When borrowing equipment, the ID card of the student will be scanned. Hence his Name, TP Number will be stored in the system. The borrowed apparatus will be scanned along to store the model type, name of the apparatus and its location saved in the database under the name of the student. Specifically, the time and date of the borrowed item are added which will help to maintain the history. Similarly when the student is returning it, the date, and time is updated as well as the borrowed apparatus will be replaced to its original place. This is done on a weekly basis so that the person can easily check the system as well as all the equipment in the laboratory.

The working principle of the database is such that when the QR code on the apparatus is scanned, the decoded result should match with database. Therefore it is easier to know its location and specifications. As such the graphical user interface helps the user to find and search for equipment they need for their experiment.

I am very sorry. My internt is very slow. I didnt know i get all this response.

I dont get any update in my hotmail. 

Thank you very much all of u who helped me.

 

Actually i can conclude that its better to use TDMS right. but what do u mean by the actual database.

My supervisor told me not to use the SQL server. Make somthing easy concerning labview itself.

[Stobber]

My supervisor told me not to use the SQL server. Make somthing easy concerning labview itself.

 

1. SQLite is a serverless DBMS. You just work with a file directly. It's ridiculously easy.
   
2. "LabVIEW itself", by which I assume your supervisor means the native APIs, doesn't have a good solution for this. "LabVIEW itself" will in fact make this harder. Your supervisor needs to let you use an appropriate tool for the job.
   

[Zahrah Chaumun]

Dear stobber

I Went through the link you told. Actually, i just download the software SQL server and tried to code the system.

But actually i don't know how to do it. What is the meaning of the native APIs.

Do you have a program of a database based on the SQLite

[Stobber]

Son, I can't teach you all of programming. You need to step back and read some tutorials (and probably entire books). Here's a start: http://sql.learncodethehardway.org/

[Zahrah Chaumun]

Thank you . Okay i will try and see 

Thank again

[ShaunR]

Actually i can conclude that its better to use TDMS right. 

 

Wrong. Your specification actually states:

 

A full featured database system is implemented

 

TDMS is not a "fully featured" database system.

 

My supervisor told me not to use the SQL server. Make somthing easy concerning labview itself.

 

None of this is "easy" in LabVIEW.

 

1. LabVIEW doesn't support QR codes. There are some threads on this forum for some 3rd party solutions

2. You will not find an "off the shelf" solution as this is a specific requirement that many would outsource - you will have to write it..

3. If you have the Database Connectivity Toolkit - use that (you probably do as you are a student).

4. If you don't have the Database Connectivity Toolkit - use SQLite.

5. You will have to learn SQL.

[Zahrah Chaumun]

Thank you very much

[AlexA]

Hi Sorry to necro this thread.

Direct DMA streaming from channels to disk including channel scaling and settings (my understanding is this happens without CPU intervention).

 

Is this quote from @Hoovahh correct? Specifically the part about not requiring CPU intervention?

 

If that is true, does it mean that when running parallel processes, it's better to let them maintain their own .tdms files, rather than message the data to a central "File IO" process which is maintaining a number of independent .tdms files? If it helps make the context clearer, I'm saving data from 3 different sources at different rates into 3 different .tdms files on a solid state drive.

[JamesMc86]

Hi Alex,

 

I suspect it's not quite true but there are a couple of shortcuts.

 

As long as you are reading data into your application there is CPU intervention then. There are a couple of techniques with TDMS where this doesn't happen, I don't think it is direct DMA but it is lower level:

1. Using the DAQmx logging the data is logged in the DAQmx driver layer rather than your application, it is also logged as raw data with scaling information making it very fast.
2. You can read a DVR from FPGA targets instead of data and there are corresponding write functions for TDMS which can mean better performance.
   

One thing I would say for your situation either way is that TDMS files don't do great with being streamed at different rates, it keeps rewriting the header portion causing the file to be fragmented which means worse write performance and file sizes increase rapidly. You may be worth keeping them as different files anyway.

 

Cheers,

James

[AlexA]

Hi James,

 

I was imagining that perhaps a performance boost could be obtained by keeping it as 3 different files, but making the TDMS files the responsibility of the process driving each relevant hardware component, rather than having a process (or actor) which functions purely as a File IO engine.

 

For the most part, I don't imagine there is much to be gained. In either case, a data copy must be made as the data is simultanesouly being displayed to the user (at reduced rate) and saved to disk (at full rate). If there is a difference, I'd be surprised and would attribute it to some low-level functionality like the ones you mentioned.

 

Incidentally, I've never heard of being able to read a DVR from an FPGA target. Do you have a link to a white paper or similar where I could read about how this works? I don't understand how one passes DVR's of data from FPGA to host? Or am I misunderstanding the level at which the DVR is created? Is it instead created at the point the host first receives the data (a DMA fifo perhaps)?

 

Many thanks for your help.

 

Cheers,

Alex

[JamesMc86]

Hi Alex,

 

SUre it was a new feature in 2012 I think. The DVR is created in the RIO driver and is really intended to be fired straight into the TDMS function. The help page is at http://zone.ni.com/reference/en-XX/help/371599J-01/lvfpgahost/fpga_method_fifo_acqread/ but can't find much else. There is an example of it under Hardware Input Output>>FlexRIO>>High Throughput>>High Throughput Streaming.lvproj

 

One thing I have never worked out is that everything references it as an "external" DVR but I never found any documentation about what this differentiation means. The one important comment is that it must be deleted before the driver can acquire more data, it must be this.

 

Cheers,

James

[hooovahh]

Very neat I didn't know this feature existed.

 

Back to the TDMS question.  I'd recommend you stick with the three separate logs at different rates because of the mentioned issue with huge log files, and the TDMS header being written all the time.  The good news is that TDMS files can be combined easily using a command line "copy /b" command.  So you can log to 3 separate files, each in their own group, then combine them into one file with 3 groups.

[AlexA]

Thanks for the info guys,

 

I'll put a couple of hours into it this weekend and see what I can figure out about the DVR from FPGA. Will report back here.

 

Cheers

[hooovahh]

Topic Split, new topic being discussed.

 

http://lavag.org/topic/18305-fpga-dvr-tdms-log/

[AlexA]

A couple of follow up questions on the nature of TDMS files.

 

1) Do TDMS Files need to have their pointer maintained? As in, update the reference to the file after a write? (In a similar fashion to reading/writing files with low level vis).

 

2) If they don't need to have the pointer maintained (opened in append mode or something), how expensive is an open operation assuming the file already exists? In other words, can I get away with calling open every time I have data to stream?

 

3) Finally, if the answer to question 2 is yes you can get away with it, does the same hold true for the special Asynchronous versions which must be configured on open?

[hooovahh]

A couple of follow up questions on the nature of TDMS files.

 

1) Do TDMS Files need to have their pointer maintained? As in, update the reference to the file after a write? (In a similar fashion to reading/writing files with low level vis).

 

2) If they don't need to have the pointer maintained (opened in append mode or something), how expensive is an open operation assuming the file already exists? In other words, can I get away with calling open every time I have data to stream?

 

3) Finally, if the answer to question 2 is yes you can get away with it, does the same hold true for the special Asynchronous versions which must be configured on open?

After a Read, Write, Get Property, Set Property, or List Contents I know that the reference is not changed.  Meaning the TDMS reference going in, is the same as the one going out and there is no need to update the reference.  I think that is what you mean by is the pointer maintained.

 

An open operation does take a long time on large files, and is likely very hard drive intensive.  Like most file I/O it is best to open it once, at the start of your application, perform all the read and writes with the reference, and then close once.  I wouldn't recommend doing an open, then operation, then close, every time you have any data you want to read or write.

 

I've never used the Asynchronous versions but I would assume they work the same way where the TDMS reference doesn't change between operations.

[AlexA]

Thanks Hooovah