Not enough memory to complete this operation error when converting large .tdms to .txt

[sts123]

My .vi works ok until input file is about 1GB in size. Otherwise, it doesn't run, throwing the error about memory.

 

 

to_Asci_3_range.vi

[cordm]

Two possibilities: use 64 bit version and have more memory, or use the offset and count terminals of TDMS Read to read data subsets.

[cordm]

Also, if you are exporting large TDMS files to even larger text files, you should think hard if that is really the right move. Find a way to handle TDMS files in the next program. People have written importers for e.g. MATLAB and Python.

[sts123]

16 hours ago, cordm said:

Also, if you are exporting large TDMS files to even larger text files, you should think hard if that is really the right move. Find a way to handle TDMS files in the next program. People have written importers for e.g. MATLAB and Python.

So, MATLAB or Python are more efficient to convert TMDS to .txt?

[codcoder]

There should be a way to work with very large files in LabVIEW without having to keep the entire file in memory.

Many years ago I worked with a very large file in Matlab (well, back then it was a very large file) and I extensively used the function memmapfile: https://se.mathworks.com/help/matlab/ref/memmapfile.html

It is a way to map a file on the harddrive and access its content without having to keep the entire file in workspace memory. A bit slower I assume but far less load on the RAM!

There must be a similar method in LabVIEW.

 

EDIT: I found this old thread: https://forums.ni.com/t5/LabVIEW/Is-there-a-way-to-read-only-a-portion-of-a-TDMS-file-without/td-p/1784752

This is something similar to what cordm refers to:

On 6/30/2025 at 9:33 AM, cordm said:

or use the offset and count terminals of TDMS Read to read data subsets.

Best practice regardless of language must always be to handle large files in chunks.

Edited 5 hours ago by codcoder

[Rolf Kalbermatter]

7 hours ago, sts123 said:

So, MATLAB or Python are more efficient to convert TMDS to .txt?

Of course not! But TDMS is binary, text is ... well text. And that means it needs a lot more memory. When you convert from TDMS to text, it needs temporarily whatever the TDMS file needs plus for the text which is requiring even more memory.

Your Matlab and Python program is not going to do calculation on the text, so it needs to read the large text file, convert it back to real numbers and then do computation on those numbers. If you instead import the TDMS data directly to your other program it can do the conversion from TDMS to its own internal format directly and there is no need for any text file to share the data.

Edited 3 hours ago by Rolf Kalbermatter