A Mat-file IO Library

[cookjr]

Hello,

I made a library for reading/writing MATLAB Mat-files in LabVIEW, and posted it on Sourceforge. It's written in pure G, with no external DLLs. Anyone is welcome to use the code if they want. It can be downloaded at http://matio-labview.sourceforge.net/. If you do try it out, I would appreciate any feedback you could provide.

Here is some sample code. I'm writing a cluster of various items to the file test.mat. This will create a structure in MATLAB with field names derived from the labels of the items in the cluster.

After running "load test.mat" in MATLAB, here is the result (shown in the variable editor):

Thanks,

Jim

[Anders Björk]

Jim

Nice work! I will definitely test this, I have been looking for a matlab file handler that could handle structs and cell arrays. I have made some preliminary test but any thing conclusive yet.

Edited September 13, 2009 by Anders Björk

[cookjr]

Hi Anders,

Thanks! I'm glad you might get use out of the library.

Regarding structs and cell arrays, I was thinking of changing how the "Append Data Element (Anything)" VI handles arrays of clusters. The current version writes them to file as a cell array of structs. I'm thinking a struct array might make more sense. Of course you would always be able to use the low-level VIs in the case that you want a cell array of structs. Does anyone have an opinion on this?

Jim

[Anders Björk]

Hi Anders,

Thanks! I'm glad you might get use out of the library.

Regarding structs and cell arrays, I was thinking of changing how the "Append Data Element (Anything)" VI handles arrays of clusters. The current version writes them to file as a cell array of structs. I'm thinking a struct array might make more sense. Of course you would always be able to use the low-level VIs in the case that you want a cell array of structs. Does anyone have an opinion on this?

Jim

Yes matlab structs for labview clusters make more sense and the reversed. Have you thought making a whos -file subVI? Would it be hard?

[cookjr]

A whos subVI would not be too hard. I'm assuming you're talking about a subVI that returns a list of variables/attributes from a mat file. I'll add that as a feature request. I'm busy right now making unit tests for the library, so I won't be able to get to it right away though.

Jim

[cookjr]

I added a whos-like VI to the latest version of the library. It returns the following information for each variable in a mat-file.

• variable name
• array dimensions
• data type (class)
• attributes (global?, sparse?, complex?)

I'm still working on the unit tests. Having some trouble passing strings in/out of the MATLAB Script Node...

Jim

[kinimod]

Nice work.

Unfortunately writing an array of clusters does not work.

There seems to be no variable in the mat file. If i understood correctly

an array of clusters should be mapped to a cell array of structs or

a struct array in Matlab.

Another issue (maybe related to the one above): Writing a waveform

does not work as well, although this is essentially just a LV cluster.

[cookjr]

Nice work.

Unfortunately writing an array of clusters does not work.

There seems to be no variable in the mat file. If i understood correctly

an array of clusters should be mapped to a cell array of structs or

a struct array in Matlab.

Another issue (maybe related to the one above): Writing a waveform

does not work as well, although this is essentially just a LV cluster.

Hi kinimod,

Thank you for taking the time to report this.

I hope we can isolate exactly why it didn't work for you, so the library can be improved for future users. It would be very helpful if you could provide more details about the steps you took to write the data.

1. What version of the matio library did you use? Some bugs were found in earlier versions that have since been fixed.
2. Did you install using the OpenG package format, or the zip package format?
3. What steps did you take to write the array of clusters? A screenshot or some sample code would be helpful here.
4. Could you send a sample of your data?

Regarding the waveform data: I think that the waveform data structure is a little different internally than a cluster, although they do some similar things. I don't normally use waveforms, so my original intent was to store them so that LabVIEW could read them back in properly instead of transforming them into something that MATLAB could use (i.e. a timeseries object). Were you able to read the waveform data back into LabVIEW?

Thanks again for your help.

Jim

[kinimod]

Hi Jim,

thanks for the quick reply.

Hi kinimod,

Thank you for taking the time to report this.

I hope we can isolate exactly why it didn't work for you, so the library can be improved for future users. It would be very helpful if you could provide more details about the steps you took to write the data.

1. What version of the matio library did you use? Some bugs were found in earlier versions that have since been fixed.
   
2. Did you install using the OpenG package format, or the zip package format?
   
3. What steps did you take to write the array of clusters? A screenshot or some sample code would be helpful here.
   
4. Could you send a sample of your data?
   

[\quote]

• Version 0.1.6. Should be the most recent one.
  
• I used the OpenG package and installed it with the VI package manager. My LabView version is 2009 (9.0)
  
• I simply modified the example VI given by the screenshot in your fist post (see attached image).
  
• Due to forum limitations i could not upload the resulting 'test.mat', but attached the VI used to create the data.
  

Regarding the waveform data: I think that the waveform data structure is a little different internally than a cluster, although they do some similar things. I don't normally use waveforms, so my original intent was to store them so that LabVIEW could read them back in properly instead of transforming them into something that MATLAB could use (i.e. a timeseries object). Were you able to read the waveform data back into LabVIEW?

Having waveforms as timeseries objects would be nice. But writing arrays of clusters would

already be fine, since converting waveforms to clusters is straightforward.

Writing and subsequent reading of waveforms from LabView works (see attached test_matio_waveform.vi).

Dominik

test_matio.vi

test_matio_waveform.vi

Edited September 24, 2009 by kinimod

[cookjr]

Hi Dominik,

• Version 0.1.6. Should be the most recent one.
• I used the OpenG package and installed it with the VI package manager. My LabView version is 2009 (9.0)
• I simply modified the example VI given by the screenshot in your fist post (see attached image).
• Due to forum limitations i could not upload the resulting 'test.mat', but attached the VI used to create the data.

Looks like you did everything right. The only thing that stands out to me is LabVIEW 2009. I do all my testing with LabVIEW 8.2.1. It won't be a problem for me to run the tests using LabVIEW 2009 when I get back to work.

Writing and subsequent reading of waveforms from LabView works (see attached test_matio_waveform.vi).

Very good to know. Thank you for testing this.

Unfortunately, I'll be away from work until Monday, so I can't even look at your VIs until then. But I will definitely look into this problem and hopefully get it fixed asap. If you need MAT-files right away, you could use the lower-level VIs (Append Data Element (1D Double, N-D Struct, 1D Cell, etc.)). These give you more fine control over the result anyway.

Thanks

Jim

[cookjr]

Hi Dominik,

I should have caught this before by looking at your screenshot.

All variables in MATLAB must have a name. The build array primitive creates an unnamed array. The current version of the matio library does not enforce the naming of data inputs (basically because sub-elements of cell and struct arrays cannot have a name). So the "Append Data Element" VI saved your data appropriately, but MATLAB didn't know what to do with it, since it didn't have a name. Here is one way to fix it using the "Insert Into Array" primitive:

Future versions of the library will have some logic that will at least throw an error if there's a problem with element naming. Obviously some documentation would be beneficial for the current version.

Thanks again for helping test the library.

Jim

[kinimod]

Hi Jim,

thanks for the quick reply. This solved the previously reported problem.

Unfortunately there is another issue: when column vectors are created in MATLAB,

e.g. n x 1 arrays, they are not correctly read by the library.

The variable returned just contains a single element.

This does not occur, if row vectors, e.g. 1 x n arrays are created.

Further, when chaining multiple read-mat VIs it would be convenient to have a

copy of the endianess input as output.

And maybe it would be nice if one could specify the name of a variable to

read from the file instead of having to read them sequentially in alphabetic order.

Nevertheless the library is very useful already .

Best regards

Dominik

[cookjr]

Hi Dominik,

Unfortunately there is another issue: when column vectors are created in MATLAB,

e.g. n x 1 arrays, they are not correctly read by the library.

The variable returned just contains a single element.

This does not occur, if row vectors, e.g. 1 x n arrays are created.

matio-0.1-7 was just uploaded and it should have fixed this bug. For your info, the bug was given ID #2869727.

Further, when chaining multiple read-mat VIs it would be convenient to have a

copy of the endianess input as output.

This feature request was given ID # 2869732.

And maybe it would be nice if one could specify the name of a variable to

read from the file instead of having to read them sequentially in alphabetic order.

This feature request was given ID # 2869734. This is a great idea. I think a separate "Extract Data Element By Name" polymorphic VI could be created. FYI, I think MATLAB stores variables to a MAT-file in the order that they were created, not in alphabetical order. Currently, you have to read the variables using the Extract Data Element VI in the order that they occur in the file. The List Variables VI can be used to determine the order of the variables.

Nevertheless the library is very useful already .

Thanks! And thank you for your bug reports and feature requests. The library is getting better, because of your help. Any new features will be added in version 0.2.

Jim

[StevenT]

I tried to use the 1D to Cell function and it does not work. The data in cell one spills over to the next array element. The output does not load in MATLAB so I tried a data in data out LabView vi. The results are below. Using matio-0.1-7 release 45, LV8.6, MAT7.3.0 on XPSP3.

I justed tried the 2D to Cell with the same results.

[cookjr]

I tried to use the 1D to Cell function and it does not work. The data in cell one spills over to the next array element. The output does not load in MATLAB so I tried a data in data out LabView vi. The results are below. Using matio-0.1-7 release 45, LV8.6, MAT7.3.0 on XPSP3.

I justed tried the 2D to Cell with the same results.

Hi Steven,

The Append 1D Cell function requires a 1D array of "data elements". Data elements are the string output of any of the Append Data Element functions. This allows you to have different data types inside your cell array. All of the cell and struct functions work with data elements. Here's an example:

Please note that the data elements inside a cell or struct array must not be compressed, and they must not have a name.

Jim

cellArrayOfStrings_example.vi

Edited October 2, 2009 by cookjr

[kinimod]

Hi Jim,

now I have problems converting large tdms files (about 70MB) to a mat file.

The matio library fails with an "Out of memory" error.

Is some part of the library leaking memory or is this simply a misusage of

the library VIs on my side?

Attached is one minimal VI that reproduces the problem.

The waveform_array_to_cluster_array SubVI transforms the

waveform array read from the tdms file into a cluster array

to be written by the matio library.

Dominik

test_tdms_to_mat.vi

waveform_array_to_cluster_array.vi

[cookjr]

now I have problems converting large tdms files (about 70MB) to a mat file.

The matio library fails with an "Out of memory" error.

Is some part of the library leaking memory or is this simply a misusage of

the library VIs on my side?

Hi Dominik,

Try the attached edited VIs. This won't give you the same structure as the "Anything" VI (cell array of structs), but it has the best chance at completing without throwing "out of memory" errors.

Note that the only thing I changed in the waveform_array_to_cluster_array VI was that I switched the "insert into array" primitive with the "replace array subset" primitive, since you were preallocating an array.

Efficiency is something that I would like to see improved in the library. It's not leaking memory, but it's not as efficient as it could be either. It's possible that using byte arrays and queues instead of strings would improve memory efficiency. This would be a major rewrite though.

Anyway, I hope the workaround VIs work for you.

Jim

jrc2_test_tdms_to_mat.vi

waveform_array_to_cluster_array.vi

[sparklejc]

Hi Jim,

Can you write a simple vi to read a mat file with a 1D array of double? (I am using Labview 8.6 and Matlab 2008). I can write a mat file and read it in matlab, but I can't seem to get read a mat file in labview, I got"Error 74 occured at Unflatten From String in level5.lvlib:Extract MAT-File Tag Field.vi->level5.lvlib:Extract Data Element (2D Array of Double).vi". Thanks very much.

Jason

[cookjr]

Hi Jim,

Can you write a simple vi to read a mat file with a 1D array of double? (I am using Labview 8.6 and Matlab 2008). I can write a mat file and read it in matlab, but I can't seem to get read a mat file in labview, I got"Error 74 occured at Unflatten From String in level5.lvlib:Extract MAT-File Tag Field.vi->level5.lvlib:Extract Data Element (2D Array of Double).vi". Thanks very much.

Jason

Hi Jason,

Could you send me a mat file that is similar to one that you're having trouble opening? It doesn't have to have real data in it. Then I'll see what I can do.

Jim

[sparklejc]

Hi Jason,

Could you send me a mat file that is similar to one that you're having trouble opening? It doesn't have to have real data in it. Then I'll see what I can do.

Jim

Hi Jim,

Thank you for your quick response. Here's a test .mat I'm trying to read.

Jason

Pls see attached

test.zip

[cookjr]

Hi Jim,

Thank you for your quick response. Here's a test .mat I'm trying to read.

Jason

Pls see attached

Try the attached VI.

There is some documentation here, but it only has info on writing to a file right now.

If anyone wants to help out, they're welcome to add to it.

Jim

read_test.mat.vi

[Jhow88]

Hey Jim,

I've been using your book (LabVIEW For Everyone) and your openG software for a few weeks now, and I recently discovered the MatIO library you created and it looks exactly like what I need to store my data using clusters. I am currently running LabVIEW 8.6, and I was able to successfully install the VI manager on my computer and recreate an example VI posted on this forum (the VI was called test_matio.vi) just to see if my system was storing the data correctly .I ran into a few problems,mainly when I open the generated file, the elements of the clusters don't have names, so I end up with an error when I try to access them in Matlab. I noticed someone else had a similar error so I made sure to use "insert into array" function instead of "append data element' , but I am still having issues. I am pretty confident that the installation on my computer is correct, because I followed the same process on a different computer (a macintosh running v9.0, whereas this computer uses windows and runs v8.6) and I got everything to work.Unfortunately the computer which is not currently working is the one that needs to be setup to the DAQ, so I need to resolve this issue on the machine. If you could take a quick look at my VI to let me know if you see anything obvious I would really appreciate it..test_cluster_structure.vi

All the best,

-Jon

[cookjr]

Hi Jon,

I think you're confusing me with Jim Kring.

I can see if I can help you with the MatIO problem though. I'll get back to you when I have an answer.

Jim

[Jhow88]

Hi Jon,

I think you're confusing me with Jim Kring.

I can see if I can help you with the MatIO problem though. I'll get back to you when I have an answer.

Jim

Jim,

Thanks for getting back to me so quickly, and I apologize for the confusion. I had read that Jim Kring was very active in the open G community so when I saw the MatIO library and how well it was working my mind immediately made an association. That being said, this library is still fantastic and exactly what I am looking for, and I thank you for your work and development of it. I look forward to hearing feedback from you when you get the chance.

-Jon

[cookjr]

Hi Jon,

Thank you for the complements on the library. I hope it does everything you need.

I edited your VI and it seems to be working now. The problem was the elements of the cluster array constant were unnamed. This was ambiguous in the earlier post. I attached the fixed VI.

Let me know if you have any other questions.

Jim

test_cluster_structure.vi