43 replies to this topic

[cookjr]

Hello,

I made a library for reading/writing MATLAB Mat-files in LabVIEW, and posted it on Sourceforge. It's written in pure G, with no external DLLs. Anyone is welcome to use the code if they want. It can be downloaded at http://matio-labview.sourceforge.net/. If you do try it out, I would appreciate any feedback you could provide.



Here is some sample code. I'm writing a cluster of various items to the file test.mat. This will create a structure in MATLAB with field names derived from the labels of the items in the cluster.





After running "load test.mat" in MATLAB, here is the result (shown in the variable editor):





Thanks,
Jim

[Anders Björk]

Jim

Nice work! I will definitely test this, I have been looking for a matlab file handler that could handle structs and cell arrays. I have made some preliminary test but any thing conclusive yet.

Edited by Anders Björk, 13 September 2009 - 02:05 PM.

[cookjr]

Hi Anders,

Thanks! I'm glad you might get use out of the library.

Regarding structs and cell arrays, I was thinking of changing how the "Append Data Element (Anything)" VI handles arrays of clusters. The current version writes them to file as a cell array of structs. I'm thinking a struct array might make more sense. Of course you would always be able to use the low-level VIs in the case that you want a cell array of structs. Does anyone have an opinion on this?

Jim

[Anders Björk]

Hi Anders,

Thanks! I'm glad you might get use out of the library.

Regarding structs and cell arrays, I was thinking of changing how the "Append Data Element (Anything)" VI handles arrays of clusters. The current version writes them to file as a cell array of structs. I'm thinking a struct array might make more sense. Of course you would always be able to use the low-level VIs in the case that you want a cell array of structs. Does anyone have an opinion on this?

Jim

Yes matlab structs for labview clusters make more sense and the reversed. Have you thought making a whos -file subVI? Would it be hard?

[cookjr]

A whos subVI would not be too hard. I'm assuming you're talking about a subVI that returns a list of variables/attributes from a mat file. I'll add that as a feature request. I'm busy right now making unit tests for the library, so I won't be able to get to it right away though.

Jim

[cookjr]

I added a whos-like VI to the latest version of the library. It returns the following information for each variable in a mat-file.

• variable name
• array dimensions
• data type (class)
• attributes (global?, sparse?, complex?)

I'm still working on the unit tests. Having some trouble passing strings in/out of the MATLAB Script Node...

Jim

[kinimod]

Nice work.

Unfortunately writing an array of clusters does not work.
There seems to be no variable in the mat file. If i understood correctly
an array of clusters should be mapped to a cell array of structs or
a struct array in Matlab.

Another issue (maybe related to the one above): Writing a waveform
does not work as well, although this is essentially just a LV cluster.

[cookjr]

Nice work.

Unfortunately writing an array of clusters does not work.
There seems to be no variable in the mat file. If i understood correctly
an array of clusters should be mapped to a cell array of structs or
a struct array in Matlab.

Another issue (maybe related to the one above): Writing a waveform
does not work as well, although this is essentially just a LV cluster.

Hi kinimod,

Thank you for taking the time to report this.

I hope we can isolate exactly why it didn't work for you, so the library can be improved for future users. It would be very helpful if you could provide more details about the steps you took to write the data.

• What version of the matio library did you use? Some bugs were found in earlier versions that have since been fixed.
• Did you install using the OpenG package format, or the zip package format?
• What steps did you take to write the array of clusters? A screenshot or some sample code would be helpful here.
• Could you send a sample of your data?

Regarding the waveform data: I think that the waveform data structure is a little different internally than a cluster, although they do some similar things. I don't normally use waveforms, so my original intent was to store them so that LabVIEW could read them back in properly instead of transforming them into something that MATLAB could use (i.e. a timeseries object). Were you able to read the waveform data back into LabVIEW?

Thanks again for your help.

Jim

[kinimod]

Hi Jim,

thanks for the quick reply.

[quote name='cookjr' date='24 September 2009 - 06:46 PM' timestamp='1253817975' post='66305']
Hi kinimod,

Thank you for taking the time to report this.

I hope we can isolate exactly why it didn't work for you, so the library can be improved for future users. It would be very helpful if you could provide more details about the steps you took to write the data.

• What version of the matio library did you use? Some bugs were found in earlier versions that have since been fixed.
  
• Did you install using the OpenG package format, or the zip package format?
  
• What steps did you take to write the array of clusters? A screenshot or some sample code would be helpful here.
  
• Could you send a sample of your data?

[\quote]

• Version 0.1.6. Should be the most recent one.
  
• I used the OpenG package and installed it with the VI package manager. My LabView version is 2009 (9.0)
  
• I simply modified the example VI given by the screenshot in your fist post (see attached image).
  
• Due to forum limitations i could not upload the resulting 'test.mat', but attached the VI used to create the data.

[quote name='cookjr' date='24 September 2009 - 06:46 PM' timestamp='1253817975' post='66305']
Regarding the waveform data: I think that the waveform data structure is a little different internally than a cluster, although they do some similar things. I don't normally use waveforms, so my original intent was to store them so that LabVIEW could read them back in properly instead of transforming them into something that MATLAB could use (i.e. a timeseries object). Were you able to read the waveform data back into LabVIEW?
[/quote]

Having waveforms as timeseries objects would be nice. But writing arrays of clusters would
already be fine, since converting waveforms to clusters is straightforward.

Writing and subsequent reading of waveforms from LabView works (see attached test_matio_waveform.vi).

Dominik


 test_matio.vi   11.69K   292 downloads

Attached Files

•  test_matio_waveform.vi   24.44K   271 downloads

Edited by kinimod, 24 September 2009 - 07:52 PM.

[cookjr]

Hi Dominik,

• Version 0.1.6. Should be the most recent one.
• I used the OpenG package and installed it with the VI package manager. My LabView version is 2009 (9.0)
• I simply modified the example VI given by the screenshot in your fist post (see attached image).
• Due to forum limitations i could not upload the resulting 'test.mat', but attached the VI used to create the data.

Looks like you did everything right. The only thing that stands out to me is LabVIEW 2009. I do all my testing with LabVIEW 8.2.1. It won't be a problem for me to run the tests using LabVIEW 2009 when I get back to work.

Writing and subsequent reading of waveforms from LabView works (see attached test_matio_waveform.vi).

Very good to know. Thank you for testing this.

Unfortunately, I'll be away from work until Monday, so I can't even look at your VIs until then. But I will definitely look into this problem and hopefully get it fixed asap. If you need MAT-files right away, you could use the lower-level VIs (Append Data Element (1D Double, N-D Struct, 1D Cell, etc.)). These give you more fine control over the result anyway.

Thanks

Jim

[cookjr]

Hi Dominik,

I should have caught this before by looking at your screenshot.

All variables in MATLAB must have a name. The build array primitive creates an unnamed array. The current version of the matio library does not enforce the naming of data inputs (basically because sub-elements of cell and struct arrays cannot have a name). So the "Append Data Element" VI saved your data appropriately, but MATLAB didn't know what to do with it, since it didn't have a name. Here is one way to fix it using the "Insert Into Array" primitive:


Future versions of the library will have some logic that will at least throw an error if there's a problem with element naming. Obviously some documentation would be beneficial for the current version.

Thanks again for helping test the library.

Jim

[kinimod]

Hi Jim,

thanks for the quick reply. This solved the previously reported problem.

Unfortunately there is another issue: when column vectors are created in MATLAB,
e.g. n x 1 arrays, they are not correctly read by the library.
The variable returned just contains a single element.
This does not occur, if row vectors, e.g. 1 x n arrays are created.

Further, when chaining multiple read-mat VIs it would be convenient to have a
copy of the endianess input as output.

And maybe it would be nice if one could specify the name of a variable to
read from the file instead of having to read them sequentially in alphabetic order.

Nevertheless the library is very useful already .

Best regards
Dominik

[cookjr]

Hi Dominik,

Unfortunately there is another issue: when column vectors are created in MATLAB,
e.g. n x 1 arrays, they are not correctly read by the library.
The variable returned just contains a single element.
This does not occur, if row vectors, e.g. 1 x n arrays are created.

matio-0.1-7 was just uploaded and it should have fixed this bug. For your info, the bug was given ID #2869727.

Further, when chaining multiple read-mat VIs it would be convenient to have a
copy of the endianess input as output.

This feature request was given ID # 2869732.

And maybe it would be nice if one could specify the name of a variable to
read from the file instead of having to read them sequentially in alphabetic order.

This feature request was given ID # 2869734. This is a great idea. I think a separate "Extract Data Element By Name" polymorphic VI could be created. FYI, I think MATLAB stores variables to a MAT-file in the order that they were created, not in alphabetical order. Currently, you have to read the variables using the Extract Data Element VI in the order that they occur in the file. The List Variables VI can be used to determine the order of the variables.

Nevertheless the library is very useful already .

Thanks! And thank you for your bug reports and feature requests. The library is getting better, because of your help. Any new features will be added in version 0.2.

Jim

[StevenT]

I tried to use the 1D to Cell function and it does not work. The data in cell one spills over to the next array element. The output does not load in MATLAB so I tried a data in data out LabView vi. The results are below. Using matio-0.1-7 release 45, LV8.6, MAT7.3.0 on XPSP3.

I justed tried the 2D to Cell with the same results.

Attached Thumbnails

• 
• 
• 
• 

[cookjr]

I tried to use the 1D to Cell function and it does not work. The data in cell one spills over to the next array element. The output does not load in MATLAB so I tried a data in data out LabView vi. The results are below. Using matio-0.1-7 release 45, LV8.6, MAT7.3.0 on XPSP3.

I justed tried the 2D to Cell with the same results.

Hi Steven,

The Append 1D Cell function requires a 1D array of "data elements". Data elements are the string output of any of the Append Data Element functions. This allows you to have different data types inside your cell array. All of the cell and struct functions work with data elements. Here's an example:



Please note that the data elements inside a cell or struct array must not be compressed, and they must not have a name.

Jim

Attached Files

•  cellArrayOfStrings_example.vi   25.03K   219 downloads

Edited by cookjr, 02 October 2009 - 03:43 PM.

[kinimod]

Hi Jim,

now I have problems converting large tdms files (about 70MB) to a mat file.
The matio library fails with an "Out of memory" error.
Is some part of the library leaking memory or is this simply a misusage of
the library VIs on my side?

Attached is one minimal VI that reproduces the problem.
The waveform_array_to_cluster_array SubVI transforms the
waveform array read from the tdms file into a cluster array
to be written by the matio library.

Dominik

Attached Files

•  test_tdms_to_mat.vi   17.31K   203 downloads
•  waveform_array_to_cluster_array.vi   15.08K   186 downloads

[cookjr]

now I have problems converting large tdms files (about 70MB) to a mat file.
The matio library fails with an "Out of memory" error.
Is some part of the library leaking memory or is this simply a misusage of
the library VIs on my side?

Hi Dominik,

Try the attached edited VIs. This won't give you the same structure as the "Anything" VI (cell array of structs), but it has the best chance at completing without throwing "out of memory" errors.

Note that the only thing I changed in the waveform_array_to_cluster_array VI was that I switched the "insert into array" primitive with the "replace array subset" primitive, since you were preallocating an array.

Efficiency is something that I would like to see improved in the library. It's not leaking memory, but it's not as efficient as it could be either. It's possible that using byte arrays and queues instead of strings would improve memory efficiency. This would be a major rewrite though.

Anyway, I hope the workaround VIs work for you.

Jim

Attached Files

•  jrc2_test_tdms_to_mat.vi   35.15K   226 downloads
•  waveform_array_to_cluster_array.vi   21.92K   207 downloads

[sparklejc]

Hi Jim,

Can you write a simple vi to read a mat file with a 1D array of double? (I am using Labview 8.6 and Matlab 2008). I can write a mat file and read it in matlab, but I can't seem to get read a mat file in labview, I got"Error 74 occured at Unflatten From String in level5.lvlib:Extract MAT-File Tag Field.vi->level5.lvlib:Extract Data Element (2D Array of Double).vi". Thanks very much.

Jason

[cookjr]

Hi Jim,

Can you write a simple vi to read a mat file with a 1D array of double? (I am using Labview 8.6 and Matlab 2008). I can write a mat file and read it in matlab, but I can't seem to get read a mat file in labview, I got"Error 74 occured at Unflatten From String in level5.lvlib:Extract MAT-File Tag Field.vi->level5.lvlib:Extract Data Element (2D Array of Double).vi". Thanks very much.

Jason

Hi Jason,

Could you send me a mat file that is similar to one that you're having trouble opening? It doesn't have to have real data in it. Then I'll see what I can do.

Jim

[sparklejc]

Hi Jason,

Could you send me a mat file that is similar to one that you're having trouble opening? It doesn't have to have real data in it. Then I'll see what I can do.

Jim

Hi Jim,

Thank you for your quick response. Here's a test .mat I'm trying to read.

Jason

Pls see attached

Attached Files

•  test.zip   289bytes   204 downloads