user-friendly, Excel-ready, quick file save

[torekp]

I often save files in a format like this:

Each row and column can have a header, and sometimes each row has multiple headers, as in this example. By allowing multiple headers per row, I can save a more-than-2-dimensional array, with identifiers for each volume, page, etc. I prefer to save a text file rather than a true Excel file, because it slows things down too much to open and close Excel using ActiveX and so on.

Sometimes I also want a general description of the whole file, which could come at the beginning or end.

I have written too many semi-specialized file writers and readers, with different kinds of headers, etc. I want to "settle this once and for all" by writing two big polymorphic VIs, a write and a read. I'm looking to y'all for advice. Anyone done this? What should I bear in mind?

I'm thinking that there would be a two-D and a many-D version. Within each of those, the main data array could be integer, DBL, or string. The headers could be either same-type-as-data, or string (and if nothing is wired for the given header, it is skipped and we start right in with the data). That's 10 polymorphs of the write VI, and 10 of the read VI. Optional general description of the file, would be preceded and followed by some code like <file header>, </file header>. Whaddaya think?

[ShaunR]

I've got one of these that I wrote before we had the spreadsheet vi's.

You don't need that many. Just pre-process with a for loop (to give the page and you can check the dimensions to bypass) and use the build array just before you save to to append/prepend a header and footer (your description can be part of the header or footer or you can have a separate one). Then just wrap that as a poly vi for your different types.

You can also do things like checking that the file exists and only writing the header if its a new one and appending instead of overwriting so you can use it as a logger (don't forget an override switch though .

I actually prefer to save different "pages" to separate files as you can load each one up as a separate sheet and it makes graphing and things easier.

[gleichman]

Have you looked at TDMS? I do exactly what you are talking about in TDMS. You can save descriptions as properties and it won't interfere with your data. You can have mutiple pages (groups). I use three groups in my data files: hardware information, test configuration and test data.

To view TDM(S) data you can use the Excel add on tool to import the data into Excel. Unfortunately, this tool can sometimes stop working (click on icon in Excel ... nothing). I wrote my own export vi that allows users to select the data of interest. And of course TDMS works well with Diadem.

[torekp]

QUOTE

To view TDM(S) data you can use the Excel add on tool to import the data into Excel.

I just heard about this a few days ago, watching an NI presentation. I tried it and thought the way the data is organized in Excel looked ugly. But at your suggestion I looked into it more - the ugliness was my fault. With a little more effort on my part I could make it pretty. When I have used TDMS in the past, I had a need for speed, so I flattened 2D arrays into a single channel. I could create a For loop instead and label each column, and I could make sure the first column is the time, or whatever it is that I want as row labels.

You mentioned your own export vi - would you post it? That would make it easier to distribute my data: I wouldn't have to tell people to download the add-on Excel tool. Plus, I might "improve" it and you might even agree that my changes are improvements.

QUOTE ( @ May 21 2009, 08:52 PM)

You don't need that many. Just pre-process with a for loop (to give the page and you can check the dimensions to bypass) [...]

Can you explain more, please? Wouldn't there be a variable number of For loops depending if it's a 3D vs 4D (vs ...) array? Or, are you implying that I should flatten all arrays first, to reduce the # of polymorphs? Hmm, that might be smart... What do you mean about dimensions to bypass?

QUOTE

I actually prefer to save different "pages" to separate files as you can load each one up as a separate sheet and it makes graphing and things easier.

That probably makes sense. Then volumes can be folders, and shelves can be folders one level up, and ... and if I'm dealing with more-than-5D arrays then things are getting ugly! With TDMS, pages become Excel pages, volumes become different files, and so on.