Importing text file

[postformac]

Hi,

I have a piece of equipment that is logging hex data to a text file and I'm having problems splitting the test file up into array data. The problem is that the text file uses a varying number of spaces as delimiters and I don't know how to handle this to get the file arranged properly in my array.

I want to split each "visually obvious" column into a separate column in the array. I have tried doing this by using the import from spreadsheet file function using a single space as the delimiter which almost works (although it gives a load of blank columns that I could cut out) but it offsets the columns by 1 each time the logged counter goes up by a factopr of 10 (as this effectively removes one blank space between the data fields).

Is there any way to use blank space in general as a delimiter without speciying how many spaces? I have attached a copy of a cut down file so you can see the format.

Thanks!

Data.txt

[Francois Normandin]

Hi,

I have a piece of equipment that is logging hex data to a text file and I'm having problems splitting the test file up into array data. The problem is that the text file uses a varying number of spaces as delimiters and I don't know how to handle this to get the file arranged properly in my array.

I want to split each "visually obvious" column into a separate column in the array. I have tried doing this by using the import from spreadsheet file function using a single space as the delimiter which almost works (although it gives a load of blank columns that I could cut out) but it offsets the columns by 1 each time the logged counter goes up by a factopr of 10 (as this effectively removes one blank space between the data fields).

Is there any way to use blank space in general as a delimiter without speciying how many spaces? I have attached a copy of a cut down file so you can see the format.

Thanks!

Your file seems based on a fixed width for each columns, so you can take advantage of this and scan each line. Though this filetype is limited to 999999 lines.

[dblk22vball]

Try this vi (saved in LV2009)

Basically you read each line, and parse out all of the spaces, adding each column to an array.

My method uses the OpenG vis (you will have to download them if you do not have them, although I highly recommend them). You could do it with NI vis too.

Read Hex Data File.vi

Edited January 27, 2010 by dblk22vball

[postformac]

Hi,

Thanks very much for the replies.

Francois - I also noticed the columns were fixed width after I posted originally and I have since made a program that works in a similar way to yours (although not quite as neatly...). What is your item above where it says "each columns pre-determined width"? I have not seen that before but it looks useful!

I did it as shown below where the file is first opened as a text file read by line and then re-shaped into a 2D array before being passed to my function to split the columns I want out of it. It seems to take a very long time to process the file though, is this because I am repeatedly passing so much data from one side of the loop to the other? The time is spent in the function that splits the strings up rather than the calling function.

dblk - I don't have the openG VIs and had never heard of them, I am downloading them now though so I will look at your example when it has finished, thanks.

Main.vi

Split strings.vi

[dblk22vball]

I tested my vi against Francois', and his was twice as fast as mine

But at least mine will give you another way to do it, if you dont mind waiting, haha (0.4 sec vs 0.95 sec)

[Francois Normandin]

Francois - I also noticed the columns were fixed width after I posted originally and I have since made a program that works in a similar way to yours (although not quite as neatly...). What is your item above where it says "each columns pre-determined width"? I have not seen that before but it looks useful!

It's simply an array constant with integer as the datatype.

Check out the array palette to get an empty container. Drag your datatype in it to set it to I32 or anything you wish.

I tested my vi against Francois', and his was twice as fast as mine

But at least mine will give you another way to do it, if you dont mind waiting, haha (0.4 sec vs 0.95 sec)

But yours can parse any file with an arbitrary number of spaces in-between numbers.

Mine is limited to fixed-width columns.

[postformac]

Thank you Francois, I will have a play with that.

Dblk - I tried your file and although it may not be as quick as Francois method (still way quicker than mine!) it is useful in that it does not rely on fixed column widths. I tried a number of different configurations with the piece of equipment that is logging the data and actually the width of the first column is adjusted to suit however many samples you set as a maximum on the hardware. For this reason I think I will go with your method as it is more adaptable regardless of how the hardware is set up.

Thanks both very much for your help!