Jump to content

String to 1D String Array, painfully slow


Recommended Posts

Hi,

I have a project to bring out data from a log file.

To make it more manageable deeper in the desig I will convert the String to a 1D string array.

To make the function faster and to avoid memory copy I use reference and controller/indicator is made "hade" on front panel to speed up.

But even this the function is painfully slow.

My test file i only 600kB.

The final Array size is 23455 index.

And this is taking about 4 minutes!!!

A sharp file can bee 20-30MB and consist of 500 000 of row or more.

That can I do to speed up?

I have used allocate memory, and calculate no of row neaded and make rooms for 120byte in each index. Only doing this to make ONE allocate and not one allocate on every index I nead to create.

The time is spending on copy into array. The read file, allocate xxxx no of row is quick.

The system is a regular PC, of newer section.

Attach code and an ex logg file.

Labview 2011.

Hope I can get ideas on this.

Best regards.

Support_string_to_array.zip

Link to comment

I fully agree with Philip. You can right-click on the Read from Text File and select "Read Lines". The glyph will change. Set the count to -1 to read the entire file in lines (the default becomes 1 line).

The other option you have is to use the Spreadsheet String to Array.

Link to comment
To make the function faster and to avoid memory copy I use reference and controller/indicator is made "hade" on front panel to speed up.

Your deeper problem is one of abstraction; your taking the “wire branches make copies” abstraction too literally. Actually, the compiler will only make copies when necessary. And even if copies are made, it will take a lot of copies to match the low performance of storing data in UI elements (which mostly comes from necessary thread-switching into the UI thread). A DVR is what you should actually have chosen to avoid copies, although the best choice may have been to just let the compiler worry about about it.

Link to comment

Big thanks!

I use the recommended function "read from textfile" and use seting read lines.

Works great!

I have miss this function.

Thanks to drjdpowell, ok I will let the compiler to worry about internal memory instead of I worry.

Now the function read the file in a few seconds instead of min.

Big thanks to all.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.