Jump to content

Recursive Directory Listing VI


sam

Recommended Posts

This VI give the directory listing of files and directories, it is also able to do a recursive listing.  I was using the OpenG VI but that could not handle very large directories efficiently.

Any comments are welcome.

-Sam

Hi Sam,

:lightbulb: I was wondering if you are interested in helping to make the OpenG version more efficient. Perhaps you would want to contribute some of your improvements.

Cheers,

-Jim Kring

Link to comment
Any comments are welcome.

It took me a while to grasp why your VI would be more efficient than other implementations. I have my own version that is more efficient than the OpenG version as well. I compared my approach to yours and noticed no performance benefit on several thousand files and directories.

I noticed that you are initializing an array of filepaths to 500 items. You resize and build an array of another 500 items at a time when you require it. This seems like an excellent approach however as i said, this didn't show up in the performance comparisons I did between my version and yours. When is the benefit evident?

I also noticed that Windows does some caching of directory access. This seems apparent when you run a recursive search for the first time regardless of what implementation you choose.

I've attached my version (LV7). Let's compare notes... :)

Download File:post-10-1077255628.vi

Link to comment
  • 1 month later...

Hi Mike, Sam, Jim

In my recursive directory listing I ignore Windows shortcuts. We had a Coop student accidentally create a a shortcut to the same directory the shortcut was in. Caused a good infinite loop and few hours to figure out why the software was filling all the memory. You can steal a prototype from the Tree example (called Directory Hierarchy in Tree Control.vi) in LabVIEW 7. The shortcut check is called Validate.vi.

Dean Mills

Meikle Automation

Link to comment

Hi Michael

I just took a look at your List Directory and thought there would be something to improve.

Look at the attached version. On my computer it is almost two times faster in enumerating large directory trees even if it is called first in the timing test to not give it advantage through Windows directory list caching.

Download File:post-10-1081496463.vi

Link to comment
In my recursive directory listing I ignore Windows shortcuts.  We had a Coop student accidentally create a a shortcut to the same directory the shortcut was in.  Caused a good infinite loop and few hours to figure out why the software was filling all the memory.  You can steal a prototype from the Tree example (called Directory Hierarchy in Tree Control.vi) in LabVIEW 7.  The shortcut check is called Validate.vi.

Yes Dean this is a great idea. I took a look at the VI you were writing about. I have an issue however with the label that NI gave to the indicator. It should not be labeled "Shortcut or not a path?". That implies that the boolean will be true if it's a shortcut or not a path, which it is not. It would be better to call it "Valid Path?". If you look at the icon it says "Valid?" right on it. I know this is minor, but still unacceptable in my opinion. Either you apply standards to your coding or you don't.

In any case...

It seems simple enough to integrate this code into the Recursive Directory Listing VI. It would make sense to add it into Rolf's version since it appears to be the fastest. I guess a switch to turn link checking on or off would be nice too. I will take all the suggestions and enhancements in this topic and create a VI which can then be submitted to the OpenG library. Whoever has submitted versions or suggestions to this topic will be given credit. Personally I use the OpenG library quite a bit however I don't use certain VI's in it because they are not optimized. This is our chance to help out and create a VI that we can all use.

Since most of you who have posted here are already willing to share your code, I will assume that you agree with the OpenG licensing. However if you are not familier with it, please read it over here. You must agree to this license if you want to participate.

So, on that note, can anyone suggest any other improvements to the VI's listed in this topic? You can take a previously submitted VI and re-write it (like Rolf did) or create another approach. I will work on creating a test routine to provide performance checking. If you have suggestions on a test VI please let me know or just submit it here.

post-10-1081821927.gif?width=400

Link to comment

One thing Michaels' code allows to do is to be able to add the capability to define file patterns.

If you pass anything but *.* or leave unwired you will not get back any Folder names.

I myself use file patterns.

I have changed mine and removed the FOR loop for the file names and brought the whole thing inside the WHILE loop and the same steps for folder is done on files.

I would post it again but your approach is much easier on the eyes.

-Sam

Link to comment
One thing Michaels' code allows to do is to be able to add the capability to define file patterns.

If you pass anything but *.* or leave unwired you will not get back any Folder names.

Well that is not entirely true. It's only true because you usually don't have folders with endings but if you name your folders for instance Folder.dir a search pattern of *.dir will return those folders. Of course I agree with you that looking for particular file patterns is a nice feature and I thought the OpenG version does that also. The problem with this is the requirement of the two List Directory Nodes for each hierarchy level which will almost take double the time than one node does.

I think with preallocating the arrays and such one can win another few percent but the real time spent is in the List Directory node.

Link to comment
  • 1 month later...

All this talk on Info-LabVIEW lately regarding recursive algorithms has prompted me to post a version of the Recursive directory listing VI that uses true recursion in LabVIEW. By this I mean using a VI that lists the directory structure of one directory and then have that VI call itself for any subdirectories it may find along the way.

I realize this setup creates a huge performance hit however I think it should be included in this posting for completeness so it can be compared to other implementations.

Enjoy.

Download File:post-10-1086381407.zip

Link to comment
  • 5 months later...
I realize this setup creates a huge performance hit

Here's a modified version of Michael's VI that doesn't have a "huge" performance hit. The trick is to minimize the number of times that the VI calls itself (due to the Open VI reference call). On my computer the listing of the LabVIEW directory took the following times:

Michael"s VI 2399 (ms)

Rolf K's VI 478 (ms)

Michael's VI modified 552 (ms)

OpenG 6717 (ms)

The test VI as well as Michael's and Rolf's original VIs are included.

Brian

Download File:post-987-1100028178.zip

Link to comment
  • 2 months later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.