I do this slightly differently (in Project Probe). Yours might be faster if you only look at link info from the file rather than loading the hierarchy but I have to look at lots of other VI stuff that require opening the VI anyway too (like re-entrancy, debugging enabled, icon, description etc). I also use it to sort VI refnums when force compiling hierarchies.
I create the file list similarly to you then I open each VI and check if it has dependencies (Get VI Dependencies Method Node). The VI opened and its dependencies are added to a [found] list. If a VI in the file list is already in the list or in memory then it doesn't need to be opened and could be considered already processed (it has a parent that was previously opened). If a VI has no parent and has no dependencies then it is an orphan. It can also find VIs in conditional disables.
So I deduce if it is an orphan from the dependencies of all the VIs