Missing external function in LV2019 64-bit executable

[Neil Pate]

I am pulling my hair out over this one...I am trying to run a LV2019 64-bit application as an executable on another PC. I have done all the normal things I do like install the runtime engine etc, but as soon as any of my code contains any VI which calls into lvanlys.dll I get the following error. 

 

This is the code:

I have done everything I can think of including re-installing the runtime engine, creating an installer from the build PC, copying all DLLs from the Runtime directory to the application data directory etc, none of this works.

This is just a regular PC, the only thing that is a bit out of the ordinary is that it does not have internet access, but surely this is not the cause? Could it be related to 64-bit?

Anyone else seen this in 2019 64-bit?

[Neil Pate]

Tried Dependency Walker (which I cannot say I really totally understand) and it looks like it is looking for LV18000_BLASLAPACK.DLL which I have located and tried copying to the directory holding the analys.dll, still no luck. I don't know what the hourglass next to this DLL is, maybe it means lazy-loaded?

[Neil Pate]

Seems there is a modern version of Dependency Walker called Dependencies, see it here: https://github.com/lucasg/Dependencies

Running this does not show any problems with my DLL and if I hover over the BLASPACK DLL it gives me a popup confirming my suspicion that it is delay loaded.

So weird...

[Neil Pate]

Just did the same test with LV2019 32-bit and it works fine on the other PC. 🥵

Also works fine in a VM so I guess its probably something borked on the PC. Time to reinstall the OS (as obviously NI Package manager refuses to cooperate and now I have a bunch of orphaned software).

Edited February 23, 2020 by Neil Pate

[Neil Pate]

So this just gets weirder!

Reinstalled the OS and the basic 2019 64-bit SP1 RTE. Still get the error.

Installed 2019 64-bit SP1 IDE, open a new VI and try and drop down Mean.vi and it gives me an error message saying A Dynamic Link Library Initialisation Failed!

So after a bit of poking around I dig into the Mean.vi and take a look at the DLL and it is configured to have the path e:\builds\penguin\labview\branches\2019\dev\dist64\resource\lvanalys.* Obviously this will not work as I dont have an E: or builds\penguin directory!

 

Edited February 25, 2020 by Neil Pate

[Neil Pate]

Installed LV2019 32-bit SP1 and it works fine, so this looks like something weird in 64-bit

[bmoyer]

This might be worth looking at but no solution was given:

https://forums.ni.com/t5/LabVIEW/Failure-to-initialize-lvanlys-DLL/td-p/3759330?profile.language=en

 

They were having the same trouble but with LV2017.

[Neil Pate]

Thanks, this is exactly the same problem I am getting! 

I have no solution yet and have tried everything I can think of apart from not using those functions which seems like a terrible idea. 

[Michael Aivaliotis]

Can't you just add the numbers and divide?

[Neil Pate]

On 2/28/2020 at 8:53 PM, Michael Aivaliotis said:

Can't you just add the numbers and divide?

Absolutely. This is what I have had to do to get things working. Unfortunately this is not the only VI I used that calls into this DLL and this is definitely a game of whack-the-mole I don't have the energy to play.

The problem is more... "why does this happen?". I am sure you know how frustrating it is when something works perfectly in the IDE but then has trouble in an executable. I thought I was at the point in my LabVIEW journey that I had experienced first-hand most of the weirdness that sometimes comes with the territory. This is altogether a new one for me though. 

 

[Michael Aivaliotis]

i get it. Sometimes you gotta patch and move on. In parallel, though, I'd open a ticket with NI to get them to spend some resources on worrying about your problem too.

[Aristos Queue]

Nothing looks broken to me in LV 2019.

I've attached my 64-bit LabVIEW 2019 Mean.vi... does it work for you?

I tried several different ways of moving the files around without any problems.

Mean (DBL).vi

[Aristos Queue]

Here's what should be happening...

The path saved inside Mean.vi is (should be) a symbolic path to the analysis library. The exact thing saved should be
       <resource>:\lvanlys.*
which you would see if you use the private Application method. (Actually, you'll see the "*" replaced by "dll" because the method does that small bit of platform-OS interpretation of the path.)

That symbolic path gets interpreted when the VI loads into memory. That means that...

• ...if you open up the Mean.vi and open the configuration dialog of the Call Library Node, you should see an absolute path that leads to the lvanalys.* on your machine.
• ...if you move the VI out of vi.lib and to some other location on disk, you should still see the same absolute path.
• ...if you open that VI with a different copy of LabVIEW, you should see the path of that other LabVIEW's analysis library.

But given what you're seeing...

It seems like whatever the absolute path is being saved is not being saved as a symbolic path.
          "e:\builds\penguin\labview\branches\2019\dev\dist64\resource\lvanalys.*"
is the directory on the build machine where the installer is created. I don't know of any way for this edit to be made, but clearly, it is happening.

It seems impossible that we could be making this mistake everywhere... we would have lots of customers complaining about an inability to build apps. Heck, the VI would be broken in the editor!

Very strange.

[Aristos Queue]

@NeilPate : please run the attached VI on your misbehaving Mean.vi and tell me what path it shows.

Read path to analysis lib.vi

[Neil Pate]

14 hours ago, Aristos Queue said:

.Heck, the VI would be broken in the editor!

It is broken in the editor! That is how I know about the e:\penguin directory, I am opening the VI directly in the LabVIEW IDE and having a look at the DLL configuration.

Unfortunately the offending PC is now at a customer site, as soon as I get access to it again I will try your experiments.

Thanks for your help, I would really love to get to the bottom of this.

[Aristos Queue]

Is there anything you can think of that has been done special to this machine? Did you get someone from NI to send you a custom patch that has never been generally released? Like, I'm seriously grasping at straws here... we grep'd the binary of Mean.vi -- that path does not appear anywhere in the shipping copy. So somehow you have a copy of Mean.vi that is not the one that is installed by an installer. I would say "it's a fluke" except for that post earlier that someone else had this happen to their LV2017! Now I have a real mystery.

[Aristos Queue]

Is this desktop LV or target LV? What is the actual path on disk to that Mean.vi?

[Neil Pate]

AQ, this was a brand new PC. I personally installed the most recent version of Win10 Pro.

I then installed LV2019 64-bit SP1 using the offline ISOs from ni.com. No fancy patches or special versions.

Then I open LabVIEW, start a new VI and drop down the Mean.vi and it is broken with the error as I have described. Several other VIs that link to analys.dll are also broken in the same way (I looked at some of the Cartesian shift VIs)!

I tried this same experiment on the same PC using LabVIEW 2019 32-bit SP1 and the problem did not exist.

Edited March 5, 2020 by Neil Pate

[Neil Pate]

@Aristos Queue I have the PC back in my possession and have run the VI you asked.

Here is the result.

 

Now, if I immediately try and drop Mean.vi I can see it is trying to load it from the correct directory

 

 

But if I do into the VI and open the Call Library Function node I get the e:\builds\penguin stuff

 

 

The directory on disk is correct:

 

 

Does this make any sense to you?

 

 

Edited March 17, 2020 by Neil Pate

[Aristos Queue]

3 hours ago, Neil Pate said:

@Aristos Queue

Does this make any sense to you?

Weirdly, yes, it does make sense. This is what would happen if the DLL was corrupt and couldn't load. The DLL is found on disk, but it could not load. I don't know exactly how the node is getting the build machine's path, but that would be the path it had the last time it resolved successfully, so I'm betting it is encoded in there somewhere and being used as a fallback location.  

Has someone been rewriting your DLL on disk? Can you try copying that DLL from a clean installation? 

[Dataflow_G]

Out of interest, what are the MD5/SHA1 hashes of the problematic lvanlys.dll? I have a LV2019 SP1 64-bit install (via NIPM, not the offline iso), and analysis functions are working. The copy of lvanyls.dll I have match the file size and modification date in your screenshot. Its hashes are:

MD5: 8b57b1ab47c00387b370d3d0fac0a246
SHA1: 7d45297d3e1d10f1de5960c116c58b7dc186fd2e

[Neil Pate]

Same MD5 as yours.

@Aristos Queue this is a fresh installation of 2019 SP1 64-bit installed using the offline installer.

I think that DLL calls into other DLLS (like BLASPACK or something like that), but I am not sure.

 

 

 

[Aristos Queue]

I have no idea what other DLLs that one calls. 

This is really weird. 

[Rolf Kalbermatter]

On 3/19/2020 at 1:53 AM, Aristos Queue said:

I have no idea what other DLLs that one calls. 

This is really weird. 

lvanlys.dll is for the most part a thin wrapper around the Intel Math Kernel (MKL) library since many moons. It used to be a fully NI private implementation of the analysis functions but at some point NI realized that they never can beat the guys from Intel in making hyper-optimalized versions of those functions that work virtually on any CPU from early Pentiums to the latest iCore monsters with optimal performance (and also work on AMD CPUs as well).

So if the Intel MKL for some reason refuses to load then such an error could occur. Incidentially I do remember some thread over on the blue side that mentions problems with the MKL on special more modern CPUs with many cores (8+). There is a possibility to configure some configuration file in the form of an ini file I believe to tell the MKL to initiailize in a specific way that allows it to load on such CPUs too. Maybe that is the problem here at hand?

Edited March 22, 2020 by Rolf Kalbermatter

[Aristos Queue]

The MKL problem is in the wild? I thought that was something that was only affecting LV 2020 (now in beta) because we were updating to the latest MKL. The bug isn't on my team's plate... I just sit near the people who are handwringing a lot about it. If that's in the wild affecting already shipping MKL versions, then, yeah, that could be it. I still don't know how the node is rolling back to its last known good resolve path... I can't find that path stored anywhere... but if we assume it is binary encoded *somewhere* in the node's saved attributes, then this makes plausible sense. 

Workaround is to a) get a new CPU or b) wait until LV issues a new version where we do whatever we are doing to avoid calling certain CPU instructions (I'm unclear what the planned solution looks like).