Installer with 3rd party DLLs

[Tomi Maila]

Hi,

I need to distribute a LabVIEW application that uses several open source shared libraries (DLLs). Some of the DLLs I can statically link to but the ones with LGPL license I need to distribute as DLLs. Now the problem I'm facing is that the there are practically many versions of the same DLLs out thereTthe problem arises when a newer version of the DLL is released. I don't want my old installer to overwrite a new version of a DLL that may be installed by another application. Also I don't want uninstallation to remove a DLL that is shared by other applications.

So what is the best way to achieve these goals when distributing LabVIEW applications or LabVIEW VI libraries? Has anyone already solved the issue?

Tomi

[jcabrer]

One solution is to create your installer with a tempDLL folder. Then include an executable or script run by the installer that copies the dlls into the final destination only if they do not exist.

Since the (un)installer is not aware of the dlls final destination, it will not remove those copies of the dlls, and since the folder is not empty, uninstall will not delete it.

[Rolf Kalbermatter]

QUOTE (Tomi Maila @ Mar 27 2008, 11:02 AM)

Hi,

I need to distribute a LabVIEW application that uses several open source shared libraries (DLLs). Some of the DLLs I can statically link to but the ones with LGPL license I need to distribute as DLLs. Now the problem I'm facing is that the there are practically many versions of the same DLLs out thereTthe problem arises when a newer version of the DLL is released. I don't want my old installer to overwrite a new version of a DLL that may be installed by another application. Also I don't want uninstallation to remove a DLL that is shared by other applications.

So what is the best way to achieve these goals when distributing LabVIEW applications or LabVIEW VI libraries? Has anyone already solved the issue?

Tomi

It's not a very good idea to install Open Source DLLs globally exactly because of the version problem and the fact that many Open Source projects reserve the right to make binary incompatible changes to the API even between minor versions. So someone else installing a newer version might break your app too.

Instead copy the DLLs in question into the main directory of your application. That will always make your application use the version of the DLL it was built for and avoid any conflicts with other applications that might use a different version of your application, even if some other application had the great idea to install a newer and possibly binary incompatible version in system32.

For .Net DLLs the GAC and the main application directory are really the only valid directories although LabVIEW allows to link to .Net DLLs by path name, but that will usually bring you into problem when distributing your app or otherwise when upgrading between LabVIEW versions, since they still try to figure out the best way to deal with that issue.

It may sound a bad advice because DLLs were meant to avoid code duplication on the disk. But it is the way Microsoft also deals with DLL hell in .Net and considering nowadays harddisk sizes saving a few MB of HD space is a very bad excuse to turn yourself into dealing with DLL hell.

And are you really saying that you are linking Open Source DLLs statically in your app? I find that a very bad idea unless you have some strong interest to hide that you use such DLLs. But even that usually won't work since the most popular license that would allow that (BSD) does require you to credit the use of such software in the doc and application such as in a splash or about screen.

Rolf Kalbermatter

[Tomi Maila]

QUOTE (rolfk @ Mar 28 2008, 11:17 AM)

It's not a very good idea to install Open Source DLLs globally exactly because of the version problem and the fact that many Open Source projects reserve the right to make binary incompatible changes to the API even between minor versions. So someone else installing a newer version might break your app too.

Instead copy the DLLs in question into the main directory of your application. That will always make your application use the version of the DLL it was built for and avoid any conflicts with other applications that might use a different version of your application, even if some other application had the great idea to install a newer and possibly binary incompatible version in system32.

For .Net DLLs the GAC and the main application directory are really the only valid directories although LabVIEW allows to link to .Net DLLs by path name, but that will usually bring you into problem when distributing your app or otherwise when upgrading between LabVIEW versions, since they still try to figure out the best way to deal with that issue.

It may sound a bad advice because DLLs were meant to avoid code duplication on the disk. But it is the way Microsoft also deals with DLL hell in .Net and considering nowadays harddisk sizes saving a few MB of HD space is a very bad excuse to turn yourself into dealing with DLL hell.

And are you really saying that you are linking Open Source DLLs statically in your app? I find that a very bad idea unless you have some strong interest to hide that you use such DLLs. But even that usually won't work since the most popular license that would allow that (BSD) does require you to credit the use of such software in the doc and application such as in a splash or about screen.

Rolf Kalbermatter

Rolf, thanks for your answer. Your guidelines sound good, however I'm not sure if I get everything to work as expected. Actually I've no problems to describe the issue in more detail to get maybe better insight into the issue.

I'm building a LabVIEW VI library that relies directly on my own non-default build of an open source library A.dll and my own LabVIEW specific wrapper LV_A.dll on some of the functions on A.dll. The open source library A.dll then depends on two other open source libraries B.dll and C.dll and one closed source library D.dll. All shared libraries A, B, C and D are from third parties.

I don't know how a windows DLLs calling other DLLs locates the other DLLs. The distributors of shared library A instruct to place B, C and D under Windows system directory and that is what I've done so far. When they are placed there A seems to properly find the DLLs. I would be more than happy to place the DLLs elsewere to avoid the possible linkage issues with wrong version of the library.

Rolf, are you saying that if all the DLLs A,LV_A,B,C and D are in the same directory, say under vi.lib, then A.dll will load B,C and D from that directory and not a possibly different incompatible version from Windows\System32.

I've understood that only one version of a single DLL can be in memory at once under Windows. Does this however mean that Windows regards DLLs with same name but different locations as different DLLs and allow as such to have multiple versions of the same shared library to be in the memory at the same time as long as they are located in different folders but can have the same name. So can I have for example two different versions of zlib.dll under vi.lib and under Windows\System32 at the same time and Windows would allow them both to be simultaneously loaded to memory by different applications.

The picture below illustrates the dependencies of the libraries

		VI Library			|		   / \		LV_A  |		   \  |			\ |			  A			 /|\			B C D

[Rolf Kalbermatter]

QUOTE (Tomi Maila @ Mar 28 2008, 07:56 AM)

Rolf, thanks for your answer. Your guidelines sound good, however I'm not sure if I get everything to work as expected. Actually I've no problems to describe the issue in more detail to get maybe better insight into the issue.

I'm building a LabVIEW VI library that relies directly on my own non-default build of an open source library A.dll and my own LabVIEW specific wrapper LV_A.dll on some of the functions on A.dll. The open source library A.dll then depends on two other open source libraries B.dll and C.dll and one closed source library D.dll. All shared libraries A, B, C and D are from third parties.

I don't know how a windows DLLs calling other DLLs locates the other DLLs. The distributors of shared library A instruct to place B, C and D under Windows system directory and that is what I've done so far. When they are placed there A seems to properly find the DLLs. I would be more than happy to place the DLLs elsewere to avoid the possible linkage issues with wrong version of the library.

Rolf, are you saying that if all the DLLs A,LV_A,B,C and D are in the same directory, say under vi.lib, then A.dll will load B,C and D from that directory and not a possibly different incompatible version from Windows\System32.

I've understood that only one version of a single DLL can be in memory at once under Windows. Does this however mean that Windows regards DLLs with same name but different locations as different DLLs and allow as such to have multiple versions of the same shared library to be in the memory at the same time as long as they are located in different folders but can have the same name. So can I have for example two different versions of zlib.dll under vi.lib and under Windows\System32 at the same time and Windows would allow them both to be simultaneously loaded to memory by different applications.

The picture below illustrates the dependencies of the libraries

		VI Library			  |			 / \		  LV_A  |			 \  |			  \ |				A			   /|\			  B C D

Ok I'm assuming that you use the Call Library Node and call a DLL with exported standard C functions here. ActiveX and .Net DLLs are an entirely different beast and not my real speciality.

Although there are two ways to locate a function in a DLL (static linked import library vs. dynamic linking through LoadLibrary/GetProcAddress) the rest of this story is all the same for both, except that dynamic linking allows the linking application (or DLL) to specify an absolute path for loading a DLL and that will fail the load if the DLL is not located there even if it could be found in other default search locations (except I believe if a DLL with the same name is already loaded into the current process image).

So when Windows gets a request to load a library (either through LoadLibrary() without absolute path or implicitedly through the import table of a statically linked import table inside an executable or DLL) it will look for that DLL in following places and search order:

1) DLL already mapped into current process with same name

2) The directory where the current process was started from (aka where LabVIEW.EXE or YourApp.exe is located)

3) System Directory (often called System32)

4) Windows Directory (sometimes also called WinNT)

5) Any directory present in the PATH Environment variable for the current user

Now LabVIEW does do a little more than just passing the DLL name itself and remembers the actual (relative) path of a DLL inside a VI and first tries to load that DLL with the resulting absolute path. If that fails because the DLL is not there it tries it again with only the DLL name. That is why it can work for the application builder to put support files including directly called DLLs into a data subdirectory inside your application directory. LabVIEW does remember that relative location and will try to load the DLL explicitedly from there first. However there is no way short from scanning the import table of a DLL to find out if it has other dependencies and I'm not aware of any application or programming environment that would go to these lengths ever.

So while you could put your directly dependant DLL into a data subfolder inside your application you can not really put dependancies this DLL has into the same folder since here only the Windows standard search order will be used and that would mean it can't find the DLL at all. However by putting all those DLLs (and most probably your directly called DLLs too for the simplicity and clearness of it) into the same folder as your executable you can quite simply ensure that your application will always use the DLLs it was designed for without clashing with other versions of the same DLL.

You could try to trick LabVIEW into explicitedly loading those DLLs too from a subdirectory directly by adding a dummy VI to your startup VI that references those DLLs directly without ever really calling them. That way when your own DLL that depends on these DLLs is loaded, Windows will find them already mapped into memory and not look any further. But that can be a bit tricky because you do need to know the exact dependancy tree and start to load the VIs referencing those DLLs in the reverse order of the DLLs in order of their dependancies. Not a big problem if the dependency is as you draw it but it can get tricky if those subDLLs also have dependancies on each other. Personally I have never felt the need to do such tricky stuff but instead just throw all the DLLs into the same directory as the executable. Yes it clutters that directory a little but hey it's quite common to have a few DLLs in the executable directory of nowadays applications.

Of course for your development machine or when you would want to distribute the VI library for development to others, things could get a bit more tricky. Since you do have a custom DLL already you could try to make those sub DLLs loaded explicitedly from the same location as your DLL itself is loaded but that would mean that you would have to reference your A.DLL from inside LV_A.DLL completely dynamically and also wrap every direct A.DLL import into your VI library through LV_A.DLL making it call A.DLL dynamically too to ensure that the DLLs are loaded properly already before A.DLL gets loaded and Windows tries to satisfy it's direct imports. Quite a hassle indeed.

So here again the most simple approach would be to just drop everything into the LabVIEW root folder (since your LabVIEW.EXE that is starting your LabVIEW development process is located here). Alternatively you could consider telling people to put it in system32 if they want to develop in LabVIEw with yout library but you will have to make sure to document to them how they need to adjust the built process to make sure those DLLs get added as support files to the built and to make sure to put them into the same directory as the executable itself.

PS: you are not by any way incorerating something that is using the Apache Runtime Library Support? Although I didn't think that was LGPL.

PPS: And no there can't be only one DLL of the same name in memory at once at least technically spoken. The limitation is that only one DLL with the same name can be mapped into a specific process space. A differnt executable and therefore different process space does reference it's own DLL version and if the paths do not match it will be two different DLLs loaded into memory, one for each process.

Rolf Kalbermatter