OpenG LabVIEW Zip 5.0.0-1 - stuck at the readme

[PA-Paul]

Hi I'm trying to install the OpenG Zip package via VIPM. But when it gets to the LVZIP readme page - the "Yes, install this library" button remains greyed out no matter how many times I scroll to the top or bottom of the dialog... It worked fine on a different computer, but on mine I cannot get past this (and its a tad frustrating!).

I'm on LabVIEW 2021.0.1f2, VIPM 2023.1

I've tried uninstalling the old version, making sure its not installed in any other LabVIEW version, but I get the same results on any version of LabVIEW... Hoping someone can help!

Thanks

Paul

[Rolf Kalbermatter]

Hi Paul

I'm aware of this problem. It has to do with different default fonts in modern Windows machines, which changes the entire text format. The main reason I haven't yet updated the installer was that I was on vacation and there is also a problem to install the package under Linux, which has to do with write permissions to certain file directories. Once I figured that out I will release a new package 5.0.1.

[PA-Paul]

Thanks Rolf, glad to hear its on your radar already.

It's not holding me up in the end as I'm not actively developing anything using the toolkit and fortunately it did work in the place I need it most (the build server!)... but I'll keep my eyes open for the update so I can get it on the dev machines too!

Thanks

Paul

[Rolf Kalbermatter]

5.0.1 and in the meantime 5.0.2 has been since released. One issue, but that is not really new and existed before: Don't disable mass compile after install, it may take some time but it sure fixes stale shared library paths in the VI and I have so far not found a way that makes those paths automatically fixup at package creation, since the path seems to need to be absolute.

The two possible approaches I'm currently considering:

1) use a so called Symbolic Path (/<user.lib>/_OpenG.lib/lvzip/lvzip.*).

Disadvantage:

- only works if installed into default location

2) use Specify Library Name on diagram for the Call Library Node and calculate its path at runtime.

Disadvantage:

- makes the shared library be not a visible component to the VIs, so that the shared library needs to be added explicitly in every application/shared library/assembly/source distribution build in order to be available in such

- extra execution time for the dynamic calculation of the path

Edited April 15 by Rolf Kalbermatter

[ShaunR]

3 hours ago, Rolf Kalbermatter said:

I have so far not found a way that makes those paths automatically fixup at package creation, since the path seems to need to be absolute

This is why it takes me hours to make an ECL build that works and is one of the many reasons only Windows is now supported (can load from same dir). Even then. I have to fight VIPM to get things in the right places.

I refuse to do #2.

[Rolf Kalbermatter]

The problems with the paths to the shared libraries should be fixed with version 5.0.3. It's a combination of using the <library_name>*.* format to let LabVIEW resolve to the right shared library with the first * being replaced with 32 or 64 and the second with the platform specific shared library file extension (dll and so). Also needed to fixup the linker info in the VIs after creating the renamed (with oglib postfix) VI hierarchy through the OpenG Builder functions. LabVIEW, after creating the renamed hierarchy, will store the fully resolved library name into all VIs, which of course is not very helpful as it pins the VIs then to only use the shared library that matches the LabVIEW platform used to create the package.

Version 5.0.4 was adding a working low level API for supporting a progress interface to the storing of files into an archive.

Version 5.0.5 was adding an extended interface to the Inflate and Deflate functions to compress and uncompress binary streams and control the header used (none, zlib or gzip). Previously only zlib was supported for Deflate and Inflate.

Version 5.0.6 fixes a bug when dealing with UNC names that contain numeric characters such as an IP address in the server name.

https://www.vipm.io/package/oglib_lvzip/

 

Edited November 14 by Rolf Kalbermatter

[ShaunR]

On 11/11/2024 at 10:06 AM, Rolf Kalbermatter said:

Also needed to fixup the linker info in the VIs after creating the renamed (with oglib postfix) VI hierarchy through the OpenG Builder functions.

Can the JKI builder be modified to do this? I've already hacked some of their VI's in ogb_2009.llb so it didn't take 6 hrs to build. It's a huge problem for me when building. I have a solution that sort of works, sometimes,  but not a full proper solution.

Can you detail your process?

[Rolf Kalbermatter]

1 hour ago, ShaunR said:

Can the JKI builder be modified to do this? I've already hacked some of their VI's in ogb_2009.llb so it didn't take 6 hrs to build. It's a huge problem for me when building. I have a solution that sort of works, sometimes,  but not a full proper solution.

Can you detail your process?

There is to my knowledge no way to modify the JKI Builder. Although I think they did fix in recent years a bug that sounded exactly like what I ran across. But the JKI Builder has many other limitations that I'm not fond of so I still rely on my own setup.

I basically use ogrsc_builder_3.0.0.11 for the renaming of the VI hierarchy with the opglib prefix with one modifications and then a heavily modified version of the OpenG Package Builder to package everything into the OpenG package. One caveat here, the ogrsc_builder_3.0.0 is from ca. 2009 times (and in 8.6 source code version). It will likely not go well with modern lvclass' and lvlib's and even more likely with lvlibp's files. It does have support for at least lvclass and lvlib but that is most likely fairly unmature seeing when it was last touched. lvclass and lvlib still were fairly new back then and had several quirks even in LabVIEW itself.

I changed deep in the belly of the OpenG Builder in OpenG\build\ogb.llb\Copy Resource Files and Relink VIs__ogb.vi, that for shared library names the file name is changed back to the previous <file name>*.* with some magic to detect the 32 or 64 in the file name if present. It's not fail safe and for that not a fix that I would propose for inclusion in a public tool, but it does the job for me. What basically goes wrong is that when LabVIEW loads the VIs, it replaces to magic place holders with the real values in the paths in the VIs in memory and when you then Read the Linker Info to massage that for renaming VIs, you receive these new fully resolved paths and when you then write back the modified linker info you cement the not-platform neutral naming into the VIs and save it to disk.

The OpenG Package Builder modifications mainly have to do with a more detailed selection of package content and special settings to more easily allow multi-platform support for shared library and other binary compiled content. In terms of user experience it is the total opposite of VIPM. It would overwhelm the typical user with way to many options and details that it could be useful for most. I had hoped to integrate the hierarchy renaming into the Package Builder too, since the information in the Package Builder would be basically enough to do that, but looking at the core of the OpenG Builder in Build Applciation__ogb.vi will for sure make you get the shivers to try to reimplement that in any useful way. 😁

And yes the naming of the tools is a bit confusing. The OpenG Builder is the tool that massages an existing hierarchy into a new on with VI renaming and relocating them into a configurable tree and fixing up relative paths to be correct for the new names and locations, while the OpenG Package Builder grabs a list of files and simply pushes them into an OpenG package (basically a ZIP file with configuration file). It would be quite useful to integrate the OpenG Builder as an extra prepare step into the OpenG Package Builder but that is a taunting exercise.

Edited November 15 by Rolf Kalbermatter

[ShaunR]

3 hours ago, Rolf Kalbermatter said:

I changed deep in the belly of the OpenG Builder in OpenG\build\ogb.llb\Copy Resource Files and Relink VIs__ogb.vi, that for shared library names the file name is changed back to the previous <file name>*.* with some magic to detect the 32 or 64 in the file name if present. It's not fail safe and for that not a fix that I would propose for inclusion in a public tool, but it does the job for me. What basically goes wrong is that when LabVIEW loads the VIs, it replaces to magic place holders with the real values in the paths in the VIs in memory and when you then Read the Linker Info to massage that for renaming VIs, you receive these new fully resolved paths and when you then write back the modified linker info you cement the not-platform neutral naming into the VIs and save it to disk.

The OpenG Package Builder modifications mainly have to do with a more detailed selection of package content and special settings to more easily allow multi-platform support for shared library and other binary compiled content. In terms of user experience it is the total opposite of VIPM. It would overwhelm the typical user with way to many options and details that it could be useful for most. I had hoped to integrate the hierarchy renaming into the Package Builder too, since the information in the Package Builder would be basically enough to do that, but looking at the core of the OpenG Builder in Build Applciation__ogb.vi will for sure make you get the shivers to try to reimplement that in any useful way. 😁

There is a "Copy Resource Files and Relink" in "<program files>\JKI\VI Package Manager\support\ogb_2009.llb" and  "<program files>\JKI\VI Package Manager\support\ogb_2017.llb".

Is it "Write Linker Info from File__ogb.vi" that you have modified?

I'll have to have a closer look.

[Rolf Kalbermatter]

1 hour ago, ShaunR said:

There is a "Copy Resource Files and Relink" in "<program files>\JKI\VI Package Manager\support\ogb_2009.llb" and  "<program files>\JKI\VI Package Manager\support\ogb_2017.llb".

Is it "Write Linker Info from File__ogb.vi" that you have modified?

I'll have to have a closer look.

Not quite! It's better to actually modify the Copy Resource Files and Relink.vi. Just add an additional case structure to handle shared libraries.

The VI in question is this one:

This will unconditionally change the linking name of all shared libraries in your build. There is a possibility that that is not desired although I can't think of a reason why that could be a problem right now.

 

Fixup Shared Library Name.vi

Edited November 15 by Rolf Kalbermatter

[ShaunR]

25 minutes ago, Rolf Kalbermatter said:

Not quite! It's better to actually modify the Copy Resource Files and Relink.vi. Just add an additional case structure to handle shared libraries.

The VI in question is this one:

This will unconditionally change the linking name of all shared libraries in your build. There is a possibility that that is not desired although I can't think of a reason why that could be a problem right now.

 

Fixup Shared Library Name.vi 31.87 kB · 0 downloads

Sweet. It's not quite the same but I'll figure the rest out. You've done the hard part  Thanks.

[Rolf Kalbermatter]

4 minutes ago, ShaunR said:

Sweet. It's not quite the same but I'll figure the rest out. You've done the hard part  Thanks.

I checked on a system where I had VIPM 2013 installed and looked in the support/ogb_2009.llb. Maybe your newer VIPM has an improved ogb_2009.llb. Also check out the actual post I updated the image.

Edited November 15 by Rolf Kalbermatter

[ShaunR]

11 minutes ago, Rolf Kalbermatter said:

I checked on a system where I had VIPM 2013 installed and looked in the support/ogb_2009.llb. Maybe your newer VIPM has an improved ogb_2009.llb. Also check out the actual post I updated the image.

Yes. The path isn't passed through but I figured out what it was supposed to be.

this is the one in my installation:

It's a trivial change though. The important part is the adding the extra case and your new VI. I was about to go all Neanderthal on the "Write Linker Info" before you posted the proper solution.

Edited November 15 by ShaunR

[Rolf Kalbermatter]

On 11/15/2024 at 4:46 PM, Rolf Kalbermatter said:

This will unconditionally change the linking name of all shared libraries in your build. There is a possibility that that is not desired although I can't think of a reason why that could be a problem right now.

It just occurred to me that there is a potential problem. If your DLLs are always containing 32 in their name, independent of the actual bitness, as for instance many Windows DLLs do, this will corrupt the name for 64-bit LabVIEW installations.

I haven't checked if paths to DLL names in the System Directory are added to the Linker Info. If they are, and I would think they are, one would have to skip file paths that are only a library name (indicating to LabVIEW to let the OS try to find them through the standard search mechanism).

This of course still isn't fail proof:

DLLs installed in the System directory (not from Microsoft though) could still use the 32-bit/64-bit naming scheme, and DLLs not from there could use the fixed 32 naming scheme (or 64 fixed name when building with VIPM build as 64-bit executable, I'm not sure if the latest version is still build in 32-bit).

[ShaunR]

On 11/19/2024 at 11:48 AM, Rolf Kalbermatter said:

It just occurred to me that there is a potential problem. If your DLLs are always containing 32 in their name, independent of the actual bitness, as for instance many Windows DLLs do, this will corrupt the name for 64-bit LabVIEW installations.

I haven't checked if paths to DLL names in the System Directory are added to the Linker Info. If they are, and I would think they are, one would have to skip file paths that are only a library name (indicating to LabVIEW to let the OS try to find them through the standard search mechanism).

This of course still isn't fail proof:

DLLs installed in the System directory (not from Microsoft though) could still use the 32-bit/64-bit naming scheme, and DLLs not from there could use the fixed 32 naming scheme (or 64 fixed name when building with VIPM build as 64-bit executable, I'm not sure if the latest version is still build in 32-bit).

I've modified your sub vi to check the actual file bitness (I think). If you target user32.dll, for example, the filename out is user32.* - which is what's expected. I need to think a bit more about what I want from the function (I may not want the full path) but it should fix the problem you highlighted (only in Windows ).

Fixup Shared Library Name.vi

 

Edited November 22 by ShaunR

[Rolf Kalbermatter]

On 11/21/2024 at 11:11 AM, ShaunR said:

I've modified your sub vi to check the actual file bitness (I think). If you target user32.dll, for example, the filename out is user32.* - which is what's expected. I need to think a bit more about what I want from the function (I may not want the full path) but it should fix the problem you highlighted (only in Windows ).

Fixup Shared Library Name.vi 37.35 kB · 0 downloads

 

That looks nasty. Checking the PE file header would be one of the last things I would resort to. It also has the potential to fail for many reasons such as LabVIEW deciding to not convert the wildcard filenames in the Linker Info anymore, although I think that's actually unlikely since those paths are meant to actually allow access the files on disk in some way. But it's a private node, so all guarantees are pretty much null and void. Also I fail to see how your method would work with a DLL called user32.dll that should stay that way when your tool is running on LabVIEW for 32-bit. It will correctly detect that the DLL is NOT 64 bit, so use the 32 to search for in the file name, correctly find that and replace it with a *. Boom, When loaded in LabVIEW for 64-bit the VI will look for user64.dll and that will not work. Also don't try to read the file for the non-Windows cases as you don't do anything with that anyhow!

The best we may be able to do here is to simply detect if it is a relative path  with only one path element (it is for system shared libraries correctly setup in the Call Library Node) and skip the renaming back to wildcard names for them. This may require more investigation for other OSes such as Linux. But there, multiarch installations are also usually solved by having different lib directories for the different architectures and not to rename the files for standard libraries. Only file names with absolute path (and hence not installed in one of these directories but located locally to to caller, MIGHT use the 32-bit/64-bit naming scheme. It's a kludge, but VIPM unfortunately has limited support to install binary files based on the bitness of the target LabVIEW version! Otherwise I wouldn't need that complete filename wildcard voodoo for my libraries.

In the past I tried to solve it by installing both versions of the shared library with different names and having a post-install hook to delete the wrong one and do the renaming to the expected standard name. But that has other complications in some situations. I may still decide to revert back to that. This nonsense is even worse to solve properly.

Edited November 25 by Rolf Kalbermatter

[ShaunR]

3 hours ago, Rolf Kalbermatter said:

Also don't try to read the file for the non-Windows cases as you don't do anything with that anyhow!

Feel free to add them in.

[Rolf Kalbermatter]

51 minutes ago, ShaunR said:

Feel free to add them in.

Not likely as there are many potential problems. For one, properly configured Call Library Nodes for system libraries will report a relative path with just the library name, so your File Open will simply error out on that as it can't find the file (it might actually magically, if you just happened to select a file through the File Dialog Box in the system directory as that will change the application global current directory variable and the Windows user space API is known to try to resolve relative names relative to that current directory). But the problem with that is that it will only work if the current directory is set to the System32 directory, which it usually isn't. An error on this level passed on to VIPM will not likely far well with the installation itself.

But the main problem is that I do not see how that could work properly when trying to build on 32-bit LabVIEW for DLL files that should remain with 32 in their name.

Edited November 25 by Rolf Kalbermatter

[ShaunR]

17 hours ago, Rolf Kalbermatter said:

But the main problem is that I do not see how that could work properly when trying to build on 32-bit LabVIEW for DLL files that should remain with 32 in their name.

It's not a problem I have. I name them x32 and x64, if necessary, so there is no issue with the likes of user32.dll.

What I do have a problem with is binaries which can be 32 or 64 bit but there is no indication in the name and the path gets "fixed" by VIPM (similar problem with TPLAT). In that scenario I want the ".*" on the end only. That is the problem it is solving and why I said I wasn't sure about the full path (I think I want just the filename). Until now I have had a similar solution to you (force changes in Post_Install and make LabVIEW search for the binaries so linking them on first load). The main difference in my solution is, perhaps, that Post_Install has the binaries in arrays on the BD and the correct bitness is saved out when installed by the user. I wanted something where I didn't update the Post_Install every time binaries changed.

Perhaps we are solving different issues due to different workflows.

Edited November 26 by ShaunR

[Rolf Kalbermatter]

On 11/26/2024 at 9:57 AM, ShaunR said:

Perhaps we are solving different issues due to different workflows.

It would seem so. You look for a way to hack VIPM to do something specific for your own shared libs and leave anything else to its own trouble. 🙂

I was trying to find a way that is a more general fix but that seems in fact impossible at this point, unless there is a way to actually have LabVIEW return the original library name for DLLs as filled in the Call Library Node.

Hmmm, thinking of this there might be, but it would involve a lot of changes in VIPM. When Reading the Linker Info earlier in the Builder, it may be possible to go through all shared library entries, get their caller name (with the link index) and then do a Find Object on the diagram of them for the Call Library Nodes, get the actual library name as filled in the Call Library Node, which hopefully hasn't yet been messed with by the loading of the VI into memory, and later on use that name to reset the linking info.

[ShaunR]

18 minutes ago, Rolf Kalbermatter said:

You look for a way to hack VIPM to do something specific for your own shared libs and leave anything else to its own trouble.

Not quite. I asked about your code to solve my issue. I offered what I thought was an improvement based on my requirements and, as I don't support Linux or Mac anymore, you could migrate it to the other systems if it was viable.

You don't like it. Fine. It works great for me but for one edge case that you have pointed out that doesn't affect me. Assuming I cannot solve that one edge case, then I have a general Windows solution with one caveat that can go in the documentation. I'd call that progress 

[Rolf Kalbermatter]

1 hour ago, ShaunR said:

Not quite. I asked about your code to solve my issue. I offered what I thought was an improvement based on my requirements and, as I don't support Linux or Mac anymore, you could migrate it to the other systems if it was viable.

You don't like it. Fine. It works great for me but for one edge case that you have pointed out that doesn't affect me. Assuming I cannot solve that one edge case, then I have a general Windows solution with one caveat that can go in the documentation. I'd call that progress 

Actually it's a little more than one edge case. It's potentially about any DLL that your library somehow references, including potentially built in LabVIEW functions that you use and that resort to calling DLL functions somewhere. Those might contain the 32 in their name or they might not, and they might also use the 32/64 naming scheme or not, and that would affect them too. Unlikely to happen? Probably, if you don't build a huge application framework library. Impossible? Definitely not.

It's your tool and you decide how you use it. Just don't come and whine that it destroyed your perfect library. 😀

At least it always works to replace the file ending with a wildcard! Unless of course you decide to use a different file ending for your special purpose libraries for whatever reason! That would have to be also caught by checking that the actual file ending is indeed the expected platform ending and only then do the file ending change to a wildcard.

It may sound like esoteric and far fetched edge cases, but I prefer to program defensive, even if I'm going to be the only final user of my functions. ☠️

Edited November 27 by Rolf Kalbermatter

[hooovahh]

On 11/26/2024 at 3:57 AM, ShaunR said:

It's not a problem I have. I name them x32 and x64, if necessary, so there is no issue with the likes of user32.dll.

I'm also in Windows only land, and renaming the DLLs in a Post Install has worked well enough so far.  If we in a 64 bit LabVIEW for instance the post install looks for if there are DLLs named [X].dll but there exists an [X]64.dll.  If so it will delete the [X].dll and rename [X]64.dll to [X].dll.  This renaming bit also works in reverse looking for [X]32.dll if we are on a 32 bit version of LabVIEW.  Development then is done in one bitness statically picking the correct DLL. I have no clue how difficult or complicated this gets supporting Linux.

[Rolf Kalbermatter]

19 hours ago, hooovahh said:

I'm also in Windows only land, and renaming the DLLs in a Post Install has worked well enough so far.  If we in a 64 bit LabVIEW for instance the post install looks for if there are DLLs named [X].dll but there exists an [X]64.dll.  If so it will delete the [X].dll and rename [X]64.dll to [X].dll.  This renaming bit also works in reverse looking for [X]32.dll if we are on a 32 bit version of LabVIEW.  Development then is done in one bitness statically picking the correct DLL. I have no clue how difficult or complicated this gets supporting Linux.

This works on Linux too, and did so in the past for the OpenG ZIP library. I'm not really sure anymore what made me decide to go with the 32/64 bit naming but there were a number of requests for varying things such as being able to not have the VIs be recompiled after installation, as that slows down installation of entire package configurations. And the OpenG Builder was also indiscriminately replacing the carefully managed lvzlib.* library name with lvzlib.dll in the built libraries so that installation on Linux systems created broken VIs. In the course of trying to solve all these requirements, I somehow come up with the different bitness naming scheme with the LabVIEW wildcard support, but I think now that that was a mistake.

In hindsight I think the "only" things that are really needed are:

• to leave the shared library name (except the file ending of course) for all platforms the same
• the simple modification in the VIPM Builder library to replace the shared library file name extension back to the wildcard
• add individual shared libraries for all supported platforms to the OGP/VIP package. But since VIPM has only really support for individual system platforms but not the bitness of individual files, we need to name the files in the archive differently for the bitness and let them install both on the target system with their respective name
• create a PostInstall hook that will rename the correct shared library for the actual bitness to the generic name and delete the other one

Of course with a bit more specific bitness support in VIPM it would be possible to define the shared library to install based on both the correct platform and bitness, but that train has probably long departed.

Will have to rethink this for the OpenG ZIP library a bit more, and most likely I will also add an extra directory with the shared libraries for the ARM and x64 cRIOs in the installation. Will require to manually install them to the cRIO target for the library to work there but the alternative is to:

• have the user add a shared network location containing the according opkg packages to the opkg package manager on the command line of the cRIO
• do an opgk install opgzip<version> on that command line

That's definitely not more convenient at all!!!

Edited November 28 by Rolf Kalbermatter

[ShaunR]

On 11/27/2024 at 3:16 PM, Rolf Kalbermatter said:

I'm not really sure anymore what made me decide to go with the 32/64 bit naming

1. Both binaries can be in the same directory.
2. No need for a Post Install or Post Uninstall.
3. No code required to choose a 32 bit or 64 bit binary in the the different LabVIEW bitnesses.

There are excellent reasons to use this nomenclature but they are thwarted by VIPM.