OpenG Zip 4.2?

[RomainP]

On 8/31/2022 at 10:02 AM, Rolf Kalbermatter said:

I can't right now work on that. But I have plans to do that in the coming months. The story behind it is that I did a little more than just to make it 64-bits.

- The file IO operations where all rewritten to be part of the library itself rather than relying on LabVIEW file IO. While LabVIEW 8.0 and newer supports reading and writing files that are bigger than 2GB, it still has the awful habit to use internally old OS file IOs that are naturally limited to only supporting characters in file names that are part of your current local and they also normally are limited to 260 character long path names. If your drive is formatted in FAT32, that is all the drive can do for you anyhow, but except for USB thumb drives, you would be hard pressured to find any FAT formatted drives anymore. So having these limitations in the library feels very bad.

These two things are specially a problem on Windows. Mac is slightly less problematic and Linux has long ago pretty much solved it all internally in the kernel and surrounding system libraries.

- Modern ZIP files support things like symbolic links and I wanted a way to support them. For Linux and Mac that is a piece of cake. For Windows I may for now not be able to seamlessly support that as creating symlinks under Windows is a privileged action, so the user has to either be elevated or you have to set an obscure Developer flag in Windows that allows all users to create symlinks.

So in summary there was a lot of work to be done, most of it actually for Windows. Most of that is done but testing all that is a very frustrating job. And the non-Windows targets will then also take some more time for additional testing and making things that were modified for Windows compile again properly.

So yes, it's still on my to-do list and I'm planning to work on it again, but right now I have another project that requires my attention.

Because of the significant changes in the underlying shared library and internal organization of the VIs it will be almost certainly version 5.0. The official library API (those nice VIs with a green gift box in them) should remain compatible but if you want to make use of the new path name feature to fully support long path names with full character support, you may have to change to the new API, with the library specific path type, although if you use high level library functions, internal long path names will be ok, you just won't be able to access them with the normal LabVIEW file functions if they contain non local ANSI characters or are to long! It's the best I could come up with without the ability to actually changing the LabVIEW source code itself to add that feature into the internal Path Manager in LabVIEW. 😀

The according File Utilities Manager functions in the library will also be available for the user in a separate palette.

OK, thank you. Please keep us informed.

[Rolf Kalbermatter]

1 hour ago, RomainP said:

OK, thank you. Please keep us informed.

I worked some on this over the holidays and made some more progress. Still quite a bit of cleanup to do, the underlaying code in the shared library was a bit a work in progress in several phases and there was some code duplication and inconsistencies. But I'm slowly getting there. 

[Mads]

I had noted a simple recipe earlier from here somewhere for manually installing lvzlib.so (OpenGZip 4.2beta) on Linux RT 2020 and newer now that the "Custom Install" option no longer exists / works in NI Max, but I seem to have lost it and cannot find it again. 

Do anyone know where that discussion was, or has the recipe? 

[Rolf Kalbermatter]

8 minutes ago, Mads said:

I had noted a simple recipe earlier from here somewhere for manually installing lvzlib.so (OpenGZip 4.2beta) on Linux RT 2020 and newer now that the "Custom Install" option no longer exists / works in NI Max, but I seem to have lost it and cannot find it again. 

Do anyone know where that discussion was, or has the recipe? 

Where you refering to this?

https://lavag.org/profile/19157-jordan-kuehn/

 

[Mads]

49 minutes ago, Rolf Kalbermatter said:

Where you refering to this?

https://lavag.org/profile/19157-jordan-kuehn/

 

The link only points to Jordan's profile, but he might have been involved back then yes...

[Mads]

Found it, here: 

 

Edited September 5, 2023 by Mads

[Rolf Kalbermatter]

52 minutes ago, Mads said:

The link only points to Jordan's profile, but he might have been involved back then yes...

Oops, sorry! 😀 But you found it already. 👍

[Mads]

The RT Image folder does not seem to be updated with the newly released version 5 of OpenG Zip...How is the process to install it on Linux RT targets now? I looked for an .so file for 5.0 or an option to install it in NI Max, but have not found it yet?

[Antoine Chalons]

I've also add issues on Linux (desktop) with the version 5.0
please see here : https://forums.vipm.io/topic/8000-unable-to-install-on-linux/

for details, I'm reverting to 4.2.1b1 for now

[Rolf Kalbermatter]

Thanks guys for actually checking out the OpenG ZIP library.

Yes there is no Realtime support yet in 5.0. I was already struggling with normal Linux support quite enough, so did not want to add yet another challenge. There will almost certainly be no support for Pharlap and VxWorks in 5.0, unless someone is really begging very nicely for it. 😁

As to the Linux installation problems, I'm sorry about that. I did move to LabVIEW 8.6 for testing and building, since that is the Linux installation that I have available, but apparently forgot that I had patched the LabVIEW 7.0 OpenG Builder to not bork the shared library names when building a distribution for a package.  So I have to investigate that. Also there is still an issue with support for Unicode ZIP archives, but since that is not happening to often it should not be a serious issue for the moment. But I'm investigating the proper fixes for that. I just hope that the NI Linux RT installations offer a similar support for dealing with Unicode as do normal Linux distributions, otherwise things will get nasty.

[Mads]

39 minutes ago, Rolf Kalbermatter said:

Thanks guys for actually checking out the OpenG ZIP library.

Yes there is no Realtime support yet in 5.0. I was already struggling with normal Linux support quite enough, so did not want to add yet another challenge. There will almost certainly be no support for Pharlap and VxWorks in 5.0, unless someone is really begging very nicely for it. 😁

As to the Linux installation problems, I'm sorry about that. I did move to LabVIEW 8.6 for testing and building, since that is the Linux installation that I have available, but apparently forgot that I had patched the LabVIEW 7.0 OpenG Builder to not bork the shared library names when building a distribution for a package.  So I have to investigate that. Also there is still an issue with support for Unicode ZIP archives, but since that is not happening to often it should not be a serious issue for the moment. But I'm investigating the proper fixes for that. I just hope that the NI Linux RT installations offer a similar support for dealing with Unicode as do normal Linux distributions, otherwise things will get nasty.

Ah. Linux RT support is the big thing for us . We used ot have some VxWorks targets, but those are obsolete now. I'll revert back to 4.2b.

[Antoine Chalons]

is there a publicly hosted repo for this package?

I'd be more than happy to help or at least test on Linux Desktop.

For us Linux Desktop is the most important and RT Linux is not critical as we are happy with 4.2b

[Rolf Kalbermatter]

1 hour ago, Antoine Chalons said:

is there a publicly hosted repo for this package?

I'd be more than happy to help or at least test on Linux Desktop.

For us Linux Desktop is the most important and RT Linux is not critical as we are happy with 4.2b

Sure: https://github.com/RolfKal/openg-lvzip

[Antoine Chalons]

Is it a must to keep this in LabVIEW 8.6?

I mean... OpenG has now moved to 2009, which is still pretty old.

[Rolf Kalbermatter]

6 minutes ago, Antoine Chalons said:

Is it a must to keep this in LabVIEW 8.6?

I mean... OpenG has now moved to 2009, which is still pretty old.

Well, do you have a LabVIEW 2009 installer for Linux (and Mac) for me to use for testing? I would be even willing to settle for 2014 but it should be an older version than 2020. I used to have 2014 on an old iMac both for 32-bit and 64-bit but that hard disk has irrecoverably died and there was no backup of it.

Edited February 14 by Rolf Kalbermatter

[Antoine Chalons]

I see... Setting up VMs for Windows, Linux and Mac testing will be a challenge

2014 sounds reasonnable

Edited February 14 by Antoine Chalons

[Rolf Kalbermatter]

5 minutes ago, Antoine Chalons said:

I see... Setting up VMs for Windows, Linux and Mac testing will be a challenge

2014 sounds reasonnable

Yes that is the main problem to support something like this. And no I don't have a 2014 installation either for Linux. I could of course download 2017 SP1, which is the oldest that can be downloaded from the NI site, but I would actually prefer to have a version that was also supporting 32-bit (which 2016 was the latest).

Edited February 14 by Rolf Kalbermatter

[Rolf Kalbermatter]

Just a heads up. I'm working on this including getting more Linux Virtual Machines setup, but it is a slow and tedious process. There seems always something that won't work.

LabVIEW installation on non-rpm systems is tedious and often almost impossible, depending on the Linux variant and version. On Fedora 34 the rpm files use a feature that a security fix to the rpm program had disabled and now it complains about invalid rpm files. Updating rpm requires kernel development files installed that can't be installed anymore. Fedora 36 allows installing of the LabVIEW 2014 SP1 that I could organize but LabVIEW segfaults on startup, likely because the kernel, libc or stdlibc is to new for it. Older Fedora images are difficult to get at and even more difficult to install.

Open source is nice but the version conflict hell seems to be still as it used to be 15 years ago, where anything but installing from source was a Russian roulette game.

So while I'm slowly getting towards something that eventually, hopefully will work, I'm for now going for one week of ski vacation and work on this will have to wait a bit.

 

 

[Antoine Chalons]

sure, in the recent years NI has really improved Linux Desktop support back in 2014 it was really tough!

for now 4.2b works fine so no rush, enjoy ski!

This weekend I'm ice diving ❄️🤿

[Mads]

On 2/13/2024 at 8:30 PM, Rolf Kalbermatter said:

Thanks guys for actually checking out the OpenG ZIP library.

Yes there is no Realtime support yet in 5.0. I was already struggling with normal Linux support quite enough, so did not want to add yet another challenge. There will almost certainly be no support for Pharlap and VxWorks in 5.0, unless someone is really begging very nicely for it. 😁

As to the Linux installation problems, I'm sorry about that. I did move to LabVIEW 8.6 for testing and building, since that is the Linux installation that I have available, but apparently forgot that I had patched the LabVIEW 7.0 OpenG Builder to not bork the shared library names when building a distribution for a package.  So I have to investigate that. Also there is still an issue with support for Unicode ZIP archives, but since that is not happening to often it should not be a serious issue for the moment. But I'm investigating the proper fixes for that. I just hope that the NI Linux RT installations offer a similar support for dealing with Unicode as do normal Linux distributions, otherwise things will get nasty.

There is no change when it comes to Real-Time support now with 5.0.4 right?

Still need to stick to 4.2 to use it on RIO-targets?

[Rolf Kalbermatter]

6 minutes ago, Mads said:

There is no change when it comes to Real-Time support now with 5.0.4 right?

Still need to stick to 4.2 to use it on RIO-targets?

Not yet. I'm actually working on it. Did read yesterday about the opkg package format needed and got some ideas about how to support that properly. Next step is to get the necessary gcc cross compile setup working again and create the relevant shared libraries and do some tests on several LabVIEW versions. Shouldn't be many months, but don't expect the according release next week. 😀