Open G Zip Tools on Linux RT

[Rolf Kalbermatter]

Ok guys, I managed to organize a nice iMac in order to be able to compile and test a shared library of the OpenG ZIP Toolkit. However I have run into a small roadblock. I downloaded the LabVIEW 2014 for Mac Evaluation version and despite that it tells me that it is the 32 bit version, does it contain the 64 version of the import library in the cintools directory.

 

Therefore I would like to ask if someone could send me the lvexports.a file from the cintools directory from an earlier LabVIEW for Mac (Intel) installation, so that I can compile and test the shared library in the Evaluation version on this iMac. I'm afraid that even a regular LabVIEW 2014 for Mac installation might contain a 64 bit library in both installation, so the lvexports.a file from around LabVIEW 2010 up to 2013 would be probably safer, as those versions were 32 bit only and therefore more likely will also contain a 32 bit library file

[Sam_Sharp]

Just wanted to pop in and say thanks for the updated library with better (Linux) RT support. I've written my own GZip compression VI based on the DEFLATE and CRC32 VIs in the library and I was hoping to be able to use it on the RT / Linux RT sbRIOs. I'm using it for compressing JSON data to send over the network.

 

I've posted the VI over on the NI Community (awaiting moderation) in case anyone else needs a GZip compression VI.

 

Thanks!

[Stobber]

I've posted the VI over on the NI Community (awaiting moderation) in case anyone else needs a GZip compression VI.

 

Nice, thanks! Mind re-posting them here with a more friendly license?

[hofzge]

I already replied on NI and would like to ask here for help. Using Sam_Sharps method i try to decopress a zipped stream of and run out of memory, which seems strange as the string is only about some about 1 MB long.

 

Do I need to work on parts of the string? I see there is a calculation for the compression and the memory allocated to the containing the uncompressed string. How can this be so large for such a small stream?

 

What I did is similar to this:

 

Edited March 1, 2016 by hofzge

[Rolf Kalbermatter]

Please provide some sample data and VIs. As you refer to the GZIP VIs on the NI thread you need to make sure you have a proper GZIP stream you want to deflate. If you rather have a ZIP file stream you will need the ZIP functions in the ZLIB library instead. The Inflate algorithm simply assumes that the incoming data stream is a properly compressed (deflate) stream and if it is something else it certainly can get upset and cause all kinds of possible problems.

[hofzge]

Sorry for being so cryptic: What i try to replace is this:

1. # Converts from base64 to binary   
2. function ConvertFrom-Base64($string)    
3. {   
4.    $bytes  = [system.Convert]::FromBase64String($string);   
5.    $decoded = [system.Text.Encoding]::UTF8.GetString($bytes);    
6.    return $decoded;   
7. }   
8.   
9. # Does buffered read of all bytes from a stream   
10. # returns a char array   
11. function ReadAllBytes($stream)    
12. {   
13.     $buffer = New-Object byte[](4096)      
14.     $ms = New-Object System.IO.MemoryStream   
15.   $bytesRead = 0   
16.   do   
17.     {   
18.         $bytesRead = $stream.Read($buffer, 0, $buffer.Length)   
19.     if ($bytesRead -gt 0)   
20.     {   
21.       $ms.Write($buffer, 0, $bytesRead)   
22.     }   
23.   } while ($bytesRead -gt 0)   
24.   return [system.Text.Encoding]::UTF8.GetChars($ms.ToArray())   
25. }   
26.   
27. # Removes 'Zipped:', Converts from base64 and unzips   
28. function FromBase64AndUnzip($string)   
29. {   
30.     $string = $string.Replace("Zipped:", "")   
31.     $dataBytes = [system.Convert]::FromBase64String($string)   
32.     $memStream = New-Object System.IO.MemoryStream $dataBytes, 0, $dataBytes.Count   
33.     $memStream.Position = 0   
34.     $gzipStream = New-Object System.IO.Compression.GzipStream $memStream, ([iO.Compression.CompressionMode]::Decompress)   
35.     $uncompressed = ReadAllBytes($gzipStream)   
36.     return New-Object String(($uncompressed), 0, $uncompressed.length)   
37. }  

From what i understand this is in fact a gzip stream. Am I wrong?

Edited March 1, 2016 by hofzge

[Rolf Kalbermatter]

Well I see a GZipStream in there, but not knowing the actual language you use it is hard to judge what specifics that may imply. Are you sure you did proper preparation of the byte stream, namely first removing the "Zipped:" string from the stream and then doing the correct Base64 decoding before trying to pass the resulting byte stream to the GZIP function?

[hofzge]

Well I see a GZipStream in there, but not knowing the actual language you use it is hard to judge what specifics that may imply. Are you sure you did proper preparation of the byte stream, namely first removing the "Zipped:" string from the stream and then doing the correct Base64 decoding before trying to pass the resulting byte stream to the GZIP function?

Thanks - being an idiot i forgot to convert from Base64 to String and this helped.

[Dan B]

I am using the ZLIB Library in my project. Originally it was used in 32 bit LabVIEW. Now I am porting it for 64 bit LabVIEW, and I am having issues with ZLIB.

Apparently, the DLL is not valid in 64 bit LabVIEW.

Has this been updated?

Specifically,  the ZLIB CRC32__ogtk.vi will not execute. The Call Library Node is pointing to an invalid library, and when I browse to the Library in the 64 Bit NI folder (C:\Program Files\National Instruments\LabVIEW 2014\user.lib\_OpenG.lib\lvzip\lvzlib.dll)  I get the error "The Library is not valid for the current platform. For example, you are running 64-bit LabVIEW and the library might be a 32-bit library.

I installed this via VI Package Manager for the 64 bit operating system.

[ShaunR]

1 hour ago, Dan B said:

I am using the ZLIB Library in my project. Originally it was used in 32 bit LabVIEW. Now I am porting it for 64 bit LabVIEW, and I am having issues with ZLIB.

Apparently, the DLL is not valid in 64 bit LabVIEW.

Has this been updated?

Specifically,  the ZLIB CRC32__ogtk.vi will not execute. The Call Library Node is pointing to an invalid library, and when I browse to the Library in the 64 Bit NI folder (C:\Program Files\National Instruments\LabVIEW 2014\user.lib\_OpenG.lib\lvzip\lvzlib.dll)  I get the error "The Library is not valid for the current platform. For example, you are running 64-bit LabVIEW and the library might be a 32-bit library.

I installed this via VI Package Manager for the 64 bit operating system.

Follow the link I gave you in the other thread. It is the droid you are looking for.

[Dan B]

1 hour ago, ShaunR said:

Follow the link I gave you in the other thread. It is the droid you are looking for.

Got it. Everything works now, and my program is now primed for 64 bit use.

It's strange this isn't in the VI Manager as the most recent release, but whatever works.

[GM1811]

On 1.8.2016 at 10:30 PM, ShaunR said:

Follow the link I gave you in the other thread. It is the droid you are looking for.

Hello ShaunR,

 

can you post the Link of the thread here? I have the same problem as Dan B.

Thank you.

[demilk]

This is the post referred to:

And the download link :

oglib_lvzip-4.1.0-b2.ogp

 

[Demoman]

Hi guys

I'm trying to use the lvzip library on a cRIO-9039 target... but I have a problem.
I'm using LabView 2017 + RT + FPGA modules installed. I installed the lvzip-4.1.0-b2 library from the link above, went smoothly.
When I try to run the main VI on the RT target, I got an error from the first attached image. I'm using the "ZLIB Extract All Files To Dir" VI (second attached image). If I disable the VI, the deploy & run is succesfull, with no errors.
Any ideas how to solve this? Should I manually install some files/libraries on the target? If yes then how can I do it i.e. any manual/thread available about this?

Thanks & regards!
Alex

 

[Rolf Kalbermatter]

Yes you need to install the shared library too. If you run VIPM to install the package you should have gotten a prompt somewhere along the install that asked you to allow installation of an additional setup program. This will install the NI Realtime extensions for the LVZIP library.

After that you need to go into NI MAX and go to your target and select to install additional software. Deselect the option to only show recommended modules and then there should be a module for the OpenG ZIP library. Install it and the shared library should be on your controller.

[Demoman]

Thanks, this worked!
I initially run the library installation on a non-admin acount. So that's why it skipped the part with installing the setup program.

1. Always install the libraries under administrator acount.
2. Enable the Secure Shell Server under NI MAX (attached screen).

[Jordan Kuehn]

On 10/12/2017 at 4:59 AM, Rolf Kalbermatter said:

Yes you need to install the shared library too. If you run VIPM to install the package you should have gotten a prompt somewhere along the install that asked you to allow installation of an additional setup program. This will install the NI Realtime extensions for the LVZIP library.

After that you need to go into NI MAX and go to your target and select to install additional software. Deselect the option to only show recommended modules and then there should be a module for the OpenG ZIP library. Install it and the shared library should be on your controller.

Is this still true in LV2021 with Linux RT? I am getting the same error, but the software installer has changed in current max and utilizes packages. In opkg I see a few perl and a python zlib package available, but not lvzlib.

Edited February 4, 2022 by Jordan Kuehn

[Rolf Kalbermatter]

On 2/4/2022 at 11:42 PM, Jordan Kuehn said:

Is this still true in LV2021 with Linux RT? I am getting the same error, but the software installer has changed in current max and utilizes packages. In opkg I see a few perl and a python zlib package available, but not lvzlib.

Actually I have to admit that I have no idea how this works in 2020 and/or 2021. I haven't been working with these versions so far together with RIO projects. I know they changed the package format for installation of software packages from the CDF to the nipkg format either with 2020 or 2021 but I have no idea how one would go about building such a package. I checked the NI Package Builder but so far it only seems to support building packages for Windows installations. Not sure about how the package part for the RT distribution would work.

[Mads]

29 minutes ago, Rolf Kalbermatter said:

Actually I have to admit that I have no idea how this works in 2020 and/or 2021. I haven't been working with these versions so far together with RIO projects. I know they changed the package format for installation of software packages from the CDF to the nipkg format either with 2020 or 2021 but I have no idea how one would go about building such a package. I checked the NI Package Builder but so far it only seems to support building packages for Windows installations. Not sure about how the package part for the RT distribution would work.

Everything works just as before in LabVIEW 2020 at least. We install it from NI Max using the custom software option. 

[Rolf Kalbermatter]

4 minutes ago, Mads said:

Everything works just as before in LabVIEW 2020 at least. We install it from NI Max using the custom software option. 

So is there a problem with installing the OpenG ZIP tools for LabVIEW 2020 and/or 2021 on a cRIO at all or was that inquiry from Jordan something else? I can not place packages on the NI opkg streams so yes this library will always have to be sideloaded through NI-MAX in some way, unless you want to copy it over yourself by hand into the right directory and create the necessary symlinks on a command line and then run ldconfig. 😀

[Jordan Kuehn]

3 hours ago, Rolf Kalbermatter said:

So is there a problem with installing the OpenG ZIP tools for LabVIEW 2020 and/or 2021 on a cRIO at all or was that inquiry from Jordan something else? I can not place packages on the NI opkg streams so yes this library will always have to be sideloaded through NI-MAX in some way, unless you want to copy it over yourself by hand into the right directory and create the necessary symlinks on a command line and then run ldconfig. 😀

I do not see a way to do this in 2021. Perhaps it is because I have the Linux RT image installed and not a custom image to begin with, but right now, even in MAX, I only see the options to configure feeds and to install packages from those feeds. If a package were available or I could build my own (I’m not sure all of the details for this library installation) I can put it in my project and install it as a dependency without it being in the official NI feed. 
 

Am I missing something here? I remember the old way of installing software, but I do not see where that is available anymore. I’ll grab some screenshots when I get to my desk if that would help. 
 

Edit// I think it was like this in 2020 as well now that I’m looking back. I believe it’s the Linux RT image that switches the software installation dialog. I can play with that some too a little later today to see if I can get it working like Mads described. But that would not work long term for me since all of my development is utilizing packages (and SystemLink) now. 

Edit 2// Here is what I see when selecting the base system image to use. I believe if I go with the "Custom Software Installation" it will give the old method back, but that is described as "Legacy".

 

And as you can see here, I only have the ability to add packages via the configured feeds. Some NI feeds, some my own.

Edited February 7, 2022 by Jordan Kuehn

[Rolf Kalbermatter]

Well the OpenG ZIP tools don't really represent a lot of elements on the realtime. Basically you have the shared library itself called liblvzlib.so which should go into /usr/local/lib/liblvzlib.so and then you need to somehow make sure to run ldconfig so that it adds this new shared library to the ldcache file.

When you install the Beta version of the ZIP Tools package you should get a prompt at some point for administrative login credentials (or an elevation dialog if you are already logged in as administrator) which is caused by the ogsetup.exe program being launched as PostInstall hook of the OpenG package.

This ogsetup.exe program does nothing more than extract the different shared libraries into C:\Program Files (x86)\National Instruments\RT Images}\OpenG ZIP Tools\4.2.0.

Depending on your target you need to copy the according liblvzlib.so from either the LinuxRT_arm or LinuxRT_x64 subdirectory to /usr/local/lib/liblvzlib.so on your target. That should be all that is needed although sometimes it can be necessary to also run ldconfig on a command line to have the new shared library get added to the ldcache for the elf loader. With the old installation method in NI-MAX this was taken care of  by the installer based on the according *.cdf file in the OpenG ZIP Tools\4.2.0 directory.

I tried to checkout the NI Package Builder but can't see how one would make a package for RT targets. I also only see the 20.5 and 20.6 versions of the NI Package Builder as last version, maybe that is why?

Edited February 8, 2022 by Rolf Kalbermatter

[Jordan Kuehn]

4 hours ago, Rolf Kalbermatter said:

Well the OpenG ZIP tools don't really represent a lot of elements on the realtime. Basically you have the shared library itself called liblvzlib.so which should go into /usr/local/lib/liblvzlib.so and then you need to somehow make sure to run ldconfig so that it adds this new shared library to the ldcache file.

When you install the Beta version of the ZIP Tools package you should get a prompt at some point for administrative login credentials (or an elevation dialog if you are already logged in as administrator) which is caused by the ogsetup.exe program being launched as PostInstall hook of the OpenG package.

This ogsetup.exe program does nothing more than extract the different shared libraries into C:\Program Files (x86)\National Instruments\RT Images}\OpenG ZIP Tools\4.2.0.

Depending on your target you need to copy the according liblvzlib.so from either the LinuxRT_arm or LinuxRT_x64 subdirectory to /usr/local/lib/liblvzlib.so on your target. That should be all that is needed although sometimes it can be necessary to also run ldconfig on a command line to have the new shared library get added to the ldcache for the elf loader. With the old installation method in NI-MAX this was taken care of  by the installer based on the according *.cdf file in the OpenG ZIP Tools\4.2.0 directory.

I tried to checkout the NI Package Builder but can't see how one would make a package for RT targets. I also only see the 20.5 and 20.6 versions of the NI Package Builder as last version, maybe that is why?

I would be happy to see if I can build a package to share or at the very least move the files over to get it working in my application. However, I do not see that directory on my machine. I did reinstall while running VIPM in admin mode but still nothing.

 

I opened the vipm package via winrar and looked around inside and this is what my post install vi looks like (I'm in Windows):

[Rolf Kalbermatter]

LabVIEW realtime support is only existing in the never officially released OpenG ZIP Library 4.1. That package is only available as download from earlier in this discussion thread here on LavaG and over at the NI forum I believe.

 

[Jordan Kuehn]

1 hour ago, Rolf Kalbermatter said:

LabVIEW realtime support is only existing in the never officially released OpenG ZIP Library 4.1. That package is only available as download from earlier in this discussion thread here on LavaG and over at the NI forum I believe.

 

Got it! I got the files installed, put them in a package build spec in the project explorer. Configured the source file to go where you said and wrote a simple script to run the ldconfig as a post install action. See screenshots below. I installed the package and ran the same test code that errored on deployment and it worked great this time. No manual moving of libraries on my end. Package attached. Thanks for all the help!

openg-zip_1.0.0-2_x64.ipk