.net decompression GZip or deflate

[X___]

That was a trick question.

[Neil Pate]

It's a real shame that more of this stuff is not open to viewing. I often wonder how something is done and try to peek inside the VI. Often it is just a bunch of CLN calls which is fine, I poke around to see what other functions are exported in the DLL but that is about it., I get bored after a short while and have bills to pay so have to move onto real work.

We know if screw with undocumented stuff it will break. No API is perfect, but at least let us see!

[Rolf Kalbermatter]

On 5/20/2020 at 2:17 PM, hooovahh said:

That's good.  I just know that for other things it can be more selective on the compression type.  I wanted to make a way to create a PNG file outside of NI's native way due to some limitations.  I read up on the documentation and it honestly wasn't all that complicated.  The only issue is that part of the file is compressed with zlib.  Using OpenG it made good normal PNGs file that could be viewed in any image viewer.  But using LabVIEW's native API for compression created an image that only some image viewers would display, chrome was not one of them.  I also made a BLF file API.  This is a logging file format for raw automotive CAN data.  It also has chunks of data that are compressed.  Using NI's native method created a file that Vector tools couldn't open and I also needed OpenG.

There are several levels in a ZIP file. A ZIP file is an archive format with things like directory entries, and stream headers. The directory entry contains records for every stream inside the archive (a stream is pretty much a single file entry in the archive) and each stream has its own header. The individual streams are compressed using the ZLIB algortithme with a small header in front. So while the ZIP file does use ZLIB to compress the individual streams it is anything but a ZLIB compressed stream in itself. GZ is closer to a single ZLIB stream but needs an additional header that tells the decomprosser about how to configure the decompression, so the file content is not a ZLIB stream but a bit more than that.

The OpenG ZIP library is mainly an implementation of a ZIP archive reader and writer but uses at its core the zlib library for the compression and decompression, just as the ZIP library does that is included in LabVIEW. While the LabVIEW provided ZIP library only exports a few high level functions to be accessed, the OpenG version goes a lot further and basically exports a lower level API on which the whole ZIP archive handling is implemented. And in addition it also exports the according ZLIB  inflate and deflate functions that allow you to implement any compression that is based on the ZLIB compression method. It would be possible to implement GZ and similar formats on top of this without a lot of effort, by simply implementing a LabVIEW library on top of these ZLIB methods and some LAbVIEW file IO methods.

[Rolf Kalbermatter]

On 5/20/2020 at 11:30 PM, Neil Pate said:

It's a real shame that more of this stuff is not open to viewing. I often wonder how something is done and try to peek inside the VI. Often it is just a bunch of CLN calls which is fine, I poke around to see what other functions are exported in the DLL but that is about it., I get bored after a short while and have bills to pay so have to move onto real work.

We know if screw with undocumented stuff it will break. No API is perfect, but at least let us see!

These ZIP VIs that come with LabVIEW are all single CLNs that call directly into LabVIEW, as NI has basically included ZLIB and the according old style ZIP file functions that come in the contributed folder of the ZLIB distribution and which were developed by Gilles Vollant. OpenG ZIP Library uses the same code, just likely a newer version. The LabVIEW inlcuded code does not allow to query the implementation version of the library so it's hard to say how up to date it is.

[Rolf Kalbermatter]

On 5/20/2020 at 10:56 PM, hooovahh said:

You'd have to ask NI, but I suspect they don't want to have to support users calling the functions directly, and instead they only want to support the users calling the VIs that they create, exposing the functions.  It gives them tighter control, and likely means less unit testing if they know a function will only ever be called a certain way.  For instance the Write PNG File has no compression.  The function internally does have a compression input, and setting it does create a more compressed PNG image.  But it is possible that compression isn't tested or supported on all platforms.  So they can either support compression on some targets and not others, either making duplicate VIs, or erroring out on those targets.  Or they just take that ability away.  Well that or someone just didn't realize that input would be requested and left it off the connector pane.

The main reason is that NI does not want to document the API for these functions. Not so much because it is secret but because once they are documented they can't change it anymore. Without having them documented (and an open VI diagram with the according CLN configuration is a sort of documentation too) they can't just go and change it easily as someone might have relied on this API and created his own VIs. With locked diagram they are free to change the API at any time and only have to update the according VIs that are shipped with LabVIEW and all is fine. If you sneaked into the diagram anyhow and used it, that is all your own fault. 😀

[X___]

55 minutes ago, Rolf Kalbermatter said:

The OpenG ZIP library is mainly an implementation of a ZIP archive reader and writer but uses at its core the zlib library for the compression and decompression, just as the ZIP library does that is included in LabVIEW. While the LabVIEW provided ZIP library only exports a few high level functions to be accessed, the OpenG version goes a lot further and basically exports a lower level API on which the whole ZIP archive handling is implemented. And in addition it also exports the according ZLIB  inflate and deflate functions that allow you to implement any compression that is based on the ZLIB compression method. It would be possible to implement GZ and similar formats on top of this without a lot of effort, by simply implementing a LabVIEW library on top of these ZLIB methods and some LAbVIEW file IO methods.

Can't use it with 64-bit LV, AFAIK...

[dadreamer]

1 hour ago, X___ said:

Can't use it with 64-bit LV, AFAIK...

I successfully used oglib_lvzip-4.1.0-b3 from this post, enhanced for 64-bit LV by Rolf. It always worked good for me.

[Rolf Kalbermatter]

16 minutes ago, dadreamer said:

I successfully used oglib_lvzip-4.1.0-b3 from this post, enhanced for 64-bit LV by Rolf. It always worked good for me.

You may possible rather use this version 4.2.0-b1 here.