Rolf Kalbermatter

I put a prerelease package under https://kalbermatter.nl/downloads/oglib_lvzip-5.0.4beta-1.ogp

The package is now officially released as version 5.0.4-1 under https://www.vipm.io/package/oglib_lvzip/

3 minutes ago, ShaunR said:

I have the binaries as arrays of bytes in the Post Install. I convince VIPM to not include the binary dependency and then write out the binary from the Post Install. You can check in the Post install which bitness has invoked it to write out the correct bitness binary. Not sure if that would work on Linux though.

Most likely not! The directories and files during installation are done inside VIPM. In VIPM 2017 you have to start it up as sudo, so the owner will be root. The Post Install script is executed inside the LabVIEW environment through VI Server. That is not launched with sudo and therefore has no write access to the root owned directory.

On 9/3/2024 at 5:28 PM, Antoine Chalons said:

On Linux NI_Unzip.lvlib:Unzip.vi can unzip normal zip files but always return error 42 when I try to unzip a lvappimg file.

Next step is trying to find out how OpenG ZIP Library fails on the lvappimg. My suspicion still is that it is an issue with the fact that the library is compiled for 64-bit. So am going to test it with 64-bit on Windows. Except that I'm using of course the most recent version 5.0 of the library which had quite a bit of changes in the C code to support 64-bit archives. So there is a very good chance that it simply will work.

I can provide you with a prerelase of 5.0.4 that should install under Linux with both VIPM 2017 and 2022 if that helps. That would show if it is still a problem in the current code base. As it is I have not a working 64-bit version of LabVIEW for Linux at this point.

31 minutes ago, ShaunR said:

Installing both binaries huh? Don't blame you. It takes me forever to get VIPM to not include a binary dependency so I can place the correct bitness at install time with the Post Install (64bit and 32bit have the same name. lol)

VIPM has limited and varying support for that depending on its version. "Linux" in VIPM 2017 means any LabVIEW version for Linux. "Linux x64" is not a thing before VIPM 2016 or so, but means 64-bit in VIPM 2017 and newer. And "Linux" in VIPM 2022 means 32-bit LabVIEW only, which is pretty stupid considering that LabVIEW for Linux is 64-bit only since 2017. Definitely an oversight by the VIPM developers that should be fixed. You don't want to require to have to explicitly specify "Linux x64" as supported target for packages that don't even have binary dependencies inside.

It's doable with a Post Install VI in the package that renames the shared libraries after installation, except with VIPM 2017 and maybe others, which needs to run with root privileges to do anything useful.

As my main Linux test system is still LabVIEW 8.6, newer versions have various troubles for me from inability to get a full installer for them to installation trouble on my Linux VMs, I did not want to abandon this version but was under the wrong assumption that it would not support the wildcard library naming to distinguish between 32-bit and 64-bit files, since the first version of LabVIEW supporting 64-bit was 2009. Obviously Brian had been starting to work on 64-bit support earlier already and not all of that support was removed by some precompile symbols it seems, so it is already present in 8.6 LabVIEW. Most likely it would have been possible to release LabVIEW 8.6 for 64-bit but probably with some bugs here and there.

Of course VIPM Package Builder doesn't really support setting specific platform and bitness limitations on individual files or file groups, only on the full package. But I'm using my own OpenG Package Builder instead which has all those possibilities, yet it doesn't really help if VIPM then goes and messes that up anyhow. But creating my own package manager is definitely one step to far for my mental health. 😁

58 minutes ago, Antoine Chalons said:

I see that it's not trivial to handle properly all the cases, to avoid error 8 when installing the zip package the first time, my suggestion is to edit the post install vi like I did below, quick and dirty, works for me

I appreciate your cooperation but there are a number of difficulties with this!

1) I try to avoid dependencies in the package including the Script VIs in OpenG ZIP Library > 5.0. It's not strictly necessary as the dependencies should be already installed at the time VIPM gets to install the real package, but I had situations where this was adding an additional problem.

2) More seriously though, it's not the fact that trying to delete it might fail if the file isn't there. That I wouldn't care and could also ignore by simply adding a Clear Error.vi after the Delete. But even with this change of yours the Delete function will still fail either by trying to clear the write protection on the file it has no access too or if I skip that by trying to delete the file it has no access to. Even ignoring that error won't help as the Move consequently will fail then too.

For VIPM 2022 ignoring any error from the Delete should however work, as it copied the files from the archive to the target location with the same user rights as what LabVIEW was started up with.

For VIPM 2017 there is unfortunately no way to avoid having the 32-bit so installed. If I mark it as Linux only it will install in both cases.

But!!!!! I just noticed that the <library name>*.* wildcard naming in the Library Name control for the Call Library Node seems to work even in LabVIEW 8.6 (which had no 64-bit release). This saves the day!! I just can install the shared library with the 32 and 64 bit postfix alongside each other and be done, without having to abandon support for 8.6!!

Thanks Brian Powell! 

Thanks, that makes a lot of sense. Unfortunately the 32/64-bit support in VIPM was for a long time more or less non-existent, just as my 64-bit version of OpenG ZIP. 😁

When it was added, ca. 2015/2016 or so, it was a quick and dirty fix but not a thorough design (design here doesn't mean it's a mega project, but that there would have been some deliberate choices and decisions). And later fixes seem to have made things rather worse than better. 😏Not that I'm complaining. Multiplatform support for a tool like VIPM or OpenG ZIP is a major pain in the ass! Especially the testing!

That permission issue seems to get a bummer! It looks to me as if the installation of the files to the LabVIEW directory is done in the context of VIPM, so the files inherit root ownership when run in VIPM 2017. Then the Post-Install hook is executed in the context of the LabVIEW installation through VI Server and that operates of course non-elevated and has therefore no rights to change the file attributes or delete the file afterwards. And the platform specific indication in the package to only install specific files depending on the target bitness is not present (pre VIPM 2016), not 32-bit specific (VIPM 2016,2017) or to specific (VIPM 2022 and maybe prior).

Having to select "Linux x64" for LabVIEW target bitness is anyhow not really correct, although understandable since the underlaying VI Server property in LabVIEW distinguishes the bitness of LabVIEW by modifying the according "App.TargetOS" property. And that was how we had implemented these package properties in OpenG Builder back in a far ago time. When adding 64-bit support to VIPM I would rather have seen them to extend the LabVIEW version property to {LabVIEW, LabVIEW 32Bit, LabVIEW 64Bit}, or something like that (technically LabVIEW x68 and LabVIEW x64 would have worked too, since the Intel platform was the only one that allowed for bitness differences in LabVIEW. But of course if you think that far already why limit it on such a thing that could have changed theoretically in the future and did with the addition of Apple Silicon Support which is not x64 but MacOS ARM64). Strictly speaking LabVIEW should only use Linux x64 (or Windows x64), if it runs on a 64-bit kernel and otherwise just Linux (or Windows NT). All those little pesky, hard to imagine, future developments!!

Will have to think a bit about this, but it's going to be tough to allow for installation in both VIPM 2017 and VIPM 2022 with the same package I'm afraid!

Just now, alireza m said:

thank you so much. and something else is bothering me. I have to make a dll out of the .c file I get by "create .c file"?

The file you get with "Create c. file" is just a function prototype with an empty function. You can of course turn that into a DLL but it won't do anything useful. You can fill in the function with some useful code and then it is a good starting point for your DLL.

My usual working method is slightly different. I use the "create c. file" function to create a file containing the matching function prototype, then copy this prototype from that file into the real project C file that I use to implement my DLL. But in most cases I don't even need that intermediate C file as I'm usually able to create the correct function prototype myself.

Seems VIPM 2022 got picky about interpreting Linux as a valid platform indication for anything but 32-bit LabVIEW for Linux? That would be in fact a bug.

Error 8 is a permission error. In that Post Install VI I try to clear the write protection on the 32-bit shared library file to be able to delete it so I can copy the 64-bit DLL over it. Does the user under which you are logged in and run VIPM have admin privileges?

I guess there is no (easy) way to see the permissions of the lvzlib.so right before that Post Install VI executes.

Does the zero-mqtt toolkit also show as disabled in the VIPM 2022 version?

If you use LabVIEW native datatypes, the Import Shared Library Wizard is NOT the right tool to use. Generally speaking if you are not able to configure the Call Library Node yourself correctly, use of the Import Shared Library Wizard is simply a way to setup your code for failure. There is absolutely no way that an automated tool could create always correct code from a C header file alone, except for really trivial APIs. The C syntax is to ambiguous to describe all aspects of an API and it does not at all describe memory management issues for any parameters that are more complex than scalars.

Generally speaking, you either have a non-LabVIEW specific DLL that hopefully uses generic C data types as parameters and is not trying to pass dynamically allocated memory buffers to the caller, or you start with a LabVIEW VI, configure a Call Library Node and connect native LabVIEW datatypes to it, configure the according parameters to Adapt To Type, right click on the Call Library Node and select "Create c. file" and then use the generated prototype and fill in the function body with what you want to implement in the C function.

For the code example 2) I provided you have to:

1) NOT use the Import Library Wizard, it can NOT deal with the complex types that LabVIEW native datatypes are.

2) configure your C compiler to look in the cintools directory inside the LabVIEW directory for header files

3) Configure a Call Library Node yourself with three parameters:

        1) Adapt to Type, pass Pointers to Handles and wiring an according 2D LabVIEW array of Int32 to it

        2) Integer, Pointer sized, Pass by Value

        3) Integer, Pointer sized, Pass by Value

As to your code, yes you can do it like that. "#include extcode.h" is however useless, there is nothing in your code that would need that header file.

You then create a Call Library Node and configure it to have these parameters:

        1) Array, int32, 2 dimensions, Pass as Array Data Pointer

        2) Integer, int32, Pass by Value

        3) Integer, int32, Pass by Value

Make sure to always use "Initialize Array" on the diagram to allocate a 2D array with the same sizes as what you pass to the function in the 2nd and 3rd parameter and wire that array to the 1st parameter!!!! If the array is not allocated or allocated smaller than what you tell the function to fill in, you just created a "crash your computer" program! It may usually not crash (right away), but it will corrupt memory and eventually your LabVIEW program will crash, eat your harddrive or do something else that is undesirable and there is nobody to blame for that but yourself.

And about the used datatypes: While int is always a 32-bit integer for all platforms on which LabVIEW can run and use of it in your C code is therefore not really a problem, other basic C datatypes such as long are not the same size on some of the platforms. Therefore it is useful to use explicitly sized datatypes such as int32_t for parameters that interface with the LabVIEW diagram. LabVIEW is very strictly typed and always uses explicitly sized datatype, the only exception are the two pointer sized integer datatypes, but they only exist in the Call Library Node, never on the diagram.

3 hours ago, alireza m said:

thanks for your opinion but I want to find a way to pass the actual array not some pointer to it

LabVIEW arrays are NOT C array pointers. You can NOT allocate an array with malloc() and hope to return that to LabVIEW in a way that LabVIEW knows that this is an array. You could return it as pointer sized opaque pointer but LabVIEW could not do anything with it directly.

You have two options:

1) Preallocate an array in LabVIEW with the necessary size and pass it to the function. Configure the according pointer to pass the array as C data pointer.

Important: The array MUST be allocated on the LabVIEW diagram. You can NOT use C Library memory management functions to allocate, resize and free memory buffers that you want LabVIEW to use on the diagram!

#inclide <stdint.h>
  
void GetArray(int32_t *arr, size_t dim_x, size_t dim_y)
{
	for (size_t x; x < dim_x; x++)
    {
	    for (size_t y; y < dim_y; y++)
        { 
             arr[x + y * dim_x] = 0;
        }
    }
}

2) Use a native LabVIEW array.

#include "extcode.h"

#include "lv_prolog.h"
typedef struct
{
    int32_t dims[2];
    int32_t elms[];
} Int2DArrRec, *Int2DArrPtr, **Int2DArrHdl;
#include "lv_epilog.h"

MgErr GetArray(Int2DArrHdl *arr, size_t dim_x, size_t dim_y)
{
    MgErr err = NumericArrayResize(iL, 2, (UHandle*)arr, dim_x * dim_y);
    if (!err)
    {
        int32_t *ptr = (**arr)->elms;
        for (size_t x; x < dim_x; x++)
        {
            for (size_t y; y < dim_y; y++)
            { 
                ptr[x + y * dim_x] = 0;
            }
        }
        (**arr)->dims[0] = dom_x;
        (**arr)->dims[1] = dom_x;
    }
    return err;
}

23 hours ago, Antoine Chalons said:

if I can help in any way with testing on Linux, DM me.

I need a bit more info here!

First: versions before 4.2 had no support for 64-bit, neither Windows nor Linux. Unless you use LabVIEW 2016 or earlier, your LabVIEW for Linux version is 64-bit. Since you can use 4.0.0 in Windows, I have to assume that you use the 32-bit version of LabVIEW there.

Is there a chance that you could try it with 64-bit LabVIEW under Windows with the >= 5.0.0 version of OpenG ZIP Library?

Second: it seems VIPM treats Windows differently to Linux as allowed target indicator:

Windows NT: means all NT derived Windows versions indpendent of bitness

Windows x86: is only Windows 32-bit

Windows x64: is only Windows 64-bit

Linux: is only Linux 32-bit

Linux x64: is 64-bit Linux

I tried to verify some of the packages on VIPM.io and if they want to explicitly say that they are for Linux, they seem to all only specify "Linux" as allowed target. Yet according to your results such packages couldn't install on LabVIEW 2017 and newer on Linux!

Could you try for a quick laugh if you can install one of the following packages on your Linux system?

- labview-zmq-3.6.2.112

- denso_robotics_lib_dpam_robot_library-2.1.0.75

- electric_power_research_institute_lib_opendss_library_x64-1.0.0.7

- national_synchrotron_radiation_laboratory_lib_nsrl_cas_interface_linux-1.0.0.5

- ogrsc_restart-1.3-1

- ovak_technologies_lib_labview_universal_xml_parser-1.0.0.13

- pantherlab_lib_panther_dashboard-x.x.x.x

- sas_workshops_lib_clearlvcache_for_g_cli-1.0.x.x

- sas_workshops_lib_clearlvcache_for_g_cli-1.0.x.x

I will try this evening at home to see why that is. It SHOULD actually work but the whole building of the package has turned out to be a lot more hassle than I remember having had in the past.

49 minutes ago, Antoine Chalons said:

in 4.0.0 the CLFN had an absolute path to the shared lib that was like /C/Work/OpenG/....
and I couldn't find the actual so file under ~/user.lib and didn't feel like relinking all the CLFNs anyway as I knew that 4.2 would work.

The size of the lvappimage I tested with is ~360 MB

Well, you should definitely try 5.0.3, 5.0.0 to 5.0.2 had various problems in terms of proper installation on specific platforms and the shared library path having been messed up by the build tools in the sources for the package.

8 hours ago, Antoine Chalons said:

oh, and with 4.2.0b1-1 on Windows, it works, it can successfully unzip an lvappimg.

I’m intrigued. Linux support for > 5.0.0 is definitely in the package, why it doesn’t allow you to select it is not yet clear to me.

I have a hunch why it might not work in pre 5.0 version but that that doesn’t affect the Windows version is strange. How large is the lvapp image?

On 9/3/2024 at 7:42 PM, Antoine Chalons said:

i sure will

It does look like the unzip.vi is using a CLFN that calls functions of the LabVIEW app so it might be doing some special stuff.

It could, but generally, the unzip functions in the LabVIEW runtime are simply a (somewhat older) version of the ZLIB code plus MiniZIP, the same I'm using in OpenG ZIP. Of course OpenG ZIP 5.0 has been updated to use the latest ZLIB sources, an alternate extended version of MiniZIP that supports 64-bit ZIP files and ZIP streams, and Unicode file name support.

Since ZLIP and MiniZIP in the contributed part of ZLIB are under the extremely liberal ZLIB license, similar to the 3 clause BSD license, NI is free to include it directly in the LabVIEW runtime.

1 hour ago, Antoine Chalons said:

On Windows I use the NI_Unzip.lvlib:Unzip.vi to unzip normal zip files as well as lvappimg file (cRIO image file created by MAX or RAD) and it works.

On Linux NI_Unzip.lvlib:Unzip.vi can unzip normal zip files but always return error 42 when I try to unzip a lvappimg file.

I've reported this as a bug to NI

 

Anyone knows another way to unzip a lvappimg file?

cross-posted to NI Forums

Did you try with OpenG ZIP library?

https://www.vipm.io/package/oglib_lvzip/

No guarantee that it works as I never considered testing that, but it is a quick test.

On 8/27/2024 at 7:35 PM, hooovahh said:

Oh yeah it sucks. I do what I can, I contact the admin when there are issues I can't resolve. I appreciate your patients.

More than 1000 posts further after the long weekend! I really would appreciate if Michael could step in for a minute here. Do you have any idea if he is still around?

4 hours ago, Novgorod said:

I'm not complaining about an undocumented/non-public API change after 20 years, just being curious .. I poked around a bit more - the prototype looks alright, it's just that the function is not intended to provide an MD5-hash of a caller-supplied message. The "message" parameter is actually an output that returns 16 random bytes (there are RNG/GUID calls at least under Windows), which get MD5 hashed. So it outputs a 16-byte random string (GUID) and its MD5 hash for some very specific application. The actual MD5 subroutine is not exposed, so it was all a red herring stinking up the attic...

The most easy solution of all: But likely not extremely fast although fairly fast should do it!

https://forums.ni.com/t5/LabVIEW/Any-GUID-generating-VI-available/m-p/4392240#M1293609

Even this should be considered LabVIEW attic area as indicated by the poop brown color of the VI Server node, but with considerably less rusty nails. Worst case, it simply stops working one day. It's most likely using the MD5 code that the old GetMD5Hash() was using. Anything else would have been traditionally at least potentially trouble to implement on non Windows platforms without help of extra library installations such as OpenSSL, EmbedTSL or coreutils, etc.

18 hours ago, Novgorod said:

I tried using the GetMD5Digest function but I'm getting strange behavior (unsurprisingly, I know)... The MgErr type is not defined in extcode.h, but I assume it's just an int32 error code (right?), so that shouldn't be the issue. Here's a minimal implementation:

The digest output string seems to require at least a 16-byte pre-allocated buffer or it returns an empty string. The confusing part is that the input string buffer gets overwritten with a seemingly random 16-byte string and the output MD5 digest is the correct MD5 hash for that random 16-byte string and NOT for the actual input string. That random string also changes every time the function is called. Maybe it needs some kind of initializer? I'd appreciate if you have any more info on that, just for the sake of curiosity... 😉

Well, that's what you get for strolling in the LabVIEW attic. Seems the prototype and functionality as I posted it was correct at LabVIEW 5.x times until sometimes later, but later got changed by NI for some reasons. Since it was an undocumented API, they are of course free to do that anytime they like (and this is also one of the main reasons to not document such APIs). The only one that could get hurt is themselves when they happen to forget to adapt a caller somewhere to the new function interface (and of course people like me and you who try to sneak in anyways).

It's also while this is called the LabVIEW attic with many rusty nails laying in the open.

43 minutes ago, hooovahh said:

I did try to implement inflate/deflate in pure G.  There was a website that explained in pretty easy to understand english how the compression works, and gave example code on how to implement it.  I went through it step by step but it still wasn't producing the correct data.  After trying in my free time for a bit I gave up. I thought it would be helpful on Linux RT for web server stuff where it could make PNG compressed images. There are alternatives.

Believe me, while it is certainly possible, this is one area where you do not want to use LabVIEW to implement it. Main reason are twofold:

It's a lot of bit twiddling and while LabVIEW supports rotate and boolean operators on numbers and is quite good at it, there is an inherent overhead since LabVIEW does do some extra safe keeping to avoid errors that you can easily do in programming languages like C. This typically is negligible except when you do it in a loop 8 * many million times. Then these small extra checks really can add up. So a LabVIEW inflate or deflate algorithm will always be significantly slower than the same in C, compiled as optimized code. This is because the boolean and rotate operations take an extremely short time to execute and the extra LabVIEW checks each time to be always safe are significant in comparison.

The second reason is the fact that bit twiddling is an extremely easy thing to get wrong and mess up. You may think to have copied the according C code exactly but in most cases there are more or less subtle things that will turn your results upside down. Debugging on that level is extremely frustrating. I have run code many many times in single step mode trying to find the error in those binary operators and missed the actual issue many times until finally nailing it. It's an utterly frustrating experience and it made me long ago promise to myself to not try to implement compression or decompression as well as encryption routines in LabVIEW except the most trivial ones. ZLIB however is not trivial, not at all!

11 hours ago, Novgorod said:

Interesting topic.. Has anyone successfully played around with the built-in (undocumented) Labview manager function "GetMD5Digest"? I assume it's used internally for VI block hashing and other stuff and I'm wondering what's the performance compared with the "public" G code MD5 utilities. If it's faster, it could be a very practical alternative because the manager functions are platform-independent (right?).

Yes that function is used for generating hashes in LabVIEW itself for various things including password protection. It was never meant for hashing huge buffers and is basically the standard Open Source code for MD5 hashing that floats on the internet for as long as the internet exists. It is similar to what the OpenSSL code would do, although OpenSSL is likely a little more optimized/streamlined. This function exists in LabVIEW since about at least 5.0 or thereabout and likely has never been looked at again since, other than use it.

You could of course call it, and dig in the LabVIEW attic with its many rusty nails, but why? Just because you can does not mean you should! 🙂

If you absolutely insist you might try yourself. This is the prototype:

MgErr GetMD5Digest(LStrHandle message, LStrHandle digest);

1 hour ago, LogMAN said:

You mean this?

It's still visible for me.

Not for me for most accounts, but that is a minor nuisance. But just checked and funnily I can view it on yours.

But the current situation really breaks the camels back. I refuse to be racist, but the obvious Indian influence in it simply makes me very angry.

I propose to simply take it offline until something can be done about it. This is less than not useful as is.

1 hour ago, Rolf Kalbermatter said:

It's starting to get really ridiculous. They seem to have optimized their spam bots to the point where they spam two forums with 120 posts so far, and others incidentally, most likely as trial to get them flooded too. If this continues like this you can shutdown this forum altogether! Nobody is coming here to see this shit and it drives the few that come here definitely away.

I really hope Michael can find his admin login credentials and do something about this.

 

Please let me ban Baba's this and Baba's that, visha's and all those other accounts. Pretty pretty please!!!!!!!!! 

Obviously disabling account creation does nothing! They are currently creating new accounts about every 10 minutes. And then start spamming a few minutes later. There still are some active logins here and there from accounts with names that are fairly telling, and that obviously have been banned already, with 0 recent activity, created a few days ago, but supposedly having 10s of posts according to the account page.

What seems disabled is the Recent Profile Visitors block on the account pages for everybody. Not sure what that helps with, but it definitely is not the solution.

At the current rate, banning accounts after they posted dozen or hundreds of messages, is simply not a manageable solution anymore. If this can't be changed, this forum is dead.

23 minutes ago, mooner said:

Can anyone tell me how to tune out the first VI missing selection? Or explain to me how I can hide the options in the red box?

Maybe in VI Properties: Execution: Allow Debugging?

It's starting to get really ridiculous. They seem to have optimized their spam bots to the point where they spam two forums with 120 posts so far, and others incidentally, most likely as trial to get them flooded too. If this continues like this you can shutdown this forum altogether! Nobody is coming here to see this shit and it drives the few that come here definitely away.

I really hope Michael can find his admin login credentials and do something about this.

Please let me ban Baba's this and Baba's that, visha's and all those other accounts. Pretty pretty please!!!!!!!!!