Slow MD5

[Rolf Kalbermatter]

10 minutes ago, Lipko said:

Never mind, maybe I used File/directory info.vi.

That does take some time as LabVIEW has to enumerate the directory contents for all files to get the  size which is the number of files in the directory.

[ShaunR]

5 hours ago, Rolf Kalbermatter said:

The LabVIEW file IO nodes maintain internally a file offset (actually it's the underlying OS file IO functions which do and advance that pointer along as you read)

Error 4 (EOF)still really annoys me. I still maintain it should be a warning and not an error. Every time I have to use my filter error to prevent passing it through I curse and wish a plague on NI. lol

[Rolf Kalbermatter]

55 minutes ago, ShaunR said:

Error 4 (EOF)still really annoys me. I still maintain it should be a warning and not an error. Every time I have to use my filter error to prevent passing it through I curse and wish a plague on NI. lol

Error handling is always a heated discussion topic. You could argue about the same for timeout errors on network and VISA nodes. And some people get in their frillies about the VISA Read returning a warning when it reads as many characters as you have specified it to read.

A warning wouldn't be better as you still would have to read both the status=FALSE and code==4 to detect it. Also I never really work with the EOF error status as I don't read a file until it errors out but until I reach its size. And if you want to work with the EOF status there is a very easy thing. Using the Clear Errors.vi for error 4 you actually get a boolean status if this error was removed from the error cluster if you need that. Otherwise just terminate the loop on the error cluster anyways, clear error 4 in all cases and go on.

[hooovahh]

Here is my minor contribution.  Following a thread on the NI forums it seems Open SSL support when installed (through MAX as an optional software package) the compatible binary is libeay32.so.  So I updated the code to remove the Get/Set file positions, and specify the file path using a conditional disable structure.

I tested it this morning on my RT VM and it worked great.  The same 1.4GB file took 20s using native G and only 2.2s using the Open SSL version.

MD5_my8.vi

[ShaunR]

3 hours ago, Rolf Kalbermatter said:

Using the Clear Errors.vi for error 4 you actually get a boolean status if this error was removed from the error cluster if you need that.

Not mine. It's just two unconnected error clusters. For this reason I have a VI that I wrote in 2009 similar to what NI eventually eventually implemented (except it defaults to 4).

. This is the diagram 

 

3 hours ago, hooovahh said:

I tested it this morning on my RT VM and it worked great.  The same 1.4GB file took 20s using native G and only 2.2s using the Open SSL version.

[ShaunR]

6 hours ago, hooovahh said:

Here is my minor contribution.  Following a thread on the NI forums it seems Open SSL support when installed (through MAX as an optional software package) the compatible binary is libeay32.so.  So I updated the code to remove the Get/Set file positions, and specify the file path using a conditional disable structure.

I tested it this morning on my RT VM and it worked great.  The same 1.4GB file took 20s using native G and only 2.2s using the Open SSL version.

MD5_my8.vi 26.1 kB · 1 download

Some things that need to be done so it doesn't crash arbitrarily (not particularly you, just commenting on the latest incarnation).

EVP_DigestFinal_ex mdlen parameter needs to be pointer to value instead of value as it returns the length,

Return values need to be used (and checked). Currently the functions return Void when they should be I32.

Need to check the MD_CTX pointer isn't 0 for each function.

 

Edit.

almost forgot.

Remove EVP_cleanup(). In versions prior to 1.1.0 it will crash other functions that use EVP and after 1.1.0 it's a no-op.

Edited September 10, 2021 by ShaunR

[JKSH]

21 hours ago, hooovahh said:

the compatible binary is libeay32.so

That means it is using OpenSSL 1.0.x or earlier.

In OpenSSL 1.1.0, "libeay32" was renamed to "libcrypto"

[hooovahh]

On 9/11/2021 at 7:42 AM, JKSH said:

In OpenSSL 1.1.0, "libeay32" was renamed to "libcrypto"

Thanks for the info.  Looks like a wildcard can't be used to select the new binary based on the naming.  I could add code to say if we are on Linux, and the first call to "OpenSSL_add_all_digests" returns an error 7 (file not found) when using the path "libeay32.so" to the binary, then to try one more time using the path of "libcrypto.so".  I mention this method because I don't want to specify the full path, but the Check If File or Folder Exists doesn't search for a file in global path locations.  Not sure if there is a more elegant solution.

Speaking of hard coded paths the Windows method is obviously not ideal at the moment too since it is a hard codded full path.  What would be the best solution here?  Reading registry information on first call?

[hooovahh]

Okay attached is an updated version that I think supports the changes I made.  It reads the registry in 32 or 64 bit LabVIEW finding the path to the shared installation location.  In Linux it tries both the libeay32.so and libcrypto.so.  In both cases it keeps the found path in a feedback node for later calls.  I also added outputs instead of void for calls, but am now realizing this version doesn't do anything with them yet.

MD5_my9.vi