Jump to content

Slow MD5


Recommended Posts

5 hours ago, Rolf Kalbermatter said:

The LabVIEW file IO nodes maintain internally a file offset (actually it's the underlying OS file IO functions which do and advance that pointer along as you read)

Error 4 (EOF)still really annoys me. I still maintain it should be a warning and not an error. Every time I have to use my filter error to prevent passing it through I curse and wish a plague on NI. lol

Link to comment
55 minutes ago, ShaunR said:

Error 4 (EOF)still really annoys me. I still maintain it should be a warning and not an error. Every time I have to use my filter error to prevent passing it through I curse and wish a plague on NI. lol

Error handling is always a heated discussion topic. You could argue about the same for timeout errors on network and VISA nodes. And some people get in their frillies about the VISA Read returning a warning when it reads as many characters as you have specified it to read.

A warning wouldn't be better as you still would have to read both the status=FALSE and code==4 to detect it. Also I never really work with the EOF error status as I don't read a file until it errors out but until I reach its size. And if you want to work with the EOF status there is a very easy thing. Using the Clear Errors.vi for error 4 you actually get a boolean status if this error was removed from the error cluster if you need that. Otherwise just terminate the loop on the error cluster anyways, clear error 4 in all cases and go on.

Link to comment

Here is my minor contribution.  Following a thread on the NI forums it seems Open SSL support when installed (through MAX as an optional software package) the compatible binary is libeay32.so.  So I updated the code to remove the Get/Set file positions, and specify the file path using a conditional disable structure.

I tested it this morning on my RT VM and it worked great.  The same 1.4GB file took 20s using native G and only 2.2s using the Open SSL version.

MD5_my8.vi

  • Like 1
Link to comment
3 hours ago, Rolf Kalbermatter said:

Using the Clear Errors.vi for error 4 you actually get a boolean status if this error was removed from the error cluster if you need that.

Not mine. It's just two unconnected error clusters. For this reason I have a VI that I wrote in 2009 similar to what NI eventually eventually implemented (except it defaults to 4).

. This is the diagram :D

image.png.d152ae602728d235ebfa52b00bf87e52.png

 

3 hours ago, hooovahh said:

I tested it this morning on my RT VM and it worked great.  The same 1.4GB file took 20s using native G and only 2.2s using the Open SSL version.

;):thumbup1:

Link to comment
6 hours ago, hooovahh said:

Here is my minor contribution.  Following a thread on the NI forums it seems Open SSL support when installed (through MAX as an optional software package) the compatible binary is libeay32.so.  So I updated the code to remove the Get/Set file positions, and specify the file path using a conditional disable structure.

I tested it this morning on my RT VM and it worked great.  The same 1.4GB file took 20s using native G and only 2.2s using the Open SSL version.

MD5_my8.vi 26.1 kB · 1 download

Some things that need to be done so it doesn't crash arbitrarily (not particularly you, just commenting on the latest incarnation).

EVP_DigestFinal_ex mdlen parameter needs to be pointer to value instead of value as it returns the length,

Return values need to be used (and checked). Currently the functions return Void when they should be I32.

Need to check the MD_CTX pointer isn't 0 for each function.

 

Edit.

almost forgot.

Remove EVP_cleanup(). In versions prior to 1.1.0 it will crash other functions that use EVP and after 1.1.0 it's a no-op.

Edited by ShaunR
Link to comment
On 9/11/2021 at 7:42 AM, JKSH said:

In OpenSSL 1.1.0, "libeay32" was renamed to "libcrypto"

Thanks for the info.  Looks like a wildcard can't be used to select the new binary based on the naming.  I could add code to say if we are on Linux, and the first call to "OpenSSL_add_all_digests" returns an error 7 (file not found) when using the path "libeay32.so" to the binary, then to try one more time using the path of "libcrypto.so".  I mention this method because I don't want to specify the full path, but the Check If File or Folder Exists doesn't search for a file in global path locations.  Not sure if there is a more elegant solution.

Speaking of hard coded paths the Windows method is obviously not ideal at the moment too since it is a hard codded full path.  What would be the best solution here?  Reading registry information on first call?

Link to comment

Okay attached is an updated version that I think supports the changes I made.  It reads the registry in 32 or 64 bit LabVIEW finding the path to the shared installation location.  In Linux it tries both the libeay32.so and libcrypto.so.  In both cases it keeps the found path in a feedback node for later calls.  I also added outputs instead of void for calls, but am now realizing this version doesn't do anything with them yet.

MD5_my9.vi

Link to comment
  • 11 months later...

Has anyone been able to make this work on Linux desktop?
I use LV 2020 SP1 (therefore 64 bit) on Ubuntu 20

I ran

Quote

apt-get update
apt-get install libssl-dev

to install libcrypto.so but at the first call of the so "OpenSSL_add_all_digests" I get error 15 "resource not found" and indeed if in the CLFN I put the full libcrypto.so path, the "OpenSSL_add_all_digests" is not in the "function" list

edit : I guess the answer is RTFM

With help from a co-worker we could make it work on Linux, multiple functions have different names, will post the VI soon.

 

Edited by Antoine Chalons
Link to comment
39 minutes ago, Antoine Chalons said:

Has anyone been able to make this work on Linux desktop?
I use LV 2020 SP1 (therefore 64 bit) on Ubuntu 20

I ran

to install libcrypto.so but at the first call of the so "OpenSSL_add_all_digests" I get error 15 "resource not found" and indeed if in the CLFN I put the full libcrypto.so path, the "OpenSSL_add_all_digests" is not in the "function" list

That's most likely because of this in the OpenSSL headers:

# if OPENSSL_API_COMPAT < 0x10100000L

These functions were required to be called in OpenSSL before 1.1.0but since 1.1.0 OpenSSL automatically initializes its engines on the first call of any function that creates a context of similar session.

And since 1.1.0 is already EOL too and you should either use 1.1.1 or even better 3.0.x, it would be indeed strange if your OpenSSL library still included those APIs. I think the old libssleay.so still contained it in 1.0.1 but it was unnecessary to call that, but when changing the shared library names they also axed many of those compatibility hacks too.

You should probably call

OPENSSL_init_crypto(OPENSSL_INIT_ADD_ALL_CIPHERS | OPENSSL_INIT_ADD_ALL_DIGESTS | OPENSSL_INIT_LOAD_CONFIG, NULL);

and yes this means reading the headers to see what numeric values those constant defines have.

 

Edited by Rolf Kalbermatter
Link to comment
Just now, Rolf Kalbermatter said:

That's most likely because of this in the OpenSSL headers:

# if OPENSSL_API_COMPAT < 0x10100000L

These functions were required to be called in OpenSSL before 1.1.1 but since 1.1.1 OpenSSL automatically initializes its engines on the first call of any function that creates a context of similar session.

And since 1.1.0 is already EOL too and you should either use 1.1.1 or even better 3.0.x, it would be indeed strange if your OpenSSL library still included those APIs. I think the old libssleay.so still contained it in 1.0.1 but it was unnecessary to call that, but when changing the shared library names they also axed many of those compatibility hacks too.

You should probably call

OPENSSL_init_crypto(OPENSSL_INIT_ADD_ALL_CIPHERS | OPENSSL_INIT_ADD_ALL_DIGESTS | OPENSSL_INIT_LOAD_CONFIG, NULL);

 

and yes this means reading the headers to see what numeric values those constant defines have.

 

Your reply came as I was typing my edit... Thanks

Link to comment
  • 5 months later...
On 9/13/2021 at 4:59 PM, hooovahh said:

Okay attached is an updated version that I think supports the changes I made.  It reads the registry in 32 or 64 bit LabVIEW finding the path to the shared installation location.  In Linux it tries both the libeay32.so and libcrypto.so.  In both cases it keeps the found path in a feedback node for later calls.  I also added outputs instead of void for calls, but am now realizing this version doesn't do anything with them yet.

MD5_my9.vi 32.87 kB · 9 downloads

why on earth did you use a ring and a property node on that ring to get the method (md4, md5, etc.)?
an enum with a format into string seems nicer.

A, also, if your VI used on Linux and built as a shared library (in order to run the app as a service) then it causes a crash.
With enum+format into string, it works.

Link to comment
2 hours ago, Antoine Chalons said:

why on earth did you use a ring and a property node on that ring to get the method (md4, md5, etc.)?
an enum with a format into string seems nicer.

A, also, if your VI used on Linux and built as a shared library (in order to run the app as a service) then it causes a crash.

That is some ugly ass code for sure.  I'm fairly certain I didn't create that ring, and instead just copied it from some other example set of code.  I can never see myself center justifying a control like that so I'm guessing I just got it from something else, and then cut and pasted code until it worked.  Enum and format into string is the way to go.  That being said I'm pretty sure I would have tested this on a Linux RT machine and didn't see a crash at least running in source.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.