TCP_NODELAY.llb library

[Mark Yedinak]

Does anyone know the password for the GetSocketInfo VI in the above library? I would like to make my own library with several other functions. I would like to use the new lvlib format instead of the llb format. However, the one VI is password protected and therefore I cannot add it in any other library.

[Phillip Brooks]

I'm not sure, but isn't "GetRawSocketFromConnectionID.vi" found in the TCP_NODELAY.LLB of that KB the the same as "vi.lib\Utility\tcp.llb\TCP Get Raw Net Object.vi"?.

[ShaunR]

I'm not sure, but isn't "GetRawSocketFromConnectionID.vi" found in the TCP_NODELAY.LLB of that KB the the same as "vi.lib\Utility\tcp.llb\TCP Get Raw Net Object.vi"?.

I believe it is (or at least a wrapper for it).

This is the prototype it (TCP Get Raw Net Object.vi) uses as the call to LabVIEW

int32_t NCGetRawNetObject(uint32_t nc, uintptr_t *netobject);

It is just a typecast to get the nc parm from the connection ID.

Edited September 30, 2011 by ShaunR

[Mark Yedinak]

I will have to look at that. If it uses a standard library in the LabVIEW distribution I can modify the code to use that. The problem with an llb file is that it uses absolute paths which can cause problems when using source code control. I would like to avoid that issue hence my desire to break the VIs out and add them to a lvlib file instead.

[Rolf Kalbermatter]

I will have to look at that. If it uses a standard library in the LabVIEW distribution I can modify the code to use that. The problem with an llb file is that it uses absolute paths which can cause problems when using source code control. I would like to avoid that issue hence my desire to break the VIs out and add them to a lvlib file instead.

The VI in VI lib should work. It may use a different API internally, if NI hasn't updated it, but should in fact do the same. Early on when networking was really sort of an addon API in LabVIEW (~ V 4) there existed a function to retrieve all kinds of internal parameters. One of them was to get the associated network socket of a network refnum. But that function ultimately called the predecessor of NCGetRawNetObject() which wasn't exported back then.

Being a standard VI lib function it should solve your source code control issues.

[ShaunR]

On 9/30/2011 at 5:07 PM, ShaunR said:

I believe it is (or at least a wrapper for it).

This is the prototype it (TCP Get Raw Net Object.vi) uses as the call to LabVIEW

int32_t NCGetRawNetObject(uint32_t nc, uintptr_t *netobject);

It is just a typecast to get the nc parm from the connection ID.

Does anyone know how to do the reverse? i.e. turn a socket into a LabVIEW refnum? I want to replace the LabVIEW Connect primitive with one that supports IPv6 but don't really want to replace the read and write.

[Rolf Kalbermatter]

1 hour ago, ShaunR said:

Does anyone know how to do the reverse? i.e. turn a socket into a LabVIEW refnum? I want to replace the LabVIEW Connect primitive with one that supports IPv6 but don't really want to replace the read and write.

That's not a publically exported API unfortunately.  This is an internal function called by the NCConnect(), NCCreateListerer() and NCWaitOnListener() functions when a socket needs to be wrapped into a network refnum. And no, the according APIs with these names that are exported are just stubs that return an error. The real implementation is in non-exported functions too, as someone back in the LabVIEW 4.0 or 5.0 days decided that the Network Manager API should not be exported from the LabVIEW kernel for some reasons. Most probably a left over from the LabVIEW for Mac days when the TCP IP library was an external library implemented with CINs.  Rather than just removing the functions from the export table they renamed them internally and all network functionality uses those new names and empty stubs returning "Manager Call not Supported" status were exported instead.

Edited January 13, 2020 by Rolf Kalbermatter

[ShaunR]

3 hours ago, Rolf Kalbermatter said:

That's not a publically exported API unfortunately.  This is an internal function called by the NCConnect(), NCCreateListerer() and NCWaitOnListener() functions when a socket needs to be wrapped into a network refnum. And no, the according APIs with these names that are exported are just stubs that return an error. The real implementation is in non-exported functions too, as someone back in the LabVIEW 4.0 or 5.0 days decided that the Network Manager API should not be exported from the LabVIEW kernel for some reasons. Most probably a left over from the LabVIEW for Mac days when the TCP IP library was an external library implemented with CINs.  Rather than just removing the functions from the export table they renamed them internally and all network functionality uses those new names and empty stubs returning "Manager Call not Supported" status were exported instead.

Indeed. I thought maybe there might have been some moveblock trickery that could have sufficed. Oh well. I've already replaced the listen, connect and close so might as well add read and write.

[Rolf Kalbermatter]

On 1/13/2020 at 6:39 PM, ShaunR said:

Indeed. I thought maybe there might have been some moveblock trickery that could have sufficed. Oh well. I've already replaced the listen, connect and close so might as well add read and write.

Depends how you implement them but to support asynchronous behaviour is a little nasty as you need to basically create a temporary structure that gets initialized on first call and then you need to call repeatedly into the shared library function to call poll() or select() to check the status of the socket and update the structure until the operation completes either because an error occured, the timeout elapsed or the requested data has been sent/received. Without that your call will be synchronous, consuming the calling LabVIEW thread and limiting potential parallelization which the native node does support out of the box.

Synchronous operation is not a problem as long as you only process one or two connections at the same time but will certainly cause problems for server application or clients that need to process many seperate connections quasy parallel.

Why I know that? I did the asynchronous implementation in this library: 

 

 

Edited January 15, 2020 by Rolf Kalbermatter

[ShaunR]

1 hour ago, Rolf Kalbermatter said:

Depends how you implement them but to support asynchronous behaviour is a little nasty as you need to basically create a temporary structure that gets initialized on first call and then you need to call repeatedly into the shared library function to call poll() or select() to check the status of the socket and update the structure until the operation completes either because and error occured, the timeout elapsed or the requested data has been sent/received. Without that your call will be synchronous consuming the calling LabVIEW thread and limiting potential paralleliziation which the native node does support out of the box.

Synchronous opearation is not a problem as long as you only process one or two connections at the same time but will certainly cause problems for server application or clients that need to process many seperate connections quasy parallel.

Why I know that? I did the asynchronous implementation in this library: 

 

Yes. I already do all of that for TLS (using the OpenSSL bios rather than the OS. of course) but I used the LabVIEW Open. Connect and Close to keep it cross platform when obtaining the socket. Now I've replaced those with OS sockets so was looking at completeness (of the sub-library) for sockets-I already have things like Linger and TCP_Nodelay and have added open, close, listen. I know I don't really have to worry about cross platform since I've given up on Linux/Mac support in the commercial package but I do still make it all cross platform so that I can use it on them 

As for your Network library. I would never have written mine if you had released it . Unfortunately you added a death timer so it' was unusable when the time came for me to use SSL (and no source for the binaries which hide some functionality). I did try to encourage you to release (in that thread) it but eventually had to write my own. Knowing what I do now about openSSL, I don't blame you for not releasing it but at the time (and for a long time after including at the time of this post) it left a huge hole in capabilities that NI weren't interested in filling.

Edited January 13, 2020 by ShaunR

[Rolf Kalbermatter]

On 1/13/2020 at 10:22 PM, ShaunR said:

Yes. I already do all of that for TLS (using the OpenSSL bios rather than the OS. of course) but I used the LabVIEW Open. Connect and Close to keep it cross platform when obtaining the socket. Now I've replaced those with OS sockets so was looking at completeness (of the sub-library) for sockets-I already have things like Linger and TCP_Nodelay and have added open, close, listen. I know I don't really have to worry about cross platform since I've given up on Linux/Mac support in the commercial package but I do still make it all cross platform so that I can use it on them 

As for your Network library. I would never have written mine if you had released it . Unfortunately you added a death timer so it' was unusable when the time came for me to use SSL (and no source for the binaries which hide some functionality). I did try to encourage you to release (in that thread) it but eventually had to write my own. Knowing what I do now about openSSL, I don't blame you for not releasing it but at the time (and for a long time after including at the time of this post) it left a huge hole in capabilities that NI weren't interested in filling.

You are aware that there is a LabVIEW Idea Exchange entry about SSL TLS support in LabVIEW that is since about 6 month in development. Most likely not something to appear in LabVIEW 2020 though. I was considering reviving my library but when I saw that I abandoned the idea.

Edited January 15, 2020 by Rolf Kalbermatter

[ShaunR]

1 hour ago, Rolf Kalbermatter said:

You are aware that there is a LabVIEW Idea Exchange entry about SSL TLS support in LabVIEW that is since about 6 month in development. Most likely not something to appear in LabVIEW 2020 though. I was considering reviving my library but when I saw that I abandoned the idea.

Yes. All it means is that I'm moving focus from the NI binaries to supplying my own. NI have usually been a couple of years behind the latest but my customers are demanding the latest (hence my questions about TLS1.3). Couple that with OpenSSL having recently made some ABI breaking changes between versions means that multi-version LabVIEW support is a little tricky if continuing with the NI binaries. So in LabVIEW 20XX you will have [probably] the long term support version of binaries and with mine you'll have the latest with backward compatibility to LabVIEW 2012.

Edited January 15, 2020 by ShaunR

[Rolf Kalbermatter]

On 1/15/2020 at 5:28 PM, ShaunR said:

Couple that with OpenSSL having recently made some ABI breaking changes between versions means that multi-version LabVIEW support is a little tricky if continuing with the NI binaries.

Recently? OpenSSL does that all the time. There have been incompatible API changes all the time. Some between 0.9 and 1.0. a few more serious ones between 1.0 and 1.1 including renaming of the shared library itself to "mitigate" the version incompatibility problem. And expect some more when they go to 3.0. Wait 3.0? what happened to 2.x? 😀

And when you look at their examples they are riddled with

#if OpenSSLVer <= 0968
   call this API
#else
   call this other API
#endif

That's definitely not going to work well unless you always can control exactly which version of OpenSSL is installed on a computer for your specific app and then you are stuck with doing all the maintenance yourself when a new version is released. I sort of managed to make the code of my library autodetect changes between 0.9 and 1.0 and adapt dynamically but definitely gave up when they started 1.1. That together with the fact that IPv6 and TLS support beyond what the LabVIEW HTTP VIs offered wasn't even any priority anywhere.

Now with 1.0.2 definitely gone in obsolete mode my library wouldn't even compile properly for the time being. 😀

PS: Just recently read somewhere that IPv4 address range has been now officially depleted, so no new IPv4 address ranges can be given out anymore. This still isn't the end of the internet as many internet providers use dynamic IP address assignment and nobody should be even thinking about connecting his coffee machine or fridge directly to the internet 😂 but it definitely shows that IPv6 support by internet providers should be something they care about. But while we here in the Netherlands have one of the highest internet connectivity rates of the world, the majority of internet providers still doesn't provide a "working"  IPv6 connectivity itself. You have to use tunnels to test that!

Edited January 17, 2020 by Rolf Kalbermatter

[ShaunR]

3 hours ago, Rolf Kalbermatter said:

That's definitely not going to work well unless you always can control exactly which version of OpenSSL is installed on a computer for your specific app and then you are stuck with doing all the maintenance yourself when a new version is released. I sort of managed to make the code of my library autodetect changes between 0.9 and 1.0 and adapt dynamically but definitely gave up when they started 1.1. That together with the fact that IPv6 and TLS support beyond what the LabVIEW HTTP VIs offered wasn't even any priority anywhere.

I was failry well insulated (historically) by the NI libraries, from that, since all the LabVIEW versions share the same binaries. NI wanted the binaries to be backwards compatible so they compiled them with the compatibilty flag (and a couple of their own functions). At the time I ummed and ahhd about using my own binaries and took the easy route-especially because of cross platform support where you had no idea what versions of openSSL where installed in Linux distributions on a day-to-day baisis. Not to mention the ball-ache of actually compiling them. Your preference for an intermediary binary definately has the advantage here by being able to dynamically load function pointers wheras I'm pretty stuck with one function breaks everything.

3 hours ago, Rolf Kalbermatter said:

Now with 1.0.2 definitely gone in obsolete mode my library wouldn't even compile properly for the time being.

I've just spent 3 days trying to get OpenSSL 1.1.1 to compile. Perl? Really? I thought CMake was bad  I always do things the hard way, though, because I don't want any dependencies. If you look at all the Windows tutorials they are mainly for Visual Studio (dependency on the MS Runtime). Try and find one for MingW  Add to that that I want the same names for 32 and 64 bit and voila! Configure nightmare. I refuse to use a thunking DLL for a library that size that changes the ABI on a whim. But finally I got a VM set up to build it (after finding out far more than I wanted to about Perl). I'm not touching that VM ever again. It will remain a whole VM just to build one library on one OS.

3 hours ago, Rolf Kalbermatter said:

but it definitely shows that IPv6 support by internet providers should be something they care about.

Those that do are being really stingy with them too. Considering the exponential increase in address space you would have thought if you rented/bought a VPS they would give you more than one IPv6 out-of-the-box. Not a chance! 

 

[smithd]

libressl seems to have a better focus on stability, plus their api is much better.

https://en.wikipedia.org/wiki/LibreSSL

When I wrote my little tls library i wanted to avoid the dll issue so what I did was use the callback variant of the api here and here and just used the built-in labview tcp api. The callbacks are run when the library needs more data to enter or leave, so I just made the callbacks write to a queue which I then pull from to feed the labview primitives. Its a much much much nicer api than all that openssl stuff. Openssl docs make my soul cry.

[ShaunR]

5 hours ago, smithd said:

libressl seems to have a better focus on stability, plus their api is much better.

https://en.wikipedia.org/wiki/LibreSSL

When I wrote my little tls library i wanted to avoid the dll issue so what I did was use the callback variant of the api here and here and just used the built-in labview tcp api. The callbacks are run when the library needs more data to enter or leave, so I just made the callbacks write to a queue which I then pull from to feed the labview primitives. Its a much much much nicer api than all that openssl stuff. Openssl docs make my soul cry.

Although it may be easier from the User end, it's still fundamentally a port of OpenSSL but without FIPS support. LibreSSL doesn't support TLS1.3, currently, and according to their Git it's sitting at OpenSSL 1.0.1 so it will be a while before it has TLS1.3.

[smithd]

On 1/18/2020 at 3:17 AM, ShaunR said:

Although it may be easier from the User end, it's still fundamentally a port of OpenSSL but without FIPS support. LibreSSL doesn't support TLS1.3, currently, and according to their Git it's sitting at OpenSSL 1.0.1 so it will be a while before it has TLS1.3.

Some would say avoiding FIPS is a feature

And yeah, it will be a while for 1.3 -- they refused to do anything until it was actually standardized and now they are working on it. Seems like their attitude is "1.2 isn't broken yet" which makes sense. Their focus from the start was to fork openssl and clean it all up, which they seem to be making good progress on. I wouldn't have expected any correlation to the openssl version numbers at this point.

Edited January 21, 2020 by smithd

[ShaunR]

12 hours ago, smithd said:

Some would say avoiding FIPS is a feature

And yeah, it will be a while for 1.3 -- they refused to do anything until it was actually standardized and now they are working on it. Seems like their attitude is "1.2 isn't broken yet" which makes sense. Their focus from the start was to fork openssl and clean it all up, which they seem to be making good progress on. I wouldn't have expected any correlation to the openssl version numbers at this point.

That [attitude] is somewhat problematic for me in that the toolkit isn't just used for connecting safely to servers but often used in laboratories for testing device vulnerabilities and compliance. Limiting the protocols and features because "it's not good practice" or "we don't think XYZ" isn't a good fit for me and this extends outside of just TLS (for example hashes) as the toolkit is also capable of generating certificates, signing/verification and the low-level stuff like raw Diffie Hellman calculations. Of course, I'm limited by the OpenSSL implementation but limiting further based on precieved wisdom isn't what I need.

Edited January 21, 2020 by ShaunR

[smithd]

Eh, I'm totally fine with security experts making it so I can't shoot myself in the foot. My main goal for my secure networking layer is to give me the most reasonably secure connection to another device without me having to know. They do have some protocols available but disabled by default (I think this includes SSL3).

FIPS, as I understand it, is less about the algorithms and more about certification of a specific version of an implementation of the algorithm, which is then ossified and never changes even for security fixes. So to their point, its about checking a box, not providing security.

Edited January 22, 2020 by smithd

[ShaunR]

15 hours ago, smithd said:

Eh, I'm totally fine with security experts making it so I can't shoot myself in the foot. My main goal for my secure networking layer is to give me the most reasonably secure connection to another device without me having to know.

Sure. But you're not my target audience