Running zmq package on Linux Ubuntu20 (c library build)

[Łukasz]

Hello,

I'm tring to make zmq package (https://sourceforge.net/projects/labview-zmq/) work on Linux Ubuntu 20.04. 

VIs api in this package relates on shared library creating in c and are builded for Windows but not for Linux. However we can find sources as well as a few tips how to build this library. 

$ ls
bonzai.c  debug.c  makefile         rename_calls.vi  win32  zmq-errors.txt
bonzai.h  debug.h  parse_errnos.py  rename_lib2.vi   win64  zmq_labview.c

where zmq_labview.c is our main source file which could be build with makefile.

As a source of knowledge I used Brandon Jones post on forum about how to build this package for NI Linux RT: https://sourceforge.net/p/labview-zmq/discussion/general/thread/d5d0b49987/  However, after many hours spend I sill getting error 13 when I tring to call this library from LabVIEW. Following steps show what I have tried, however in every step I could make a mistake as I'm not regular C programmer. 

___________________________________________________________________________________

The first step is to check if two librarys which are needed by zmq_labview.c are avaiable in system: 

libzmq.so 

$ ldconfig -p | grep libzmq.so
	libzmq.so.5 (libc6,x86-64) => /lib/x86_64-linux-gnu/libzmq.so.5
	libzmq.so (libc6,x86-64) => /lib/x86_64-linux-gnu/libzmq.so

liblvrt.so (I dont undestant why its needed as its not included in zmq_labview.c but its used in makefile)

$ ldconfig -p | grep liblvrt.so
	liblvrt.so.20.0 (libc6,x86-64) => /usr/local/lib64/liblvrt.so.20.0
	liblvrt.so.15.0 (libc6) => /usr/local/lib/liblvrt.so.15.0
	liblvrt.so (libc6,x86-64) => /usr/local/lib64/liblvrt.so

NOTE: this output is from system where I didn't modyfied /etc/ld.so.conf.d. I have tired to add new file with paths to liblvrt.so as Brandon Jones describes, but I didn't recognized any difference. 

Next step is to modify source, so I set: #define LVALIGNMENT 8 and pthread_t thread;

And then makefile modyfications are needed. I have simply removed all windows realted stuff and thats a result:

# LVZMQ GNU-MAKE BUILD SCRIPT
# for both Windows and Linux targets

SRC = zmq_labview.c

# LINUX BUILD RULES

# omg so much simpler!
LABVIEW = /usr/local/natinst/LabVIEW-2020-64
LDFLAGS = -L $(LABVIEW) -L /lib/x86_64-linux-gnu/
CFLAGS =  -Wall -O3 -fpic -I $(LABVIEW)/cintools -I $(LABVIEW)
LDLIBS = -lzmq -llvrt
ifdef DEBUG
	CFLAGS += -DDEBUG
endif
	
all : lvzmq64.so
# copy the product to the labview directory
	@cp $< $(LABVIEW)/vi.lib/addons/zeromq


# Linux shared-library target
lvzmq64.so : $(SRC)
	$(CC) -shared -o $@ $^ $(CFLAGS) $(LDFLAGS) $(LDLIBS)

running make gives this results:

In file included from bonzai.h:29,
                 from zmq_labview.c:12:
bonzai.c: In function ‘bonzai_grow_leaf’:
bonzai.c:74:2: warning: this ‘for’ clause does not guard... [-Wmisleading-indentation]
   74 |  for ( i = 0; i < tree->n; ++i );
      |  ^~~
bonzai.c:76:3: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘for’
   76 |   if (( t = tree->elem[i] ) && ( t->id == x ))
      |   ^~
zmq_labview.c: In function ‘lvzmq_close’:
zmq_labview.c:108:11: warning: variable ‘i’ set but not used [-Wunused-but-set-variable]
  108 |  int ret, i; void *sock;
      |           ^
zmq_labview.c: In function ‘lvzmq_ctx_create_reserve’:
zmq_labview.c:223:27: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
  223 |  *pinstdata = bonzai_init(( void* )ninits );
      |                           ^
zmq_labview.c: In function ‘lvzmq_send’:
zmq_labview.c:485:25: warning: implicit declaration of function ‘zmq_msg_set_group’; did you mean ‘zmq_msg_set’? [-Wimplicit-function-declaration]
  485 |  if ( group && *group ) zmq_msg_set_group( &msg, group );
      |                         ^~~~~~~~~~~~~~~~~
      |                         zmq_msg_set
zmq_labview.c: In function ‘lvzmq_receiver_thread’:
zmq_labview.c:605:11: warning: variable ‘err’ set but not used [-Wunused-but-set-variable]
  605 |  int ret, err;
      |           ^~~
zmq_labview.c: In function ‘lvzmq_join’:
zmq_labview.c:860:8: warning: implicit declaration of function ‘zmq_join’; did you mean ‘lvzmq_join’? [-Wimplicit-function-declaration]
  860 |  ret = zmq_join( s->sock, group );
      |        ^~~~~~~~
      |        lvzmq_join
zmq_labview.c: In function ‘lvzmq_leave’:
zmq_labview.c:868:8: warning: implicit declaration of function ‘zmq_leave’; did you mean ‘lvzmq_leave’? [-Wimplicit-function-declaration]
  868 |  ret = zmq_leave( s->sock, group );
      |        ^~~~~~~~~
      |        lvzmq_leave

so there are several warrnings, but the library is build and now I can test VIs which will use lvzmq64.so but first library call with finish with error 13.

At this point, I tried to figure out how the Call Library Function is working and how to reproduce its task with c code, so I will get more info about the problem. The library is buit without .h file so in case of c call we need to use extern:

#include <stdio.h>
extern int lvzmq_ctx_create( int, int); //first function to call from labview to create context of zmq connection

int main(){
    printf("Test start\n");
    int a;
    int b;
    lvzmq_ctx_create( a, b);
    return 0;
}

gcc compilation of this test gives an output:

gcc test.c -L/home/user/Desktop/Lukas-Linux-folders/shared2/ -llvzmq64
/usr/bin/ld: /home/user/Desktop/Lukas-Linux-folders/shared2//liblvzmq64.so: undefined reference to `zmq_leave'
/usr/bin/ld: /home/user/Desktop/Lukas-Linux-folders/shared2//liblvzmq64.so: undefined reference to `zmq_join'
/usr/bin/ld: /home/user/Desktop/Lukas-Linux-folders/shared2//liblvzmq64.so: undefined reference to `zmq_msg_set_group'
collect2: error: ld returned 1 exit status

ok, so I got this error. It looks similar to warrnings reported by compiler when I build shared library. Now I stuck, because how it possible that those function calls are wrong, as its working on windows? Maybe the zmq package on Ubuntu doesn't contain those functions? Or I made a mistake when I build library. Any hint what to check next will be appreciated.

 

[Rolf Kalbermatter]

Your missing a few general differences between Windows PE files and Linux ELF format.

That DLL you build is just a thin wrapper around libzmq.dll/so to help interface to it from LabVIEW. The LabVIEW Call Library Node is not a generic Interop replacement and has certain limitations that are mostly due to the fact that NI did not want to create a Call Library Node configuration for which a user first has to study rocket science, physics and philosophy together with math and astrophysics in order to be able to operate it. So they chose certain limitations in what you can configure for the various parameters.

Some lower level C APIs like to expect certain things to be done with its parameters that are hard or even impossible to achieve with the possibilities of the Call Library Node alone. It can be often worked around by creating very convoluted code in the LabVIEW diagram to prepare the correct memory buffers with pointers at the right place etc. etc, but in order to be able to create such code you need to understand exactly how the C API works and has to be programmed and then even some more, such as how a C compiler aligns these elements in memory. People who have that knowledge will very quickly decide to write the according code in C(++) and let the C compiler fiddle with the details about memory layout etc, and then create around that a functional interface that limits itself to use LabVIEW friendly parameters so that the Call Library Node configuration gets straightforward and simple. This requires to write an extra shared library in C(++) and compile it with some compiler but if you don't have that knowledge anyways you are certainly doomed when trying to create the according C compiler like voodoo code on the LabVIEW diagram. And would be similarly doomed to know how to properly configure the parameter in the Call Library Configuration even if it allowed for such involved operations directly.

Now on Windows when you incorporate some shared library (DLL) in your own DLL you usually link a so called import library with your project. This import library contains functional stubs that locate the specific DLL where the function is implemented and link to the according function. If the dependent DLL can be found on the target system when trying to load your own DLL, all is well, otherwise Windows will fail the load of your DLL (and LabVIEW tells you that Windows couldn't find the DLL or one of its dependencies, with most users overlooking the mentioning of "dependencies" and getting upset since the DLL is obviously in the location where they told LabVIEW it should be).

Linux elf libraries are slightly different, there is no import library. The C linker will simply put all symbols it couldn't find in the source code for the current shared library into an external reference list. The elf loader when asked to load that shared library will go through this external reference list and try to resolve the symbols with any known shared library in its ld cache.

And here you seem not to have installed libzmq on your system. But it is referenced by this custom wrapper shared library, not incorporated into it, so if the Linux elf loader can't find a shared library that exports these symbols it posts above message. 

Edited August 19, 2022 by Rolf Kalbermatter

[Łukasz]

Thank you for your response. I see this topic will be more and more interesting. 

So in relation to that Call Function Node doesn't find the library you suggest that libzmq cannot be found by ELF file (The question is: which one when I run vi in LabVIEW ide? just ./labview ?). But ok, not best but "a test" will be to check if the default loader can find dynamically load library (libzmq) which is used by the wrapper (lvzmq64.so) 

/usr/local/natinst/LabVIEW-2020-64/vi.lib/addons/zeromq$ ldd lvzmq64.so 
	linux-vdso.so.1 (0x00007fffac9f1000)
	libzmq.so.5 => /lib/x86_64-linux-gnu/libzmq.so.5 (0x00007f3034e3e000)
	liblvrt.so.20.0 => /usr/local/lib64/liblvrt.so.20.0 (0x00007f303315a000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3032f68000)
	...

so we are sure that it could be found (we know that it could be found in ld-cache because we checked it with ldconfig in previus post). However, that doesn't mean that EFL file can find it, because NODEFLIB flag could prevent that. 

*I have checked if ./labview has set this flag, but no. No response - it's off. But as I wrote before I am not sure if this is a real caller of wrapper library.

$ readelf -d ./labview | grep NODEFLIB

Another option where EFL file will look for the library is LD_LIBRARY_PATH and RPATH / RUNPATH. Changing LD_LIBRARY_PATH is not good practice but I have to checked it:

export LD_LIBRARY_PATH=/lib/x86_64-linux-gnu/

which looks silly, doesn't change anything but I feel better that I've checked all ideas. 

RPATH and RUNPATH are set on compilation porcess, so I cannot change them easly. 

At this point, I cant see any problem that libzmq cannot be find as a resource. I think the issue is in wrapper itselfs, what is shown by this simple c app test. 

 

Edited August 20, 2022 by Łukasz

[JKSH]

On 8/19/2022 at 6:22 PM, Łukasz said:

zmq_labview.c:868:8: warning: implicit declaration of function ‘zmq_leave’; did you mean ‘lvzmq_leave’? [-Wimplicit-function-declaration]
  868 |  ret = zmq_leave( s->sock, group );

These warnings look weird to me. The problematic functions are all declared in zmq.h which is included by zmq_labview.c, so you should have no "implicit declarations". I haven't used this library before, but I suspect something went wrong during compilation of lvzmq64.so.

 

3 hours ago, Łukasz said:

RPATH and RUNPATH are set on compilation porcess, so I cannot change them easly. 

You can check it with chrpath --list. If it exists, you can change it with chrpath --replace.

[Rolf Kalbermatter]

38 minutes ago, JKSH said:

These warnings look weird to me. The problematic functions are all declared in zmq.h which is included by zmq_labview.c, so you should have no "implicit declarations". I haven't used this library before, but I suspect something went wrong during compilation of lvzmq64.so.

Those implicit warnings could be actually due to version conflicts. Those libxxxxxx libraries are open source and tend to be developed over time. It is not very common that functions just disappear when a new version is released, usually there are at least macros to declare a mapping to a new function name, so that backwards compatibility can be offered. So if you get these errors and it is not because you simply have buggy headers, the most likely cause is that your source code was developed against a newer version of the library.

Under Linux it is common that you install a library and its headers at the same time. Sometimes a package is divided into the library only part which contains all binary components and sometimes documentation, a development part which contains the headers and source code as well as tests for the library. Sometimes the headers are also installed by the binary package to allow developing depending code.

So likely the problem is not that you do not have libzmq installed at all but instead you have an older version installed than what was used to develop the LabVIEW bindings. That means that zmq.h is present on your system and can be correctly included by the source code for the wrapper but it doesn't declare certain functions that are used by the LabVIEW wrapper and the according binary library doesn't contain those functions either of course.

You have two solutions:

1) try to install a more up to date libzmq for your system. This simply solves all your problems but may be not always trivial. NI does not go and rebuild libraries for older NI Linux RT systems and unfortunately the nature of Linux makes it necessary to build the libraries and tools with the kernel headers that match the kernel version used for a particular Linux system. Since the NI Linux RT version is specifically tied to the used LabVIEW version, you can't just go and download a different NI Linux RT version for your target without also updating your LabVIEW version. And you can't simply take a shared library and header from a newer NI Linux RT system and copy it onto your system as that might simply move the symbol mixup one dependency level higher but not resolve it. So if you want to go this path without moving the a newer LabVIEW version you may actually have to download the sources for libzmq, copy them to your cRIO and compile them yourselves into the correct shared library and then install it together with the headers. With the typical make toolchain used for many libraries that is basically a move into the source directory for a library with "cd ........", then typing "./configure", "make", and "make install" on the command line, but in true Linux manner there are many other make toolchains a library developer can choose from (or none at all and just leave everything to the end user) and you will have to read the documentation for your library to see how you have to compile it from source. Of course you also have to make sure that the GNU C compiler tools are installed on your system. This is done on the command line by typing "opkg update", "opkg install packagegroup-core-buildessential" and of course your device needs to have a valid internet connection to be able to contact the NI package feeds.

2) Modify the LabVIEW shared library C wrapper code to make use of different library functions that your installed library provides. This might be straightforward or not, depending on the features used and functionality involved. It certainly requires you to look into the actual library functions, how they are meant to work and be used and get your hands dirty with real programming work.

Edited August 20, 2022 by Rolf Kalbermatter

[ensegre]

It's a complete shot in the dark, but have you tried

LD_PRELOAD=/lib/x86_64-linux-gnu/libzmq.so ./labview

?

[Dataflow_G]

The problem looks to be zmq_labview.c is calling functions in the zeromq library which are specified as draft functions, and are disabled in stable release versions. These functions are listed in the compilation output as implicitly declared, and when configuring the CLFN in LabVIEW, it throws a warning it can't find them. This is likely the cause of error 13.

If you check zmq.h, it has the following comment:

/******************************************************************************/
/*  These functions are DRAFT and disabled in stable releases, and subject to */
/*  change at ANY time until declared stable.                                 */
/******************************************************************************/

#ifdef ZMQ_BUILD_DRAFT_API
...
/*  DRAFT Socket methods.                                                     */
ZMQ_EXPORT int zmq_join (void *s, const char *group);
ZMQ_EXPORT int zmq_leave (void *s, const char *group);
...
#endif

If -D ZMQ_BUILD_DRAFT_API is added to the compile flags, the implicit declaration warnings are no longer present. But you still have the problem that the stable release of zeromq doesn't have these functions available. So you'll either need to find and install a draft release (these don't appear to be pre-built for Ubuntu), compile it yourself with draft APIs enabled (possibly using -DENABLE_DRAFTS=ON), or replace the functions from the draft spec in zmq_labview.c.

[Rolf Kalbermatter]

On 8/22/2022 at 5:13 AM, Dataflow_G said:

The problem looks to be zmq_labview.c is calling functions in the zeromq library which are specified as draft functions, and are disabled in stable release versions. These functions are listed in the compilation output as implicitly declared, and when configuring the CLFN in LabVIEW, it throws a warning it can't find them. This is likely the cause of error 13.

If you check zmq.h, it has the following comment:

/******************************************************************************/
/*  These functions are DRAFT and disabled in stable releases, and subject to */
/*  change at ANY time until declared stable.                                 */
/******************************************************************************/

#ifdef ZMQ_BUILD_DRAFT_API
...
/*  DRAFT Socket methods.                                                     */
ZMQ_EXPORT int zmq_join (void *s, const char *group);
ZMQ_EXPORT int zmq_leave (void *s, const char *group);
...
#endif

If -D ZMQ_BUILD_DRAFT_API is added to the compile flags, the implicit declaration warnings are no longer present. But you still have the problem that the stable release of zeromq doesn't have these functions available. So you'll either need to find and install a draft release (these don't appear to be pre-built for Ubuntu), compile it yourself with draft APIs enabled (possibly using -DENABLE_DRAFTS=ON), or replace the functions from the draft spec in zmq_labview.c.

That's the other possibility. The LabVIEW wrapper is basically using futures (not like Java futures, which are more like an object callback, but more like Python __future__ which are features that are expected to be introduced with the next version as a stable function or interface or were discontinued).

As to getting pre built binaries from a distribution like Ubuntu for such draft builts, that is very unlikely unless someone at Ubuntu had decided that it is a totally and completely unmissable feature for their platform, which I doubt the zmq draft items every could amount to.

Edited August 23, 2022 by Rolf Kalbermatter

[Łukasz]

So I removed the defualt version of libzmq3-dev package from Ubuntu and installed manualy the version with draft api enabled (for Ubuntu 20.4)

echo 'deb http://download.opensuse.org/repositories/network:/messaging:/zeromq:/release-draft/xUbuntu_20.04/ /' | sudo tee /etc/apt/sources.list.d/network:messaging:zeromq:release-draft.list
curl -fsSL https://download.opensuse.org/repositories/network:messaging:zeromq:release-draft/xUbuntu_20.04/Release.key | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/network_messaging_zeromq_release-draft.gpg > /dev/null
sudo apt update
sudo apt install libzmq3-dev

That fixes the problem and the basic functions ZMQ LabVIEW library works. 

Edited September 9, 2022 by Łukasz

[Dataflow_G]

2 hours ago, Łukasz said:

That fixes the problem and the basic functions ZMQ LabVIEW library works.

Thanks for confirming its working 👍

[TLH]

Hey everyone, I'm trying to do a similar build/install on a Linux RT device (cRIO-9045), and am also getting error 13, I think it could be for similar reasons as this thread.  Since Linux RT isn't Ubuntu based, any idea on how to get libzmq with draft APIs downloaded on there?