zeromq - simple fast lightweight communication - does it fill a gap in NIs offering?

[Bob Edwards]

I've tried datasocket, too limiting. Tcpip - there's got be a higher level than that. Shared network variables - daunting in complexity and still proprietary to NI. I found 0mq or zeromq the other day, which ticks a lot of boxes. I've written an Excel based test executive but wanted a robust and simple interface to control distributed slave instrument apps. It prototyped ok in datasocket if it wasn't for the fact that controls all appear as integers whatever their real type before they are written to. This makes auto-discovery of resources a pain.

I don't see any discussion of zeromq on lavag, has anyone got experience of using the labview version - http://sourceforge.net/p/labview-zmq/wiki/Home/ ? It would be comforting to know it had proved to be reliable and scaled ok before spending any more time on it. I'm encouraged by some prototypes i've written for fault tolerance.

Would welcome any experience with 0mq and other suggestions for simple networking without going completely back to basics.

Cheers bob, uk

[JackDunaway]

ØMQ is on my radar, and a few other's (we can talk about it offline, but I'm not ready to endorse it publicly). I can say, I'm eyeballing ØMQ as a particularly intriguing solution for actor-oriented design (and generally, distributed messaging) both in LV and between LV and the rest of the world.

[Bob Edwards]

For anyone who hasn't come across omq before, this book was very useful in getting started. http://zguide.zeromq.org/page:all

Yes, between LV and the rest of the world is particularly useful as the safety critical products I'm working around are mainly programmed in C. The number of language bindings for 0mq is impressive. Not something that NI seems all that interested in, which is shame (for us that is)

[ShaunR]

For anyone who hasn't come across omq before, this book was very useful in getting started. http://zguide.zeromq.org/page:all

Yes, between LV and the rest of the world is particularly useful as the safety critical products I'm working around are mainly programmed in C. The number of language bindings for 0mq is impressive. Not something that NI seems all that interested in, which is shame (for us that is)

It's interesting that they use a length header which is the same as the transport.lvlib so they could be compatible (or at least made to be compatible-transport.lvlib has a compression and encryption byte).

I don't think it really brings that much to the table over native labview implementations. From my cursory look. Pub/sub seems to be it's main "killer" property and there are native labview implementations which are safer and x-platform, This is probably the reason for NIs lack of enthusiasm. I'm not being dismissive, I think I just need to spend more time looking at it and maybe something will jump out as being an advantage. for my use cases - there has to be a really, really good reason to rely on 3rd party compiled binaries.

I think the library could be improved by allowing binding to the labVIEW event system (a big bugbear of mine with VISA).If an intermediate binary is used, (as it is in this case) then adding events so that developers can hook into the event structure is trivial (even more so than in native labVIEW). I find event driven asynchronous comms, vastly superior to the normal methods (

)..

I had a few issues with the library but I don't think they are insurmountable. I kept getting an error (too many sockets) and pressing abort in LV x64 causes it to hang (on the Multi example, without a stop button). But the person who wrote the wrapper seems eu fait with binding to labview, so I expect the latter will get worked out.

Edited November 17, 2013 by ShaunR

[Bob Edwards]

I hadn't tried transport.lvlib. Will download and see how it compares with 0mq for this application - thanks for the prompt. I want my controller app to be able to detect slave apps by their inputs and outputs a la datasocket, but probably not pubsub. I was going to fit my test exec app with a 0mq 'router' port and the satellite hardware driver apps fitted with a 'request' port. At start up, each driver app would send a 'request' that contained a description of I/O ports to the test exec. The test exec would compile these into a dictionary of possible connections - like the datasocket connection popup. The 'router'/'request' combination allows each driver app to set an 'identity' for itself. The user can then connect to 'frequency in' port of 'siggen 1' and so on from a selection tree. I can see how the conversation would work and that it could recover from a failure of either slaves or exec ok.More labview 0mq examples are needed to illustrate proper recovery from failure and also graceful shutdown. I agree about having it event based.

[ShaunR]

I hadn't tried transport.lvlib. Will download and see how it compares with 0mq for this application - thanks for the prompt. I want my controller app to be able to detect slave apps by their inputs and outputs a la datasocket, but probably not pubsub. I was going to fit my test exec app with a 0mq 'router' port and the satellite hardware driver apps fitted with a 'request' port. At start up, each driver app would send a 'request' that contained a description of I/O ports to the test exec. The test exec would compile these into a dictionary of possible connections - like the datasocket connection popup. The 'router'/'request' combination allows each driver app to set an 'identity' for itself. The user can then connect to 'frequency in' port of 'siggen 1' and so on from a selection tree. I can see how the conversation would work and that it could recover from a failure of either slaves or exec ok. More labview 0mq examples are needed to illustrate proper recovery from failure and also graceful shutdown. I agree about having it event based.

 

Transport.lvlib is just a protocol leveler (it just means that you can send data via UDP, TCP, Bluetooth etc transparently). Dispatcher, however, is a Pub/Sub implementation that uses Transport.lvlib and is probably closer to what you are looking for.

 

Woohoo. Links are working again!

Edited November 18, 2013 by ShaunR

[Yair]

I find event driven asynchronous comms, vastly superior to the normal methods (

)..

 

My keen senses are detecting class wires in that example. Shall I call a doctor?

[drjdpowell]

As an aside, there is also LabbitMQ, a LabVIEW interface to RabbitMQ, to consider.  

[ShaunR]

My keen senses are detecting class wires in that example. Shall I call a doctor?

 

Ssshhhhh. Don't look closely at the MDI toolkit either

[Bob Edwards]

The MDI toolkit's ok for local but not distributed. LabbitMQ looks interesting - RabbitMQ has been quite popular. I found nanomsg http://nanomsg.org/ (son of 0mq) but still only alpha so not interesting yet.

 

I wonder if a truly dominant messaging standard will appear in this age of distributed 'things'? We still don't agree which end the MS digit goes half the time. The other issue is 'more complicated' is 'better'. At least I can take 0mq in quickly because the api is tiny.

[aart-jan]

I need to write an application that handles data from an IBM Websphere MQ server. Although apparently there are several client applications available for Websphere MQ, I have no idea yet how I can get the data out of the client and into Labview. OMQ seems a client/server library in itself, not as an api for any messaging middleware based on a certain standard such as AMQP, MQTT, STOMP etc.

[ShaunR]

The MDI toolkit's ok for local but not distributed.

 

I think you misunderstood. I didn't proffer it as a solution. It was a retort to Yairs dig at me and classes (a kind of in-joke). You should instead take a look at the Dispatcher (that I highlighted in a previous post) which is very similar to your requirement, but native labview.

[drjdpowell]

ØMQ is on my radar, and a few other's (we can talk about it offline, but I'm not ready to endorse it publicly). I can say, I'm eyeballing ØMQ as a particularly intriguing solution for actor-oriented design (and generally, distributed messaging) both in LV and between LV and the rest of the world.

I looked into zeroMQ with the idea of using for actors, but I couldn’t really identify how it would work for me better than just using TCP connections.  ZeroMQ has many communication patterns, but none really matched up to what I need in an “Actor’s address”.

[Bob Edwards]

I'll take another look at Dispatcher, Shaun, I had a brief play with it a while back. Like Jack says, it all depends which is the 'closest fit' to any one requirement. Keeping things native labview would be a bonus.

 

One thing the 0MQ author says might be true Jack - if 0MQ doesn't seem to fit your needs, you may be looking at the requirement from the wrong angle. All the various port type Pub Sub Request Reply Router Dealer and so on can be mixed and matched. There may be an unconvenstional combo that does fit - just a thought.

 

I thought you'd like to hear a little about my application which is a deliberately very simple test exec with all the horsepower that comes with spreadsheets. As such it's relatively slow but pretty versatile: -

 

1. One or more excel spreadsheets contain data, program logic or analysis logic

2. 'Spreadpump' is a test executive that just moves ranges of excel data around sheets and between sheets and instrument controls and indicators

3. Zero or more instrument control applications that can slave to Spreadpump. The user can name these, so that more than one instance of an instrument driver can used

4. Spreadpump is programmed with around 20 instructions using a table based editor with drop-down user selection where possible so the learning curve is tiny. Includes conditional goto and gosub based on whether a range is all blank or not.

5. 'In' and 'Out' instructions move desired ranges to and from the instrument control apps. A control or indicator is thus defined as <user appname>/<control name>. A range can be one value, 1D or 2D array of one type of data - string, integer, float, boolean.

6. Spreadpump optionally waits for spreadsheet recalc before reading from a sheet. The user can control display refresh to improve speed or hide intermediate states.

 

So each of my instrument apps needs to identify itself as healthy. If I use a 'multipart 0MQ message', the 'identity' feature of 0MQ can be set with the unique app name as a string and id's the sender to the test exec. The 'first message' from any instrument driver  would be to tell the test exec the names, control refs, data types and direction for all controls and indicators requiring remote control. Each instrument control app would be fitted with a 'request' port to enable that.

 

The test exec needs to talk with 0-N instrument control apps. The 0MQ 'router' port appears to fill that bill. In any multipart message received, the first part would be the user name for the app, than an empty part, then the data required. If reading from a control, Spreadpump would send a message to the relevent app with the control ref. as data. The remote app would return the data as a variant. Writing to an Indicator, Spreadpump would send to the relevent app the control ref and data as variant. The remote app would acknowledge with maybe just the control ref. The router takes care that the reply returns to the requester.

 

To make sure all apps are still alive, if the remote instrument controllers get sent a 'heartbeat' message, they return the 'first message' again (say) with a timeout in Spreadpump to spot dead links.

 

I haven't completely worked out what happens when things fail, but it's not going to be sophisticated - just wait and retry until the user says stop.

 

It's been working well using datasocket reading indicators from some pretend instrument drivers, but as I say datasocket is quirky in that when an instrument controller published it's inputs, they all appear is integers. That's a bit naff, since Spreadpump can't easily discover what data type to deliver to an input.

[ShaunR]

It's been working well using datasocket reading indicators from some pretend instrument drivers, but as I say datasocket is quirky in that when an instrument controller published it's inputs, they all appear is integers. That's a bit naff, since Spreadpump can't easily discover what data type to deliver to an input.

Hang on. I missed this in your first post. Are you sending control refs? Control refs are integers and have no meaning outside of the application instance (they are basically pointers, but not quite). Have you tried,using the Variant To Flattened String. That will give you the type info that you can send and examine the other side. No messaging system can help you with that. It is a serialization issue.

Edited November 19, 2013 by ShaunR

[Bob Edwards]

Yes you're right, passing around ctrl refs is probably more brittle than using the control names. In the proposed 0MQ solution, when an instrument controller starts, it sends a registration message to the test exec defining the appname and also for each remoteable control: controlname / datatype / direction. So a number of such instrument drivers start and the test exec accumulates a list of this information.

 

The user edits a test script and can select the required control or indicator for data transfer from a tree style menu derived from the list. (He can also type in <appname/controlname>)

 

When the script runs, the 'output' instruction takes data from a defined spreadsheet range and transfers it to <appname> <controlname>

 

The remote apps could pass the ctrl refs to the test exec to store in the same list and the test exec could use those to define where the data is going, but this is just more complex. The control names alone are enough - it's just a string versus an integer.

[Bob Edwards]

The datasocket prototype allowed the user to select using the 'DS select' menu. When test exec script ran, all datasockets were opened and data transfer was by DS Read and Write. Whne the script stopped all sockets were closed. Very simple, if it weren't for the fact inputs don't show up correctly. You try it, put a string input in a while loop. Make sure the input is bound to datasocket. Start datasocket server and run the VI.  Open the datasocket server diagnostics. Check the input you've just created. It's shown as an integer not a string. So if another app wishes to discover what datatype that input is, it can't tell. Bah!

[ShaunR]

The datasocket prototype allowed the user to select using the 'DS select' menu. When test exec script ran, all datasockets were opened and data transfer was by DS Read and Write. Whne the script stopped all sockets were closed. Very simple, if it weren't for the fact inputs don't show up correctly. You try it, put a string input in a while loop. Make sure the input is bound to datasocket. Start datasocket server and run the VI.  Open the datasocket server diagnostics. Check the input you've just created. It's shown as an integer not a string. So if another app wishes to discover what datatype that input is, it can't tell. Bah!

Nope. Shows up with a big letter A for me denoting a string.

[Bob Edwards]

Shaun, I still think datasocket is 'broken' on this obscure point. It's possible you configured your input string to be a publisher not a subscriber. Labview allows you to do this.

 

To see the effect for sure: -

 

1. Run datasocket server and open the diagnostics window

2. Search labview examples for datasocket and open 'Front Panel Datasocket read.vi' and 'Front Panel Datasocket Write.vi'.

3. Run 'Front Panel Datasocket read' and check the 'fpwave' variable that appears - on my Win XP machine running Labview 2010 that appears as a blue '2' for integer

4. Run 'Front Panel Datasocket Write' and the new variable changes to the correct type

 

Thats my problem I think - datasocket subscribers don't show as the proper data type until someone publishes data to the variable. This probably has to be - If you create a datasocket subscriber using 'datasocket open' rather than binding a control, that subscriber is capable of accepting any kind of data.

 

I've tried a lot of dodges with datasocket properties, writing data once from inside the instrument control apps etc. but so far they all result in remote inputs being unusable or datasocket misbehaviour.

 

As NI hint that datasocket is non-preferred, I thought it would be better to move on to a comms standard with some life left in it. I'd been looking to adopt something that was cross platform, a lot of language bindings and an active developer group. As has been pointed out, though, adopting third pary carries risks, hence the debate.

[ShaunR]

Shaun, I still think datasocket is 'broken' on this obscure point. It's possible you configured your input string to be a publisher not a subscriber. Labview allows you to do this.

 

To see the effect for sure: -

 

1. Run datasocket server and open the diagnostics window

2. Search labview examples for datasocket and open 'Front Panel Datasocket read.vi' and 'Front Panel Datasocket Write.vi'.

3. Run 'Front Panel Datasocket read' and check the 'fpwave' variable that appears - on my Win XP machine running Labview 2010 that appears as a blue '2' for integer

4. Run 'Front Panel Datasocket Write' and the new variable changes to the correct type

 

Thats my problem I think - datasocket subscribers don't show as the proper data type until someone publishes data to the variable. This probably has to be - If you create a datasocket subscriber using 'datasocket open' rather than binding a control, that subscriber is capable of accepting any kind of data.

 

I've tried a lot of dodges with datasocket properties, writing data once from inside the instrument control apps etc. but so far they all result in remote inputs being unusable or datasocket misbehaviour.

 

As NI hint that datasocket is non-preferred, I thought it would be better to move on to a comms standard with some life left in it. I'd been looking to adopt something that was cross platform, a lot of language bindings and an active developer group. As has been pointed out, though, adopting third pary carries risks, hence the debate.

 

Well. If datasocket isn't quite doing what you need. Perhaps try network streams? They have a property node whereby you can retrieve the datatype of the endpoint. I've never used them, as I have Dispatcher, but from what I can tell they can operate in a number of configurations (one to one, one to many etc).

Edited November 21, 2013 by ShaunR

[Bob Edwards]

Network streams and your dispatcher look good - nice simple apis. Network streams seems to be only 1:1 - isn't a problem. Slave instrument controllers could signal the test exec using UDP (say) for i/o discovery together with two network streams for the data. From the advice above there's two or three candidate comms solutions - I'll prototype and let you know how it turned out.

 

What's the best way of sharing code snippets here?

[GoGators]

 Not something that NI seems all that interested in, which is shame (for us that is)

Just came across this port.  Sys Eng at NI is interested in zeromq and rabbitmq.  We've been playing around with both and are considered making an open source library if it fits a need.  One of the SEs even built it for the cRIO-9068 platform for alliance day to talk about reuse on NI Linux RT.  Don't count us out yet

[M_P]

I'm particularly looking into this (I'm also known as MattP on the NI forums, I work in the SE group like GoGators). From my perspective, it fills a gap that was partially filled by the AMC library (using same API for inter-loop and inter-node communication), network streams (automatic reconnect in background abstracted from the read/write, good performance), and shared variables (the pub-sub use case).  

 

What it handles better, IMHO, is that it handles not just the reconnect, but also the initial connect and intermediate send/receive queues entirely asynchronously.  It also easily handles N:1 or 1:N using several different transports.  It's not so much the base set of patterns that are useful, but the combinations of them that perrmit complex problems to be solved with very small quantities of efficient code.  Me likey.

 

Currently I don't see support for ZMQ on VxWorks, which makes usage on cRIO rather tricky, and I know it hasn't been tested on Pharlap but 'appears to work.'

 

I'm contemplating either liberally borrowing the design philosophy of ZMQ for my own comms framework (which would not be easily usable with non-NI tools but would permit NI-specific transports such as FPGA DMA FIFOs), attempting to port and test the C source over to VxWorks, or possibly to attempt a full ZMQ implementation in native LV code.  

[Darin]

... or possibly to attempt a full ZMQ implementation in native LV code.  

 

Been there, done that.  I have G implementations of ZMTP (at least most bits I need) and AMQP (0.9.1 not the 1.0 trainwreck).  I tend to use AMQP most often since my code is more complete (and there was more adoption of it when I wrote it), but they are both great.  The brokerless nature of ZMQ can be great when you need it, turns out most times I was implementing a de facto broker and could have stuck to AMQP.

 

Unfortunately the specs are really laid out seductively in terms of classes and methods so I gummed up the works by using a bit too much LVOOP (not to mention busted my carpal tunnels).  Performance is ok, but as I de-LVOOP it is finding another gear.  I create worker pools consisting of simple  LV workers and similar C++, Java, or Python workers and see what share each worker does.  LV is behind, but getting up there with the big boys.  The LV9 to LV12 performance kick was also welcome here.

 

The AMQP implementation gave rise to one of my new mantras:  No LVOOP in a loop.

[M_P]

Been there, done that.  I have G implementations of ZMTP (at least most bits I need) and AMQP (0.9.1 not the 1.0 trainwreck).  I tend to use AMQP most often since my code is more complete (and there was more adoption of it when I wrote it), but they are both great.  The brokerless nature of ZMQ can be great when you need it, turns out most times I was implementing a de facto broker and could have stuck to AMQP.

 

Unfortunately the specs are really laid out seductively in terms of classes and methods so I gummed up the works by using a bit too much LVOOP (not to mention busted my carpal tunnels).  Performance is ok, but as I de-LVOOP it is finding another gear.  I create worker pools consisting of simple  LV workers and similar C++, Java, or Python workers and see what share each worker does.  LV is behind, but getting up there with the big boys.  The LV9 to LV12 performance kick was also welcome here.

 

The AMQP implementation gave rise to one of my new mantras:  No LVOOP in a loop.

 

I figured somebody had to have already done this - it's too enticing of a project not to.

 

Would you happen to have actual benchmark data or code that you'd be willing to share?