Lapdog over the network?

[Daklu]

There is no state... so it is much simpler. 

 

Yeah, stateless functions do make using it much easier.  I noticed how your TCP vi opens a connection every time a message needed to be sent instead of having an open and closed state.  What the overhead associated with that?  How frequently would you have to send messages before you would consider having a stateful TCP vi?

 

(Of course, the tradeoff of having stateless functions is that you have to pass *everything* as arguments, which in this case tends to pull the code towards onioning and structural dependencies.)

 

 

It helps, of course, if the "target" describes the modules' capabilities (like DVM, FILE, DB etc) and that would be a "convention" I practice since it allows conversations such as:

FILE->LOAD->c:tempcfg.cfg

DVM->SET->VOLT->3.9

DB->QUERY->SELECT * FROM test;

 

As long as the strings are relatively simple like that I could probably handle it.  There were a few things that left me scratching my head until I traced through the code several times:

1. UI.vi claims the message format is TARGET->CMD->PAYLOAD, but the Msg Send function actually converts it to SENDER->CMD->PAYLOAD.  Kind of a loose interpretation of the word "target," huh? 
   
2. For the longest time I assumed the Msg Send name terminal was for the target's name instead of the sender's name.  (Using the vi name to identify named queues for the message targets would never work for me; I rename stuff all the time during active development and that would wreck havoc on my code.)
   
3. Mostly due to number 2, I couldn't figure out how you were getting messages from the TCP listening loop to the TCP display loop.
   

 

 

@Daklu This is not really the right place to go query this, but I was just wondering why you didn't do the required casting for the various messages within each "Get XXX Message.vi"?

 

Steve Chandler asked me the same thing way back when I first released LapDog.  Let's use the StringMessage class as an example.  Currently the Get String.vi has input and output terminals for the StringMessage class on the conpane.  If I want to do the downcasting inside Get String.vi, the class input terminal has to be of the Message class.  However, doing that eliminates the ability to override Get String.vi in a StringMessage child class.

 

In my opinion, that's the kind of feature that is intended to help users (and may in fact help many of them) but ultimately ends up sacrificing power and limits the api's usefulness to expected use cases.  I'd fully support anybody who wanted to create a package of add-on vis that do that, but the base api needs remain flexible.

 

 

Remember, I need RemoteTCPMessenger to work in an extensive body of code that works with the generic “Send” and “Messenger” classes, thus I need to have it work through a predefined simple API (Create, Send, etc.), not some new TCP-specific API.  Large parts of the codebase must be blind to the fact that it is using TCP.  If the internals of RemoteTCPMessenger need to be slightly unclear to achieve this, that is no big problem, as the user of the messaging package doesn’t need to know this internal detail.

 

I'll try to figure out another way to express what I'm trying to say.  I don't think we're on the same page yet.

 

 

Could you expand further?

 

On which part?  100% visible/invisible?  Overriding Send?  Inverting the Message/Transport dependency?

[AlexA]

Steve Chandler asked me the same thing way back when I first released LapDog.  Let's use the StringMessage class as an example.  Currently the Get String.vi has input and output terminals for the StringMessage class on the conpane.  If I want to do the downcasting inside Get String.vi, the class input terminal has to be of the Message class.  However, doing that eliminates the ability to override Get String.vi in a StringMessage child class.

 

In my opinion, that's the kind of feature that is intended to help users (and may in fact help many of them) but ultimately ends up sacrificing power and limits the api's usefulness to expected use cases.  I'd fully support anybody who wanted to create a package of add-on vis that do that, but the base api needs remain flexible.

 

Hmmm, I don't really understand why it precludes over-riding "Get String.vi"? Wouldn't you just change the cast object in the child method (making "Get Payload.vi" a must over-ride method)?

As a secondary point, why would you want to create a child of 'StringMessage' rather than just creating your new message directly from 'Message' itself? Perhaps because you want to bundle string parsing into the get method? Even then, as above, couldn't you just do your parsing on the data from your child "Get String.vi" which casts to the child message object and parses the data as you would like?

 

Do you have an imagined use-case for inheriting from a native-type message?

[Daklu]

Hmmm, I don't really understand why it precludes over-riding "Get String.vi"? Wouldn't you just change the cast object in the child method (making "Get Payload.vi" a must over-ride method)?

 

If you open Get String.vi and replace the StringMessage input control with a Message control (which you must do if you want the downcast to be inside Get String,) you'll end up with an error:

 

LabVIEW Object 'Message in': Dynamic terminal must be of the same class as owner of VI.

 

Since Get String.vi is a member of the StringMessage class, if I want to allow people to override the method it has to have a StringMessage input terminal, not a Message input terminal.  Changing the terminal to a Message control breaks dynamic dispatching because the compiler has no way of guaranteeing the object going into the terminal at runtime is a StringMessage object or one of its children.

 

I could add a DowncastAndGet<Payload> to each of the message classes while preserving the ability to override Get<Payload>, but that adds complexity to the api and makes it less approachable for new users.  Furthermore, if I did that then DowncastAndGetxxx and Getxxx would have overlapping functionality, and I prefer to avoid that in my apis.

 

Given my own druthers and assuming I had the time and motivation, I would probably write a single vi that downcasts and calls Getxxx for each message type, put them in a separate library, and tie them all together with a polymorphic vi.  That way users could easily do 99% of their message payload retrieval using a single vi.  (See attached project for example and sample polymorphic vi.)

 

 

 

Do you have an imagined use-case for inheriting from a native-type message?

 

Sure.  Suppose I am serializing large amounts of data to send over a network.  I can, if I think it is necessary, create a CompressedString class or an EncryptedString class that is completely transparent to the actor that receives the message.  I have also, on occasion, created native message subclasses as unit test mocks so I can detect what is happening internally in a component I am interested in.

LapDog GetPayload Example.zip

[ShaunR]

Yeah, stateless functions do make using it much easier.  I noticed how your TCP vi opens a connection every time a message needed to be sent instead of having an open and closed state.  What the overhead associated with that?  How frequently would you have to send messages before you would consider having a stateful TCP vi?

 

(Of course, the tradeoff of having stateless functions is that you have to pass *everything* as arguments, which in this case tends to pull the code towards onioning and structural dependencies.)

 

Don't get me wrong. you can have stateful FUNCTIONS. I'm saying that the HANDLER should not be stateful. Sending  a message should execute a function every time rather than conditionally executing if a message was received earlier....... there be monsters Nothing to stop you having a class in a frame that maintains its state but it should be encapsulated within the scope of the function, not the handler or, even better, just don't send a message at all (it depends what state we are talking about- file open, read,close........bury it in the function. Move slide if position is greater than 3mm, thats a sequencers job and it shouldn't send "move").

 

The TCPIP code I placed was just a quick and dirty way of sending commands. From experience it is quite adequate for the local loop at medium rates (10s of ms) or for an internal  GB network of hundreds of ms, but forget it for internet or high speed comms. It is also useful if you are talking to many destinations with short packets infrequently (e.g. 50 cRIOs with  "RESTART"). It is up to the developer to define the functions and features, the handler just provides a messaging interface.

 

I think I said earlier that in the real world I use a comms handler and can send via a number of methods. That module actually maintains connections (whatever the interface) but I have made it function so that they die if not used for a couple of minutes so if they are used often they persist, but I don't have to explicitly close them. You can do whatever you like in the handlers, just KISS

 

As long as the strings are relatively simple like that I could probably handle it.  There were a few things that left me scratching my head until I traced through the code several times:

[*]UI.vi claims the message format is TARGET->CMD->PAYLOAD, but the Msg Send function actually converts it to SENDER->CMD->PAYLOAD.  Kind of a loose interpretation of the word "target," huh? 

[*]For the longest time I assumed the Msg Send name terminal was for the target's name instead of the sender's name.  (Using the vi name to identify named queues for the message targets would never work for me; I rename stuff all the time during active development and that would wreck havoc on my code.)

[*]Mostly due to number 2, I couldn't figure out how you were getting messages from the TCP listening loop to the TCP display loop

1. Actually. The generic symantics are TARGET->SENDER->CMD->PAYLOAD. So no. It's not loose. Target and sender are purely for routing. It could have been Destination IP address and Source IP address but for this implementation I chose VI names and relieved the burden on the user of typing in the sender on every message. Additionally, you don't have to use queues. It's just convenient that queues can be isolated by names so the target is consumed for that purpose. Queues also break the need for wires running all over the place, or shared storage (which is required for events). You could have used TCPIP primitives in the Send instead of Queues and it would have worked much the same way directly over a network (which is pretty much how dispatcher in the CR works). In this respect it is a messaging strategy realised with queues and handlers. But the messaging itself would be identical with TCPIP listeners and opens. This is what makes it easy to traverse boundaries both in hardware and in software languages since it could be a webserver written in python at the other end (not easy to do with labview coded messages).

 

2. Well. You can't beat discipline into programmers. Only the army can do that But you have perhaps missed the "reuse" aspect. Once you have a TCPIP handler, for example, then why do you need to rename it? It's like renaming Lapdog. Besides renaming is just a search and replace string on all VIs. Not as if you have to go and rename a shedload of VIs,and have to re-link all over the place is it?

 

3. I don't understand the question. There is no TCPIP display loop. There is a TCPIP handler which happens to be showing its panel. And there is an IMG handler. The listener just forwards the message using Send.

Edited June 18, 2013 by ShaunR

[AlexA]

Don't get me wrong. you can have stateful FUNCTIONS. I'm saying that the HANDLER should not be stateful. Sending  a message should execute a function every time rather than conditionally executing if a message was received earlier....... there be monsters Nothing to stop you having a class in a frame that maintains its state but it should be encapsulated within the scope of the function, not the handler or, even better, just don't send a message at all (it depends what state we are talking about- file open, read,close........bury it in the function. Move slide if position is greater than 3mm, thats a sequencers job and it shouldn't send "move").

 

@ShaunR Just to clarify, when you talk about having a stateful function, what mechanism are you proposing to maintain this state? Is the state encapsulated via something like an FGV? 

@Daklu: Ahh, thanks for helping me understand that decision, it makes sense when you put it like that.

[drjdpowell]

On which part?  100% visible/invisible?  Overriding Send?  Inverting the Message/Transport dependency?

How would you do it?  (Though you can explain what you mean be the Inverted dependency thing also.)

[ShaunR]

@ShaunR Just to clarify, when you talk about having a stateful function, what mechanism are you proposing to maintain this state? Is the state encapsulated via something like an FGV? 

 

Could be. Could also be a class, or action engine-depends on your level of abstraction. The goal is to have one VI per frame that is not dependent on the message order and is atomic. For a trivial example. You could have a JKI style statemachine set of messages for opening, reading and closing a file and have to pass the reference from frame-to-frame. I'm suggesting this is a bad idea and you should have a one read message and it invokes a READ vi which opens reads and closes the file.

[AlexA]

Ahh ok, I thought you were specifically advocating against having something like an enum (state) which is maintained in the process data cluster and is passed into a function (subVI) to change the way it performs.

Edited June 18, 2013 by AlexA

[Daklu]

Once you have a TCPIP handler, for example, then why do you need to rename it? It's like renaming Lapdog. Besides renaming is just a search and replace string on all VIs. Not as if you have to go and rename a shedload of VIs,and have to re-link all over the place is it?

 

Actually I was thinking about using vi names for target ID in combination with onion routing.  There would be too many long dependency chains for me to have confidence I'm making all the corrections that are necessary.

 

 

I don't understand the question. There is no TCPIP display loop. There is a TCPIP handler which happens to be showing its panel.

 

It wasn't a question, just comments on what I initially had some trouble figuring out.  I meant the TCP listening loop.

 

 

 

How would you do it?  (Though you can explain what you mean be the Inverted dependency thing also.)

 

I'll try to gin up some diagrams to show how I think about the problem.

 

Regarding the inverted dependency...

In LapDog.Messaging, the MessageQueue class depends on the Message class, but not the other way around.  Slide #3 of your presentation shows the Msg class having a dependency on the Send class.  Originally I though you were implementing MyMsg.Send methods similar to some of the early AF prototypes.  However, if I understand correctly that dependency exists only because you built the ability to reply into the your base Msg class.

[ShaunR]

Actually I was thinking about using vi names for target ID in combination with onion routing.  There would be too many long dependency chains for me to have confidence I'm making all the corrections that are necessary.

 

You only really need to onion route when you traverse a boundary (like forwarding across TCPIP) so message depth is usually limited to 2 (arguably it could be considered chaining at that point). Everything else you send directly. If you were to onion route (i.e. concatenate) to twenty levels, then I can see that may be an issue.I just don't see the point if I can send directly. What's your use case?

Edited June 19, 2013 by ShaunR

[drjdpowell]

Regarding the inverted dependency...

In LapDog.Messaging, the MessageQueue class depends on the Message class, but not the other way around.  Slide #3 of your presentation shows the Msg class having a dependency on the Send class.  Originally I though you were implementing MyMsg.Send methods similar to some of the early AF prototypes.  However, if I understand correctly that dependency exists only because you built the ability to reply into the your base Msg class.

Ah yes, each message contains a reply address of “Send” class.  So messages depend on “Send”, but not (I think) on any of the child classes of “Send”.

[Daklu]

Ah yes, each message contains a reply address of “Send” class.  So messages depend on “Send”, but not (I think) on any of the child classes of “Send”.

 

Do all messages include the address of the sender automatically, or is it only for the subset of messages that are intended to be request/reply?

[drjdpowell]

You supply a reply address, and its optional.  And the reply address does not have to be the sender’s; replies can be directed to any third party that the sender has the address of.

[AlexA]

Ok, so I just hit against the limitation of Onion routing that I think you were talking about earlier Daklu, and it comes as a result of this question: What about if I want to script the operation of the app and run it headless?

 

With my current implementation, I can't script it headless and still save data. I have to open up the UIs for the respective processes to save the data (the decision whether or not to save data has been made in the UIs so far). I've easily got 100+ messages, really, really don't want to re-factor them to not use Onion routing . I think I can get away with just refactoring a couple of data messages that need to be either displayed or saved, or both.

Anyways, just jumping on to say I understand why you'd avoid Onion now .