Decoupling message based systems that communicate across a network

[John Lokanis]

Goal:

Find the best methods (or at least some good options) for message decoupling that are simple to implement and efficient to execute.

Background:

Messaging architectures in LabVIEW that use a class for each unique message type are strongly coupled to the recipient process (aka ‘Actor’).  This is due to the need within the message execution code (a method of the message class) to access the private data of the recipient process or to call methods in the recipient process in order to do the work that the message intends.  (See Actor Framework for an example of this).

The problem arises when you wish to send these messages across a network between two separate applications.  In most cases, these applications are not duplicates of each other, but rather serve completely separate purposes.  An example would be a client and a server.  The client needs to message requests to the server and the server needs to push data to the client.  For a process to send a message, it needs to package the inputs in the private data of the message class and then transmit it via the network transport (which can be implemented in multiple different ways and is not material to this discussion).  In order to construct the message, the sender will need a copy of the message class included in their application.  This will mean they will need to also load the class of the message recipient since it is statically linked to the message class within the method that executes the message.  And since that will trigger many other class dependencies to load, the end results is the majority of the classes in the recipient application will need to be included in the sending application.  This is not an optimal solution.

So, we need to find a way to decouple messages from their recipients but still be able to execute them.

My solution:

The way I have been handling this is for each message that needs to cross the network I create a message class whose execute method calls an abstract method in a separate class (call this my network message execution class).   Both the sender and the recipient will have a copy of these message classes and the network message execution class.  Inside the message class’s execution method, I access a variable that stores an instance of the network message execution class and then calls the specific abstract method in the network message execution class for this particular message.

In each recipient application, I create a child of the network message execution class and override the abstract methods for the messages I intend to receive, placing the actual execution code (and static links to the recipient process) within the child class methods.

Finally, when each application initializes, I store its child network message execution class in the aforementioned variable so it can be used to dynamically dispatch to the actual method for message execution.

The advantages of this are:

Messages are decoupled between my applications.

The disadvantages are:

For each message I wish to transmit, I must create a new message class, a method in the network message execution class and a method override with the real implementation in the recipient’s network message execution class child and then edit the message class to call the method in the network message execution class.

This also means that each application must have a copy of all the message classes passed between applications and the network message execution class.

The problem arises when you add a 3^rd or fourth application or even a plugin library to the mix and wish to decouple those from the other applications.  You must either extend the current set of messages and the abstract methods in the network message execution class, having each entity maintain a copy of all of the messages and the network message execution class even though they never send or receive most of those messages, or you must add additional variables to your application to store different implementations of the network message execution class for each link between entities.

So, now that you have read my long explanation, does anyone have ideas for a better way to solve this?  I would love to simplify my code and make it easier to maintain, while retaining the functionality that class based message architectures offer.  But decoupling must be addressed somehow.

[ShaunR]

Use strings (the ultimate variant) and you won't have any problems either crossing the network boundary or language implementation.

Don't straight jacket yourself with LabVIEW classes which are tedious for encoding and completely rubbish for decoding.

[John Lokanis]

Thanks but that defeats the goal of "retaining the functionality that class based message architectures offer".  I realize there are many ways to implement a message based architecture and each has its trade offs.  Since I am working with class based messages, I want to solve the problems that this architecture poses.  I actually like the benefits it has so I plan to stick with it for now.

[Aristos Queue]

(I'm trying a new notation... message classes are double letters. Classes representing senders and receivers are full names, like Client or Server. I think this convention may aid communication.)

 

Here is what you say you want to do:

1. Client sends message class AA to server and says "please use message class BB to give me information Y".
2. Server eventually is ready to send information Y. It creates an instance of message BB and sends that across the network.

The challenge everyone runs into is how to instantiate BB on the server. But you don't actually need to do this.

This is a technique that I should have put together sooner, but just flat out couldn't see until Dr. James Powell (user drjdpowell) walked me into it.

 

Rephrase the problem:

1. Client sends message class AA to Server and says "Please send me information Y using the standard protocol message CC. Inside CC, include the name 'BB' in its data."
2. Server eventually is ready to send information Y. It creates an instance of CC and puts Y and the string 'BB' into the payload.
3. CC is totally agnostic of any particular receiver. It's "on arrival do this" code [which in Actor Framework is called "Do.vi" but may be something else in your systems] says "Create an instance of BB and put information Y in it, then immediately call BB's "on arrival do this" code." Class BB is a child of CC, which means that you can have a method on class CC that uses the Swap Values primitive to easily move the data from one object to the other without a data copy.

The CC class can be loaded on both sides without requiring any particular recipient classes to be loaded. Only class BB is specific to the receiver, and it is never on the sender's side.

Also, ShaunR isn't wrong in his assessment about strings. The classes obviously cannot actually cross the network boundary -- no data structure can. The only thing that *can* cross the boundary is strings. So you have to encode the objects on one side and decode them on the other. If you use the process I laid out above, that allows you to use the built-in serialization mechanisms of LabVIEW classes. Or you can create VIs to encode and decode your data structures. This is a problem regardless of whether you use LabVIEW classes or any other data structure. My Character Lineator was created specifically to investigate the problem of serialization over time, where you have multiple protocols in the same channel. It is overkill for simple data movement that will never change between any given pair of client/server, but becomes increasingly necessary as your data serialization requirements increase, which they generally do over time. If you do take ShaunR's advice and build a string representation of your data, please review the CL and make sure you include all the necessary version checks, region encodings, and formatting control that your situation calls for.

[John Lokanis]

Ok, but the data Y could be a data class and could be specific to msg class BB, so even though server does not need a copy of BB, it still needs a copy of Y.

 

But I see your point.  You are decoupling by using the class name in text format to select how to execute received data.  You still need to come up with a way to package data Y (and X and Z and etc) on the server side since each unique message has the potential to have unique data types.  And I don't see how to make CC as generic as you indicate.  I understand that CC can load an instance of class BB by name from disk and then call it's do method, but how can it translate the data from a generic type to the type Y that BB requires as input without having some way to select the class Y to cast it to?

[BrianGShea]

I'm going to spitball some ideas here. I have not attempted the conversation part of this but  I have been very successful with the dynamic loading and abstract classes (and i hear LV 2014 will make this easier with some new packed project library options)

 

Statically linking messages may be the first of your issues. If you could dynamic load messages using the encoding that AQ mentioned then deserialize the actual message data into the class, that would be a step in the right direction. At that point, all dynamically loaded messages would need to be a child of some abstract message that you main applications (client or server) could statically link to, I.e. the Do.vi

 

The beauty part of classes are their ability to override default behavior via Dynamic Dispatching rather than having to add a new method for each message.

 

You will also need an API to extract/inject the information from your client and server or mainly just the server. This API will need to be separate from you application, but both your application and conversation modules will need to reference this. That way you can build new conversation modules for the server without statically linking them inside the server application.

 

Now on both ends the conversation module would need to be implemented, once on the client to request/receive information and once on the server to receive/reply. Once the app is compiled, adding conversations, would be as easy as building a client side and server side conversation handler. Using AQ's suggestions for encoding something like this might occur "AA:How are you today?". The server would load AA class dynamically and send the remaining parts of the string to the deserialize function, AA class would take the message "How are you today?" and reply "AA:I am very good, Thank you!". On the client side, the client would load AA dynamically and deserialize the remaining part of the message and invoke some message in the API or inject the data to some VI in the API.

 

Obviously the AA Classes on the client and server end would have different implementations and possibly methods for send and receive or receive and reply, but i think i got my idea across.

 

Keeping the class names the same on both ends will help keep which client message corresponds to which server message, we'll just call it a conversation. This may be bad because having two classes in memory of the same name is not a good idea unless scoped differently via a library.

 

I have not talked about the API to extract/inject data into either client or server, and I'm not going to since it is our of scope of the main discussion. 

 

Since we have the ability to dynamically load messages and inject/extract data from an application, we can now create new conversations without changing at least the server side application. The client would either have to be scripted to initiate new conversions or statically linked to the messages of the conversion. Otherwise who would initiate the conversation?

 

Let me know if i'm crazy, so i don't attempt this in the future.

[Aristos Queue]

Ok, but the data Y could be a data class and could be specific to msg class BB, so even though server does not need a copy of BB, it still needs a copy of Y.

 

The data Y cannot be specific to msg class BB because the data class has to be filled in by the sender. By definition, it cannot be anything that is specific to the receiver, and therefore you can create class CC that is totally independent of the receiver.  It also cannot be anything specific to the sender because the sender doesn't know the receiver.

Edited July 25, 2014 by Aristos Queue

[John Lokanis]

Well, maybe I am doing things the *wrong* way, but the data sent between 'actors' in my systems is almost a class.

For example:

My server loads a hierarchical set of data from a database and stores it in a class that is a composition of several classes that represent the various sub elements of the data's hierarchy.  When a client connects to the server, the server needs to send this data Y to the client so it can be formatted and displayed to the user.  So, both will need the ability to understand this Y data class.  And the client's BB class must accept this Y class as input (normally by having it be an element of the message class's private data).

Now I suppose I could flatten the class on the server side and send it as a string using the generic CC class, then on the client side I could write the BB class to take a string data input so the CC class could pass the data to the child 'do' method in the BB class, but at that point I would have to unflatten the string to the Y data type so it could be used in the client.

How is this any better than the old enum-varient cluster message architecture?  You still need to cast your data.  You are just casting classes instead of variants.  One of the advantages of class based message architectures was the use of dynamic dispatch to eliminate the case selector for incoming messages and the use of a common parent message class so all the different class data could ride on the same wire and you would never have to cast it in your 'do' methods.

There is another advantage to keeping the data in its native format.  I can send the same message using the same functions to an external application or an internal 'actor' and the sender does not need to know the difference.  The architecture will automatically send it to the right destination and use the right method for sending it based on how I have setup my application.  This makes it very easy to break an application apart into two separate entities with very little code changes.

 

But I think we are getting away from the original question.  I accept the fact that there are other ways to send messages across a network that do not use classes or that convert the data into a string and use some other method than class based messaging to send the message.  But if I REALLY want to stick to class based messaging, is there a better or simpler way to decouple a message from its sender and receiver than the way I outlined in my original post?  So far, my method is working for me, but I would love to find a way to re-factor it to something simpler.

[mje]

I do very little inter-process communication, but I tend to come at it from a different angle when I do. It's not the components (classes) that need to be shared between client and server, it is the interface. It's really a serialization thing.

 

The client may package up a message in class AA which gets serialized through the transport layer. Server gets the data. Maybe it results in another instance of the same AA class being created on the server. Maybe it's the same class but a different version. Maybe the server's deserialization implementation has been updated and creates an entirely different class. The server may as well be a black box-- what happens inside doesn't matter so long as the interface contract is satisfied.

 

For example when I contact lavag.org I don't care what platform the website is running on. All that maters is my interface to the server, in this case the HTTP protocol. I make my GET request and I get my response.

 

I grant you though that if one uses the built-in class serialization then coupling is inevitable. So my only real suggestion to your question on a "way to decouple a message from its sender and receiver" is to settle on a serialization method that doesn't demand this coupling.

[ShaunR]

My server loads a hierarchical set of data from a database and stores it in a class that is a composition of several classes that represent the various sub elements of the data's hierarchy.  When a client connects to the server, the server needs to send this data Y to the client so it can be formatted and displayed to the user.  So, both will need the ability to understand this Y data class.  And the client's BB class must accept this Y class as input (normally by having it be an element of the message class's private data).

Now I suppose I could flatten the class on the server side and send it as a string using the generic CC class, then on the client side I could write the BB class to take a string data input so the CC class could pass the data to the child 'do' method in the BB class, but at that point I would have to unflatten the string to the Y data type so it could be used in the client.

 

Hmmm. So let me get this right. You have taken an implementation agnostic storage mechanism (a DB), queried it and stuffed it into an implementation and language specific set of classes which you then wish to transmit to another set of language and implementation specific classes that may or may not be the same and now want to break the enforced cohesion due to using the aforesaid classes?.

 

Why have you not just sent the SQL queries instead of faffing around with LV classes (a bit bit like this) and leave the client to figure out how it wants to display it (basically just as string indicators-use the DB to strip type) If you have a DB in the mix, then you can always craft SQL queries for the data and prepend instructions for functional operations. If you want to wrap all that formatting into message classes; fine. Like I said, encoding is tedious, but for decoding, the fact you cannot dynamically instantiate a class in LabVIEW and strict typing will ensure you cannot write a generic decoder (which is what AQs discombobulator thingy attempts to address).

 

Using message classes for this type of thing is exactly what I was saying about straight jacketing and not using a class messaging system will be quicker smaller and easier to maintain and extend. In fact, in this instance, as you are using a DB you will find that you will be able to extend it sometimes without writing any LV code - just changing and expanding the string messages via a file or via the client as the server acts just as a message parser.

[LogMAN]

You should take a look into this: http://lavag.org/files/file/220-messenging/

It is the first thing that got into my mind while reading this topic. Here is some more information about the inner workings: http://lavag.org/topic/17783-continuous-measurement-and-logging-redone-with-“messengingâ€-actors/

[John Lokanis]

Hmmm. So let me get this right...

I knew it was a bad idea to give a specific example in a discussion like this.  Inevitably, someone would read way too much into it and make a ton of unfounded assumptions that just go off in a tangent away from the original topic of the thread.

So, to quickly answer your question: sometimes the DB is on the other side of the planet and it is not accessible to be queried by the client.  Sometimes there is no DB but ranter a few XML files on a file server or on the machine the LV based server app is running on and those are not accessible to the client.  And the client has no need to know the source of the data.  It just needs a copy of the data so it can fulfill it's role as the VIEW in this MVC architecture.  Without a full understanding of the requirements of a system, suggesting solutions completely outside the context of the topic of the thread is not helpful.

And as for the string based message solution, as I stated before, that is a different solution with it's own set of complications.  I have chosen class based messaging for the benefits it offers, like easy message construction without using variants or flattened strings and automatic message execution without having to decode and select the implementation.  Thanks to inheritance and dynamic dispatch, this makes a nice clean implementation, with the drawback of strong coupling when crossing the network.

 

If anyone has some ideas relevant to decoupling an LVOOP class in a cleaner or simpler way than I proposed, please share your thoughts.  But let's keep this thread on topic and not degenerate it into a comparison of architecture styles.  We can start a different thread for that elsewhere if other wish to discuss it.

[ned]

I'm guessing this is somewhat a continuation of http://lavag.org/topic/16714-class-dependency-linking/?

 

What's the reason for storing the "network message execution class" in a variable (a functional global or some such, I assume) rather than wiring it in as a terminal? It might be easier to switch between implementations if it was a wire.

 

Can the messages be grouped together, such that there's a set that covers core functionality, then add additional groups? If so, you could inherit from the "network message execution class" with each successive child adding an additional set of handled messages.

[John Lokanis]

I'm guessing this is somewhat a continuation of http://lavag.org/topic/16714-class-dependency-linking/?

Yup.  Should have linked to that old thread.  I am revisiting my solution in hopes of finding ways to improve it.

 

What's the reason for storing the "network message execution class" in a variable (a functional global or some such, I assume) rather than wiring it in as a terminal? It might be easier to switch between implementations if it was a wire.

If I hard wire it in the parent method of the message, then it is statically linked again.  By setting it at runtime in an init routine, I can break the static link.  Here is how it works:

I have a parent class of type 'network message execution class'.  This is part of my architecture.  It has no methods.  IT just exists to give a type for my implementation variable storage and to inherit from.

For each App-to-App communication path I need, I create a child of 'network message execution class'.  This child has only abstract methods for each message that App A and App B might send to each other.  I then create a grandchild in App A that just overrides the methods for each message App A can receive.  I do the same in App B.  So, I now have a parent, a child and two grandchildren.  Only the grandchildren have code in their methods that statically link to other methods within their respective App. Both apps need a copy of the parent and the child but only App A has a copy of the grandchild for App A.

I store an instance of the grandchild in a variable (of my generic parent type 'network message execution class') in each App on init.

I then want to send message class X to App B.  So, App A and App B need a copy of message class X.  App A uses the class to construct the message (sets the values of the private data) and then sends it over the network (via some transport).

In the execute (or Do.vi for AF people) method of message class X, I access the local grandchild from the variable, but cast it as the shared child with the abstract methods for each message.  I then call the specific (abstract) method in the child class for message X.  At runtime this gets dynamic dispatched to the override method in the grandchild that is actually on this wire (from the variable I stored) and I get the desired behavior executed in the receiver (in this case, App B).

I hope that was not too cumbersome to read through.  If you refer to my original diagram, it might make more sense.

As for why all this variable stuff, I am trying to make a generic architecture where you can create different message execution implementations for each project.  So the architecture just supports some generic execution class but the specific one with the abstract methods is customized to the project.

I also actually store this in a variant attribute so when the message accesses it, it can choose from several by name so I can support multiple separate message groups between App A and App B, C, D, etc...

Can the messages be grouped together, such that there's a set that covers core functionality, then add additional groups? If so, you could inherit from the "network message execution class" with each successive child adding an additional set of handled messages.

That is exactly what I am doing.

 

So, maybe my implementation is the best solution but I was hoping for something simpler.  I would like to have less pieces to it if possible.  As it is now, for each message I want to add, I need to:

• Create the message from my network message template.
• Add the abstract method to the 'network message execution class' child.
• Edit the message execute code to call the new abstract method.
• Override the abstract method in the grandchild class of the receiver and implement the specific actions the message recipient is required to do.

I just have this feeling that there is some OOP trick or design pattern that I am missing that could make this cleaner.

[ShaunR]

I knew it was a bad idea to give a specific example in a discussion like this.  Inevitably, someone would read way too much into it and make a ton of unfounded assumptions that just go off in a tangent away from the original topic of the thread.

 

Not true. No assumptions, just pure string messages and an example of how it can work based on your own description (DB, XML, Queues, Events, Classes - it doesn't matter because it is decoupled). And I've already told you how to decouple the classes - serialise to strings - but you don't like that answer

 

When you insist on only using a hammer; everything looks like a nail so don't jump down my throat because a screw bends when you try to hammer it into concrete and I explain why and easier alternatives using a screwdriver.

Edited July 29, 2014 by ShaunR

[John Lokanis]

I should have entitled the original thread 'Decoupling LVOOP class based message systems that communicate across a network' or perhaps been even more generic and asked: 'How to best decouple a class from it's implementation?'

As stated, I am not interested in changing the whole architecture and abandoning message classes.  For the most part, they work very well and make for efficient and clean code.  But every architecture has its issues.

And serialization (as far as I understand it) really does not help anything because you still need the class on both ends to construct and execute the message.

I did not intend to jump down anyone's throat but if you re-read your responses, they seem a bit pushy instead of being helpful for solving the problem.  I would prefer to focus on the OOP decoupling problem and solve that then to 'pull out all the nails and replace them with screws'.

[ned]

I don't have a brilliant way to simplify. If you wanted to get into scripting, you could try scripting the addition of a new message, so you would just need to write the logic for the message itself (unavoidable) and the logic for how to handle that message (also unavoidable). The script would automate adding the abstract and specific calls to the parent and child of the network message execution class.

 

I still have an issue with using a variable inside the Execute method to get the correct network message execution class. There's no problem storing the class in a variable, but to follow the Visitor pattern, you should move the variable outside the entire message structure, and pass the class into the Execute method on a wire. This way you eliminate all the calls to the variable inside the various Execute methods. Maybe this is what you're doing already, and I've misunderstood - it would help to see code. I put together the diagram below, taking pieces from my Visitor example project, that I hope helps to show how this works:

There are two "Visitors" (which are like your Network Message Execution classes). They each perform an action on a DataItem (analogous to your Message). DataItem has a Do method with two parameters: dynamic dispatch on the type of DataItem, and static dispatch on the Visitor (the action). Inside the Do method is the reverse - dynamic dispatch on the Visitor, static dispatch on the DataItem since it's type is already known due to the previous dynamic dispatch. The message class only has a dependency on the parent Visitor class, not on any of the child implementations.

 

EDIT: and also note, there's no casting of the Visitor at all, whereas in your description you say you need to cast it to the parent type, which I don't understand.

Edited July 29, 2014 by ned

[LogMAN]

Okay, I start again by reading your initial post and ignoring any reply (except yours). Please be aware that this post is a pure brainstormer from my side and is in no way implemented by myself. Also you should know that I always brainstorm beginning with the basics and without limitations on changing entire structures. Since I don't have your current implementation at my disposal, my brain might get a bit out of control, but I try to stick as close to your situation as possible:

First of all, we want to decouple implementations between sender and receiver. This requires a common 'language' between the systems. We don't want to go down to string handling and already have some sort of messaging system implemented which allows transmission of objects. Let me jump into your last post for now:
 

I should have entitled the original thread 'Decoupling LVOOP class based message systems that communicate across a network' or perhaps been even more generic and asked: 'How to best decouple a class from it's implementation?'
...

Decoupling a class from it's implementation is just not possible by design! A class provides its own set of variables and implementation of behaviour. From an opposite point of view:

Since a couple of years my collegues and I implement anything in LabVIEW without classes. Instead we define a typedef and a couple of functions that work on that typedef. As you can tell, a class would be a far better solution to implement such functions as it defines fields (typedef) and methods.

You currently have a set of data that is common between server and client, packed into the private field of a class. As you mentioned the MVC before, that would be our Model. I see two ways of implementing that kind of model. First a class which must only define the data and provides no methods (except for accessors). The second option is a plain typedef. We'll settle for the class from here on.

Next we want two implementations for the same model (server/client). That is easily archived if we just share the model as we specified above. We don't inherit and just use the class as model which is a parameter for our Controller.

So the flow would be:

Server Side : Controller -> Model >> [NetworkMessage] >> Model -> Controller : Client Side
* Notice that NetworkMessage relates to your entire messaging system.

That somewhat 'covers' the data transfer (will make more sense in a moment; maybe less). Next we want the receiver to do something with these information. I'll try to stick as much to your example as possible:
 

...
An example would be a client and a server. The client needs to message requests to the server and the server needs to push data to the client.
...

The server doesn't know what the client intends to do with the information. So the client must provide some 'meta-data' to the server which is send back in the reply. The client knows how to handle the 'meta-data' in the reply and will act accordingly. (It's the client telling itself what to do with the reply from the server)
 

...
The problem arises when you add a 3^rd or fourth application or even a plugin library to the mix and wish to decouple those from the other applications. You must either extend the current set of messages and the abstract methods in the network message execution class, having each entity maintain a copy of all of the messages and the network message execution class even though they never send or receive most of those messages, or you must add additional variables to your application to store different implementations of the network message execution class for each link between entities.
...

I also want to get rid of the ridiculous amount of messages the client needs to implement...

Basically there should only be a single receiving loop that receives messages from the server and checks if the client implements a method to handle the given data. If that's not the case, it could either throw an error or just ignore the message.

Assume we implement a class. That class is for use by the Messaging system and provides the means to transport a Model. Let's call it RelayMessage. The RelayMessage class has two fields, the Model (we need a parent Model!) and a Message (we need a parent Message similar to the Actor Framework). As the RelayMessage is received by the server, it'll do whatever it has to do, insert the Model and send the RelayMessage back to the caller. The caller can than execute whatever is implemented in the Message class.
* I hope that makes sense to you...

So we need to share a couple of classes between server and client:

MessageBase :      Is overridden by the client to implement a specific action (Do.vi); The server does not know or care about this! <-- This is actually the decoupling!
ModelBase :           Defines the base class of a Model which is used by the RelayMessage (as data)
RelayMessage :     Defines a single Message between server and client (with MessageBase and ModelBase as fields)
<Model> :               The specific Models which are supported by the server and may or may not be implemented by the clients. Of course the server must know about all Models (not necessarily, but we'll stick to that assumption)

If my guessing is somewhat sane, this should give you a more or less decoupled system, where the server knows all Models. The clients implement the necessary models and define their own actions.

The missing link right now is how to tell the server what to do...
My idea is that the RelayMessage implements a method for each function of the server (we require an implementation of MessageBase for each method, which is put on another field of the RelayMessage). Each method can have parameters and require an implementation of MessageBase from the client (for reply-handling of the client).

Now that I think about it, this can even be used to implement a couple of MessageBase classes that are common throughout all systems and allow for the server to issue tasks without request from the clients... Last but not least, if the server should be flexible and scalable in functionality depending on the client (like the client specified what the server has to do), the server could implement a Messaging system where the client implements the functions, which the server executes (so basically the actor framework, but the actor is on the server).

Now please don't ask me how to implement all of this... It sounds like fun to me, but also like a lot of work
If this post makes no sense, forget it

[John Lokanis]

EDIT: and also note, there's no casting of the Visitor at all, whereas in your description you say you need to cast it to the parent type, which I don't understand.

Thanks.  Your example is actually similar to what I am doing.  I will try to find some time to build an example to post.  The real project is already well over 2000 VIs and not something I could share here.

 

To answer your question, my architecture differs from AF in that I have a singleton object wrapped in a DVR that represents the communication buss allowing my processes (actors) to communicate.  This make it a flat message system and not a hierarchy like AF.  To add network messaging to a project, you can instantiate the system using a child class of the system object that adds the ability to store the local implementation of the 'network message execution class'.  Since the architecture has no idea what messages the project will need to execute, the value of this variable in the system class is the 'network message execution class' top parent with no methods.  The version for the project that contains the abstract methods is a child of this class.  So, when we write the execution code in the message, we read the variable (which is of type 'network message execution class' top parent), but cast it to the child with the abstract methods.  This then allows us to call the specific abstract method for this message.  Now, if this is executed in a specific system where the variable has been set to the grandchild class with the concrete implementation of the execute methods, then dynamic dispatch will call the grandchild method and we will get our desired execution behavior.

I just read that three times and I think it is as clear as I can make it.  Sorry if it is not.

 

If this post makes no sense, forget it

No worries.  I kinda follow your thinking (I think).  I will have to ponder this some more.

[Neil Pate]

Jack and Michael, if you are "listening", I think this kind of stuff would make a great topic for VI Shots live.

[ShaunR]

I should have entitled the original thread 'Decoupling LVOOP class based message systems that communicate across a network' or perhaps been even more generic and asked: 'How to best decouple a class from it's implementation?'

As stated, I am not interested in changing the whole architecture and abandoning message classes.  For the most part, they work very well and make for efficient and clean code.  But every architecture has its issues.

And serialization (as far as I understand it) really does not help anything because you still need the class on both ends to construct and execute the message.

I did not intend to jump down anyone's throat but if you re-read your responses, they seem a bit pushy instead of being helpful for solving the problem.  I would prefer to focus on the OOP decoupling problem and solve that then to 'pull out all the nails and replace them with screws'.

 

I think you read the words, but didn't understand the content. My writing style can seem adversarial sometimes but it is usually with the genuine intent to help or, in this case, stop you going off on a unicorn hunt.The point I'm trying to get across is you are chasing a unicorn and only because the solution must be LV classes. If you drop that requirement, it's just an ordinary horse and we have loads of them.

 

A better heading would have been. "How do I reconstitute a LV object from a string sent over a network like I can in Python, Javascript or PHP". That would have got you the unanimous response of "You can't!" and is the reason why your "decoupling" search will ultimately be a dead end when using classes.

 

Anyway. I said my bit. I've clarified and re-clarified and so its time for me to shut up with a final apology for coming across as "pushy".

[drjdpowell]

The problem arises when you wish to send these messages across a network between two separate applications.  In most cases, these applications are not duplicates of each other, but rather serve completely separate purposes.  An example would be a client and a server.  The client needs to message requests to the server and the server needs to push data to the client.  For a process to send a message, it needs to package the inputs in the private data of the message class and then transmit it via the network transport (which can be implemented in multiple different ways and is not material to this discussion).  In order to construct the message, the sender will need a copy of the message class included in their application.  This will mean they will need to also load the class of the message recipient since it is statically linked to the message class within the method that executes the message.  And since that will trigger many other class dependencies to load, the end results is the majority of the classes in the recipient application will need to be included in the sending application.  This is not an optimal solution.

So, we need to find a way to decouple messages from their recipients but still be able to execute them.

 

Note: I was on vacation when this conversation was going on, and I haven’t read it in great detail.

 

I only use Command-Pattern messages in limited cases, favouring a variation on string-variant messages mostly.  But the framework I use can do Command Pattern, and can decouple the command from the “serverâ€.   So I made a test example.  In it, the Client sends a request (non-command-pattern for simplicity) for a data message (DataAA.lvclass).   As part of its reply “addressâ€, it specifies an “envelope message†(CommandAA.lvclass) that it wishes to receive the data message inside of.  The TCP Framework keeps the reply address in flattened form while on the server side, so the server never needs to load class CommandAA.  The “Do.vi†method of CommandAA.lvclass invokes the “Read.vi†static method of DataAA.lvclass.

 

 

The methods one needs to write are Write and Read for the Data message, and Do for the Command (with an optional method to add metadata to the Command if desired).  [aside: AQ proposed a slightly different form, where the Command class is a child of the Data Message Class; he would need an additional method to support building the Command from the Data message.]  

Command Pattern Uncoupling.zip

drjdpowell_lib_messenging-1.2.1.29.vip

drjdpowell_lib_cyclic_table_probes-1.1.0.8.vip

[John Lokanis]

Thanks for all the replies on this.  After some discussions with other DEVs at NI Week, I concluded that the way I am decoupling my network messages is the only practical solution for a command-pattern based message system.  And now this thread and those discussions are leading me to reconsider the command-pattern design altogether.  I still find it difficult to give up on because of its benefits and the fact that NI chose it for their AF architecture.  It must be good, right?  Well, I have started a different thread on this issue over here if you want to continue the discussion and give feedback.  I think we need to find a best practice for AOP designs if not a specific architecture.  Personally, I had to go down the command-pattern road myself to truly see where it led.

[drjdpowell]

On 8/13/2014 at 7:47 PM, John Lokanis said:

 I concluded that the way I am decoupling my network messages is the only practical solution for a command-pattern based message system.  

 

Didn't like my "envelope" demo, eh?  Personally I would drop the command pattern for the data message (using standard Variant or Object messages instead) and just use command pattern for actual commands.   Then there is one class per message transfer, with server decoupled from client.

Edited January 9, 2019 by drjdpowell

[John Lokanis]

It's not that I didn't like it.  I am still trying to decide what path to take overall.  I just want maximum simplicity and maximum functionality.  Somewhere they intersect and I hope to figure out where for my needs.

I did discuss the different options with Allen Smith at NI Week and the conclusion was for command pattern across a network, the abstraction (or interface) design was the cleanest or at least easiest to understand.