Jump to content

Decoupling message based systems that communicate across a network


Recommended Posts

Goal:

Find the best methods (or at least some good options) for message decoupling that are simple to implement and efficient to execute.

Background:

Messaging architectures in LabVIEW that use a class for each unique message type are strongly coupled to the recipient process (aka ‘Actor’).  This is due to the need within the message execution code (a method of the message class) to access the private data of the recipient process or to call methods in the recipient process in order to do the work that the message intends.  (See Actor Framework for an example of this).

The problem arises when you wish to send these messages across a network between two separate applications.  In most cases, these applications are not duplicates of each other, but rather serve completely separate purposes.  An example would be a client and a server.  The client needs to message requests to the server and the server needs to push data to the client.  For a process to send a message, it needs to package the inputs in the private data of the message class and then transmit it via the network transport (which can be implemented in multiple different ways and is not material to this discussion).  In order to construct the message, the sender will need a copy of the message class included in their application.  This will mean they will need to also load the class of the message recipient since it is statically linked to the message class within the method that executes the message.  And since that will trigger many other class dependencies to load, the end results is the majority of the classes in the recipient application will need to be included in the sending application.  This is not an optimal solution.

So, we need to find a way to decouple messages from their recipients but still be able to execute them.

My solution:

The way I have been handling this is for each message that needs to cross the network I create a message class whose execute method calls an abstract method in a separate class (call this my network message execution class).   Both the sender and the recipient will have a copy of these message classes and the network message execution class.  Inside the message class’s execution method, I access a variable that stores an instance of the network message execution class and then calls the specific abstract method in the network message execution class for this particular message.

In each recipient application, I create a child of the network message execution class and override the abstract methods for the messages I intend to receive, placing the actual execution code (and static links to the recipient process) within the child class methods.

Finally, when each application initializes, I store its child network message execution class in the aforementioned variable so it can be used to dynamically dispatch to the actual method for message execution.

The advantages of this are:

Messages are decoupled between my applications.

The disadvantages are:

For each message I wish to transmit, I must create a new message class, a method in the network message execution class and a method override with the real implementation in the recipient’s network message execution class child and then edit the message class to call the method in the network message execution class.

This also means that each application must have a copy of all the message classes passed between applications and the network message execution class.

post-2411-0-58036100-1406312917_thumb.pn

The problem arises when you add a 3rd or fourth application or even a plugin library to the mix and wish to decouple those from the other applications.  You must either extend the current set of messages and the abstract methods in the network message execution class, having each entity maintain a copy of all of the messages and the network message execution class even though they never send or receive most of those messages, or you must add additional variables to your application to store different implementations of the network message execution class for each link between entities.

So, now that you have read my long explanation, does anyone have ideas for a better way to solve this?  I would love to simplify my code and make it easier to maintain, while retaining the functionality that class based message architectures offer.  But decoupling must be addressed somehow.

Link to post
Share on other sites

Use strings (the ultimate variant) and you won't have any problems either crossing the network boundary or language implementation.

Don't straight jacket yourself with LabVIEW classes which are tedious for encoding and completely rubbish for decoding.

  • Like 1
Link to post
Share on other sites

Thanks but that defeats the goal of "retaining the functionality that class based message architectures offer".  I realize there are many ways to implement a message based architecture and each has its trade offs.  Since I am working with class based messages, I want to solve the problems that this architecture poses.  I actually like the benefits it has so I plan to stick with it for now.

Link to post
Share on other sites

(I'm trying a new notation... message classes are double letters. Classes representing senders and receivers are full names, like Client or Server. I think this convention may aid communication.)

 

Here is what you say you want to do:

  1. Client sends message class AA to server and says "please use message class BB to give me information Y".
  2. Server eventually is ready to send information Y. It creates an instance of message BB and sends that across the network.

The challenge everyone runs into is how to instantiate BB on the server. But you don't actually need to do this.

This is a technique that I should have put together sooner, but just flat out couldn't see until Dr. James Powell (user drjdpowell) walked me into it.

 

Rephrase the problem:

  1. Client sends message class AA to Server and says "Please send me information Y using the standard protocol message CC. Inside CC, include the name 'BB' in its data."
  2. Server eventually is ready to send information Y. It creates an instance of CC and puts Y and the string 'BB' into the payload.
  3. CC is totally agnostic of any particular receiver. It's "on arrival do this" code [which in Actor Framework is called "Do.vi" but may be something else in your systems] says "Create an instance of BB and put information Y in it, then immediately call BB's "on arrival do this" code." Class BB is a child of CC, which means that you can have a method on class CC that uses the Swap Values primitive to easily move the data from one object to the other without a data copy.

The CC class can be loaded on both sides without requiring any particular recipient classes to be loaded. Only class BB is specific to the receiver, and it is never on the sender's side.


Also, ShaunR isn't wrong in his assessment about strings. The classes obviously cannot actually cross the network boundary -- no data structure can. The only thing that *can* cross the boundary is strings. So you have to encode the objects on one side and decode them on the other. If you use the process I laid out above, that allows you to use the built-in serialization mechanisms of LabVIEW classes. Or you can create VIs to encode and decode your data structures. This is a problem regardless of whether you use LabVIEW classes or any other data structure. My Character Lineator was created specifically to investigate the problem of serialization over time, where you have multiple protocols in the same channel. It is overkill for simple data movement that will never change between any given pair of client/server, but becomes increasingly necessary as your data serialization requirements increase, which they generally do over time. If you do take ShaunR's advice and build a string representation of your data, please review the CL and make sure you include all the necessary version checks, region encodings, and formatting control that your situation calls for.

Link to post
Share on other sites

Ok, but the data Y could be a data class and could be specific to msg class BB, so even though server does not need a copy of BB, it still needs a copy of Y.

 

But I see your point.  You are decoupling by using the class name in text format to select how to execute received data.  You still need to come up with a way to package data Y (and X and Z and etc) on the server side since each unique message has the potential to have unique data types.  And I don't see how to make CC as generic as you indicate.  I understand that CC can load an instance of class BB by name from disk and then call it's do method, but how can it translate the data from a generic type to the type Y that BB requires as input without having some way to select the class Y to cast it to?

Link to post
Share on other sites

I'm going to spitball some ideas here. I have not attempted the conversation part of this but  I have been very successful with the dynamic loading and abstract classes (and i hear LV 2014 will make this easier with some new packed project library options)

 

Statically linking messages may be the first of your issues. If you could dynamic load messages using the encoding that AQ mentioned then deserialize the actual message data into the class, that would be a step in the right direction. At that point, all dynamically loaded messages would need to be a child of some abstract message that you main applications (client or server) could statically link to, I.e. the Do.vi

 

The beauty part of classes are their ability to override default behavior via Dynamic Dispatching rather than having to add a new method for each message.

 

You will also need an API to extract/inject the information from your client and server or mainly just the server. This API will need to be separate from you application, but both your application and conversation modules will need to reference this. That way you can build new conversation modules for the server without statically linking them inside the server application.

 

Now on both ends the conversation module would need to be implemented, once on the client to request/receive information and once on the server to receive/reply. Once the app is compiled, adding conversations, would be as easy as building a client side and server side conversation handler. Using AQ's suggestions for encoding something like this might occur "AA:How are you today?". The server would load AA class dynamically and send the remaining parts of the string to the deserialize function, AA class would take the message "How are you today?" and reply "AA:I am very good, Thank you!". On the client side, the client would load AA dynamically and deserialize the remaining part of the message and invoke some message in the API or inject the data to some VI in the API.

 

Obviously the AA Classes on the client and server end would have different implementations and possibly methods for send and receive or receive and reply, but i think i got my idea across.

 

Keeping the class names the same on both ends will help keep which client message corresponds to which server message, we'll just call it a conversation. This may be bad because having two classes in memory of the same name is not a good idea unless scoped differently via a library.

 

I have not talked about the API to extract/inject data into either client or server, and I'm not going to since it is our of scope of the main discussion. 

 

Since we have the ability to dynamically load messages and inject/extract data from an application, we can now create new conversations without changing at least the server side application. The client would either have to be scripted to initiate new conversions or statically linked to the messages of the conversion. Otherwise who would initiate the conversation?

 

Let me know if i'm crazy, so i don't attempt this in the future.

Link to post
Share on other sites

Ok, but the data Y could be a data class and could be specific to msg class BB, so even though server does not need a copy of BB, it still needs a copy of Y.

 

The data Y cannot be specific to msg class BB because the data class has to be filled in by the sender. By definition, it cannot be anything that is specific to the receiver, and therefore you can create class CC that is totally independent of the receiver.  It also cannot be anything specific to the sender because the sender doesn't know the receiver.

Edited by Aristos Queue
Link to post
Share on other sites

Well, maybe I am doing things the *wrong* way, but the data sent between 'actors' in my systems is almost a class.

For example:

My server loads a hierarchical set of data from a database and stores it in a class that is a composition of several classes that represent the various sub elements of the data's hierarchy.  When a client connects to the server, the server needs to send this data Y to the client so it can be formatted and displayed to the user.  So, both will need the ability to understand this Y data class.  And the client's BB class must accept this Y class as input (normally by having it be an element of the message class's private data).

Now I suppose I could flatten the class on the server side and send it as a string using the generic CC class, then on the client side I could write the BB class to take a string data input so the CC class could pass the data to the child 'do' method in the BB class, but at that point I would have to unflatten the string to the Y data type so it could be used in the client.

How is this any better than the old enum-varient cluster message architecture?  You still need to cast your data.  You are just casting classes instead of variants.  One of the advantages of class based message architectures was the use of dynamic dispatch to eliminate the case selector for incoming messages and the use of a common parent message class so all the different class data could ride on the same wire and you would never have to cast it in your 'do' methods.

There is another advantage to keeping the data in its native format.  I can send the same message using the same functions to an external application or an internal 'actor' and the sender does not need to know the difference.  The architecture will automatically send it to the right destination and use the right method for sending it based on how I have setup my application.  This makes it very easy to break an application apart into two separate entities with very little code changes.

 

But I think we are getting away from the original question.  I accept the fact that there are other ways to send messages across a network that do not use classes or that convert the data into a string and use some other method than class based messaging to send the message.  But if I REALLY want to stick to class based messaging, is there a better or simpler way to decouple a message from its sender and receiver than the way I outlined in my original post?  So far, my method is working for me, but I would love to find a way to re-factor it to something simpler.

Link to post
Share on other sites

I do very little inter-process communication, but I tend to come at it from a different angle when I do. It's not the components (classes) that need to be shared between client and server, it is the interface. It's really a serialization thing.

 

The client may package up a message in class AA which gets serialized through the transport layer. Server gets the data. Maybe it results in another instance of the same AA class being created on the server. Maybe it's the same class but a different version. Maybe the server's deserialization implementation has been updated and creates an entirely different class. The server may as well be a black box-- what happens inside doesn't matter so long as the interface contract is satisfied.

 

For example when I contact lavag.org I don't care what platform the website is running on. All that maters is my interface to the server, in this case the HTTP protocol. I make my GET request and I get my response.

 

I grant you though that if one uses the built-in class serialization then coupling is inevitable. So my only real suggestion to your question on a "way to decouple a message from its sender and receiver" is to settle on a serialization method that doesn't demand this coupling.

Link to post
Share on other sites

My server loads a hierarchical set of data from a database and stores it in a class that is a composition of several classes that represent the various sub elements of the data's hierarchy.  When a client connects to the server, the server needs to send this data Y to the client so it can be formatted and displayed to the user.  So, both will need the ability to understand this Y data class.  And the client's BB class must accept this Y class as input (normally by having it be an element of the message class's private data).

Now I suppose I could flatten the class on the server side and send it as a string using the generic CC class, then on the client side I could write the BB class to take a string data input so the CC class could pass the data to the child 'do' method in the BB class, but at that point I would have to unflatten the string to the Y data type so it could be used in the client.

 

Hmmm. So let me get this right. You have taken an implementation agnostic storage mechanism (a DB), queried it and stuffed it into an implementation and language specific set of classes which you then wish to transmit to another set of language and implementation specific classes that may or may not be the same and now want to break the enforced cohesion due to using the aforesaid classes?.

 

Why have you not just sent the SQL queries instead of faffing around with LV classes (a bit bit like this) and leave the client to figure out how it wants to display it (basically just as string indicators-use the DB to strip type) If you have a DB in the mix, then you can always craft SQL queries for the data and prepend instructions for functional operations. If you want to wrap all that formatting into message classes; fine. Like I said, encoding is tedious, but for decoding, the fact you cannot dynamically instantiate a class in LabVIEW and strict typing will ensure you cannot write a generic decoder (which is what AQs discombobulator thingy attempts to address).

 

Using message classes for this type of thing is exactly what I was saying about straight jacketing and not using a class messaging system will be quicker smaller and easier to maintain and extend. In fact, in this instance, as you are using a DB you will find that you will be able to extend it sometimes without writing any LV code - just changing and expanding the string messages via a file or via the client as the server acts just as a message parser.

Link to post
Share on other sites

You should take a look into this: http://lavag.org/files/file/220-messenging/

It is the first thing that got into my mind while reading this topic. Here is some more information about the inner workings: http://lavag.org/topic/17783-continuous-measurement-and-logging-redone-with-“messengingâ€-actors/

Link to post
Share on other sites

Hmmm. So let me get this right...

I knew it was a bad idea to give a specific example in a discussion like this.  Inevitably, someone would read way too much into it and make a ton of unfounded assumptions that just go off in a tangent away from the original topic of the thread.

So, to quickly answer your question: sometimes the DB is on the other side of the planet and it is not accessible to be queried by the client.  Sometimes there is no DB but ranter a few XML files on a file server or on the machine the LV based server app is running on and those are not accessible to the client.  And the client has no need to know the source of the data.  It just needs a copy of the data so it can fulfill it's role as the VIEW in this MVC architecture.  Without a full understanding of the requirements of a system, suggesting solutions completely outside the context of the topic of the thread is not helpful.

And as for the string based message solution, as I stated before, that is a different solution with it's own set of complications.  I have chosen class based messaging for the benefits it offers, like easy message construction without using variants or flattened strings and automatic message execution without having to decode and select the implementation.  Thanks to inheritance and dynamic dispatch, this makes a nice clean implementation, with the drawback of strong coupling when crossing the network.

 

If anyone has some ideas relevant to decoupling an LVOOP class in a cleaner or simpler way than I proposed, please share your thoughts.  But let's keep this thread on topic and not degenerate it into a comparison of architecture styles.  We can start a different thread for that elsewhere if other wish to discuss it.

Link to post
Share on other sites

I'm guessing this is somewhat a continuation of http://lavag.org/topic/16714-class-dependency-linking/?

 

What's the reason for storing the "network message execution class" in a variable (a functional global or some such, I assume) rather than wiring it in as a terminal? It might be easier to switch between implementations if it was a wire.

 

Can the messages be grouped together, such that there's a set that covers core functionality, then add additional groups? If so, you could inherit from the "network message execution class" with each successive child adding an additional set of handled messages.

Link to post
Share on other sites

I'm guessing this is somewhat a continuation of http://lavag.org/topic/16714-class-dependency-linking/?

Yup.  Should have linked to that old thread.  I am revisiting my solution in hopes of finding ways to improve it.

 

What's the reason for storing the "network message execution class" in a variable (a functional global or some such, I assume) rather than wiring it in as a terminal? It might be easier to switch between implementations if it was a wire.

If I hard wire it in the parent method of the message, then it is statically linked again.  By setting it at runtime in an init routine, I can break the static link.  Here is how it works:

I have a parent class of type 'network message execution class'.  This is part of my architecture.  It has no methods.  IT just exists to give a type for my implementation variable storage and to inherit from.

For each App-to-App communication path I need, I create a child of 'network message execution class'.  This child has only abstract methods for each message that App A and App B might send to each other.  I then create a grandchild in App A that just overrides the methods for each message App A can receive.  I do the same in App B.  So, I now have a parent, a child and two grandchildren.  Only the grandchildren have code in their methods that statically link to other methods within their respective App. Both apps need a copy of the parent and the child but only App A has a copy of the grandchild for App A.

I store an instance of the grandchild in a variable (of my generic parent type 'network message execution class') in each App on init.

I then want to send message class X to App B.  So, App A and App B need a copy of message class X.  App A uses the class to construct the message (sets the values of the private data) and then sends it over the network (via some transport).

In the execute (or Do.vi for AF people) method of message class X, I access the local grandchild from the variable, but cast it as the shared child with the abstract methods for each message.  I then call the specific (abstract) method in the child class for message X.  At runtime this gets dynamic dispatched to the override method in the grandchild that is actually on this wire (from the variable I stored) and I get the desired behavior executed in the receiver (in this case, App B).

I hope that was not too cumbersome to read through.  If you refer to my original diagram, it might make more sense.

As for why all this variable stuff, I am trying to make a generic architecture where you can create different message execution implementations for each project.  So the architecture just supports some generic execution class but the specific one with the abstract methods is customized to the project.

I also actually store this in a variant attribute so when the message accesses it, it can choose from several by name so I can support multiple separate message groups between App A and App B, C, D, etc...

Can the messages be grouped together, such that there's a set that covers core functionality, then add additional groups? If so, you could inherit from the "network message execution class" with each successive child adding an additional set of handled messages.

That is exactly what I am doing.

 

So, maybe my implementation is the best solution but I was hoping for something simpler.  I would like to have less pieces to it if possible.  As it is now, for each message I want to add, I need to:

  • Create the message from my network message template.
  • Add the abstract method to the 'network message execution class' child.
  • Edit the message execute code to call the new abstract method.
  • Override the abstract method in the grandchild class of the receiver and implement the specific actions the message recipient is required to do.

I just have this feeling that there is some OOP trick or design pattern that I am missing that could make this cleaner.

Link to post
Share on other sites

I knew it was a bad idea to give a specific example in a discussion like this.  Inevitably, someone would read way too much into it and make a ton of unfounded assumptions that just go off in a tangent away from the original topic of the thread.

 

Not true. No assumptions, just pure string messages and an example of how it can work based on your own description (DB, XML, Queues, Events, Classes - it doesn't matter because it is decoupled). And I've already told you how to decouple the classes - serialise to strings - but you don't like that answer

 

When you insist on only using a hammer; everything looks like a nail so don't jump down my throat because a screw bends when you try to hammer it into concrete and I explain why and easier alternatives using a screwdriver.

Edited by ShaunR
Link to post
Share on other sites

I should have entitled the original thread 'Decoupling LVOOP class based message systems that communicate across a network' or perhaps been even more generic and asked: 'How to best decouple a class from it's implementation?'

As stated, I am not interested in changing the whole architecture and abandoning message classes.  For the most part, they work very well and make for efficient and clean code.  But every architecture has its issues.

And serialization (as far as I understand it) really does not help anything because you still need the class on both ends to construct and execute the message.

I did not intend to jump down anyone's throat but if you re-read your responses, they seem a bit pushy instead of being helpful for solving the problem.  I would prefer to focus on the OOP decoupling problem and solve that then to 'pull out all the nails and replace them with screws'.

Link to post
Share on other sites

I don't have a brilliant way to simplify. If you wanted to get into scripting, you could try scripting the addition of a new message, so you would just need to write the logic for the message itself (unavoidable) and the logic for how to handle that message (also unavoidable). The script would automate adding the abstract and specific calls to the parent and child of the network message execution class.

 

I still have an issue with using a variable inside the Execute method to get the correct network message execution class. There's no problem storing the class in a variable, but to follow the Visitor pattern, you should move the variable outside the entire message structure, and pass the class into the Execute method on a wire. This way you eliminate all the calls to the variable inside the various Execute methods. Maybe this is what you're doing already, and I've misunderstood - it would help to see code. I put together the diagram below, taking pieces from my Visitor example project, that I hope helps to show how this works:
post-3989-0-55750300-1406662917.png

There are two "Visitors" (which are like your Network Message Execution classes). They each perform an action on a DataItem (analogous to your Message). DataItem has a Do method with two parameters: dynamic dispatch on the type of DataItem, and static dispatch on the Visitor (the action). Inside the Do method is the reverse - dynamic dispatch on the Visitor, static dispatch on the DataItem since it's type is already known due to the previous dynamic dispatch. The message class only has a dependency on the parent Visitor class, not on any of the child implementations.

 

EDIT: and also note, there's no casting of the Visitor at all, whereas in your description you say you need to cast it to the parent type, which I don't understand.

Edited by ned
Link to post
Share on other sites

Okay, I start again by reading your initial post and ignoring any reply (except yours). Please be aware that this post is a pure brainstormer from my side and is in no way implemented by myself. Also you should know that I always brainstorm beginning with the basics and without limitations on changing entire structures. Since I don't have your current implementation at my disposal, my brain might get a bit out of control, but I try to stick as close to your situation as possible:

First of all, we want to decouple implementations between sender and receiver. This requires a common 'language' between the systems. We don't want to go down to string handling and already have some sort of messaging system implemented which allows transmission of objects. Let me jump into your last post for now:
 

I should have entitled the original thread 'Decoupling LVOOP class based message systems that communicate across a network' or perhaps been even more generic and asked: 'How to best decouple a class from it's implementation?'
...


Decoupling a class from it's implementation is just not possible by design! A class provides its own set of variables and implementation of behaviour. From an opposite point of view:

Since a couple of years my collegues and I implement anything in LabVIEW without classes. Instead we define a typedef and a couple of functions that work on that typedef. As you can tell, a class would be a far better solution to implement such functions as it defines fields (typedef) and methods.

You currently have a set of data that is common between server and client, packed into the private field of a class. As you mentioned the MVC before, that would be our Model. I see two ways of implementing that kind of model. First a class which must only define the data and provides no methods (except for accessors). The second option is a plain typedef. We'll settle for the class from here on.

Next we want two implementations for the same model (server/client). That is easily archived if we just share the model as we specified above. We don't inherit and just use the class as model which is a parameter for our Controller.

So the flow would be:

Server Side : Controller -> Model >> [NetworkMessage] >> Model -> Controller : Client Side
* Notice that NetworkMessage relates to your entire messaging system.

That somewhat 'covers' the data transfer (will make more sense in a moment; maybe less). Next we want the receiver to do something with these information. I'll try to stick as much to your example as possible:
 

...
An example would be a client and a server. The client needs to message requests to the server and the server needs to push data to the client.
...


The server doesn't know what the client intends to do with the information. So the client must provide some 'meta-data' to the server which is send back in the reply. The client knows how to handle the 'meta-data' in the reply and will act accordingly. (It's the client telling itself what to do with the reply from the server)
 

...
The problem arises when you add a 3rd or fourth application or even a plugin library to the mix and wish to decouple those from the other applications. You must either extend the current set of messages and the abstract methods in the network message execution class, having each entity maintain a copy of all of the messages and the network message execution class even though they never send or receive most of those messages, or you must add additional variables to your application to store different implementations of the network message execution class for each link between entities.
...


I also want to get rid of the ridiculous amount of messages the client needs to implement...

Basically there should only be a single receiving loop that receives messages from the server and checks if the client implements a method to handle the given data. If that's not the case, it could either throw an error or just ignore the message.

Assume we implement a class. That class is for use by the Messaging system and provides the means to transport a Model. Let's call it RelayMessage. The RelayMessage class has two fields, the Model (we need a parent Model!) and a Message (we need a parent Message similar to the Actor Framework). As the RelayMessage is received by the server, it'll do whatever it has to do, insert the Model and send the RelayMessage back to the caller. The caller can than execute whatever is implemented in the Message class.
* I hope that makes sense to you...

So we need to share a couple of classes between server and client:

MessageBase :      Is overridden by the client to implement a specific action (Do.vi); The server does not know or care about this! <-- This is actually the decoupling!
ModelBase :           Defines the base class of a Model which is used by the RelayMessage (as data)
RelayMessage :     Defines a single Message between server and client (with MessageBase and ModelBase as fields)
<Model> :               The specific Models which are supported by the server and may or may not be implemented by the clients. Of course the server must know about all Models (not necessarily, but we'll stick to that assumption)

If my guessing is somewhat sane, this should give you a more or less decoupled system, where the server knows all Models. The clients implement the necessary models and define their own actions.

The missing link right now is how to tell the server what to do...
My idea is that the RelayMessage implements a method for each function of the server (we require an implementation of MessageBase for each method, which is put on another field of the RelayMessage). Each method can have parameters and require an implementation of MessageBase from the client (for reply-handling of the client).

Now that I think about it, this can even be used to implement a couple of MessageBase classes that are common throughout all systems and allow for the server to issue tasks without request from the clients... Last but not least, if the server should be flexible and scalable in functionality depending on the client (like the client specified what the server has to do), the server could implement a Messaging system where the client implements the functions, which the server executes (so basically the actor framework, but the actor is on the server).

Now please don't ask me how to implement all of this... It sounds like fun to me, but also like a lot of work :)
If this post makes no sense, forget it :D

Link to post
Share on other sites

EDIT: and also note, there's no casting of the Visitor at all, whereas in your description you say you need to cast it to the parent type, which I don't understand.

Thanks.  Your example is actually similar to what I am doing.  I will try to find some time to build an example to post.  The real project is already well over 2000 VIs and not something I could share here.

 

To answer your question, my architecture differs from AF in that I have a singleton object wrapped in a DVR that represents the communication buss allowing my processes (actors) to communicate.  This make it a flat message system and not a hierarchy like AF.  To add network messaging to a project, you can instantiate the system using a child class of the system object that adds the ability to store the local implementation of the 'network message execution class'.  Since the architecture has no idea what messages the project will need to execute, the value of this variable in the system class is the 'network message execution class' top parent with no methods.  The version for the project that contains the abstract methods is a child of this class.  So, when we write the execution code in the message, we read the variable (which is of type 'network message execution class' top parent), but cast it to the child with the abstract methods.  This then allows us to call the specific abstract method for this message.  Now, if this is executed in a specific system where the variable has been set to the grandchild class with the concrete implementation of the execute methods, then dynamic dispatch will call the grandchild method and we will get our desired execution behavior.

I just read that three times and I think it is as clear as I can make it.  Sorry if it is not.

 

If this post makes no sense, forget it :D

No worries.  I kinda follow your thinking (I think).  I will have to ponder this some more.

Link to post
Share on other sites

I should have entitled the original thread 'Decoupling LVOOP class based message systems that communicate across a network' or perhaps been even more generic and asked: 'How to best decouple a class from it's implementation?'

As stated, I am not interested in changing the whole architecture and abandoning message classes.  For the most part, they work very well and make for efficient and clean code.  But every architecture has its issues.

And serialization (as far as I understand it) really does not help anything because you still need the class on both ends to construct and execute the message.

I did not intend to jump down anyone's throat but if you re-read your responses, they seem a bit pushy instead of being helpful for solving the problem.  I would prefer to focus on the OOP decoupling problem and solve that then to 'pull out all the nails and replace them with screws'.

 

I think you read the words, but didn't understand the content. My writing style can seem adversarial sometimes but it is usually with the genuine intent to help or, in this case, stop you going off on a unicorn hunt.The point I'm trying to get across is you are chasing a unicorn and only because the solution must be LV classes. If you drop that requirement, it's just an ordinary horse and we have loads of them.

 

A better heading would have been. "How do I reconstitute a LV object from a string sent over a network like I can in Python, Javascript or PHP". That would have got you the unanimous response of "You can't!" and is the reason why your "decoupling" search will ultimately be a dead end when using classes.

 

Anyway. I said my bit. I've clarified and re-clarified and so its time for me to shut up with a final apology for coming across as "pushy".

Link to post
Share on other sites

The problem arises when you wish to send these messages across a network between two separate applications.  In most cases, these applications are not duplicates of each other, but rather serve completely separate purposes.  An example would be a client and a server.  The client needs to message requests to the server and the server needs to push data to the client.  For a process to send a message, it needs to package the inputs in the private data of the message class and then transmit it via the network transport (which can be implemented in multiple different ways and is not material to this discussion).  In order to construct the message, the sender will need a copy of the message class included in their application.  This will mean they will need to also load the class of the message recipient since it is statically linked to the message class within the method that executes the message.  And since that will trigger many other class dependencies to load, the end results is the majority of the classes in the recipient application will need to be included in the sending application.  This is not an optimal solution.

So, we need to find a way to decouple messages from their recipients but still be able to execute them.

 

Note: I was on vacation when this conversation was going on, and I haven’t read it in great detail.

 

I only use Command-Pattern messages in limited cases, favouring a variation on string-variant messages mostly.  But the framework I use can do Command Pattern, and can decouple the command from the “serverâ€.   So I made a test example.  In it, the Client sends a request (non-command-pattern for simplicity) for a data message (DataAA.lvclass).   As part of its reply “addressâ€, it specifies an “envelope message†(CommandAA.lvclass) that it wishes to receive the data message inside of.  The TCP Framework keeps the reply address in flattened form while on the server side, so the server never needs to load class CommandAA.  The “Do.vi†method of CommandAA.lvclass invokes the “Read.vi†static method of DataAA.lvclass.

 

post-18176-0-69546800-1407254804_thumb.p

 

The methods one needs to write are Write and Read for the Data message, and Do for the Command (with an optional method to add metadata to the Command if desired).  [aside: AQ proposed a slightly different form, where the Command class is a child of the Data Message Class; he would need an additional method to support building the Command from the Data message.]  

Command Pattern Uncoupling.zip

drjdpowell_lib_messenging-1.2.1.29.vip

drjdpowell_lib_cyclic_table_probes-1.1.0.8.vip

Link to post
Share on other sites
  • 2 weeks later...

Thanks for all the replies on this.  After some discussions with other DEVs at NI Week, I concluded that the way I am decoupling my network messages is the only practical solution for a command-pattern based message system.  And now this thread and those discussions are leading me to reconsider the command-pattern design altogether.  I still find it difficult to give up on because of its benefits and the fact that NI chose it for their AF architecture.  It must be good, right?  Well, I have started a different thread on this issue over here if you want to continue the discussion and give feedback.  I think we need to find a best practice for AOP designs if not a specific architecture.  Personally, I had to go down the command-pattern road myself to truly see where it led.

Link to post
Share on other sites
On 8/13/2014 at 7:47 PM, John Lokanis said:

 I concluded that the way I am decoupling my network messages is the only practical solution for a command-pattern based message system.  

 

Didn't like my "envelope" demo, eh?  Personally I would drop the command pattern for the data message (using standard Variant or Object messages instead) and just use command pattern for actual commands.   Then there is one class per message transfer, with server decoupled from client.

Edited by drjdpowell
Link to post
Share on other sites

It's not that I didn't like it.  I am still trying to decide what path to take overall.  I just want maximum simplicity and maximum functionality.  Somewhere they intersect and I hope to figure out where for my needs.

I did discuss the different options with Allen Smith at NI Week and the conclusion was for command pattern across a network, the abstraction (or interface) design was the cleanest or at least easiest to understand.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Similar Content

    • By Marko Hakkarainen
      I had some time to learn about new interfaces and finally I could implement my collection class as I had envisioned. I didn’t want to use iterable and iterator names, because I thought that would have been too bold a claim.
      The original version of the collection class was (and is) used as a collection of sequence steps. Each element can be either a sequence command (send message, wait timer, wait complete etc.) or another collection of commands (sub-sequence). That’s the reasons for the labels and search method. Otherwise it is just a fancy (Rube Goldberg) array.
      Next method is recursive and it steps through all elements in the collection. Execute is only method, which requires override.
      For now, it’s at least an exercise in new interfaces. I don’t know if it’s useful enough to be in the code repository, but I can polish it up if needed.
       
      --
      Marko H
      Certified LabVIEW Architect
      www.optofidelity.com
      Iterable Collection LV2020.zip
    • By Ryan Vallieu
      I have seemingly found an issue with the shipping example code for Nested Malleable VIs.  Another user has verified that he saw the same behavior in 2019.
       
      I am working through the examples and the presentation from NIWeek 2019.  In running the Lesson 2b code (C:\Program Files (x86)\National Instruments\LabVIEW 2019\examples\Malleable VIs\Nested Malleable VIs) I found the Equals.vi in the class was not being leveraged and the search failed.  When I went to my LabVIEW 2018 machine and ran the Lesson 2b.vi the code worked to find the element by correctly leveraging the in-class Equals.vi.
      One difference I see is that in the 2018 example the Equal.vi is in the example folder with the code, and in 2019 the Equal.vi has been moved to VI.lib - otherwise the code looks to be the same.  The Equals.vi code looks identical, and the calling VIM look identical.  I posted on the LabVIEW NI.com forum here: 
      https://forums.ni.com/t5/LabVIEW/LabVIEW-2019-Malleable-VIs-Shipping-Examples-Lesson-2b-Nested/m-p/3966044/highlight/false#M1129678
       
      I am trying to determine what may have broken or changed between the implementation in 2018 and 2019, visually the code looks the same.
    • By Zyl
      Hi everybody,
       
      I'm running into something I don't really understand. Maybe you can help me here !
      I've got a LVLIB that is used as an 'Interface': it exposes public VIs which wrap around public functions of a private class (see code attached) . The class is private because I want to force the users to use the 'interface' functions.
      In one of my interface VI, I create a DVR on the private class (Interface_init). The DVR is stored into a typedef (FClass_DVR.ctl) and this typedef is the 'reference' that link all the interface public functions.
      In TestCode.vi (which is not part of the lvlib and illustrates the standard code that a user can create to use my driver), I can call my public interface functions and link them without any problem.

      But as soon as I create an indicator on that reference (to create a state-machine-context-cluster for example), my TestCode VI breaks !

      The error returned is : This VI cannot use the LabVIEW class control because of library access scope. The LabVIEW class is a private library item and can only be accessed from inside the same library or libraries contained in that library.
      I understand that the class is private. But the DVR is contained into a public control. Using an In Place structure on that DVR into TestCode would not work, since the class is private. So why is the DVR control problematic at that point ? Creating it do not breaks any access protection...
      Am I missing something ?
      DVR Private POC.zip
    • By Brains
      Hi,
      Does anybody know the best way to make a copy of a byref object (open gds v4) at runtime and pass all the attributes values (including inherited attributes) to the new object?
      Thank you!
      Craig
    • By GregFreeman
      I currently have a project that I am refactoring. There is a lot of coupling that is not sitting well with me due to typedefs belonging to a class, then getting bundled into another class which is then fired off as event data.
      Effectively, I have class A with a public typedef, then class B contains ClassA.typedef and then class B gets fired off in an event to class C to be handled. Class C now has a dependency on class A which is causing a lot of coupling I don't want.
      For my real world example I query a bunch of data from our MES, which results in a bunch of typedef controls on the connector panes of those VIs. Those typedefs belong to the MES class. I then want to bundle all that data into a TestConfig class and send that via an event to our Tester class. But, now our tester has a dependency on the MES.
      I see a few ways to handle this. First is move the typedefs currently in the MES class, to the TestConfig class. The MES VIs will now have the typedefs from the TestConfig class on their connector panes, but at least the dependency is the correct "direction." Or, I can move the typedefs out of classes all together, but then I am not sure the best way to organize them. Looking for how others have handled these sorts of dependencies.
       
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.