John Lokanis

Thanks guys! It's been on my to-do list for years (even been on my performance review at work to get this done!). Finally got around to it. I must say it was harder than I expected. Mainly in the time constraints. The works is straightforward, but I seem to be getting slower in my old age... Looking forward to the new options for maintaining my CLA. May have to do an NI Week presentation every other year...

On second thought, is this really all that bad? Since they will not contain any data, how much performance penalty will there really be? I took a look at AQ's project you mentioned. While I am not sure I totally understand it, it does seem to have one thing in common with mine: every message is a class. Is this a bad idea? Is having a large class hierarchy a bad thing if the hierarchy is what controls the execution of your application?

Good point. I will need to find an alternate solution for that. This is all good feedback. In the end, I might abandon this architecture but I have definitely learned some things along the way. Another problem I discovered is the need to pass the channel class object through all the methods on its parent's wire type. Not sure why but it would not let me do it any other way. And as a result, I cannot use the accessor methods on the channel data without casting it to the channel class type. So, this looks like a dead end.

Why would I want to bring the messages from one channel over to another? The whole point of this is each channel responds to unique messages. If you create a new channel, you then need to create the unique messages for it. Is there a better solution to eliminate the need to modify the architecture to extend it? This is easily remedied. All I need to do is pass the channel object thru the execute and store it in a shift register. Here is an updated version of the project that does this. MessageQ.zip Now, if I need state data preserved on a channel, I can add the variables to the channel's private data and create accessors that the message classes can use. Yes, I tested this and see I am stuck with the current implementation.

I am wondering if I can't simplify this code some more. If I pass a child object on a parent wire to a create queue function, will I get a queue of type child or parent?

I think where we are misunderstanding each other is in what a message is. My concept is a message is data and an action for a specific channel (process) to take. So, there would normally be no reason for two channels to use the same message, with the possible exception of a Stop or Quit command. And even in that case, each channel might need to have a special version of a Stop or Quit to do it's own cleanup. In my architecture, I have multiple parallel loops (processes) doing specific jobs. They each have their own unique set of actions. For example, one might be processing calls to a database. Another might be updating a UI display. Another might be launching plug-in VIs. And another might be receiving messages from the plugin VI and then messaging the other processes to write to the database or update the display or...etc...Perhaps if there were common commands, there would be a way to have those handled in the channel or MessageQ classes instead of each message. I will need to look into how that would work. One other thing I missed when looking at your non-LVOOP version: it does not support multiple versions of the architecture simultaneously. Since you are using named queues, if you have more than one copy of the architecture in memory at once, there will be cross-talk. I often (in the original non-LVOOP version of this architecture) need to have multiple instances of this code running in parallel and they must not know about each other. They also must be spawned from the same reentrant code base. Perhaps there is a way to eliminate the named queues in your code, but then we still have the issues of having to edit the VIs that support the architecture each time we want to extend it. So, on a multi-developer project, that can lead to some issues. In my version, I still need to edit the message classes to inherit from their proper channel, which means editing the project. I will need to look into a way around this so the messages become independent plug-ins to the project.

Here is how it works: The top level parent class (MessageQ) owns the messaging system. It stores the queues in it's class data. All channels must inherit from this class *and* override its methods. The channel class represents a single actor in your parallel architecture. It has the ability to receive messages from anyone within the application. It can also send messages to anyone within the application since it has access to the top level MessageQ class data. Additional channels can be created in your application by a simple Save As operation on an existing channel class. All messages (actions, functions, etc) are implemented as their own class. They must inherit from the channel class that executes them. If you have common code that more than one channel needs to call, you can easily wrap this code with a message class under each channel that needs it. The message classes only need to override the execute method of the parent. A message can have class data that is totally unique to itself. There is no need to ever modify the MessageQ class or the channel class when adding new messages in the future. You can also create new messages by doing a Save As on an existing message and then modifying it to meet the needs of the application. The Execute method of any message can send a message to any other channel (or even itself) by simply sending the destination message's object to the Send function. The developer does not even need to know what channel will execute the message. The class hierarchy figures that out automatically. So, to create an application using this architecture, you first determine how many parallel processes you need and create the channels. Then, as you add functionality, you create classes for each function. If you decide to add additional parallel processes later, simply duplicate an existing channel class and the process loop that uses it, then change the input object to the new class and you are done! Since the queue data is stored in the DVR in the top level MessageQ class, you can run multiple copies of your application in the same application instance without worry that they will interact. And, if you want other parts of your code to have access to all the channels, you simply pass them a copy of MessageQ object. And since this is a DVR, there is very little data copied. That is fine, but it fails to meet the design goals. You are using named queues. So, when you send a message, you must know the destination of the message and choose the right queue. Also, to add more messages, you must modify the execute function to add more cases. And, if you misspell the message command text, it will not work. So, it might be smaller, but it really is not the same architecture and does not have the advantages that LVOOP gives you. The hierarchy is what enforces the message path. This is a technique I learned in the LVOOP class that NI teaches. I would like to have a better way to choose the queue, but this seems the most obvious. Also, since I don't anticipate this architecture being used with larges sets of channels, I don't think there is must performance penalty here. But, if you have a better solution, I am interested. Keep in mind my target for this architecture is user applications where speed is not an issue. Again, this is by design and is a technique I learned in the LVOOP class. Why is this a bad idea? When you say each class should do one thing, I thought that is what it was doing. Each channel class sends and receives messages. Each message class acts on a message. Isn't this basically what the factory pattern does? I suppose one improvement would be to make a single channel class and them implement some way to create channels with a method in that class. My current design uses the channel class itself as the differentiator between the channels. Perhaps I could make a friend class to hold the common components of the channel class. But, I still need a way to differentiate channels from each other so messages automatically go to the proper process. Anyways, my original goal was to implement an architecture replacement for the common producer-multiple consumer design I have used in the past. And to eliminate the need to pass a cluster of queue refs to each consumer (forcing me to update all of them if I add a new consumer and its queue ref). Also, to eliminate the need to have a type-def controlled enum and variant as the payload for each consumer to control execution. Meaning I would have to update the enum to add a message. Which in turn changes the type of the queue, which changes the cluster of queue refs which causes another recompile of the whole project practically... Not fun when using source control... (I'm sure AQ is going to chime in any moment and tell me why this is all a horrible idea... )

Added and improved the first version. Added code to release the queues and the DVR on exit. Added better error checking. Added error cases to all methods. Moved create method call to process section of code. This demonstrates that processes can be placed anywhere in the application, as long as the original MessageQ object is created first and an initial write is performed to generate the DVR. -John MessageQ.zip

Ok, I think I solved it. Please take a look at the attached project and give me your thoughts. This seems like a nice clean way to implement cross messaging between multiple parallel processes using OOP to hide the queue refs. It also meets my requirements of ease of adding additional processes and messages without making changes to the parent classes. I especially like how the generic Send will adapt the the message type and automatically choose the proper channel based on the class hierarchy. I am sure I missed some details since this is my first attempt at OOP in LabVIEW. So, please let me know what I can/should do to improve this. thanks, -John MessageQ.zip

Nice. I will have to test this option out. I would still like to do this with only queues and no events but maybe that is not necessary.

Ok, here is what I am thinking: Create A Queue: Child (channel type) overrides parent dynamic dispatch method. In the child method, it gets the array of variant from the parent (must block updates). It then creates a queue of type 'child', converts this to a variant and then adds it to the parent's array, writing the result back and unlocking the parent's data. Get the Queue for a specific channel: Child overrides parent dynamic dispatch method. In the child method, it gets a copy of the array of variant from the parent (no blocking). It then creates a queue of type child and uses that to cast each variant in the array until it succeeds. It returns the found queue as a variant (since it cannot return it as the specific queue ref or the datatypes would not be consistent). Convert Queue Ref Variant to Actual Queue Ref: Each child needs to have a method that converts the variant of the queue ref to an actual working queue ref. This can be done similar to how we tested each queue ref variant in the Get method. This cannot be a dynamic dispatch method since it must have an output unique to the child's type. But each child class must include one of these. This will only be used inside the loop that responds that child's channel and since we know we are building this loop for this purpose, that is ok. Send Message: This will be similar to the Get Queue method. In this case, we are sending an object of type 'child of channel' or child of the child (grandkid). This needs to somehow dispatch to a method of the child (channel) that does something similar to the Get but instead of returning the variant, it enqueues the grandkid object in the proper queue for the channel and returns nothing. Obviously the sender needs to know the type of message and needs to call any init methods to setup the data in the message before passing it to the send. Execute Message: This will be a dynamic dispatch method that dispatches from the parent down to the grandkid, where the specific implementation for this message is contained. I realize this all sounds complex, but once it is constructed, I think it will scale well. You can add as many channels as you want. You can add messages (behaviors) by adding grandkid classes to any channel, without touching the code above it. The main top level VI will simply initialize a channel for each loop (could be done inside the sub-vi that holds the loop). The queue itself will be hidden except in the loop subvi where it needs to wait for a message. I suppose this could be put inside a method as well. so, where are the holes in my plan?

Where can I find an example of the Command Pattern? I will take a look at the code to see if this works. Definitely do not want to use named queues, however. Is there some way to have an array of queues of child objects and then create a method that I can pass in a child object and compare it to all the queues in the array to find the one that uses the same data type? I think this is the key and might not be possible. Here is why: 1. the array of queues will need to be stored in the parent as an array of variant since they types are incompatible. 2. The input to the 'find' method will need to be of type parent class so any child can be passed in, but then I need to cast it to the child type, create a queue of that type and then try to cast each variant in the parent's array to the new type of 'queue of child' until I find one that works. The problem is, I don't think there is any way to cast the object passed in to its child type without knowing what it is first! Some ideas are: try casting it to each child type, but how do you get a list of all child types and an example of each? Or, do you make the find method dispatch to the child, passing it the array of variant and have it return the correct item in the array? And then, how do you convert that to the proper child typed queue ref? Can a child access the array in the parent's datatype, use it in the dynamic dispatched code and then put it back in the parent array? this is getting quite convoluted...

I might be missing something, but those examples seem to demonstrate the Factory Pattern but do not address the messaging problem I am trying to solve.

How do I create a generic message passing class that allows me to map specific messages to specific receivers? Basically, think of one of those pneumatic message tubes that used to be used to pass paperwork around a large building. The tube is generic, but the message is specific and has a specific destination. The Problem: I have an application with 7 parallel loops. Each of these loops monitors a queue and when it gets a message it de-queues it and takes the appropriate action (executes the proper case). Each loop can send a message to the other 6 loops. To make this work, I create 7 queues in the top level application and then pass all 7 queue references to each loop (they are each in their own sub-vi). Also, each case in each loops uses unique data. So, the data type of the queues is a cluster of an enum (the message type) and variant (the data). In each case I have to cast the variant to the proper data type before I use it. The Solution: I want to have a single class hierarchy that can handle all messages. I want each loop to have its own child class. I want each message in each loop to be a child of the loop's child class so I can use dynamic dispatch to choose the action taken. I want a single class wire to be sent to each loop. I will initialize the parent object to create a ‘channel’ for each loop. This must be done by passing an object of the loop's child type to a create method. Inside each loop sub-vi, it will call a method to get the queue reference for the ‘channel’ it is supposed to respond to. It will do this by passing an object of the loops’ child type to the get method. When a loop receives a message, it will de-queue it and pass it to an execute method that will dispatch to the correct action. To send a message, a send method will be called and the ‘action’ object (initialized with any specific data) will be passed in. This will cause it to be placed in the correct queue for the loop that responds to that type of message. (The action object is an object of one of the loop's children) The result will be a clean way to pass messages around the application without the need to pass large sets of references to each sub-vi. Also, there will be no need to type cast the data from a variant or use enums to set the message type. To add new messages, I will just need to create a new child class of the loop’s class. No need to modify any of the rest of the class hierarchy or change any typedefs. The Question: Is this even possible? If so, how do I do it? I have been trying to figure it out for a while with little progress. It seems feasible but I am just learning LVOOP and am not sure if there is an existing design pattern for this or a way to combine existing patterns. Has anyone done something similar? Bonus feature: It would be great if this would work in parallel with multiple copies of the top level VI. In other words, all the communication channels would be unique to the top level caller and not store any data in globals of any type. Thanks for any ideas or help with this… -John

I don't know where you are going with this good vs bad thing. Decoupled code is not necessarily good and strongly coupled code is not necessarily bad. No one is trying to make those kind of judgments here. What we are trying to do is define what coupling is and what the benefits and pitfalls are. Most of the discussion has been surrounding what decoupling means. In that area, there has been some disagreement but I think we have come to a reasonable conclusion. For me, this has been helpful because I can now look at my designs and ask myself: "will decoupling offer any benefits to this project?" and "what steps will I need to take to decouple the UI from the rest of the code". Right now I have a strongly coupled design that I would like to change to a client-server architecture in the future, decoupling the UI from the functional code. From this discussion, I know that the best way to approach that is to define an API for the two to communicate that can be passed over a network. Now the fun part starts where I need to figure out how to best achieve that with the tools that LabVIEW offers. -John

I think this sums it up for me. Especially the bit about a defined interface. That is what I was going for. I would only add that the interface should be able to cross any boundary (interprocess, inter-application, internal network or internet) as dictated by the needs of the user.

What I really want to achieve is a single file that contains a group of plugins and all their dependent sub-vis. I call this a library of plug-ins. And I want to be able to have multiple libraries that are installed over time and as needed. I want a single EXE that can call any plugin in any of these libraries if provided with the name of the library and the plugin within. The libraries should be able to be built from the same pool of source code sub-vis but once built, they must own their version of the sub-vis so there can be no cross linking at runtime. This way, we do not need to re-validate the EXE or older libraries when we release a new one with potentially updated sub-vis. I can currently do this using .LLBs and the OpenG builder. I am looking for a native NI solution that uses LVOOP and lvlibs or ppls or ??. I just want to understand the methods before diving into the re-architecture.

BTW: don't use fade in on a splash screen if your app will normally be run remotely using RDP. For some reason, fades take a long time to transmit over RDP and the effect is mostly lost.

Yes. I do something like this right now with Web Services. These also do not natively support a version number. My solution involves running a pre-build action in the build spec. This calls a VI that uses scripting to modify the default value of an indicator in another VI in the project. So, each time you build, the pre-build VI increments the value. You can then implement some code in your project to return the value of this indicator when you need to query the version number programatically. Setup your build script like this: Then modify the attached files to work for your project. Pre-Build Action.vi Web Service Version.vi Hope that helps! -John

Any chance you can build an example project to demonstrate this? thanks, -John

The second one. I want to be able to develop modules that will be runnable in the LV environment without the presence of the exe's services. In otherwords, the module should be as decoupled from the exe as possible so that a simple wrapper can be used to allow it's execution. I need this so that many team members can create and test their modules without needing to run the main exe's code. Looks like I just need to caches an instance of each module class to hlod it in RAM. I will have to think about the rest of your responses and get back to you...

I am working on an architecture to support a dynamically loaded set of test modules using LVOOP techniques. Since I am new to LVOOP, I am not sure the best way to proceed. Here are some details: The exe needs to be able to call a test module from the external library using only the name of the library and the module. Assume that all libraries will be stored in a fixed location relative to the exe. The exe will spawn multiple parallel processes that could each call the same module at the same time. The module therefore must be completely reentrant to allow this with no blocking or cross contamination of state data. Each of the parallel processes is allow to call multiple copies of the same module simultaneously, if the module allows for this type of operation. So, each process must allow this with no blocking or cross contamination of state data. The exe needs to pass information to the module in a standard format to control its execution (parameters). The exe must be able to determine if the module is still running (or crashed). The module must be able to monitor a set of Booleans controlled by the exe in order to respond to abort or error flags. The module must be able to pass back messages to the exe while it is still running to update its status. The module must be able to pass back its results to the exe and those result must remain accessible and readable after the module has completed and left memory. The module must be executable outside of the exe for debugging and development purposes with minimal wrapping. Each library of modules must be packaged into a single file that removes diagrams, type defs, etc. Each library of modules much include all sub-vis calls by any module in the library and those must be name-spaced to prevent cross linking with similar named Vis from other modules in other libraries that might be in memory at the same time. This way a common set of source code can be used by all modules but is frozen to the version used at compile time (of the library) Once a module is loaded from disk and called by the exe, a template of it will remain in memory so that future calls will not incur a disk access as long as the exe remains in memory. My first inclination is to make a common ancestor class for all test modules. Each test module would then be a child class of this ancestor. I could then call the module by instantiating an instance of that class. The ancestor could then have must override methods for parsing the parameter inputs and for converting the modules output into a standard format for the exe to read. Not sure how to implement the flag monitoring or status updates. I really have no idea how to package the modules into libraries. Packed Project Libraries? Right now in my non-LVOOP implementation I use the OpenG builder to create an LLB with the same characteristics. Any ideas you can offer are appreciated! -John

As far as I know and have experienced, Open Reference in VI Server is always blocked by the root loop, regardless of what you pass to it. This makes for some interesting headaches if you have a system that uses dynamically loaded plug-in test modules that are opened as needed while a process runs. The user can effectively hit the 'pause' button by simply dropping down an menu and not making a selection. I have no idea how to work around the problem. Maybe one of the NI gurus can chime in...

Turns out it is local to my company's ISP. They say they are working on it. Thanks for testing for me!