managing multiple instrument drivers - best architecture?

[automationtx]

What is the the best way to organize code when your core program has to manage several instruments and device drivers? I need to have several instrument drivers up and running while my state machine operates. I have to talk to about 10 devices and coordinate their actions. I have to handle the case where I loose communications with some of them, in which case I need to be able to try to re-initiate communications. How do you handle this when you have to communicate with 100 devices? Surely this is not unique? I am looking for a modular approach so we can test and debug without too much going on in one vi.

It seem like this is the problem that LabView is made for, but I have not come up with a good architecture. It could be I just don't know how to phrase the problem correctly for a google or NI search! Thanks for any suggestions you can give!

[Tim_S]

Best is very dependent on what you are doing.

I'm guessing you are looking to communicate (and keep alive/monitor) devices in parallel to your main task. There's currently a thread discussing Producer/Consumer and similar architectures; it's a good read. That is only a suggestion for the instruments and not for the remainder of your system.

I also recommend you look at state machines as a general item to know about. I like to use them when I have a loop talking to an instrument, but that's my style and certainly not the only way to interface with instruments.

Talking to 1, 10, or 100 devices is a matter of scaling and PC resources. I find resources is a more limiting factor in there is only so much bandwidth in a RS232 or CAT5 connection, and each loop takes more processor and memory.

[todd]

This is exactly the kind of problem I've been trying to solve. I really like using classes, since the instruments must sometimes be swapped out. But I can't quite figure out where to keep the class wires so that they're available to the tests AND to some other processes (simulation, for example). Some are kept in AEs, which makes it easier to access them from tests with a set of AE wrapper VIs. Now I'm trying to instantiate an AE class in order to create multiple AEs - but that brings back the original problem of where to keep that class wire.

[Daklu]

Thanks for any suggestions you can give!

I doubt this is the kind of suggestion you're looking for, but I recommend hiring a consultant for a week or so. The questions you're asking can't really be answered without a much better understanding of the requirements and constraints.

[automationtx]

This is exactly the kind of problem I've been trying to solve. I really like using classes, since the instruments must sometimes be swapped out. But I can't quite figure out where to keep the class wires so that they're available to the tests AND to some other processes (simulation, for example).

Have you found an example anywhere? I've found it very odd that the most complex example out there just talks to two drivers, and does not do it very well. Meaning, no restart ability ect

[Daklu]

But I can't quite figure out where to keep the class wires so that they're available to the tests AND to some other processes (simulation, for example).

I'm not quite sure what you mean by "where to keep the class wires." Are you using the instruments in the tests and simulation at the same time?

I've found it very odd that the most complex example out there just talks to two drivers, and does not do it very well. Meaning, no restart ability ect

There's nothing particularly difficult with writing auto reconnect code, though it isn't something I'd put directly in the instrument class. Just give the class a way to report the connection has been dropped and let the calling vis be responsible for monitoring the connection and reestablishing it when you want. There are two basic ways you can handle connection monitoring: Inline or as a parallel process.

Inline--meaning the connection is checked as part of retrieving a data point--is easier to implement, but if a connection is dropped you won't know it until you try to get a data point. Doing the reconnect operations directly in your data processing loops may cause unacceptable delays depending on the project specifics.

Parallel process connection monitoring is just a separate loop dedicated to the instrument that periodically checks to see if the connection is still valid even if you're not actively collecting data. When it discovers the connection has been dropped it will attempt to reestablish it. That way the reconnection process doesn't cause your data processing loop to hang. You might be able to use a by-ref class if the instrument's device driver supports it. Personally I prefer to stick with by-val classes and just route all data exchanges with the instrument through that loop.

[todd]

Have you found an example anywhere?

No. For the last few applications, I managed to keep the code modular enough that refactoring was basically painless - but I can see a ceiling to the scalability. Reading the discussions, here, of everyone's design patterns has helped immensely. If you have time, create a bunch of classes with fake instruments and do trial-and-error. If you have money, follow Daklu's advice.

I could not find suitable examples, either. The hardware abstraction examples are wonderful for teaching inheritance. I get the feeling that people who use LabVIEW to develop applications have their own methods, and that people who use it develop one-panel data-grabbing VIs are overwhelmed by (and don't need) complex examples. A few experts have shown me their code, and it seems to me that it is more complicated than it needs to be - lots of upcasting and downcasting, etc. One gentleman has a base class called Instrument from which all other instrument classes inherit. This allows him a huge amount of flexibility, but it is hard for me to keep it all in my head. (What's the G equivalent of the old Linux book's quote: "400 lines of code is all most people can keep in their head at once"? 400 VIs? 400 classes?)

My plan of attack, at the moment, is a single cluster of Somethings that can be accessed by name in any test (and in an emulator). Each Something is an Action Engine class, and includes an instrument class in its private data. Create the single cluster at the top-level and pass it around to the tests. Perhaps this is easier to do with some of the messaging systems mentioned on LAVA, but I'm not there, yet.

Plenty of words from me, but no code, this time. A VI's worth 400 lines.

[Daklu]

One gentleman has a base class called Instrument from which all other instrument classes inherit.

Interesting. I'd love to see what he did. When I started LVOOPing I tried something similar because it seemed like the "right" way to do it. In the long run I found myself fighting against the limitations of inheritance instead of benefitting from it. Unless the instruments are very similar I don't see much point in putting them all in the same class hierarchy, especially if it's intended to be reused across multiple applications.

This allows him a huge amount of flexibility, but it is hard for me to keep it all in my head. (What's the G equivalent of the old Linux book's quote: "400 lines of code is all most people can keep in their head at once"? 400 VIs? 400 classes?)

It varies depending on complexity, but I can only keep details of maybe a half dozen block diagrams in my head at once. (Or less than 1 QSM block diagram.) Well-named and well-segregated sub vis are your friend.

[drjdpowell]

What is the the best way to organize code when your core program has to manage several instruments and device drivers? I need to have several instrument drivers up and running while my state machine operates. I have to talk to about 10 devices and coordinate their actions. I have to handle the case where I loose communications with some of them, in which case I need to be able to try to re-initiate communications. How do you handle this when you have to communicate with 100 devices? Surely this is not unique? I am looking for a modular approach so we can test and debug without too much going on in one vi.

One thing to think about is the possibility of organizing instruments into "subsystems", so the higher level program controls subsystems and each subsystem deals with a limited number of instruments that work together. Then no part of the program is trying to juggle too many parts, and you can test the subsystems independently. It depends if your application can be organized into subsystems ("VacuumSystem", "PowerSystem"). I think that is called the "Facade Pattern" in OOP.

Personally, I usually find it worthwhile to give each instrument a VI running in parallel that handles all communication with it. Then any feature like connection monitoring (as Daklu mentioned) can be made a part of this VI; this can include things like alarm conditions, or statistics gathering, or even active things like PID control (the first instrument I ever did this way was a PID temperature controller). Think of this as a higher-level driver for the instrument, which the rest of the program manipulates.

You can use a class to represent or proxy for each subsystem or instrument; this class would mainly contain the communication method to the parallel-running VI. Daklu's "Slave loop" is an example. You can either write API methods for this class (as Daklu does) or send "messages" to this object (which I've been experimenting with recently).

[todd]

I'd love to see what he did.

It was really slick. He could identify all instruments on several different busses/comm methods (including custom), show them all to the user, and allow them to select names for them all. Saved the config to a file for later automation. He's in Lynnwood.

Well-named and well-segregated sub vis are your friend.

Then I have made a lot of friends!

Okay, time to put some nodes where my mouth is. I know I've seen many discussions about this on LAVA (duplicate class wires make a new copy of the object), but I'm not ramped up, yet. All I can come up with is that I need a SINGLE VI that can access the specific instrument's class. Not there, yet.

Personally, I usually find it worthwhile to give each instrument a VI running in parallel that handles all communication with it.

...

Think of this as a higher-level driver for the instrument, which the rest of the program manipulates.

You can use a class to represent or proxy for each subsystem or instrument; this class would mainly contain the communication method to the parallel-running VI. Daklu's "Slave loop" is an example. You can either write API methods for this class (as Daklu does) or send "messages" to this object (which I've been experimenting with recently).

Yes! This is what I've converged upon. But I have resorted to using named queues with the "standard" string+variant to handle all of the instruments' handles/actors/slaves. I'm searching for a method that, for lack of a better term, "feels better than" passing commands via named queues. For example, some instruments have a method that gathers streamed data and adds it to an AE. Other instruments just have simple read/write access. Just haven't unified them, yet - not for the sake of unification for its own sake, but for ease of use of the reusable API.

[todd]

Here's what I've been mostly using - except that the two lower while loops are in subVIs, and they each have Init or Open cases that store an instrument class in a USR. Also, I name the queues and create a helper VI for each queue/loop that obtains the named queue and enqueues elements.

One problem I'd like to solve is the most useful way to return data (haven't wrapped my head around lapdog or anything, yet). Sometimes I put it in a "global" data AE in a parallel loop in the instrument's handler VI. Also needed is a cleaner (more debuggable) method of talking to the instruments' queues inside test VIs and from the simulator (a separate loop at the top level that listens to another set of buttons and sliders - so the water-spraying can be tested without first setting fire to the city).

Another problem is "instantiating" multiple instances of an instrument's loop without duplicating code. I've "classed" the instrument loop (several of the same kinds of instruments behave differently), but that pushes the instrument class creation into the top-level loop, which I don't like. I'd like to create the object in the top-level and send it as the variant to the instrument loop, except the instrument loop is a method of the class...

Robot Usage.vi

[Daklu]

You can either write API methods for this class (as Daklu does) or send "messages" to this object.

The Slave Loop post is a little outdated. I've since decided wrapping the input queue in class methods is an optional feature of slave classes. It's best used in special situations where you need to either protect your slave's input queue or need to assure messages are typed correctly. It's kind of tedious having to change them frequently during development. If I'm the sole owner of the code I'll expose the slave's input queue as an output of the creator method.

But I have resorted to using named queues with the "standard" string+variant to handle all of the instruments' handles/actors/slaves. I'm searching for a method that, for lack of a better term, "feels better than" passing commands via named queues.

There's nothing inherently wrong with standard string/variant messages. Named queues, on the other hand.... *shudder* A couple easy ways to eliminate the named queue: You can pass the class' input queue as part of its creator/constructor. Or you can create it internally as part of the constructor and wire it to an output terminal on the creator. I prefer passing it out because it reenforces the idea that the slave owns the queue, not the calling vi.

Here's what I've been mostly using - except that the two lower while loops are in subVIs, and they each have Init or Open cases that store an instrument class in a USR. Also, I name the queues and create a helper VI for each queue/loop that obtains the named queue and enqueues elements.

Pfft... see? You're already using slaves. Slave classes are just a way to encapsulate a parallel process to help manage the complexity. They have 3 core methods: Create, Execute, and Destroy. Your looping sub vi is the same as my Execute method. I use the Create method to make sure the object has all the information it needs to start executing. (Queues, etc.) Then instead of using a USR for maintaining state you store it as private class data.

One problem I'd like to solve is the most useful way to return data

You already have a queue going from the robot loops to your controller. Why don't you just send it as a message?

(haven't wrapped my head around lapdog or anything, yet).

There's a common perception LapDog.Messaging is really complicated that I don't quite understand. It's almost identical to standard string/variant queues, except the msg/data pair is boxed in an object. (Well, that and the automatic messages I included as part of the Dequeue method.) I guess the confusing part might be the downcasting requirements? That's just an OOP equivalent of a Variant to Data function (in this particular framework.)

Out of time... maybe I'll be able to post more later.

[todd]

Named queues, on the other hand.... *shudder* ... Or you can create it internally as part of the constructor and wire it to an output terminal on the creator.

Yeah, named queues haven't bit me, yet - but I've been making single application instances, so far. I like it the idea of sending a slave queue out - trying it now with non-OOP Create + Execute. Baby steps.

Pfft... see? You're already using slaves. ... Then instead of using a USR for maintaining state you store it as private class data.

Yes, exactly! Just have to figure out how to transport their control across BD lines. I'm picturing having the Execute method hold the class (with its private data) in a shift register - example below.

You already have a queue going from the robot loops to your controller. Why don't you just send it as a message?

Because I haven't figured out how to send all instruments' classes into invoked test VIs - although now I'll try bundling up all their message queues and passing them in. I like this idea because it allows me to send that bundle to the simulator loop, too!

There's a common perception LapDog.Messaging is really complicated that I don't quite understand.

It's not difficult, now that I've used it. It's just a matter of taking the time to build up an app - starting from your example. Little extra for filling in error handling, etc. I'm still at the point where I'm trying to figure out why you wired a loop's error out to the error in of ReleaseQueue.vi, and why errors are in SRs.

Here's my initial attempt. I used LapDog - just to keep your attention. I haven't yet figured out where to put the Create and Execute VIs (not yet classed, as mentioned above). Perhaps a "dispatcher" loop at the top-level that receives Create-required information and spawns Execute loop VIs - don't know, yet.

Sample Project.zip

[drjdpowell]

Yes, exactly! Just have to figure out how to transport their control across BD lines. I'm picturing having the Execute method hold the class (with its private data) in a shift register - example below.

...

Because I haven't figured out how to send all instruments' classes into invoked test VIs - although now I'll try bundling up all their message queues and passing them in. I like this idea because it allows me to send that bundle to the simulator loop, too!

...

I haven't yet figured out where to put the Create and Execute VIs (not yet classed, as mentioned above). Perhaps a "dispatcher" loop at the top-level that receives Create-required information and spawns Execute loop VIs - don't know, yet.

Hi todd, I'm having a hard time following what your doing. It sounds complicated, but things always sound complicated when one is can't tell what's going on. I'm not sure what exactly you are "classing" and calling your "instrument. So let me sketch out how I see things.

The design I normally use is this:

There is a communication reference (often a queue) that allows communication with a loop (on diagram or in a subVI). This loop contains information about, and methods to act on, a real-world instrument.

Now, there is more than one place to use a LVOOP class in the above design, and more than one thing that can be referred to as "the instrument class". The place I think your using is the latter part: the information about, and methods to act on, the real-world instrument. That's a perfectly good place to use a class, the most obvious place, and useful for various reasons, but I would never want to use that class as "the instrument" in the higher-level code. The entire structure is "the instrument", and the communication reference out front is a "reference to the instrument".

If you look at my designs, the LVOOP classes with instrument names are actually just wrappers of a communication reference (or references). And these are effectively by-reference objects; they can be easily passed around and used in multiple places (no problem with branched wires). They serve as a proxy for, and encapsulate, the entire structure of communication, loop, data/methods, and real-world hardware. To interact with the "instrument", I send a message to this proxy object (or call a class method that sends a predefined message), which passes the message to the loop, which performs the appropriate action on the internal data or communicates with the real-world instrument.

If I have second, by-value class for the instrument data and methods, it is accessed only internal to the loop, and never "passed across BD lines". If your trying to pass this by-value object between different loops, that sounds like a rather complicated and troublesome thing to do.

-- James

Edited October 22, 2011 by drjdpowell

[todd]

It sounds complicated

"If I can't explain it simply, I don't understand it well enough."

There is a communication reference (often a queue) that allows communication with a loop (on diagram or in a subVI). This loop contains information about, and methods to act on, a real-world instrument.

Yes! What is that kind of loop called? I call it the Instrument Loop. "Driver" almost works, except that term tends to be reserved for the lowest-level functions. "Actor" almost works, given AQ's definition of it as state data in a module - however, the loop itself is more like his Actor Core.vi instead of the Actor.vi which spawns it. "API" almost works, because the messages handled by that loop (which are, again, the ONLY way to act on the real-world instrument) are the only messages allowed - except the API is actually the set of VIs that wrap possible messages. In my code so far, the Instrument Loop carries an instrument's class in a shift register. The instrument's class provides the inherited methods (such as "Take a Measurement" and "Set Scan Rate") that call some level of the instrument's driver.

Now, there is more than one place to use a LVOOP class in the above design, and more than one thing that can be referred to as "the instrument class". The place I think your using is the latter part: the information about, and methods to act on, the real-world instrument. That's a perfectly good place to use a class, the most obvious place, and useful for various reasons, but I would never want to use that class as "the instrument" in the higher-level code. The entire structure is "the instrument", and the communication reference out front is a "reference to the instrument".

Please pardon my lack of clarity, and thank you for taking the time to correctly suss things out. One note that is beside the point: I do make the Instrument Loop VI a method in a class, because several of the real-world instruments require loops that are parallel to the Instrument Loop (similar to the Image Display Loop in daklu's image here: http://lavag.org/topic/15089-dequeue-element-timeout-state-control/page__view__findpost__p__90891). Also, each Instrument Loop has a "Show" message so it's front panel can be brought up - so the Instrument Loop also has a paraller User Event structure loop (which only uses the available Instrument Loop message API. But I still consider those parallel loops part of the Instrument Loop functionality.

If you look at my designs, the LVOOP classes with instrument names are actually just wrappers of a communication reference (or references). And these are effectively by-reference objects; they can be easily passed around and used in multiple places (no problem with branched wires). They serve as a proxy for, and encapsulate, the entire structure of communication, loop, data/methods, and real-world hardware. To interact with the "instrument", I send a message to this proxy object (or call a class method that sends a predefined message), which passes the message to the loop, which performs the appropriate action on the internal data or communicates with the real-world instrument.

If I have second, by-value class for the instrument data and methods, it is accessed only internal to the loop, and never "passed across BD lines". If your trying to pass this by-value object between different loops, that sounds like a rather complicated and troublesome thing to do.

I've only been back on these forums for the last few months, and have been looking at every VI I can find - ramping up on inheritance. Now that the terminology is better defined - at least in this thread - (Instrument Loop is the only place to communicate with a real-world instrument; API is the set of messages that can be sent to the Instrument Loop), I can be clear about the problem:

Think about the one-panel VIs many people start with: open a VISA session, initialize an instrument, take several measurements, save data to file, close the VISA session. Now imagine a top-level VI like the one in Daklu's post mentioned above. Add a button to the front panel called "Run Test 1". The User Event structure tells the Mediator Loop to run a one-panel VI (invoke node, do not wait for completion). I want to send the by-reference message queues to the one-panel VI so it has access to the real-world instruments via the Instrument Loop's API. I also want the by-reference message queues to be available to another top-level VI so messages can be injected (typically used for simulating instruments that are not present, or that don't have fine-enough controls for checking the test's logic).

[todd]

This works, so far:

Now to create instrument APIs that can be called in the tests (wrap the message enqueue in a VI, as James mentioned), and see how those work for returning data to the test. If that works, on to OOPifying the Loops and seeing if they work as reentrant, for multiple of the same "class" of instrument.

[Daklu]

Just have to figure out how to transport their control across BD lines.

James already commented on this, so I'll repeat what he said in a slightly different way.

Broadly speaking there are two ways of sharing a resource among parallel processes: The first is using references. You share the object directly using DVRs, FGs, globals, etc. These are effectively forms of object references. If the "reference" is maintained internally within the class, it is a "by-ref" class. GOOP, G#, and possibly some other frameworks appear to be built around by-ref objects.

The other way is to share access to the resource. In other words, have the object executing in it's own process and communicate with it via the messaging transport it exposes. I almost always use a queue, but it depends on the intent of the specific class. The Actor Framework and my coding style are based on this kind of sharing. AQ used the term "handle" for the queue. I'm inclined to stick with that instead of overloading the word "reference."

AQ pointed out the difference recently but I don't have the link at hand. Reference objects methods are synchronous and execute in the same thread as the calling code. "Messaging" objects (for lack of a better word) are asynchronous and execute their methods in a separate thread. It's much more event-oriented and a bit of a shift in thinking about how to approach the problem.

If I have to share an instrument among parallel processes I'll wrap the instrument class (which is usually a simple wrapper around the manufacturer's api) in a slave loop. I may or may not put the slave loop in a class. The slave loop api--the messages it can act on--is designed according to the amount of detail the calling loops need. Ideally they will only need a couple high-level messages that wrap a bunch of low-level details, but it's not always the case.

I'm still at the point where I'm trying to figure out why you wired a loop's error out to the error in of ReleaseQueue.vi, and why errors are in SRs.

The loop's error out is wired to ReleaseQueue for sequencing. I don't want to release the queue until the loop exits but since the MessageQueue doesn't propogate through the loop that leaves me with the error wire. I could use a flat sequence (and sometimes do if there's a lot of clean up code) but it's not necessary in this case and just clutters up the diagram.

Errors are in a SR because the Dequeue method looks for errors on its error in terminal. When it finds one, instead of dequeuing the next message it packages up the error in an ErrorMessage object and spits it out in a "QueueErrorIn" message. Then the error handler case is just like any other message handler. Some people put an error handler vi inline after the case structure. Nothing wrong with that. I prefer this way because it puts my error handling code on the same level of abstraction as the rest of the message handling code and it gives me a more coherent picture of what the loop is doing.

Overall my approach to error handling in slaves may be a little different than what is common. I don't terminate the loop on errors. That's fine for quick prototypes but I don't like it for production code. Unless you're dynamically launching your execution loops, terminating a loop is an unrecoverable error; the app has to be restarted. Good applications will gracefully self-terminate when unrecoverable errors occur rather than let the user unknowingly continue with incomplete functionality, so usually there's a cascading trigger mechanism that stops all the loops when one of them stops.

But I find that behavior rather inconvenient during active development. Lots of times I want the execution to continue so I can repro the conditions that cause the error. If the error isn't handled locally in the slave loop it propogates up to its master as a message, which can choose to act on it or pass it up the chain. Eventually someone decides to do something with it. Right now unhandled errors are eventually converted to debug messages containing the source and summary of the error and displayed in an indicator on the fp. Later on in development I might convert that to a stored error log.

My slave* loops don't get to decide on their own when to stop. The slave exits only when the master sends it an Exit message. (Exceptions noted below.**) Here's how my apps might respond to a low-level error that requires termination:

1. A terminal error occurs in a low-level slave loop.

2. If necessary the slave puts itself in a safe state, then sends a RequestShutdown message to its master.

3. The RequestShutdown message is propogated up through the app control structure (master-slave links) until it reaches the top of the chain--the Grand Master.

4. If the GM agrees termination is appropriate it sends Exit messages out to its immediate slaves. Those slaves are in turn masters of other slaves, so they copy the message for their slaves. This repeats so all slaves receive the message. The masters do not terminate yet.

5. When a leaf slave (those slaves that are not also masters) receives an Exit message it cleans up after itself and terminates the loop. The final act by any slave loop is to send an Exited message to its master.

6. Once the master receives Exited messages from all its slaves it terminates and sends its own Exited message to its master. Eventually the GM receives exited messages from all its slaves and terminates itself.

*[This is my preference. I don't consider it part of the definition of a slave.]

**[There are two exceptions to this rule. 1) If the slave's input queue is dead someone else has incorrectly released it. Since it cannot receive an Exit message the slave should terminate itself. 2) If the slave's output queue is dead the master forgot to send the slave an Exit message before terminating. Since nobody is able to listen to the slave's outputs it should terminate itself.]

Seems like a lot of extra work, doesn't it? One of my rules for master loops is they do not terminate until all their slaves have terminated. (With the same two exceptions noted above.) This rule grew out of my exploration with Active Objects a couple years back and problems with trying to shut down dynamically launched vis during active development. Eventually I decided most of my requirements were better addressed by statically launched parallel loops, but I kept the master requirement for a couple reasons.

1. It's easy to define a controlled shutdown process, regardless of the reason for the shutdown.

2. During active dev, when a loop doesn't shut down correctly it's fairly easy to find the offending loop. It's the one that didn't send an exit message, and since I can define the UI to be the last thing that terminates I'm not left guessing whether or not the messages were sent.

3. If I want to dynamically instantiate multiple objects at runtime it's dirt simple to convert it to an actor style object.

Here's my initial attempt. I haven't yet figured out where to put the Create and Execute VIs (not yet classed, as mentioned above).

The thing that immediately jumps out at me is you'e wired a MessageQueue constant into the Digitizer Execution Loop. That won't work. You have to use the Create method. I realize you probably did that so you could run the vi, but I wanted to make sure the requirement was clear.

Your simulated and custom Digitizer classes inherit from the abstract Digitizer Base class. In general I have discovered abstract base classes tend to get in the way unless you have very specific and well defined reasons for using them. I make better progress by starting with a concrete class. I'd just use the Custom Digitizer class and subclass the Simulated Digitizer from it. Others probably have different opinions.

Later on if you need other digitizer classes with very similar behaviors you can make them subclasses of the Custom Digitizer as well. If the other digitizers implement different behaviors you can then subclass them all from a common parent, or create a separate class hierarchy and delegate to the digitizer classes. (I usually like delegation better.)

Regarding where to put the Create and Execute vis, creating a DigitizerSlave class too early can slow down development because you're shuffling back and forth between the two block diagrams to make sure all your message names and message types stay synched up. I usually start the slave loop on the same block diagram as the master and push it down into a slave class if the need arises. (Though dynamically launching the execution loop does qualify as "need.")

[todd]

Ahh - it's nice to be able to follow 99% of your post! (Not sucking up - just happy that I'm getting it.)

By the way, thanks for your time.

Broadly speaking there are two ways of sharing a resource among parallel processes

I finally understood this last week - helped along by Evil AQ. My next hurdle was/is making it happen, and sharing the interface with a few other VIs.

The loop's error out is wired to ReleaseQueue for sequencing.

That makes sense. I've used an error cluster constant just inside the while-loop for that sequencing.

Errors are in a SR because the Dequeue method looks for errors on its error in terminal. When it finds one, instead of dequeuing the next message it packages up the error in an ErrorMessage object and spits it out in a "QueueErrorIn" message.

Ah! I missed that. I like it! I used to put a dequeue timeout case that just checked for errors on the SR.

Overall my approach to error handling in slaves may be a little different than what is common. I don't terminate the loop on errors.

I've just been throwing in Simple Error Handler.vi, during development. I like the possibility of logging errors after deployment, though.

The thing that immediately jumps out at me is you'e wired a MessageQueue constant into the Digitizer Execution Loop. That won't work. You have to use the Create method. I realize you probably did that so you could run the vi, but I wanted to make sure the requirement was clear.

Yes, quite clear. I was coding out loud. I'm not yet unhappy with the two-way notification shown in post #16 above.

Your simulated and custom Digitizer classes inherit from the abstract Digitizer Base class. In general I have discovered abstract base classes tend to get in the way unless you have very specific and well defined reasons for using them. I make better progress by starting with a concrete class. I'd just use the Custom Digitizer class and subclass the Simulated Digitizer from it. Others probably have different opinions.

Later on if you need other digitizer classes with very similar behaviors you can make them subclasses of the Custom Digitizer as well. If the other digitizers implement different behaviors you can then subclass them all from a common parent, or create a separate class hierarchy and delegate to the digitizer classes. (I usually like delegation better.)

The instruments I use can vary wildly (in many ways) over their development cycle, and I need to support each version. The abstract class helps me flesh out the simplest methods to expose, and to drop the simulated one in place so I can write tests before the instrument arrives, or before announced functionality is added to the real metal. I didn't really plan it that way, but I'm happy with it, so far. (I haven't seen delegation - time to study.)

Regarding where to put the Create and Execute vis, creating a DigitizerSlave class too early can slow down development because you're shuffling back and forth between the two block diagrams to make sure all your message names and message types stay synched up. I usually start the slave loop on the same block diagram as the master and push it down into a slave class if the need arises. (Though dynamically launching the execution loop does qualify as "need.")

I usually start loops that way, too, just to see what inputs they need when they're sub-VIed (before I have a standard for the current project). I have enough screen real estate to have them all open, and I have several existing loop objects that I will massage into the new message standard.

Thanks, again!

Edited October 22, 2011 by todd

[Daklu]

You slipped a couple posts in there... I guess I'm getting too long-winded.

Yes! What is that kind of loop called? I call it the Instrument Loop.

I agree the other terms don't quite fit. Sometimes people just call them threads, but that creates confusion between LV threads, OS threads, and CPU threads. Besides that it doesn't relay any information about the intent of the loop. Instrument loop is as good as anything... at least for this thread.

I also want the by-reference message queues to be available to another top-level VI so messages can be injected (typically used for simulating instruments that are not present, or that don't have fine-enough controls for checking the test's logic).

I'm not quite following this. When the app starts your instrument loops start running. The tests are top-level vis that are dynamically launched when users request it. You pass the instrument loop's input and output queues to the tests so they each can communicate with the instrument loop. You also plan on creating other top level vis to simulate instruments and stuff. Did I get that right?

Potential problems:

First, each message on a queue can only be dequeued once. How are you going to get messages from each instrument to all the top level vis that want the information? You have to create copies somewhere along the line. You can make the instrument loop observable and have other loops register their own input queues with it. Or you can have the instrument loop send all it's messages to another loop (mediator,) which in turn creates copies and sends them to the receivers. I prefer to split the functionality and keep each loop's responsibilities separate and distinct.

Second, the idea of injecting an instrument's queues into a top level vi so the vi can simulate the instrument seems backwards to me. Isn't that why you have a Digitizer Simulated class? Subclass your simulated instruments directly from the instrument they're simulating. I have a simulated Stage object I use. When the creator executes it launches a front panel allowing me to manipulate encoder feedback and see how the app responds. It's great since most of my dev work is done away from the hardware. Better yet, switching between simulated and real hardware is as easy as replacing the class creator.

Third, I'm curious how you are preventing tests from interfering with each other. If tests a, b, and c all send messages to instrument 1, are you running the risk of having the tests issue conflicting commands?

The instruments I use can vary wildly (in many ways) over their development cycle, and I need to support each version.

I see.... would you be at 'F' in Everett? I spent several years writing test software to support hardware development activties. It's a challanging environment. Speed tends to take priority over everything else and it's hard to convince managers of the long-term benefit of good code.

[drjdpowell]

I've only been back on these forums for the last few months, and have been looking at every VI I can find - ramping up on inheritance. Now that the terminology is better defined - at least in this thread - (Instrument Loop is the only place to communicate with a real-world instrument; API is the set of messages that can be sent to the Instrument Loop), I can be clear about the problem:

Think about the one-panel VIs many people start with: open a VISA session, initialize an instrument, take several measurements, save data to file, close the VISA session. Now imagine a top-level VI like the one in Daklu's post mentioned above. Add a button to the front panel called "Run Test 1". The User Event structure tells the Mediator Loop to run a one-panel VI (invoke node, do not wait for completion). I want to send the by-reference message queues to the one-panel VI so it has access to the real-world instruments via the Instrument Loop's API. I also want the by-reference message queues to be available to another top-level VI so messages can be injected (typically used for simulating instruments that are not present, or that don't have fine-enough controls for checking the test's logic).

Hi todd,

I can't comment on everything you and Daklu have been discussing, but let me make some comments on your design so far.

I would suggest that, whatever design for a parallel process you decide on, use the same one for both "Instruments" and "Tests". They are both "actors"/"active objects"/"slaves"/whatever. Currently, you have a totally different method for running tests (dynamic launch after "Set Control Value") and running instruments (class method on-diagram), which means you basically have to debug two different, quite advanced and complex, designs. Settle on one design that works for both and perfect it.

For example, I've been developing a design I call "Parallel Process". I have a parent class that contains the "Launch" (Create) VI that dynamically launches a VI, sets it up with a command messenger (message queue) and wraps that queue inside the parent object. All my... let's standardize and call them "actors"... inherit from this class (I have a few templates that I copy from). As each actor is standardized, each is created by the same parent "Launch.vi", and once that VI is debugged I never have to worry about the details of dynamic launching. And the child actor classes are very simple; generally with no additional private data items, no additional methods or overrides, and only one VI (the one that is dynamically launched) copied from a template. The complicated stuff is all in the parent class.

Note that once you have the basics of a VI dynamically launched and running in parallel, sitting behind a command queue, you have all the ability to customize it. Your "Create Test.vi" could (after launching) send your Test a message containing a cluster of "Instrument" proxy objects (ie. objects that contain the command queues of your instruments). Your "Create InstrumentA.vi could send messages containing the information needed to configure InstrumentA.

Regardless of what design you go with, try to get complicated stuff like dynamic launching and messaging standardized into clean reusable components (such as in a reuse library or a parent class) so that you can deal with the complexity once and then not have to worry about again.

A second comment I have is about running multiple "Tests" in parallel, which I get the impression you're trying to do. If that is the case, it may be a good idea to add the ability for a Test to "lock" an instrument so that it can't be simultaneously used by other Tests, to prevent potentially dangerous conflicts. This is something for which a parent "Instrument" class (child of your "Actor" class) might be useful. The locking could use a semaphore or similar feature contained in the parent class.

-- James

Errors are in a SR because the Dequeue method looks for errors on its error in terminal. When it finds one, instead of dequeuing the next message it packages up the error in an ErrorMessage object and spits it out in a "QueueErrorIn" message. Then the error handler case is just like any other message handler. Some people put an error handler vi inline after the case structure. Nothing wrong with that. I prefer this way because it puts my error handling code on the same level of abstraction as the rest of the message handling code and it gives me a more coherent picture of what the loop is doing.

I stole this idea it for my own messaging system within about 10 minutes after downloading LapDog. I would also note that Daklu treats timeouts in a similar way: outputting a "Timeout" message rather than having an extra timeout-boolean output. I really great idea.

-- James

Edited October 23, 2011 by drjdpowell

[todd]

You slipped a couple posts in there... I guess I'm getting too long-winded.

Or I'm being too impatient

I agree the other terms don't quite fit. Sometimes people just call them threads, but that creates confusion between LV threads, OS threads, and CPU threads. Besides that it doesn't relay any information about the intent of the loop. Instrument loop is as good as anything... at least for this thread.

They are both "actors"/"active objects"/"slaves"/whatever.

Actor Slave Object Loop: ASOL - everyone has one! Ha! (Nothing like sophomoric humor.)

When the app starts your instrument loops start running. The tests are top-level vis that are dynamically launched when users request it. You pass the instrument loop's input and output queues to the tests so they each can communicate with the instrument loop.

I'm just now "internalizing" that an output queue is actually required. I just have to massage my existing data queues to output data the same way as an immediate measurement.

You also plan on creating other top level vis to simulate instruments and stuff. Did I get that right?

Exactly - but just a single top level VI to control the simulated instruments.

Potential problems: First, each message on a queue can only be dequeued once. How are you going to get messages from each instrument to all the top level vis that want the information? You have to create copies somewhere along the line. You can make the instrument loop observable and have other loops register their own input queues with it. Or you can have the instrument loop send all it's messages to another loop (mediator,) which in turn creates copies and sends them to the receivers. I prefer to split the functionality and keep each loop's responsibilities separate and distinct.

I strive to make things as simple as I can. That said, I consider my current confusion a result of being mid-way through a programmer's continuous learning process that you mentioned in another thread, once.

Second, the idea of injecting an instrument's queues into a top level vi so the vi can simulate the instrument seems backwards to me. Isn't that why you have a Digitizer Simulated class? Subclass your simulated instruments directly from the instrument they're simulating.

And that is exactly the type of simple answer that I've missed. I shall do it this way.

I have a simulated Stage object I use. When the creator executes it launches a front panel allowing me to manipulate encoder feedback and see how the app responds. It's great since most of my dev work is done away from the hardware. Better yet, switching between simulated and real hardware is as easy as replacing the class creator.

Exactly my intent. I use an enum to choose the class that gets created.

Third, I'm curious how you are preventing tests from interfering with each other. If tests a, b, and c all send messages to instrument 1, are you running the risk of having the tests issue conflicting commands?

A second comment I have is about running multiple "Tests" in parallel, which I get the impression you're trying to do. If that is the case, it may be a good idea to add the ability for a Test to "lock" an instrument so that it can't be simultaneously used by other Tests, to prevent potentially dangerous conflicts.

At the moment, if an instrument is being used in one test, any other test that tries to use that instrument will throw an error. This is not entirely built out, but I consider it my responsibility to understand the complexities required to support this kind of flexibility.

I see.... would you be at 'F' in Everett? I spent several years writing test software to support hardware development activties. It's a challanging environment. Speed tends to take priority over everything else and it's hard to convince managers of the long-term benefit of good code.

PM about my location (with which you are familiar, according to local NI folks) is forthcoming. I live near 'F', though. So far, the development cycles have been beneficial in that I have time (and my manager gives me leave, as long as stuff works when needed) to iterate the test architecture - with the help of experts, here.

I would suggest that, whatever design for a parallel process you decide on, use the same one for both "Instruments" and "Tests". They are both "actors"/"active objects"/"slaves"/whatever.

...

Settle on one design that works for both and perfect it.

This is an intriguing concept, and I have been wondering how soon I would have to try to instantiate a class of test the same way as an instrument actor.

For example, I've been developing a design I call "Parallel Process". I have a parent class that contains the "Launch" (Create) VI that dynamically launches a VI, sets it up with a command messenger (message queue) and wraps that queue inside the parent object. All my... let's standardize and call them "actors"... inherit from this class (I have a few templates that I copy from). As each actor is standardized, each is created by the same parent "Launch.vi", and once that VI is debugged I never have to worry about the details of dynamic launching. And the child actor classes are very simple; generally with no additional private data items, no additional methods or overrides, and only one VI (the one that is dynamically launched) copied from a template. The complicated stuff is all in the parent class. Note that once you have the basics of a VI dynamically launched and running in parallel, sitting behind a command queue, you have all the ability to customize it. Your "Create Test.vi" could (after launching) send your Test a message containing a cluster of "Instrument" proxy objects (ie. objects that contain the command queues of your instruments). Your "Create InstrumentA.vi could send messages containing the information needed to configure InstrumentA.

Excellent! This will be by next peak to ascend.

Regardless of what design you go with, try to get complicated stuff like dynamic launching and messaging standardized into clean reusable components (such as in a reuse library or a parent class) so that you can deal with the complexity once and then not have to worry about again.

This is an excellent method, and one toward which I've iterated towards over the last two months of my re-immersion into LabVIEW.

Thank you both!

[automationtx]

One thing to think about is the possibility of organizing instruments into "subsystems", so the higher level program controls subsystems and each subsystem deals with a limited number of instruments that work together. Then no part of the program is trying to juggle too many parts, and you can test the subsystems independently. It depends if your application can be organized into subsystems ("VacuumSystem", "PowerSystem"). I think that is called the "Facade Pattern" in OOP.

Personally, I usually find it worthwhile to give each instrument a VI running in parallel that handles all communication with it. Then any feature like connection monitoring (as Daklu mentioned) can be made a part of this VI; this can include things like alarm conditions, or statistics gathering, or even active things like PID control (the first instrument I ever did this way was a PID temperature controller). Think of this as a higher-level driver for the instrument, which the rest of the program manipulates.

You can use a class to represent or proxy for each subsystem or instrument; this class would mainly contain the communication method to the parallel-running VI. Daklu's "Slave loop" is an example. You can either write API methods for this class (as Daklu does) or send "messages" to this object (which I've been experimenting with recently).

Thanks, I do like this approach. The separate VI's organized as subsystems seem like the most logical flow. I'll just have to master one of the messaging options discussed. I am not an oop programmer yet but I can see clear benifits to the approach and enforced modularity of oop. I'll check out the facade pattern. In general, the best term I've read to describe what I've been looking for is "a hardware abstraction layer".

I appreciate Todd and Daklu's running conversation. Lots to learn there.

[todd]

Here is most of the state of my noodling with this. Some obvious stuff is left out (gracefully stopping some loops; low-level VISA and other comm methods; proprietary stuff).

main.vi is the top-level VI. Some goals (maybe I should have written these BEFORE coding - just kidding, I knew this would be an iterative approach):

- Provide simple access to instruments in test VIs, as in "Test 1.vi"

- Allow one-time selection of instrument "descriptor" data from the top-level (hide the controls and use INI files for production code), as in main.vi. In fact, such an INI file would eliminate the notifiers in main. But I need to be able to choose them for now.

- Provide a live front-panel for each instrument, as in Execute.vi

- Allow simulated instruments with live front-panels and with APIs, as in Execute.vi

- All instruments must support immediate response data, as in Version.vi

- Some instruments must support streaming data with multiple versions of data manipulation, as in Execute.vi and "Read and Queue Bytes.vi". Kind of. My current real app does dynamic dispatch of Execute.vi, but does not have some aspects properly abstracted - such as the comm method (serial vs dll - each with their own versions) and data parsing.

It still "feels" as if I haven't got the traditional kind of OO philosophy, but if I can just find the right place to override methods and encapsulate data, I'll be in flexibility heaven.

It also feels as if the controls and indicators on Execute.vi could all be private class data.

Sample Project.zip