Differences between QSM, QMH, and actor MHL?

[Daklu]

(This thread is branched from a comment Shaun made here.)

 

 

...I don't see any difference personally between an actor "Do" and QMH as I don't think there is one and it seemed that you were not really convinced there was one either I do however see a huge difference between Actors,QMH and Queued State Machines (and you mentioned the JKI QSM as in reference to QMHs) and in that context I agree with many of the things you often espouse. The usual example I give for QSMs is reading a file, since it is a lot of code and a lot of messages all to do something that can be done in a single simple VI and there is little benefit in splitting out the states. That VI would then be put in a QMH or Actor.

 

........later....
Nope. I do know the difference between a QMH and Actor and I don't think it's semantics. QMHs don't contain state and the state is driven by the producer. If they do maintain state, then they are QSMs (QSMs and Actors being subsets of QMHs). So an Actor can be the same as a QSM if it does not manage it's state (unusual) but a comparison between Actors and QSMs would have been clearer.

 

I've never seen a difference between QMH/QSM and use the terms interchangeably.  Seems to me QMH was just rebranding the QSM to avoid confusion with the name "state machine."  I'm curious if/how others differentiate between QSM/QMH.

 

 

QMHs don't contain state and the state is driven by the producer.

 

This comment confused me.  The first reference to state implies state=data, but the second reference to state implies state=one frame in a case structure.  Can you explain further?

 

<Thinking out loud>

Using your definition, if the QMH doesn't maintain data state, I take that to mean it doesn't have any shift registers?  In which case the only things it can do when processing a message is exit the loop or send a message somewhere else.  Hmm... that does eliminate the meta-stability problem sequentially dependent messages create.  Functionally it is exactly the same as what I consider a MHL, except my MHLs can (and usually do) contain data state.

</End thinking >

 

I still believe QMH is a poor name.  It identifies the message transport's implementation--a queue.  The transport's implementation is irrelevant to your definition of what a QMH is.  Furthermore, I believe the transport's implementation is irrelevant to any well-written message handling loop. 

 

<Thinking out loud again>

The one thing the QSM, QMH, and actor MHL have in common is they receive and handle messages in their own logical thread.  So what variations are there of well-written message handling loops?  What are the advantages and disadvantages of each of them?  Can we come up with more intuitive names for them?

 

Here are some of the things I believe are essential for any well-written message handling loop:

-No sequential message dependencies(!!)

-Message processing time << Interval between messages

 

If we accept that as the baseline, the only two important characteristics that come to mind immediately are:

-Data State:  Does the MHL retain user data between messages?

-Behavioral State:  Does the MHL have more than one handler for any message and choose which handler to use based on its internal data?

 

Thoughts?  (Open to all, not just Shaun.)

[ShaunR]

OK. Here's how I see it, for my sins (it's rather similar to FGV and AE).

A queued message handler is a collection of frames that are basically an API of discrete functions driven by an ordered list of incoming messages. There are no inter dependencies between the frames and they can be called in any order at any time. You are sort of right in your thinking out loud, in that they generally do not have shift registers but that isn't a defining feature since it may have shift registers to maintain the state of that individual function/frame (lets say a counter). But that state is not used by any other frames and is not propagated. The module, as a whole, does not have any state. Any frame, at any time can be executed without regard to previous calls or calls to other frames.

An example of this definition would be something like a Drive controller. It has moveTo, Stop, Reset, getPosition etc. The "state" of the drive would be maintained by an external module (usually a sequence engine) which would call the methods in the right order and at the right times. The drive state is maintained external to the module - it is just a collection of functions. There are very good reasons for doing this, most notably, that the sequence engine knows of other dependent states of the system so you don't do something like drive a ream into the bed when no part is present. Externalising the state in this case helps you decrease application specific functionality of the module and reduce module interdependence.

A QSM is also a collection of frames driven by an ordered list, but they are interdependent and must be called in various orders depending of the internal state of the module. Additioanlly, the frames may add messages that were not on the original list. As said before. I use the reading of file as a trivial example as I've seen it many times and think "what a lot of code to achieve very little" There, you may have open, read, write and close frames, but the API interface is ReadFile/WriteFile. When actioned, each frame is called in the correct order dictated by the internal decisions about the state of the file. It is no longer just a collection of functions, it has interdependent frames and maintains internal state (of the file whilst executing).

Edited August 12, 2013 by ShaunR

[Val Brown]

For me the queue structure is what defines a QSM and it is different from a QMH.  The primary example of a QSM for me was the original shipping example of the ASPT.  It had a while loop that held a case structure that was fed by a build array of boolean outputs that was searched for index, that was incremented and then typecast to an enum.  Each case of that structure (from FP controls) held used build array to queue as sequence of states that was pipelined to another for loop that dequeued the array of states to be run.  The queue was the list (or queue) of states to be run in each iteration of the enclosing while loop NOT the mechanism of using an LV queue.

 

Yes, tough to put into words but it was a primitive form of "producer-consumer" based on the UI because, well, that was about the only way to implement something like that back in LV5.  And it was "hard on the eyes" as the Chinese would say, since there is no good word for "ugly" in Chinese.

 

For me a QMH would be producer-consumer kind of construct that uses LV queues.  A QSM queues states (within the each state/iteration of producer loop, whereas a QMH uses queues originating in one while loop to communicate to, or message, another loop. The difference lies in the communication used as well as there being more than one while loop, each running in parallel.

[drjdpowell]

I still believe QMH is a poor name.  It identifies the message transport's implementation--a queue.  The transport's implementation is irrelevant to your definition of what a QMH is.  Furthermore, I believe the transport's implementation is irrelevant to any well-written message handling loop. 

A “queue” is different than a specific implementation of a queue, such as the LabVIEW in-built Queue.  User Events also involve a queue, as do things like TCP or Network Streams and the like.  In contrast, things like Notifiers do not.  So, “QMH” does not specify the transport implementation.

Re QSM/QMH/MHL: I note that “queued-message handler” is ordinary English, while "message-handling loop” only makes sense if one understands the programming-specific use of the word “loop” to mean repeated code.  “Queued state machine” only makes sense if one uses "state” in a very strange way that doesn’t have anything to do the state of anything.   So I like “queued-message handler”.

 

BTW, one can make a QMH that is composed of a pool of MHL “workers”, so those two terms are not necessarily interchangeable.

[Val Brown]

One approaches queues state, the other queues messages.  It's analogous to the difference between Mealy and Moore machines.  Either CAN be converted into the other and pass a "Turing Test" of equivalence so, if you take an abstract enough view, there is no difference functionally.  This is much more of a by ref implementation whereas the other is more of a by val implementation.

 

Historical usage in the LV community, however, does support the idea that a QSM does not use the LV structure of a queue (basic function) because there is one "super loop", whereas a QMH does use the LV structure of a queue as a means to transport messages between two parallel loops, which may or may not be explicitly coded in the same VI.

[Daklu]

A queued message handler is a collection of frames that are basically an API of discrete functions driven by an ordered list of incoming messages. There are no inter dependencies between the frames and they can be called in any order at any time.

 

Have you looked at the 2012 QMH project template?  Would you consider the Initialize, Initialize Data, and Initialize Panel messages interdependent?  If so, then is that template not, in fact, a QMH?

 

 

The module, as a whole, does not have any state.

 

What do you call a module that does have state (behavioral or data,) but does not have interdependencies between the frames?  It doesn't fit into your QMH or QSM definition.

 

 

For me a QMH would be producer-consumer kind of construct that uses LV queues.  A QSM queues states (within the each state/iteration of producer loop, whereas a QMH uses queues originating in one while loop to communicate to, or message, another loop. The difference lies in the communication used as well as there being more than one while loop, each running in parallel.

 

I have talked to a few people who consider the both loops to be part of the QMH/QSM.  I don't find that definition very helpful, primarily because of its limited usefulness as a component.  I've never seen any examples illustrating how to integrate other concurrent processes into the basic QMH template.  (I suspect that's because it would reveal all the glaring problems in the implementation, but... *shrug*.)

 

 

A “queue” is different than a specific implementation of a queue, such as the LabVIEW in-built Queue.  User Events also involve a queue, as do things like TCP or Network Streams and the like.  In contrast, things like Notifiers do not.  So, “QMH” does not specify the transport implementation.

 

I'll grant you QMH doesn't necessarily specify the transport implementation, even though in many cases it does.  Regardless, QMH *does* specify the transport's behavior.  Transports are not defined as having queue-like behavior.  It certainly could behave like a queue and present messages in the order they are sent.  It could also behave like a stack and present the most recently sent messages before the earlier messages.  Or it could behave like a global and overwrite pending messages with newer ones.  Or it could randomly choose which message to present from all pending messages.

 

The point is, a well-written message handler doesn't care about the transport's behavior.**  Yes, the transport's behavior could have a significant impact on how you create systems of MHLs, but an MHL is robust against changes in that aspect of the transport's behavior.  It doesn't require a specific transport behavior to work correctly. 

 

**That statement defines the difference I see between a MHL and QMH/QSM better than anything else I've been able to come up with.  If you can arbitrarily change a transport's message ordering behavior without breaking the receiving loop's expected behavior, then it's a MHL.  Otherwise you have a QMH/QSM.

 

For example, here's a screenshot from the 2012 QSM project template.  You probably want to create your own instance from the template--it will probably be easier to follow what I'm saying.

 

 

There are four message handling cases we are concerned with--Initialize, Initialize Data, Initialize Panel, and Update Display.  The message sequencing is set up like this:

--> Initialize        |--> Initialize Data        |--> Initialize Panel                   |--> Update Display

Their responsibilities are:

Update Display - Update the fp indicators with data from the message payload.

Initialize Data - Reset the shift register data to their default values.

Initialize Panel - Self-send all the current shift register data in an Update Display message.

Initialize - High level message that initiates the the initialization process.  The implicit guarantee is immediately after this message sequence has been processed the shift register data and front panel indicators have been reset to their initial values.

 

The "Initialize" message is put on the UI queue in the "Create All Queues" vi (we'll ignore that peculiar design decision this time) so it is the first message processed by the loop.  As long as the transport behaves like a queue, it will probably do what you expect.  (We'll also ignore the race condition that exists between sending the Initialize Data and Initialize Panel messages.) 

 

What happens if you change the transport's queue behavior to stack behavior?  The Initialize Panel and Initialize Data are processed out of order and the Initialize message no longer fulfills its responsibility.  It's worse if you change the transport to global behavior--any arriving message can potentially abort the sequence of initialization messages and "break" the Initialization message.

 

As I see it, the template is very much a QMH/QSM, but is too dependent on the transport behavior to be called a MHL.

[ShaunR]

Have you looked at the 2012 QMH project template?  Would you consider the Initialize, Initialize Data, and Initialize Panel messages interdependent?  If so, then is that template not, in fact, a QMH?

Just looked now. There are only two frames. Exit and "No Event", Default.

 

 

What do you call a module that does have state (behavioral or data,) but does not have interdependencies between the frames?  It doesn't fit into your QMH or QSM definition.

Well. I'm going to shift/clarify my position, just slightly, although I maintain all I have said is correct. The QSM is as described and the other example is a QMH, but the QMH example doesn't define a QMH, but it a more common one.

As I said on another thread. A QMH just means it's a buffered message handler (buffered by a queue, shift register, lemons). How it is handled or how it is buffered is irrelevant. It is a genericism that has buffered messages and are handled by some code.

So the QSM is a QMH with specific characteristics that make it behave as a state machine and I have tried to define what those differences are in comparison to a non state machine QMH.

[todd]

"Just looked now. There are only two frames. Exit and "No Event", Default."

 

 

Try "Project -> Create Project", instead of "File -> New"

Edited August 13, 2013 by todd

[Daklu]

Well. I'm going to shift/clarify my position, just slightly, although I maintain all I have said is correct.

 

I'm not claiming you're wrong, but I do claim your interpretation is just one of many equally valid interpretations.

 

 

As I said on another thread. A QMH just means it's a buffered message handler (buffered by a queue, shift register, lemons). How it is handled or how it is buffered is irrelevant. It is a genericism that has buffered messages and are handled by some code.

 

I mostly agree, with the added restriction that messages are buffered in a FIFO and presented in a first-in-first-out order.  For better or worse the names "QMH" and "QSM" also convey expectations about the techniques used to write it and how users can and should interact with it.  Those techniques often create unexpected race conditions.

 

 

[Copied from the other thread, it's more appropriate here.]

ShaunR, on 13 Aug 2013 - 09:04, said:

 

A message handler, handles messages. Period. A queued message handler buffers the messages before it handles them. That's really all there is to it.

 

From an abstract, natural language perspective I agree with you.  From a useful programming jargon perspective I don't.  Those definitions are far too general--we know there are "good" and "bad" ways to implement a message handler.  Good message handling loops conform to a stricter set of rules and offer more guarantees than bad message handling loops.  Giving good message handling loops their own name helps users understand what they can and cannot expect from a given message handler.

 

I'm not particularly tied to MHL as a name exclusive to good message handling loops, but I can't think of a better name.  When somebody tells me they have implemented code using a QMH my first thought is, "$%#!  Now I get to spend the next two weeks inspecting their code and removing all the race conditions." 

 

As I've tried to explain, even the name "QMH" violates the basic principles of writing a message handler that can be safely used as a component in a concurrent application.  I want a name for message handling loops that rejects those techniques that are shown to be potentially thread unsafe.  No macros.  No self-messaging.  No timeouts.  No sequential dependencies.  Selfish?  Maybe.  Beneficial to the community as a whole?  I think so.

 

 

So the QSM is a QMH with specific characteristics that make it behave as a state machine and I have tried to define what those differences are in comparison to a non state machine QMH.

 

Again, not claiming your definition is wrong, just incomplete.

 

As I understand your definitions, QMHs do not have any concept of overall loop state--either in terms of data or behavior.  However, each frame can persist information between calls but do not share information with each other.  In other words, feedback loops can exist inside a frame, but not outside the case structure.  QMHs also have the characteristic of no sequential dependencies between message handlers (which I interpret as each message handling frame completes all its responsibilities prior to the execution system exiting the frame) and frames do not self-send new messages.

 

On the other hand, QSM do have loop state, sequential dependencies, and may self-send new messages.

 

My question is simply me trying to better understand your definitions.  What do you call those implementations that don't fall into either category?

-No loop state, sequential dependencies, self-sends messages.

-Has loop state, no sequential dependencies, no self-sent messages.

-etc.

[drjdpowell]

For example, here's a screenshot from the 2012 QSM project template. 

 

The thing that strikes me about that picture is, first, the fact that the queue has two writing processes, and second, that subactions of handling a message are treated as new messages for the queue.  I agree with you about the problems of this, but don’t think that this is best communicated by arguing about how to define the differences between two very similar acronyms.  I feel the same way about the “Action Engine”/FGV thing; it’s not really very productive.  

 

I think it would be better to identify and talk more directly about the issues of message ordering and perhaps introduce more illuminating terminology, for example the concept of a “critical section”, or the database-transaction concepts of “consistency” and “isolation”.  The “Initialize” transaction is a critical section, and because it isn’t protected as such it isn’t isolated from other transactions.  Adding a different (non-FIFO) transport, that may switch the order of the two subactions, would also prevent it being consistent.  

 

Note, btw, that a Get-then-Set transaction on a FGV/AE is also a critical section, and because it isn’t protected as such it isn’t isolated from other transactions.  Thus, these concepts are more widely useful than the definitions of QMH/MHL.

 

— James

[Daklu]

The thing that strikes me about that picture is, first, the fact that the queue has two writing processes, and second, that subactions of handling a message are treated as new messages for the queue.  I agree with you about the problems of this, but don’t think that this is best communicated by arguing about how to define the differences between two very similar acronyms. 

 

The purpose of defining the differences between QSM, QMH, and MHL is to try and make them meaningful.  As long as there is no agreement on definitions they have limited value in communicating ideas.  In forum language the three terms are used interchangeably, causing ambiguity and confusion.

 

I don't have strong objections to Shaun's definition of MHL:  A message handling loop handles messages.  That definition has value as an abstract idea.  When a MHL is identified it tells me something useful about its role in the application.

 

For the time being I'll refer to actor-style message handling loops as "Safe" MHLs (SMHL), meaning it can receive any message at any time without breaking the contract of its messages.  Any MHL may be safe or it may be unsafe.  The only way you can tell is by inspecting the code.

 

Calling the subset of message handling loops that use a FIFO to buffer messages a QMH may be how the term is commonly used.  If so, that's fine with me--I can adapt.  However, I don't think it's a useful distinction in the same way identifying a LIFO MHL, or SEQ MHL, or Random Access MHL, or Notifier MHL isn't particularly useful.  It gives me information about how the MHL is implemented, but from an actor-oriented viewpoint I don't care how it is implemented.  All I care about is that it is safe.

 

 

I think it would be better to identify and talk more directly about the issues of message ordering and perhaps introduce more illuminating terminology, for example the concept of a “critical section”, or the database-transaction concepts of “consistency” and “isolation”.  The “Initialize” transaction is a critical section, and because it isn’t protected as such it isn’t isolated from other transactions.  Adding a different (non-FIFO) transport, that may switch the order of the two subactions, would also prevent it being consistent.  

 

I'm not opposed to introducing new terminology, and I sometimes view sequences of messages as transactions.  But since you introduced the terms you get to explain what they mean in the context of a MHL.    I believe these terms have value as abstract concepts.  The more difficult question is how does one identify critical sections in a MHL?

[ShaunR]

The purpose of defining the differences between QSM, QMH, and MHL is to try and make them meaningful. 

 

Here you go.

 

MH: Message Handler. Executes functions based on messages.

QMH: Buffered MH that operates on an ordered list of messages.

QSM: A flavour of QMH that operates as a state machine.

MHL: a description of a Message Handler (MH) because it manifests itself in LabVIEW as a while loop.(redundant).