How many "User Events" can LabVIEW queued?

[MViControl]

Hi all,

As we all known, we can define user events in labview and then we can triger the events. I just want to known how may events can be queued in event queue of labview? the reason why I am asking this is because that I want to decide whether to use event or queque to impliment a state machine.

thx

[Ton Plomp]

Hi all,

As we all known, we can define user events in labview and then we can triger the events. I just want to known how may events can be queued in event queue of labview? the reason why I am asking this is because that I want to decide whether to use event or queque to impliment a state machine.

thx

I guess untill the memory of your computeris full.

That will happen faster with an event structure than with a queue, the overhead of the event structure includes the actual timestamp when the event happens.

Ton

[Daklu]

I just want to known how may events can be queued in event queue of labview?

I can't give you an exact number, but the answer is "more than you should ever need."

To be honest, I think you're basing your decision on the wrong criteria. Events and queues can both be used to transmit data to parallel loops, but they have different purposes. Events are used in a "one-to-many" situation. You have one event source and many event consumers. Queues are used in a "many-to-one" situation. You might have many sources enqueueing messages, but only one consumer dequeuing messages. If you have a "one-to-one" relationship you can use either, but queues are easier to work with.

I try not to think of it as an "event queue" because the event structure doesn't necessarily process events in the exact order they were received. (Ton knows more about that than I do.) Plus you have no way to manipulate the "event queue." Think of it as an "event handler" instead. When using events it is better to set up an event structure in a loop to receive events and then send messages and the event data (via a queue) to a parallel loop to do the actually processing. That way your event handler stays free to process events as they come in and you don't have to worry about events stacking up.

When is it a good idea to use events? The first is obviously when you have to process the user's fp input. The Producer/Consumer Design Pattern (Events) template is a good starting point for that. I also use events when I'm writing asynchronous code modules as a way to notify clients of things they might be interested in. Personally I haven't found events particularly useful in other situations, mainly because of the edit time overhead associated with them.

the reason why I am asking this is because that I want to decide whether to use event or queque to impliment a state machine.

Uhh... neither. I know lots of people tout the virtues of the Queued State Machine; personally I think it's a poor pattern because it doesn't promote good design decisions. Using a queue to store sequences of states to be executed can easily lead to code that contains hidden errors and is hard to maintain. You can use a queue to transmit messages and data from the UI loop to the Processing loop, but don't use that queue to control state as well.

[MViControl]

I can't give you an exact number, but the answer is "more than you should ever need."

To be honest, I think you're basing your decision on the wrong criteria. Events and queues can both be used to transmit data to parallel loops, but they have different purposes. Events are used in a "one-to-many" situation. You have one event source and many event consumers. Queues are used in a "many-to-one" situation. You might have many sources enqueueing messages, but only one consumer dequeuing messages. If you have a "one-to-one" relationship you can use either, but queues are easier to work with.

I try not to think of it as an "event queue" because the event structure doesn't necessarily process events in the exact order they were received. (Ton knows more about that than I do.) Plus you have no way to manipulate the "event queue." Think of it as an "event handler" instead. When using events it is better to set up an event structure in a loop to receive events and then send messages and the event data (via a queue) to a parallel loop to do the actually processing. That way your event handler stays free to process events as they come in and you don't have to worry about events stacking up.

When is it a good idea to use events? The first is obviously when you have to process the user's fp input. The Producer/Consumer Design Pattern (Events) template is a good starting point for that. I also use events when I'm writing asynchronous code modules as a way to notify clients of things they might be interested in. Personally I haven't found events particularly useful in other situations, mainly because of the edit time overhead associated with them.

Uhh... neither. I know lots of people tout the virtues of the Queued State Machine; personally I think it's a poor pattern because it doesn't promote good design decisions. Using a queue to store sequences of states to be executed can easily lead to code that contains hidden errors and is hard to maintain. You can use a queue to transmit messages and data from the UI loop to the Processing loop, but don't use that queue to control state as well.

Thanks for reply!

What kind of design pattern do you often use for the parallel process?

I guess untill the memory of your computeris full.

That will happen faster with an event structure than with a queue, the overhead of the event structure includes the actual timestamp when the event happens.

Ton

then from this point of view, is the queue better than events?

[jgcode]

I try not to think of it as an "event queue" because the event structure doesn't necessarily process events in the exact order they were received. (Ton knows more about that than I do.)

Can anyone provide more info on this?

Is this the Dynamic vs Static registration issue of User Events?

Is there a known bug?

It does not sound very nice that LabVIEW would not process events in the order they are received !!

Uhh... neither. I know lots of people tout the virtues of the Queued State Machine; personally I think it's a poor pattern because it doesn't promote good design decisions. Using a queue to store sequences of states to be executed can easily lead to code that contains hidden errors and is hard to maintain. You can use a queue to transmit messages and data from the UI loop to the Processing loop, but don't use that queue to control state as well.

I disagree - I guess I am one of those touters

I have found I can write cleaner code using a QSM.

As LabVIEW is a dataflow language it can get complicated if I have to share data between parallel processes, so I try to avoid this (where possible) if I don't have to.

It can be beneficial if a single loop is used, as it means I share its state within the loop, making use of passing by-value - just the way LabVIEW likes it.

I don't want to sound like a nuthugger either but this is why I find the JKI State Machine a very elegant solution, esp wrt Macros in this case.

I do agree through that as soon as you start queuing states in the QSM it can start to get tricky - but pre-buffering (i.e. Macros) states then running through them is very handy, clean and easy to debug.

Also it does not affect the UI experience as you can disable the cursor if it has to think for a while or crunch some heavy stuff if you are responding to User Interface Events as well.

You can also communicate with the QSM through an Event Structure, allowing external messages in (although I recommend creating an API so that you have a controlled interface).

There is more work in setting up Event messaging but sometimes it is nice to have Static Data defined for events rather than say e.g. a Queue with an enum and variant for the data.

(You could implement messaging using LVOOP, but that would most likely be more work)

What kind of design pattern do you often use for the parallel process?

Of course every design pattern has its limitations and if you need to separate the UI from the Engine or run a Control thread then you will require a more complex pattern but I have found that I usually just have multiple JKI State Machines.

I even like to use this as the framework for a process in an Active Object.

Anyways, I find for what I do, 80% of my use cases can use a QSM and not have to worry.

And if I need to expand I create more QSMs (on the project I am currently working on, it was decided to separate the UI from the Engine and it was quite painless).

I find I don't use Queues that much for messaging any more but rather for buffering up data etc...

Even though Events were not designed to replace queues for messaging and you can't flush, preview etc... there seem a logical evolution and I find very easy to work with.

E.g. having all messages (User Events and User Interface Events) coming through the one interface (Event Structure) helps me to share the statefullness and make use of by-value as mentioned above.

I.e. I don't have to poll another queue for messages and listen to the User Interface.

This is just how I currently like to do things.

I used to always startup a GUI using multiple loops, namely QSM-PC (Event) but I found this complicated this fast.

Plus using Queues (enum/variant) meant I had extra data types per loop to create and maintain - JKI State Machine is string based and so there are no data-types needed and therefore e.g. the State Manager (for checking if an error occurs and going to the Error Handler state etc...) can be reused. That is why I like it better than the Expression Flow QSM-PC example.

Nowdays I am like, why make things more complicated if they do not have to be...

[Ton Plomp]

This is starting to be a nice discussion.

I try not to think of it as an "event queue" because the event structure doesn't necessarily process events in the exact order they were received.

Can anyone provide more info on this?

Is this the Dynamic vs Static registration issue of User Events?

Is there a known bug?

It does not sound very nice that LabVIEW would not process events in the order they are received !!

A LabVIEW event structure is indeed listening to two event queues, one for dynamic events and one for static events. When there are events in both queues it picks the one with the earliest timestamp. However these can be duplicate (I am not sure but I guess the resolution of this timestamp is 15/16 ms on windows) and then LabVIEW just picks the one it likes best.

Ton

[jgcode]

This is starting to be a nice discussion.

A LabVIEW event structure is indeed listening to two event queues, one for dynamic events and one for static events. When there are events in both queues it picks the one with the earliest timestamp. However these can be duplicate (I am not sure but I guess the resolution of this timestamp is 15/16 ms on windows) and then LabVIEW just picks the one it likes best.

Ton

Ok, I think I have read about this on the dark side a while back (I did not locate the post with a quick search)

User Event vs Value Signalling and timestamp resolution example?

As long as it doesn't mess it up in the one queue then all is well.

All serious Event programmatic messaging should be done using User Events anyways.

[Ton Plomp]

Here's a VI that sometimes will show what I mean.

Hit the 'Fire' event that fires a user event and a value signalling event (in that specific order)

In those events I read the current iteration number of the while loop.

If the events are in order the First Iteration should be smaller than SecondIteration

Sometimes however that is not always the case.

I cannot get a resolution like I expected (it seems like the resolution is 1 ms) so I uses a different timer than the Get Ms.

Ton

EventOrder.vi

[Daklu]

I wrote up two lengthy responses to this so far... both of which got crunched by IE.

What kind of design pattern do you often use for the parallel process?

It depends on how complex the parallel process is. For simple processes almost any flavor of Producer Consumer State Machine is just fine. It's the Queued State Machine flavor that causes trouble. As long as you keep the messaging component (to pass data from the producer to the consumer) separated from the consumer's state component you probably won't have trouble. The QSM combines the messaging system and the state system into a single queue, and this creates problems.

For more complex processes or if I want to hide the parallel process' internals I'll use an Active Object. Active Objects add a considerable amount of complexity to your system though. If you don't need the flexibility or encapsulation there isn't much reason to go that route.

Is this the Dynamic vs Static registration issue of User Events?

Yep, that's what I was referring to, but I didn't remember the details.

I disagree - I guess I am one of those touters

It's probably fair to call me a QSM denier.

I have found I can write cleaner code using a QSM.

I have a challenge for you. Next time you want to implement a QSM, use a regular state machine instead and restrict yourself to one state for each message from the producer loop. For example, if you have four buttons on the fp, you have only four states (+an init, exit, and an idle state if you want one) regardless of how complex the process is for each state. If you need to use code in more than one state, put it in a sub vi instead of new state. Then compare the 4 state SM with a 20 state QSM and decide which is cleaner. (Of course I'm saying this without having examined one of your QSMs, so maybe yours don't have the problems that most of them do.)

I do agree through that as soon as you start queuing states in the QSM it can start to get tricky - but pre-buffering (i.e. Macros) states then running through them is very handy, clean and easy to debug.

Handy, yes, but I've found that for all but the most simple applications they are very difficult to debug.

Suppose you put states X, Y, and Z on the queue, but you forgot that X queues up A and B before exiting. Instead of XABYZ you end up with XYZAB. So to fix that you have X enqueue A and B on the front of the queue instead of the back. Now you have XABYZ, just like you want. All good? Maybe, maybe not. Do you have an Abort button that flushes the queue and/or throws a quit state (Q) on the front of the queue? What happens if X is executing when the user hits the Abort button? While X is executing the queue is flushed and Q is put in it. X finishes executing and put AB on the front. Now your queue has ABQ and your application isn't exiting. Pretty soon you have to start examining the queue (and possible maintaining a history of previous states) just to figure out what states you should enqueue next. Application control logic is way easier to follow when it's on the block diagram instead of the queue.

I believe QSMs are inherently flawed. They depend on the states being executed in a specific order. However, when you have independent loops adding items to the queue (as most QSM implementation do) you forfeit your ability to guarantee the order will be correct and open the door for nasty bugs that may occur rarely and are very hard to track down. You certainly can be disciplined enough to only enqueue states in your producer loop, but is everyone that works (or will work) on the app just as disciplined? QSMs practically encourage you to enqueue states in the consumer loop--it's an easy way to reuse code. It's hard to pass that up when it looks like an quick fix to a problem.

Good design patterns promote good design decisions. QSMs don't promote good design decisions.

(The one time a QSM might be justified is if you're doing a lot of operations in the consumer loop that can't easily be put in a sub vi, like using front panel property/invoke nodes. Even in those cases I'm more inclined to send user events from the consumer loop back to the UI loop and let the UI loop handle all the fp control functions.)

[jgcode]

I have a challenge for you. Next time you want to implement a QSM, use a regular state machine instead and restrict yourself to one state for each message from the producer loop. For example, if you have four buttons on the fp, you have only four states (+an init, exit, and an idle state if you want one) regardless of how complex the process is for each state. If you need to use code in more than one state, put it in a sub vi instead of new state. Then compare the 4 state SM with a 20 state QSM and decide which is cleaner.

Unfortunately that is not much of a challenge - as I already try to program like that (try being the highlighted word)!

I either put the logic in the case or stick it in a method/integration VI) if it gets too big or messy.

I strive to map each function/input to a single case (ideally).

I am thinking we are trying to achieve the same process here, I just prefer attributes of the JKI QSM to do it.

The Macros are handy as I can then call UI update methods if I have to on a single button press etc... and I get to share statefullness with the UI Structure.

I do have States that contain reuse states, but I try not to let this get out of control (there's that word again).

Because I agree with what you are saying: Stick it in a SubVI for reuse - not in a State.

I used to abuse this concept and ended up with the most complicated logic and state transitions and a mammoth number of states that was a nightmare to work with etc...

Since then I have been to State Machine Rehab and will not lapse back into my former ways.

...Of course I'm saying this without having examined one of your QSMs

Well I'll show you mine if...

This is just the current program I have open and am working on. It is a simple example, but nonetheless a QSM example:.

Comments (or bashing) welcome from anyone (always keen to learn more and improve).

Oh, and don't get me wrong - I am always up for a challenge

***

So I have my QSM with its States. The custom States (Step at the bottom) are shared states (so they can't be in Event Structure), and yes it may have been better to encapsulated in subVIs and reused, but I didn't think it warranted that in this case (or I couldn't be bothered - one of the two):

The Event Loop within the QSM handles User Events and User Interface Events. Lots of events for data entry screens (which are JKI QSM's too) and button commands and external events:

An Integration Sub VI. Other times I like to make the State cluster a Class and the Integration VIs Class Methods:

No need for an Integration VI here:

[jgcode]

Suppose you put states X, Y, and Z on the queue, but you forgot that X queues up A and B before exiting. Instead of XABYZ you end up with XYZAB. So to fix that you have X enqueue A and B on the front of the queue instead of the back. Now you have XABYZ, just like you want. All good? Maybe, maybe not. Do you have an Abort button that flushes the queue and/or throws a quit state (Q) on the front of the queue? What happens if X is executing when the user hits the Abort button? While X is executing the queue is flushed and Q is put in it. X finishes executing and put AB on the front. Now your queue has ABQ and your application isn't exiting. Pretty soon you have to start examining the queue (and possible maintaining a history of previous states) just to figure out what states you should enqueue next. Application control logic is way easier to follow when it's on the block diagram instead of the queue.

Just quickly - when I say "Macro" it doesn't have to be an actual Macro it can just be prebuffering (queuing of States). Use a Macro when those Queued States need to be called in more than one place

Yes, having States queue up other states is where cracks start appearing in the great wall.

The Macro should drive the QSM.

E.g. Button Pressed - do this, then this, then this, then this, then this.

Then this should never decide to then do that or maybe this, or maybe that? etc....

I don't condone logic like you outlined above, that has headache written all over it.

Why not have just one Macro: XABYZ

If you ever want to run a different sequence then create a new Macro e.g. XYZ

I think you also highlighted an issue with a global queue - if any one can stick stuff into it that is not that best design IMHO.

If it has to go though an interface that is always going to be much saver. It can then be vetted by the Process.

The JKI QSM queue is always internal to the Process (its by-val), and therefore can prevent bad stuff from happening.

I also find using LVOOP on top of that is where more re-usability comes in. In terms of you saying "stick it in a SubVI".

It helps it all to be much cleaner.

You certainly can be disciplined enough to only enqueue states in your producer loop, but is everyone that works (or will work) on the app just as disciplined?

Can you ever really control for this?

If you know how to 100% stop douche bag programmers from making dumb decisions please let me know.

I am pretty sure I could come in and wreck yours or any application very quickly if I wanted to regardless of measures taken

It's hard to pass that up when it looks like an quick fix to a problem.

Its called discipline (refer to douche bag comment above).

[Daklu]

If you know how to 100% stop douche bag programmers from making dumb decisions please let me know.

Yank his license?

I am pretty sure I could come in and wreck yours or any application very quickly if I wanted to regardless of measures taken

I didn't mean people intentionally trying to screw up the code. I meant less knowledgable LV programmers coming in and making changes that *appear* to be good decisions, when in fact they are introducing subtle bugs that not only aren't noticed, but can't be eliminated without significantly restructuring the app. It doesn't even need to be a coworker; maybe it's a customer breathing down your neck complaining that you're spending too much time on it when the problem would be solved if you just made the quick fix his way. (Some of our internal customers are always telling us how simple it is to change our apps to accomodate their latest request. Occasionally they're right. Usually there are significant bugs associated with their change they're not aware of. There are always structural and maintenance concerns with their proposed implementations.)

I am thinking we are trying to achieve the same process here, I just prefer attributes of the JKI QSM to do it.

You mean you're trying to discourage people from using the QSM too!? You sure have a weird way of going about it...

Ultimately we're all trying to create good software that meets our customer's needs. Some of our differences might just be due to the different environments we work in. Since I develop internal tools my customers consider my time essentially free. (They're not signing my paycheck.) I'm expected to be able to react quickly to contantly changing requirements. To some extent our tools are *always* in development. We never know when someone will come with a request for "just this one little change" to a current tool.

Last night I spent a couple hours going over JKI's state machine template and their 3 button dialog example. (The enqueue state icon is a little too phallic for me, but that's neither here nor there.) The way they've incorporated arguments into the state calls is very clever. On the other hand, the fact that you can even have arguments in your state call indicates it's more a function machine than a state machine. As far as QSMs go it avoids the biggest problem... parallel loops putting states on the same queue. As a top level vi it suffers from the same issue every nested SM has--the UI is unresponsive while states are being processed. No problem if all your state sequences execute in less than a second. Bigger problem if the user hits the Stop button while the state machine is off doing something for the next 20 seconds. And it still encourages having states queue up other states. (Have I mentioned I think that's a bad idea?)

Don't get me wrong, I'm not knocking JKI. (Except for whoever created those icons... ) They are arguably one of the top dev houses in N. America, and maybe in the world. Every one of them could code circles around me. They've used that template to create lots of applications that made their customers happy. My point is that with all their experience and knowledge put into the template, they haven't eliminated the QSM's inherent flaws.

And let's face it, you, JKI, and anyone else who hangs out here regularly is probably in the top 20% of all Labview users. When using a QSM you know what things are going to lead to problems. My observations are that the QSM is one of the first design patterns users learn once they decide the spaghetti-code-on-a-single-block-diagram technique doesn't work very well. They don't know how to correctly use a QSM. They just know that everyone is telling them to use QSMs. There is very little discussion on the larger issues that have to be addressed to create good software using a QSM. Given that there are so many ways to shoot yourself in the foot with the QSM, I think it is a design pattern best left for advanced users.

Because I agree with what you are saying: Stick it in a SubVI for reuse - not in a State.

What's the deciding factor for you when choosing between a sub vi or a state?

Since then I have been to State Machine Rehab and will not lapse back into my former ways.

Me too. I'm on my soapbox due to personal experience, not because I've issued a fatwa against QSMs. (I have issued a fatwa against them, but that's not why I'm on my soapbox.)

I don't condone logic like you outlined above, that has headache written all over it.

Why not have just one Macro: XABYZ

If you ever want to run a different sequence then create a new Macro e.g. XYZ

I agree. But that leads naturally leads to the next step... Why not just create a sub vi out of each of state ABXYZ, and make one state for each macro?

I think you also highlighted an issue with a global queue - if any one can stick stuff into it that is not that best design IMHO.

The issue isn't restricted to globally available queues. It will occur any time you have parallel loops putting stuff on a queue. If you pull the event structure out of JKI's state machine and put it in a parallel loop on the same block diagram you are opening the door to this problem, even though the queue itself is still private to that vi. The only way to avoid it is to restrict yourself to putting data on the queue from only one branch of data flow.

[jgcode]

This is a really good discussion (aside from the wanger references of course ) - would you mind posting up some example pictures of how you like to do it also?

I think that would be helpful (for me at least).

[Daklu]

aside from the wanger references of course

This from the guy who turned himself into a gigantic penis in 7 seconds...

would you mind posting up some example pictures of how you like to do it also? I think that would be helpful (for me at least).

I'll ask my wife, but I don't think she will be very keen on the idea...

Oh, oh, oh... you mean the state machines.... That'll be a little tougher since I almost never use them. When I went through QSM rehab I chose the abstinence path and swore off them. While I was exploring how to convert QSMs into regular (non-queued) state machines, I discovered my application requirements aren't really met by the state machine pattern. So I switched over to messages and message handlers.

(One thing I noticed in my QSMs is that my "states" weren't states at all; they were processes. Step back from SM implementations for a minute and think about the states of a can of soda. A few possible states are "wet," "cold," and "closed." If you shake it up and give it to your girlfriend to open it would be in the "overflowing" state. (And she'd be in the "pissed off" state.) These are all adjectives. They are describing a noun. Now look at the state names in your state machine. How many of them are describing a noun and how many are commands to do something? Is "spray soda" a valid state for a soda can?)

What's the difference between a state machine and a message handler? In my mind the main difference is when you diagram a state machine you end up with states leading to other states, similar to this. (Ben posted this quite some time ago. I hope he doesn't mind me using it here as a good illustration of a control flow diagram.)

Diagramming a message handler results in an asterisk pattern. It sits in an idle state waiting for a message to come. When it receives one it enters the case statement to process that message and immediately returns to wait for the next message. Each message is an independent and atomic operation. There is never a direct transition from one "state" to another.

There are also terminology differences that (I believe) exert some influence over how programmers use these patterns. The very nature of state machines is to enter a state, do some work, and choose the next state based on its state at that point in time. That's what makes it a state machine. In contrast, message handlers only handle messages. By calling it a message handler instead of a queued state machine (which it closely resembles when implemented) you are giving it a different role in your application; one that when implemented is less apt to result in a messy maze of mixed-up machine states. (QSMs are the demon offspring from the unholy mating of state machines and message handlers.)

Now that my excuses are out fo the way...

The only code I have at the moment is some stuff from the first MVC app I wrote a while back. These are diagrams from the process loops of the Model ("Engine") active object and the View ("UI") active object. This particular UI was a prototype I used primarily to test out the rest of the app. It has only 5 buttons and a status bar--not nearly as complicated as yours. (Turns out they didn't really need all those fancy UI things they originally asked for...) In any case, I think this at least illustrates the idea.

(In this particular application my active objects receive messages on a queue and use events to communicate to external listeners. Also, you'll notice I was quite liberal with my application of objects, going so far as to wrap every message, event, and the event's data in its own class. Neither of these details are important to the way I use a message handler. In fact, the two loops could be implemented on the same bd without the active objects.)

This is the engine's process loop. There are only four messages this loop handles. See that vi in the middle, CalTableEditor:ExecuteCalProcess? Instead of creating an endless list of states to execute each of the 2 dozen calibration processes, all that complexity is hidden from the user here because the purpose of this vi is to handle messages, not execute a calibration process. The calibration process is delegated to the CalTableEditor object. (More precisely, it is delegated to each of the 2 dozen CalTableEditor child classes. CalTableEditor is an abstract class and is never directly instantiated. Each of the child classes is responsible for executing its own calibration process.) I send out events before the cal process starts and after it finishes so the status bar on the UI can be updated.

Here's the UI's event structure to handle front panel events. If the user selects a valid calibration file from the dialog box, the path is bundled into a LoadCalFileRequestEventData object and the LoadCalFileRequestEvent is fired. (Whether or not anyone handles that event or what the actual outcome of the load request is doesn't matter to the UI. When something happens the user needs to know about, another module will send the necessary information to the UI. The UI only "decides" how to present the information to the user.)

This is where the UI handles messages that are sent to it. Since this is a simple UI there aren't many messages it exposes. If I wanted to disable controls or have other kinds of UI changes I would add new message cases here to do those things. In this example an external module (the Controller in this case) has sent a ShowDialogBoxMessage object to the UI. Contained in the ShowDialogBoxMessage is a child of DialogBox.lvclass. Like the CalTableEditor class, the DialogBox class is an abstract class and the actual act of showing the dialog box is delegated to the child classes. (BTW, this is an example of the command pattern.) This app has six different dialog box styles that can be called by simply using a different DialogBox subclass when creating the ShowDialogBoxMessage object. The notifier is there in case the caller is waiting on a response from the dialog box.

[jgcode]

You mean you're trying to discourage people from using the QSM too!? You sure have a weird way of going about it...

No! I mean this: Ultimately we're all trying to create good software that meets our customer's needs. More so the trying to create good software (much more fun).

On the other hand, the fact that you can even have arguments in your state call indicates it's more a function machine than a state machine.

Pardon my ignorance - what is a function machine? I tried to google it but came up blank. I have not heard the term before. However, passing arguments is a requirement IMO. Otherwise there is no way you can e.g. set the UI to Set Cursor Busy then Do Stuff then Set Cursor Idle in a Macro. But again you shouldn't abuse this (arguments). So you should not use it to pass data, as in application data - that can get buggy. And it is just as bad as using e.g. locals to store data. All application data should go into the State Cluster so it can be used by another state. Or more specifically (for me) I like Objects within the State Cluster only. I use the argument feature for messaging not for data transfer (I hope my distinction makes sense). By-val is always safer and easier to debug.

As a top level vi it suffers from the same issue every nested SM has--the UI is unresponsive while states are being processed. No problem if all your state sequences execute in less than a second. Bigger problem if the user hits the Stop button while the state machine is off doing something for the next 20 seconds.

Well, there are a couple of things I need to think about if the UI is going to go off and churn away for x seconds.

1) Firstly, do I really need or want the User to be doing anything else on the FP whilst this is happening? If the answer is no I don't want them to, then I just Set the Cursors busy and disable the FP whilst the thing churns away and does what it has to. Then I enable the FP once done. I probably don't want to give the user the option to queue a heap of things at one time.

2) If there is a requirement for an Abort function then:

a) Should that operation run in its own separate Process?

b) Should I separate the UI from the Engine - will that suffice?

c) Can I use the natural loopingness of the State Machine to run the process and check the UI for the Stop as well?

These would be functionally dependent on the app itself. None of these are show-stoppers to do. Programming 1) and 2c) is less work then 2a) and 2b) and then I would normally add another QSM to do this.

And it still encourages having states queue up other states. (Have I mentioned I think that's a bad idea?)

Yes you have.

Allows, yes. Encourages, no/maybe.

They haven't eliminated the QSM's inherent flaws.

Can these flaws be totally eliminated or through good programming can they altogether be avoided?

Is any design bullet proof?

When using a QSM you know what things are going to lead to problems. My observations are that the QSM is one of the first design patterns users learn once they decide the spaghetti-code-on-a-single-block-diagram technique doesn't work very well. They don't know how to correctly use a QSM. They just know that everyone is telling them to use QSMs. There is very little discussion on the larger issues that have to be addressed to create good software using a QSM. Given that there are so many ways to shoot yourself in the foot with the QSM, I think it is a design pattern best left for advanced users.

I guess that is no different to everything else - even for the most evil of things: globals, sequence structures etc...

Everything has a place IMO - (some more that others tho).

But the blame should not be on the QSM, more on the programmer.

What's the deciding factor for you when choosing between a sub vi or a state?

I don't have a cut and dry response for that. It depends on the application. Sometimes its easy to use states, and if the function of the VI is simple there is no reason (for me) to be creating a bunch of subVIs and even typedefs I have to maintain. So I try to avoid that initially. Normally for data entry screens, or simple screens that is what I do. Normally for the Main VI/GUI I can see from the design it is required or I guess and put in the extra work up front.

Of course that is what refactoring is for.

I haven't not been able to, or had major issues with, up-converting a simple approach to a more complex one.

That one of the other things I really like - its a really scalable approach.

I'm on my soapbox due to personal experience.

I always learn the best lessons from the mistakes I make.

I agree. But that leads naturally leads to the next step... Why not just create a sub vi out of each of state ABXYZ, and make one state for each macro?

Well then how to you stop the execution of ABXYZ?

If ABXYZ are separate States you could insert (into the Queue) an Abort between anyone of them.

Plus then if you wanted to create ABYZ or XYAB or BAZ if would the case of having five states then creating the required Macros.

No need in creating more SubVIs - which would be a bit more work for the same result.

The benefits would increase if if you add more Letters to this.

Adding one more state then writing another Macro is easier and more readable then more SubVIs.

Additionally it would be much less work if you ever wanted to change the order of things - just cut and paste strings rather than blocks of code.

And you can always modularise that approach too - stick a Macro in a Macro.

That way you can reorder from a top level (Macro) and maintain correct logic or you can rearrange the order of the SubMacro and it will persist across all calling Macros.

The issue isn't restricted to globally available queues. It will occur any time you have parallel loops putting stuff on a queue. If you pull the event structure out of JKI's state machine and put it in a parallel loop on the same block diagram you are opening the door to this problem, even though the queue itself is still private to that vi. The only way to avoid it is to restrict yourself to putting data on the queue from only one branch of data flow.

Yes, but other than simple apps, apps need to be able to message between parallel processes, listening for asynchronous events etc... The above approach would be very restrictive? As long as messaging is logical and has an interface it should be easier to track and debug. I.e. search for this VI as messaging can only happen through there, better yet, Events allow you to create Static messaging (datatypes defined at edit time) which can be (more work but) safer than e.g the single interface with a variant datatype. So there is usually no way around it.

[Daklu]

Pardon my ignorance - what is a function machine? I tried to google it but came up blank. I have not heard the term before.

I guess I haven't been on my soapbox enough for google to pick it up. "Function machine" is a term I coined based on a comment from mesmith here. It is what QSMs should be called because it more accurately describes what they really are. I suggest that QSMs aren't state machines at all. What we commonly refer to as "states" within the QSM don't (to me) appear to be states in the context of state machines in general. (See my previous comment on the soda can.) They are commands to do something, similar to function calls in a procedural programming language. Hence, "function machine."

However, passing arguments is a requirement IMO.

It may be a requirement for function machines. As near as I can tell state machines don't have the ability to pass arguments between states; everything is decided based on the current state. (My understanding of state machine theory is woefully incomplete, so maybe some flavors do.)

There are some good articles on state machines here, and this article talks about the difference between Moore and Mealy state machines. Here's a Moore state diagram of a microwave oven taken from the article. (For some reason I can't attach the image inline. Refer to figure at the end of the post.) There are a couple interesting points in this diagram. First, notice the states they have chosen. More importantly, notice what isn't a state. There aren't any command states--no "Start Cooking" or "Ring Bell" states. The act of actually starting the microwave would be a sub vi that runs when the Cooking state is entered. (In a Mealy state machine StartCooking.vi would be set up as exit actions in both the Idle state and the CookingInterrupted state.) Second, notice the state transitions are all (except for the Init->Idle transition) based on changes to the state variables. The state transition follows changes in the state variables. In a function machine the state transition precedes changes in the state variables. I haven't worked through all the ramifications of this but I do get the feeling it is an important difference.

Can these flaws be totally eliminated or through good programming can they altogether be avoided?

The flaws are inherent to the design, so no I don't think they can be eliminated. The impact of the flaws on your app can be reduced if you're really careful. I figure why walk along the edge of the cliff if you don't have to? I think there are better design patterns that are both more flexible and more robust, so why continue to use this one?

Is any design bullet proof?

No, but some are made of glass while others are made of steel.

That one of the other things I really like - its a really scalable approach.

My experience was exactly the opposite. I didn't find the QSM scalable at all--it gets way too complicated too quickly.

Well then how to you stop the execution of ABXYZ?

If ABXYZ are separate States you could insert (into the Queue) an Abort between anyone of them.

I'd argue that if you have to stop execution at some arbitrary point in ABXYZ, a state machine probably isn't the right design pattern to use.

Yes, but other than simple apps, apps need to be able to message between parallel processes, listening for asynchronous events etc... The above approach would be very restrictive?

The restriction only applies if the message receiver relies on messages coming in a specific order, such as with QSMs. If your receiver is a message handler instead of a QSM then it isn't an issue.

[jcarmody]

Usually there are significant bugs associated with their change they're not aware of.

You used "there", "their" and "they're" in the same sentence.  You could be kicked off of the Internets.

Thank you.     

[Daklu]

By the way Jon, I hope my comment about your state names aren't taken as criticisms of your code. Your code is extremely clean, well documented, and the way you've organized the states helps clarify things. (Likely better than mine.) My critiques are targeting the QSM design pattern, not your implementation of it.

You used "there", "their" and "they're" in the same sentence. You could be kicked off of the Internets.

Whew... thanks for catching that! I'll download the internet when I get home so I won't have to worry about it.

(My views on the QSM may be out in left field, but at least my grammar isn't. )

[jgcode]

By the way Jon, I hope my comment about your state names aren't taken as criticisms of your code. Your code is extremely clean, well documented, and the way you've organized the states helps clarify things. (Likely better than mine.) My critiques are targeting the QSM design pattern, not your implementation of it.

Hell no dude, I am here to learn homie...

And solid feedback/discussion is why I post here for.

I have been trying to digest your code, and haven't had time to reply.

Note: Even if I got shot down in a pile smouldering flames, I wouldn't care as long as I can take something away from it.

Thanks for your time posting all this stuff, i'll replay in details soon.

[Daklu]

Thanks for your time posting all this stuff, i'll replay in details soon.

One good thing for me that has come out of this discussion is I have a much better understanding of state machines. I finally know the differences between "Moore," "Mealy," and "Mixed" state machines, how those different models are implemented in Labview, and have a *much* better idea of how to correctly decompose application requirements into a state machine. Unfortunately for the rest of you, I'm now more convinced than ever that the QSM is not a state machine and shouldn't be promoted as a good general purpose design pattern.

I've actually started implementing all three models of the Microwave state machines from the article I linked to above. But like you, spare time is hard to come by and I'm not sure when I'll get them done.

[jgcode]

I guess I haven't been on my soapbox enough for google to pick it up. "Function machine" is a term I coined based on a comment from mesmith here. It is what QSMs should be called because it more accurately describes what they really are. I suggest that QSMs aren't state machines at all. What we commonly refer to as "states" within the QSM don't (to me) appear to be states in the context of state machines in general. (See my previous comment on the soda can.) They are commands to do something, similar to function calls in a procedural programming language. Hence, "function machine."

Yes, I think you are taking the name too literal. They are called QSMs cause when you open up the NI manual it calls the design pattern a QSM (I don't know who came up with the name).

As a side note the traditional name for the String Based QSM is the Queued Message Handler which is exactly what the JKI QSM is based on (string based).

Now I understand what the "Function Machine" term was referring to. But I think NI have coined the phrase Multi Functional VI (MFVI) already (sorry mesmith!) which is basically the(their) State Machine pattern combined with a FGV pattern - which when you look at the QSM is fundamentally what it is (along with the ability to queue itself). I agree the terms are confusing, bear with me here: I hold state information in USR (FGV) and runs different States based what wired into the Case Structure. The JKI combines the User Event pattern as well (all of this is NI terminology).

Now when I say state information this is really the state of the objects in the application. This is normally implemented as a cluster (or class) of objects - but it only just acts as a container.

When I say States, I am really referring to Commands (so I agree with your expression of this).

In terms of using States versus using SubVIs - I guess this goes back to our conversation about documentation in the CLD wrt MFVIs vs LVOOP: In that you can create single a single VI with many states or you can create a lot of subVIs (and I was discussing which would be faster/easier based on the time limit).

So this is what I refer to by State and is why I talk about interchanging State with SubVI and when is the best time to use one of the other etc....

I find it comes down to management and how many different files I want to manage.

I normally like to try to keep the QSM light on files (subVIs and typedefs) as long as its neat, whilst I prefer lotsa subVIs for my functionality and encapsulate that using LVOOP.

So yes, maybe calling it a State Machine is bad?

But it seems quite strongly part of the nomenclature of LabVIEW now to change it!

But by all means go for it - although I have to teach it, so I will have to stick to it

It may be a requirement for function machines. As near as I can tell state machines don't have the ability to pass arguments between states; everything is decided based on the current state. (My understanding of state machine theory is woefully incomplete, so maybe some flavors do.)

In terms of when you talk about State in this context - I usually try to encapsulate this inside an object.

So in terms of your Soda Can example the Soda Can object would manage its own State.

Having a QSM managing many objects' States would be very difficult.

But these objects (may) needs to live inside a GUI - this is where I use QSM a lot - as the architecture for the GUI.

But the QSM is not handling the Soda Can's State, it's simply calling and running the Commands from e.g. the User Interface.

In some cases it will be up to the Soda Can what to do based on its internal State.

I didn't find the QSM scalable at all--it gets way too complicated too quickly.

I have found the QSM a great architecture for the GUI, and scalable too.

From you images it looks like you have a string Command from a Queue wired into a case structure.

Is this not some form of QSM (in the above sense)?

It also seems you have taking messaging a step further and implemented the famous design pattern that Paul-At-Lowell raves about (he was threaten to post this recently, but this is the first LabVIEW example I have seen).

So I am not familiar with it, and only know what I have read in HFDPs and wiki.

But if you would like to, please feel free to post more on this as I am very interested in this as it seems you have a higher level of abstraction here.

This raises the following question of interest about your style and application including:

1) How do you handle smaller GUIs. For example if you have a small application with a main GUI and 5 smaller dialogs (for data entry etc...) do you use this pattern (and separate GUI and Engine) for all of them?

2) What about really basic dialogs?

3) In the example you posted it seems like there is a great deal of work gone into the messaging, do you have a reusable framework to build from?

Cheers

-JG

[Daklu]

Yes, I think you are taking the name too literal.

Quite possibly true. I do continue to use the term QSM because that's the name the Labview community associates that pattern. However, when I recognized my own applications based on the QSM were becoming unmanagable I started searching online for information about how to design a program using a QSM. Couldn't find much of anything. So then I started reading about state machines in general and how they apply to software. None of that info made much sense in the context of applying it to a QSM in Labview. Why? Because a QSM isn't a state machine in any common understanding of what state machines are.

They are called QSMs cause when you open up the NI manual it calls the design pattern a QSM

Yeah? Then NI is wrong too.

You hear that Dr. T!? I'm calling you out! (Albeit very quietly...)

I could start calling an apple an "orangutan" and those who know me well would understand my meaning, but when somebody in an online forum asks for a dessert suggestion because the the in-laws are in town and I respond with "orangutan pie," it's bound to cause all sorts of trouble. (Especially when I recommend using peeled orangutans and removing the stem. ) I know I'm unlikely to change the world and get everyone to start calling it a "function machine." I'm sure there are those reading this (if anyone is left now that we've completely derailed the original topic) who dismiss this as a minor semantics issue. I disagree--what we choose to call things often conveys information about that thing. "QSM" implies that pattern is, in fact, a type of state machine when in reality it is not. If instead of "QSM" we referred to that pattern as "Hector," I would consider that a better (if rather arbitrary) name. "QSM" is an extremely poor name for the pattern.

I do believe that once you recognize that it really is a function machine (or MFVI) and start thinking about it in those terms, the questions that follow will naturally lead to a more suitable pattern. The question that started me down that path was one I asked you earlier. "How do I decide if an arbitrary piece of functionality should be in a state or a sub vi?" What heuristics and guidelines would you give to a junior developer that asked you that question?

As a side note the traditional name for the String Based QSM is the Queued Message Handler which is exactly what the JKI QSM is based on (string based).

Doesn't matter if the QSM is based on strings, variants, enums, or classes. It's the architecture itself that is flawed, not the data it passes. (I suppose it wouldn't surprise you that I'm not a fan of the QMH either.)

I think NI have coined the phrase Multi Functional VI (MFVI)

I've not heard that phrase before. "Multi Functional VI" does more accurately describe what it is and, somewhat ironically, also perfectly explains why you shouldn't use it for applications. Code is easier to maintain over the long run when it is separated, not combined. This is known as separation of concerns. (QSM code is similar to procedural code, so read the sentence about C and Pascal.) I could use my favorite procedural programming language and write a generic function similar to this...

DoEverything(str funcName, var funcArgs){  Select funcName {    Case = "Func1" 	{      Cast(funcArgs, string);  	// do stuff	};	Case = "Func2"	{  	Cast(funcArgs, int);  	// do other stuff  	DoEverything("Func1", varData);    };	Case = "Func3"	{  	Cast(funcArgs, string);  	// do stuff again  	DoEverything("Func1", varData);  	DoEverything("Func2", varData);  	// do even more stuff	};	Case Else {};  };};Main(){  funcNames[] = {"Func1", "Func3", Func2"};  funcArgs[] = {"Foo", 42, "Bar"};  For i = 0 to 2  {	DoEverything(funcNames, funcArgs;  };};

(I'll be the first to admit this is an imperfect textual metaphor for the QSM. For one thing the above implementation uses a function stack (permitting a kind of recursion) instead of a function queue. For another it doesn't show many of the QSM features, such as capturing events/messages. Regardless, the metaphor is attempting to illustrate the code structure of the QSM, not the actual implementation.)

The question is why do this? Doesn't it make much more sense to have Func1, Func2, and Func3 be their own functions instead of wrapped in an arbitrary DoEverything() function? Even if your functions didn't directly call into other functions, does DoEverything() strike you as good programming?

I suspect the QMH and QSM came about from attempts to "keep all the code on one screen" and a general reluctance to create sub vis. I understand that--I used to hate having to make a bunch of sub vis too. (Still do if its only purpose is to reduce the amount of screen space requried.) There are practical advantages to using sub vis instead of states. Earlier I mentioned it is clearly (IMO) more readable when the program flow is defined by the block diagram instead of by the queue. There are other advantages too.

Using sub vis instead of states is also much safer, especially when the application gets complex. Consider three sub vis which form a cycle by calling into each other like this:

SubVI1 --> SubVI2 --> SubVI3 --> SubVI1

Labview throws a compiler error if you do this--you end up with a broken run arrow. If you use states that enqueue other states instead of sub vis Labview's compiler can't check your code for cycles. Unless your application is very simply it is unreasonable to expect to be able to write test cases for all code paths. In short, the only way to make sure any given change has not introduced this error is by inspection. Dunno about you but the thought of having to manually trace through all the state machine's execution paths for any non-trivial changes is decidedly unappealling.

Another nice thing the compiler does for us when using sub vis instead of states is ensure type safety. When you're converting all your data to a string so it can be passed to the next state (function) you're preventing the compiler from doing that. All those edit time broken wires that would have appeared with sub vis have just turned into run time errors that may or may not be caught by your testing. Once again, the only way you can be sure you've got it right is by inspection. Code inspection has a place in programming for sure, but expecting it to be sufficient for anything other than simple algorithms is a mistake imo.

(Speaking of type safety errors, there's a bug in the pseudo code I posted above. Did you notice it while looking over the code? How long did it take you to find it? Had each function been broken out on its own the compiler would have picked it up immediately.)

the(their) State Machine pattern

Hate to break it to you Scooby, but their state machine pattern appears to be woefully inadequate for implementing state machines as well. I've spent far too much time on this reply as it is... I'll try to post more later.

I have to teach it, so I will have to stick to it

You're a Labview instructor? Very cool.

[jgcode]

Hate to break it to you Scooby, but their state machine pattern appears to be woefully inadequate for implementing state machines as well. I've spent far too much time on this reply as it is... I'll try to post more later.[/size]

Who is Scooby? And yes I used parenthesis as I thought you would have issue's with NI's (theirs) basic SM implementation too

If instead of "QSM" we referred to that pattern as "Hector,"

I think you are onto something here, it has a nice ring to it.

You hear that Dr. T!? I'm calling you out! (Albeit very quietly...)

Does that help

I've not heard that phrase before. "Multi Functional VI" does more accurately describe what it is and, somewhat ironically, also perfectly explains why you shouldn't use it for applications. Code is easier to maintain over the long run when it is separated, not combined. This is known as separation of concerns. (QSM code is similar to procedural code, so read the sentence about C and Pascal.) I could use my favorite procedural programming language and write a generic function similar to this...

I suspect the QMH and QSM came about from attempts to "keep all the code on one screen" and a general reluctance to create sub vis. I understand that--I used to hate having to make a bunch of sub vis too. (Still do if its only purpose is to reduce the amount of screen space requried.) There are practical advantages to using sub vis instead of states. Earlier I mentioned it is clearly (IMO) more readable when the program flow is defined by the block diagram instead of by the queue. There are other advantages too.

Yes this is the trade off I am willing to except based on the module's size.

But that module is normally a GUI VI.

It is also the reason for my question about how do you handle small dialogs etc... (which I am still interested to know)

But the way I like to program this is not so much exposed functions like in your code example.

Functions exposed like this on the block diagram of the main VI is bad and way too hard to test.

It more a Command Case to run an Integration (another NI term) i.e. the Top Level code (GUI) is calling the bottom level code (which are methods that encapsulated functions) in the correct order and passing in input data.

If I have a basic Get/Set (Ok/Cancel) data entry screen, I really don't see the point of implementing something elaborate if I get the same result using Hector.

Also for small projects, and charge by the hour/day stuff, sometimes I honestly can't justify the time.

I want to ensure that using Hector is scalable for the future however, if it was a reusable framework (that was my other question about what you have posted) I would use it.

Also when it comes to GUIs how to you test your code?

Are you able to automate testing of it some how?

The above issues you mentioned (datatype checking etc...) are great points, but in terms of GUI I don't have any way to automate/check/test these other than Usability Tests, but am very interested to learn more or how I can do this better.

Would your implementation help towards this issue?

[Francois Normandin]

You hear that Dr. T!? I'm calling you out! (Albeit very quietly...)

First thing that popped into my head reading this:

"I'm Dr.T and I'm a Night-Elf Mohawk."

^{Sorry to be off-topic on an off-topic topic but I couldn't help.}

_{BTW, I love this off-topic topic...}

[jgcode]

First thing that popped into my head reading this:

"I'm Dr.T and I'm a Night-Elf Mohawk."

^{Sorry to be off-topic on an off-topic topic but I couldn't help.}

_{BTW, I love this off-topic topic...}

ROFL! Great find Francois - I just had to post the vid

<object width="640" height="385"><param name="movie" value="http://www.youtube.com/watch?v=bqJE5TH5jhc&hl=en_US&fs=1&"><param'>http://www.youtube.com/watch?v=bqJE5TH5jhc&hl=en_US&fs=1&"><param name="allowFullScreen" value="true"><param name="allowscriptaccess" value="always"><embed src="http://www.youtube.com/watch?v=bqJE5TH5jhc&hl=en_US&fs=1&" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="640" height="385"></object>

Get some nuts... fool