Is the day of the "Queued State machine" behind us?

[Grampa_of_Oliva_n_Eden]

After some of my fellow wire workers were looking at a very complicated QSM application I explained that architecture was fine if you have spent your whole life doing assebly programming and you did not want to change the way you design applications.

With LV's multi-threading, it seems to me that there is no need to use one of those undocumentable monsters.

So although I have already instructed them to "never develop anything life that" I thought I may want to do a sanity check with Y'all.

So....

Are the days of the QSM behind us?

Please share your thoughts, and I will sit back and learn.

Ben

[PaulG.]

Let's make sure we know what we are talking about. There is some confusion in the LV community. When I hear "queued state machine" I think of a state machine that uses queues to queue and dequeue state elements. And when I hear "queued message handler" I think of a state machine that uses string states in an array that are inserted and removed. What are we talking about?

Pardon my confusion, but one of the so-called strengths of the "string state machine", i.e. queued message handler is that it "self-documenting".

[Aristos Queue]

QUOTE(Ben @ Aug 14 2007, 03:12 PM)

Are the days of the QSM behind us?

I think we'd need some more details about what exactly you think is behind us. NI just released the new Statechart Module for building really advanced state machines. I don't think that architecture is behind us. Are you simply referring to programming individual state machines should be avoided in favor of using some reusable framework? Because that I agree with.

[Jim Kring]

QUOTE(Ben @ Aug 14 2007, 01:12 PM)

Are the days of the QSM behind us?

No, I don't think so. I'll be using them for quite a while.

[Val Brown]

QUOTE(Aristos Queue @ Aug 14 2007, 02:10 PM)

I think we'd need some more details about what exactly you think is behind us. NI just released the new Statechart Module for building really advanced state machines. I don't think that architecture is behind us. Are you simply referring to programming individual state machines should be avoided in favor of using some reusable framework? Because that I agree with.

Quick question on the Statechart Module -- is that included in 8.5 or is it a separate (purchased??) module/toolkit?

[JDave]

QUOTE(Ben @ Aug 14 2007, 01:12 PM)

After some of my fellow wire workers were looking at a very complicated QSM application I explained that architecture was fine if you have spent your whole life doing assebly programming and you did not want to change the way you design applications.

Could you show a picture of a block diagram demonstrating such a monstrous construct?

QUOTE(Ben @ Aug 14 2007, 01:12 PM)

With

LV

's multi-threading, it seems to me that there is no need to use one of those undocumentable monsters.

...

Are the days of the QSM behind us?

What would the future be?

[lavezza]

QUOTE(Val Brown @ Aug 14 2007, 06:04 PM)

Quick question on the Statechart Module -- is that included in 8.5 or is it a separate (purchased??) module/toolkit?

NI.com lists it as $1499 in the US and CAD1800 for you Canadians.

It it marketed as a Module (on the same level as Real-Time, FPGA, PDA, etc.) and not as a toolkit (like database, VI analyzer, etc.). Most other modules are available with a significant discount if purchased with the NI Developer Suite, but it doesn't look like the Statechart module has been added as a Developer Suite option yet.

Pat Lavezza

[i2dx]

QUOTE(Aristos Queue @ Aug 14 2007, 11:10 PM)

referring to programming individual state machines should be avoided in favor of using some reusable framework? Because that I agree with.

a reusable framework would be fine. If many people use that design pattern, it would help many people to develop faster, if they can use a pre defined architecture.

On the other hand: it does not talong to develop a new QSM for me, because I use VI templates and since I discovered, that they are recursive, I create a complete "sub-application" within seconds ...

QUOTE(Jim Kring @ Aug 15 2007, 12:49 AM)

No, I don't think so. I'll be using them for quite a while.

/signed

[Val Brown]

QUOTE(lavezza @ Aug 14 2007, 07:37 PM)

NI.com lists it as $1499 in the US and CAD1800 for you Canadians.

It it marketed as a Module (on the same level as Real-Time, FPGA, PDA, etc.) and not as a toolkit (like database, VI analyzer, etc.). Most other modules are available with a significant discount if purchased with the NI Developer Suite, but it doesn't look like the Statechart module has been added as a Developer Suite option yet.

Pat Lavezza

Yeah, that's what I thought and that's really disappointing on several levels.

[Michael Aivaliotis]

I'm always open to new ideas. Do you have an alternative?

I've been using QSM my whole career in one incarnation or another.

[shoneill]

QUOTE(Ben @ Aug 14 2007, 10:12 PM)

After some of my fellow wire workers were looking at a very complicated QSM application I explained that architecture was fine if you have spent your whole life doing assebly programming and you did not want to change the way you design applications.

With LV's multi-threading, it seems to me that there is no need to use one of those undocumentable monsters.

So although I have already instructed them to "never develop anything life that" I thought I may want to do a sanity check with Y'all.

So....

Are the days of the QSM behind us?

Please share your thoughts, and I will sit back and learn.

Ben

As others have already mentioned...

Depends on what you mean with QSM.

I use event-driven queued state machines quite a lot, especially in connection with UIs (otherwise events don't make so much sense) and I find it really useful, so I certainly won't be abandoning it any time soon. I also don't quite understand the problem with QSM and multi-threading. It's quite possible to have a well multi-threaded QSM if you use the right structure. Of course, I'm takling about an architecture with more than just 1 producer and 1 consumer. Goes more towards component programming than anything else, but it's core is still a QSM.

There are cases where Statemachines are simply not neccessary of course.

Ben, what do you think they should be replaced with?

I await your answer eagerly!

Shane.

[Grampa_of_Oliva_n_Eden]

QUOTE(shoneill @ Aug 15 2007, 03:42 AM)

As others have already mentioned...

Depends on what you mean with QSM.

...I await your answer eagerly!

Shane.

A code construct that maintians an array of "States" in a shift register and pops the top state of the stack and executes same. Any state can push other states onto the stack before they complete.

All state involved in the application are implemented as cases.

It only uses a single loop.

Becuase the state transtitions are only established while the code is running, it is impossible to diagram what will happen in what order. States will execute in the oder they are removed from the stack but because states can "preempt" other operation already on the stack, you have to repeatedly check what is on the stack to understand where it is going (possble if the stack is an array, impossible if the stack is in a queue). And then if you decide something bad happened in an earlier state, there is no way to trace it back.

I'll see if I can get a screen shot it this 40-50 state monster that prompted this Q.

Ben

[Grampa_of_Oliva_n_Eden]

QUOTE(Ben @ Aug 15 2007, 07:13 AM)

A code construct that maintians an array of "States" in a shift register and pops the top state of the stack and executes same. Any state can push other states onto the stack before they complete.

All state involved in the application are implemented as cases.

It only uses a single loop.

Becuase the state transtitions are only established while the code is running, it is impossible to diagram what will happen in what order. States will execute in the oder they are removed from the stack but because states can "preempt" other operation already on the stack, you have to repeatedly check what is on the stack to understand where it is going (possble if the stack is an array, impossible if the stack is in a queue). And then if you decide something bad happened in an earlier state, there is no way to trace it back.

I'll see if I can get a screen shot it this 40-50 state monster that prompted this Q.

Ben

Here is a screen shot of one of the sub-VI's after scrubbing to protect the inocent.

http://forums.lavag.org/index.php?act=attach&type=post&id=6627''>http://forums.lavag.org/index.php?act=attach&type=post&id=6627'>http://forums.lavag.org/index.php?act=attach&type=post&id=6627

This construct was natural for me when I started out and was still thinking about pushing things onto stack and poping them off. It also has a natural dual priority scheduling feature in that if you insert at the head of the queue those states pre-emt others and low priority can put at the back of the queue. When we only had a single thread and were trying to do more than a PC wanted to, these were useful.

It just strikes me that thinkinga bout pre-empting and priority of execution is falling by the wayside now that we have almost 100X the CPU from just 5 years ago.

So you still use this type of structure?

Ben

[torekp]

QUOTE(shoneill @ Aug 15 2007, 07:42 AM)

It's quite possible to have a well multi-threaded QSM if you use the right structure. Of course, I'm takling about an architecture with more than just 1 producer and 1 consumer. Goes more towards component programming than anything else, but it's core is still a QSM.

Can you explain this in more detail to someone like me? Someone who's "Not A Real Programmer, But I Play One At Work."

[shoneill]

QUOTE(Ben @ Aug 15 2007, 02:49 PM)

Here is a screen shot of one of the sub-VI's after scrubbing to protect the inocent.

http://forums.lavag.org/index.php?act=attach&type=post&id=6627''>http://forums.lavag.org/index.php?act=attach&type=post&id=6627'>http://forums.lavag.org/index.php?act=attach&type=post&id=6627

This construct was natural for me when I started out and was still thinking about pushing things onto stack and poping them off. It also has a natural dual priority scheduling feature in that if you insert at the head of the queue those states pre-emt others and low priority can put at the back of the queue. When we only had a single thread and were trying to do more than a PC wanted to, these were useful.

It just strikes me that thinkinga bout pre-empting and priority of execution is falling by the wayside now that we have almost 100X the CPU from just 5 years ago.

So you still use this type of structure?

Ben

OK, I see what u mean now.... I suppose each software architecture, when pushed far enough, will develop some cracks. I wasn't sure you were referring to a single self-feeding loop. I wasn't even aware the term "Queued state machine" referred to this.

I remember implementing something like this back in 1997 or 1998 before I even knew it had a name... Seemed like a good way to save time back then. Still, I had maybe 10 or 20 states, most of them quite simple.

I actually (I don't know why) try to avoid self-feeding loops. It can be useful in some cases, but as a general rule I avoid them.

On a side note, am I the only one who uses a seperate UI loop (hiding controls, scaling charts and so on) for their code? I guess I'm trying to prevent my main loop from switching to the UI thread for hiding and showing controls, but I don't know if it's overkill or not.....

Shane.

[Grampa_of_Oliva_n_Eden]

QUOTE(shoneill @ Aug 15 2007, 10:01 AM)

...

On a side note, am I the only one who uses a seperate UI loop (hiding controls, scaling charts and so on) for their code? I guess I'm trying to prevent my main loop from switching to the UI thread for hiding and showing controls, but I don't know if it's overkill or not.....

Shane.

No, I try to do all of the UI work that can be accomplished quickly ( < 1 second) in the GUI loop and anything that takes time gets passed off to a slave loop.

Ben

[PaulG.]

QUOTE(Ben @ Aug 15 2007, 08:49 AM)

Here is a screen shot of one of the sub-VI's after scrubbing to protect the inocent ...

NOOOOOoooo!!! Not THAT!!!

This example of a "queued message handler" (NI's design name, not mine) with 60 states is somewhat of a monster. Many QMH tools have been written by some of our associates that make managing something like this a lot easier (I see you have one or two in place) but this would still be a little too out of control for my tastes.

The QMH is fine for a design that needs only 2-10 states and you need it fast. Anything bigger than that and architecture a little more robust and predictable is in order.

I've tried a number of architectures but I always go back to the typedef enum in an array feeding a case structure. Worksforme.

[Dan Press]

QUOTE(shoneill @ Aug 15 2007, 10:01 AM)

On a side note, am I the only one who uses a seperate UI loop (hiding controls, scaling charts and so on) for their code? I guess I'm trying to prevent my main loop from switching to the UI thread for hiding and showing controls, but I don't know if it's overkill or not.....

I have been using the pattern shown here for some time now. I have seen many varaitions on this. There is one loop for Events (UI) and another for other processing. I will also usually have a subVI tucked in the upper left corner that runs in parallel and at a different priority. That subVI houses DAQ stuff or other hardware interface code. I still consider this a queued state machine (QSM). The difference between this and the "monster" above is that the data is encapsulated in a cluster and there is an event loop. Still, this has its limitations. I have been guilty of amassing a long list of cases inside both loops!

http://forums.lavag.org/index.php?act=attach&type=post&id=6630''>http://forums.lavag.org/index.php?act=attach&type=post&id=6630'>http://forums.lavag.org/index.php?act=attach&type=post&id=6630

[Mike Ashe]

I for one will continue to use QSM's in their various forms for a long time. I agree that they can get unwieldy, but I think that is more of a coding style and documentation issue than an architectural issue with QSM's. With a reusable template, a library of VIs to implement the QSM and the discipline to document properly along the way I haven't found them to be a problem.

Just my $2

Mike

[LAVA 1.0 Content]

QUOTE(Mike Ashe @ Aug 15 2007, 12:56 PM)

Just my $2

Mike

WOW... $2.00. Must be an increase because of the premium membership

[Val Brown]

QUOTE(NormKirchner @ Aug 15 2007, 01:41 PM)

WOW... $2.00. Must be an increase because of the premium membership

Clearly it was a scalable comment.

[silmaril]

ZITAT(Ben @ Aug 15 2007, 02:49 PM)

Here is a screen shot of one of the sub-VI's after scrubbing to protect the inocent.

This construct was natural for me when I started out and was still thinking about pushing things onto stack and poping them off. It also has a natural dual priority scheduling feature in that if you insert at the head of the queue those states pre-emt others and low priority can put at the back of the queue. When we only had a single thread and were trying to do more than a PC wanted to, these were useful.

It just strikes me that thinkinga bout pre-empting and priority of execution is falling by the wayside now that we have almost 100X the CPU from just 5 years ago.

So you still use this type of structure?

Isn't that exactly the structure from NI's old Advanced courses, which came from VI Engineering originally?

This example shows how a good LabVIEW programm looked like some 10 years ago.

I think a queued state machiche is still a very good idea, but there are some things to keep in mind:

I would not recommend using strings for the case selector. Better use a (typedef!) enum instead. This way you can avoid typos that take you hours to debug. I also recommend not having a default case. This way your can make shure, you have really made your own decision which case to run for each state.

I don't really think a simple shift register is the best way to save your states. Depending on your application you might want to use a functional global or Queue functions instead. If you application is event-driven, you can also use user events for this (with the drawback that you cannot insert events at the beginning of the event queue).

I like to add a variant data type to each queue element. This gives me something like function calls with parameters.

Of course, using such a queued structure doesn't mean it has to be the only loop! As mentioned by others, you can still have several other loops in parallel. For example your user interface is running in the main loop with additional parallel loops for DAQ and qriting to file. Inter-thread-communication can be done using queues, notifiers or events.

Especially with all those multi-core processors around, it really is a waste of CPU power to write single-threaded applications.

No, I don't think the days of the QSMs are over. But everyone has to keep up with the times, even old LabVIEW programmers.

[JDave]

Ben, I appreciate your description of a bad QSM. Let me restate your points to see if I understood you correctly.

QUOTE(Ben @ Aug 15 2007, 04:13 AM)

A code construct that maintians an array of "States" in a shift register and pops the top state of the stack and executes same.

This seems to be common to all QSMs, whether it is in a Queue or a shift register. Not problematic.

QUOTE(Ben @ Aug 15 2007, 04:13 AM)

Any state can push other states onto the stack before they complete.

All cases have the ability to manipulate the queue of actions/states. Hard to predict and track how program moves through states.

QUOTE(Ben @ Aug 15 2007, 04:13 AM)

All state involved in the application are implemented as cases.

No distinction between true state of the software and actions to be performed by the software. Bundled together in the architecture, making it hard to maintain or change things.

QUOTE(Ben @ Aug 15 2007, 04:13 AM)

It only uses a single loop.

No sharing of tasks between multiple loops or even multiple VIs running in parallel.

QUOTE(Ben @ Aug 15 2007, 04:13 AM)

Becuase the state transtitions are only established while the code is running, it is impossible to diagram what will happen in what order.

I thought all state transitions were established at runtime. The difference is how transparent they are to the developer, right?

QUOTE(Ben @ Aug 15 2007, 04:13 AM)

States will execute in the oder they are removed from the stack but because states can "preempt" other operation already on the stack, you have to repeatedly check what is on the stack to understand where it is going (possble if the stack is an array, impossible if the stack is in a queue). And then if you decide something bad happened in an earlier state, there is no way to trace it back.

Adding in the complexity of preemption and priority scheduling to the architecture already described is chaotic.

David

[Grampa_of_Oliva_n_Eden]

QUOTE(dsaunders @ Aug 16 2007, 02:42 PM)

...

Adding in the complexity of preemption and priority scheduling to the architecture already described is chaotic.

David

I was trying to sit back and try to learn something but your last comment has me smiling becuase of two statements from people around me.

One said "It is like using a bunch of "goto" statements". The other was my wife who has repeatedly said "Goto statements introduce chaos."

Ben

[Val Brown]

QUOTE(Ben @ Aug 16 2007, 11:52 AM)

I was trying to sit back and try to learn something but your last comment has me smiling becuase of two statements from people around me.

One said "It is like using a bunch of "goto" statements". The other was my wife who has repeatedly said "Goto statements introduce chaos."

Ben

FWIW, I used to use that as a way to talk about the difference between assignment and equality tests:

GOTO=CHAOS <> GOTO==CHAOS