Iterating over multiple variables in automated test with state machine

[Jason H]

Recently I created an automated test that must collect data over a number of variables. For example, collecting temperature data on a wifi radio over a number of test cases.

The most natural way to write the code would be nested for loops. It would look something like this

for every voltage,

for every temperature,

for every channel,

for every data rate,

take power measurement;

end

end

end

end

however my test must be robust and flexible. I cannot simply put 4 nested for loops on my block diagram. For one thing it will be huge. Also I cannot pause, run special error handling code, and do a number of other things that a state machine allows me to do. The problem is if I have a number of discrete states, such as

init

set voltage

set temp

set channel

set data rate

measure power

stream data to file

exit

I must figure out how to simulate the state transitions of nested for loops. Currently I have a separate state "update user settings" which has all the logic built into it. This logic is implemented as a mathscript node with a large number of if-statements in it, which is about 3 screens high. It works fine, but it doesn't seem very ideal. It makes my code more difficult to understand and more error prone. This seems like a common problem, so I would like to know how other people are solving it.

[2and4]

QUOTE (jasonh_ @ Apr 18 2009, 09:55 PM)

Recently I created an automated test that must collect data over a number of variables. For example, collecting temperature data on a wifi radio over a number of test cases.

The most natural way to write the code would be nested for loops. It would look something like this

for every voltage,

for every temperature,

for every channel,

for every data rate,

take power measurement;

end

end

end

end

however my test must be robust and flexible. I cannot simply put 4 nested for loops on my block diagram. For one thing it will be huge. Also I cannot pause, run special error handling code, and do a number of other things that a state machine allows me to do. The problem is if I have a number of discrete states, such as

init

set voltage

set temp

set channel

set data rate

measure power

stream data to file

exit

I must figure out how to simulate the state transitions of nested for loops. Currently I have a separate state "update user settings" which has all the logic built into it. This logic is implemented as a mathscript node with a large number of if-statements in it, which is about 3 screens high. It works fine, but it doesn't seem very ideal. It makes my code more difficult to understand and more error prone. This seems like a common problem, so I would like to know how other people are solving it.

How about using 6 parallel loops, the first being a UI loop (with an Event structure), and the other 5 are dedicated to the variables you are trying to read. Each loop is a consumer in that it receives messages on its own dedicated queue (as presented in the LabVIEW Intermediate I course).

Each loop will also be a state machine. The messages that the loop dequeues have to tell the loop 1) what to do and 2) what data to use. Flattened strings can contain both a command portion and a data portion. Variants can also be used. A cluster with an Enum and a variant/string is also commonly used.

A loop will receive commands from a previous loop and will also receive data from a subsequent loop. For example the "data rate" loop will receive commands from the "channel" loop. The "data rate" loop will also receive data back from the "power measurement" loop. In other words when the "channel" loop runs it delegates to the "data rate" loop.

Similarly the "channel" loop will receive commands from the "temperature" loop and data from the "data rate" loop, the difference being the data from the "data rate" loop includes the data that the "data rate" loop received earlier from the "power measurement" loop.

I recommend that a loop use the SAME queue to receive commands AND data.

Using an architecture like this you can incorporate error handling, PLUS it is maintainable since each loop is a state machine. You may even be able to remove your Mathscript node.

[Jason H]

2and4,

Thanks for your reply. I would not need a separate complete state machine for every variable, since for any single variable it is just a couple vi's that do all the work. So if they were separate loops, they would just be plain while loops with code in them (ie, a single state). Also, my code doesn't need to run in a parallel fashion. I don't want to change all variables at once, I just want to progress through a number of steps, similar to what you would do if you were manually testing.

In any case, regardless of whether I have a single state machine which executes the commands, or whether I have parallel consumer loops which execute the commands, I still need to generate the correct commands in the correct sequence.

After posting this thread, I have thought of another way to do it. Basically run the nested for loops but only enqueue the states. See example code below. I don't like this approach because it stores all the state information in memory. Also, I like to flush the queue when I encounter an error and I don't like having more than a couple states queued up at once. Also I don't want to put things at the front of the queue (such as pause, etc) because I would like to use the queue like a queue. I shouldn't have to break from that except perhaps for errors. Perhaps instead of enqueuing that array, I could put it in my settings cluster and just pull off the next item in my "update user settings" state. Keep the ideas coming. I know a lot of other people have solved this problem before.

[ChrisClark]

I've done something like this in the past. This is a sort of pattern where the execution is sweeping over the nested setup variables and doing the same test over and over, REPEAT -> setup all vars then run test. If you are doing this all in LabVIEW only, I prefer to separate the execution from the "generate the correct commands in the correct sequence." So you already figured that out. You can generate the correct commands and then put them in a cluster array instead of dropping them all on your state machine que. You then have a big stack and your state machine can just pop the next step and execute it. The state machine is flat and not nested so you can pause, resume, jump to the end, etc. Each element on the stack could be a cluster of VI name, VISA resource, variant of parameters. Then the state machine would use VI server to load each VI and execute it, or have a case for each VI.

You could use a LV2 functional variable or LVOOP class to encapsulate your stack with the following methods, load stack, pop element, reset to start, peek, check for end, etc. So you would run an initialization with the nested loops, or equivalent, to generate the stack and load it into your object.

You mentioned this will take up a lot of memory. For several nested variables you could generate a deep stack. A cluster array of 10,000 elements and 10 MB of memory doesn't seem too big to me. Clearly though you may cross a threshold of too much. In that case you could load only the input information

for every voltage,

for every temperature,

for every channel,

for every data rate,

take power measurement;

into the object and use a linked list, or some other data structure with a pointer to where you are in the sequence. Then instead of popping the next element from a big stack, you would generate the next element, one at a time and never have the whole stack in memory. This takes more thought than the nested loops initialization taking up lots of memory approach. Either way you just have one class or LV2 global that inits and pops the next test, all encapsulated.

cc

2and4,

...

In any case, regardless of whether I have a single state machine which executes the commands, or whether I have parallel consumer loops which execute the commands, I still need to generate the correct commands in the correct sequence.

After posting this thread, I have thought of another way to do it. Basically run the nested for loops but only enqueue the states. See example code below. I don't like this approach because it stores all the state information in memory. ....

[Cat]

I've dealt with this a few different ways. I seem to come back to having, in essence, 2 state machines. One that drives the set up (for me usually frequency and amplitude) and as its last state, an internal state machine that deals with the actual data -- cycling thru channels, collection, processing, saving, etc. All of this is driven by arrays of setup parameters and a pointer variable to each one of those arrays that get updated at the end state. I don't really see any reason why this couldn't be all one state machine, but it's easier for me wrap my brain around it all if I break it up. Plus, there have been a few situations where the setup part was quite easily satisfied with a couple nested for loops, like you mentioned.

I would love to post some code, if I could figure out how to get it from my real computer to this Big Brother controlled email machine. :thumbdown: If this method sounds useful to you, but I haven't explained it well enough, let me know and I'll see what I can do about getting some code or at least screen shots.

[mje]

There are other ways, but the devil is in the details of course, they might not work. Two possibilities:

Calculate your state as a function of just one loop counter. If your highest level loop requires 10 iterations (say your data rate and stream command), then you know that whenever i % 10 evaluates to false, you need to issue a channel command prior to the other commands. This can be applied N-levels deep if your logic is sound.

Maybe the dimensions of your states aren't well defined. Another possibility is just keeping a series of state stacks. For actions A, B, C, and D, populate 4 stacks (arrays) of settings. When the D stack is empty, you pop an element off the next lower stack C, which in turn re-populates the D stack with a new set of calculations. The D-stack will continue to be operated on until empty again. Eventually, the C stack empties, and the B stack then repopulates the C stack, which also repopulates the D stack...all things eventually unwind when all the stacks are empty.

You might even be able to combine the two methods if need be.

Of course, there's nothing inherently wrong or non-robust about nested loops either, it's up to you to figure out what's the best given the details of your application.

[asbo]

There are a couple ways you could do this with a single state machine - your choices are depend on your preference of Q's or string arrays. The important part, though, is to have two of whichever you prefer (or maybe even mix the two if you're feeling crazy). The idea is that one Q will manage the discrete steps and one will manage the automation steps. Initialize the discrete Q with Init or whatever initial preparations you need to do, then have your Default state be something to the tune of "DQ Next Automation Step". You'll need to have a few extra steps like "Start Voltage Loop" and "Next Voltage Loop", but doing it this way will let you have the opportunity to pause, change settings, whatever in between steps.

If I have a chance later, I could build you an example of what I'm talking about, but I don't have any code ready which I can distribute.

[Mark Yedinak]

Personally I am not a big fan of preloading the all of your executions states for a state machine. This type of action means that your state machine cannot be flexible during its execution to handle things like pauses, aborts and errors. I prefer to calculate the states on the fly and only queue a minimum number of states at one time. Generally I try to only queue multiple states when I am including general purpose actions which should not care about what the previous state was or what the next state will be (for example error handling or data logging). For multi-variable processing I use methods similar to what MJE suggested and will include a state such as "Determine Next State" in my state machine. Preloading all of your states isn't really that different than simply using a huge stacked sequence other than the ability to dymanically build the sequence. Also as suggested I would separate control of the UI (separeate while loop with an event structure) out from the processing state machine. If the UI needs to interact with the processing state machine it can simply queue the necessary actions. If they need to get processed immediately queue them at the opposite end of the queue.

[jhoskins]

This is the exact reason why we chose to go with Teststand for automated testing.

[crelf]

QUOTE (jasonh_ @ Apr 18 2009, 10:55 PM)

Recently I created an automated test that must collect data over a number of variables. For example, collecting temperature data on a wifi radio over a number of test cases. The most natural way to write the code would be nested for loops.

That sounds totally like a TestStand application to me.

[ShaunR]

QUOTE (crelf @ Apr 20 2009, 06:06 PM)

That sounds totally like a TestStand application to me.

Ditto.