Best way to pass data to a dynamic VI?

[John Lokanis]

I have been trying to figure out the best way to pass data to a dynamically spawned VI. Here is the setup

The VI is reentrant.

I want to have the VI run independently (don’t wait for completion), so I cannot use call by reference. I must use the Run method.

I need to pass in a bunch of input parameters of varying data types.

Option 1:

Open a reference to the VI.

Pass in input parameters by using the Ctrl Val.Set method to set the FP control values.

Run the VI

Option 2:

Open a reference to the VI.

Get the new VI’s clone name.

Create a Queue of (cluster of string and variant) type using the clone name to name the queue.

Fill the Queue with the input parameters (using the string for their name and variant for their value).

Run the VI.

Inside the VI, Obtain the queue from the clone name.

Dequeue all the elements, feed them into a for loop and use the string to select a case for each value.

The for loop has a shift register for each expected parameter value.

In each case, set the value of the appropriate shift register using the variant. (use the Variant to Data function). It helps if you initialize all the shift registers with a constant of the expected data type.

Take the output of all the shift registers and use them as the input parameters in your code.

Do a Force Destroy on the clone named queue.

So, why for option2 since it seems much more complex? Well, the idea was the Ctrl Val.Set method modifies FP terminals so it should/might/does cause a thread swap to the UI thread, something that is not good when speed is concerned. Also, option 2 means you have no FP terminals at all, so the VI should stay completely out of the UI thread (unless you do something else stupid).

What do you all think? Am I chasing my tail or is this a good idea?

-John

[Yuri33]

Option 2) I understand passing the data by a string\variant queue, but why all the extra for loop\shift register stuff in the dynamic VI? Just preview the queue elements (use Get Queue Status with Return Elements = True so that you don't lose the data by dequeuing) and form a string array and corresponding a variant array. Whenever you need the appropriate data, search the string array for the data name and use that to index the variant array, then convert the variant to the expected data type. If you encounter an error in the conversion, you obviously passed the wrong type of data for that data name.

Option 3) Would a functional global work? Even though you dynamically spawn a VI, I believe any embedded functional global should still contain whatever you wrote to it in the calling program.

[LAVA 1.0 Content]

QUOTE(Yuri33 @ Dec 13 2007, 01:45 AM)

Option 3) Would a functional global work? Even though you dynamically spawn a VI, I believe any embedded functional global should still contain whatever you wrote to it in the calling program.

I guess the reason why the dynamic VI is set to re-entrant is because it's going to be used at different places with different sets of data.

So using a functionnal global would require great care and might cause race conditions, no ?

I am always a bit scared of setting my VIs to re-entrant, the first reason is that I'm not self-confident enough in my own skills and the second is that I never really needed them. And third, now there are different type of re-entrant VIs and I didn't really understand the differences

"We are limited not by the vast domain of the possible, but by the space inside us"

[MikaelH]

Hi

I would create a referenced based active object, one object per instance you need.

The attribute the object would have could be different dynamic event, some for sending data to the process and some to receive data.

//Mikael

[LAVA 1.0 Content]

QUOTE(jlokanis @ Dec 12 2007, 06:43 PM)

So, why for option2 since it seems much more complex? Well, the idea was the Ctrl Val.Set method modifies FP terminals so it should/might/does cause a thread swap to the UI thread, something that is not good when speed is concerned.

I just used 'option 1' on a side project and it works well (just last week!). The only comment/question I have is why you would be concerned about the thread swap? This hit occurs only when you start the dynamic VI. If you're randomly starting large numbers of instances based on UI events AND these instances are CPU intensive or time critical, then you might risk delays or data acq loss.

I start the instances with one subvi (see picture). I also included two queue references; one for commands to and one for responses from the dynamic vi. I've used the Command ref, but haven't used the response queue (yet ).

Overall, I like this idea and will probably use it more :thumbup:

http://lavag.org/old_files/monthly_12_2007/post-949-1197548237.gif' target="_blank">

[Aitor Solar]

I also prefer the first method (call me traditional ). To avoid the necesity of writing down the controls' names (so error-prone), I have developed an XNode to manage the Set Value and Get Value methods automatically. Visually is similar to the Call-By-Reference function, and you can easily drag a VI or browse to the desired VI:

The XNode's generated code just gets and sets the connected values:

Next step is to automatically update the XNode pattern when the referenced VI changes, but Aristos warning about linker functions has frightened me .

Saludos,

Aitor

[Norm Kirchner]

Aitor, that rocks!! :thumbup:

[JDave]

QUOTE(Aitor Solar @ Dec 13 2007, 05:06 AM)

I also prefer the first method (call me traditional ). To avoid the necesity of writing down the controls' names (so error-prone), I have developed an XNode to manage the Set Value and Get Value methods automatically. Visually is similar to the Call-By-Reference function, and you can easily drag a VI or browse to the desired VI:

This does, indeed, rock. I created a scripting program to generate a wrapper VI to do the same thing, but what you made should just become a standard part of LabVIEW.

[Norm Kirchner]

Another thought would be a spin on the LVx concept.

Check it out below.....please..

http://forums.lavag.org/LVx-Exported-LV-Fu...lity-t9437.html

[Bobillier]

Just a small question about your vi. Why get controls values before running ?

[Yair]

QUOTE(Norm Kirchner @ Dec 13 2007, 06:13 PM)

Aitor, that rocks!! :thumbup:

Agreed, and to add to the last question - this would have the advantage over the CBR node that you can run the VI asynchronously if you accept that you will not have the outputs. That would make it much nicer.

[John Lokanis]

Thanks for the replies.

The Xnode solution looks a lot like the VI I wrote to write the values directly (using option1). I am still concerned about the thread swap to the UI thread when doing this. I am launching 100s of these VIs from several instances of a reentrant VI every minute in my system and want to avoid anything that might make a bottle neck. That is why I came up with the queue idea. Here is a screen shot of the VI that gets the data:

As you can see, all you end up with is the wires with the data on them, just like they had come from a FP control.

I have also attached the VI that is used to send the data.

-John

[LAVA 1.0 Content]

QUOTE(jlokanis @ Dec 13 2007, 06:24 PM)

I am launching 100s of these VIs from several instances of a reentrant VI every minute in my system and want to avoid anything that might make a bottle neck.

launching 100s every minute from several instances of a reentrant vi ?

Care to describe your system a bit more?

[Kevin P]

jlokanis,

I would go with Option 2 as well. I must add my curiosity to that of LV_Punk -- 100's of spawings per minute? Sure seems some other approach might be called for...

Meanwhile, for those discussing Option 1 based on the Set Control Value / Get Control Value paradigm. There are some indications that this approach isn't fully robust. There was a recent thread on the ni forums (over here) and in that thread I linked to an older thread where I encountered similar quirks a couple years ago.

I wrestled it a while, but eventually hacked up an inelegant workaround and moved on. So, either beware, or if you can, explain what's going on so I can understand why I should *expect* the observed behavior.

-Kevin P.

[Tim_S]

I've used both methods and which I've used depends on what the dynamic VI is doing. If it's something that I'm going to run once, I set the front panel control values and then run the VI. If it's something that's going to loop in the background, I may pass some initial values (configuration files path, for example) and then pass information to it using events, queues, and LV2 globals. The method I used depended on the application's structure and the nature of data to send. For example, error handling routines get their information from a queue so all the errors can be handled at once while a value that can be set from or used in multiple points in the application goes to a LV2 global.

[gb119]

Wow ! :worship: I was trying to write an XNode to do just that (well, actually I was going to have two XNodes for writing controls and reading indicators), but I couldn't work out how to probe the incoming vi reference to work out the pattern of the connector pane and the types of controls connected - I had thought that if I had a strict vi reference I could do this at scripting time, but no it doesn't seem possible

QUOTE(Aitor Solar @ Dec 13 2007, 01:06 PM)

I also prefer the first method (call me traditional ). To avoid the necesity of writing down the controls' names (so error-prone), I have developed an XNode to manage the Set Value and Get Value methods automatically. Visually is similar to the Call-By-Reference function, and you can easily drag a VI or browse to the desired VI:

Ah, I guess this the hint - you don't try and work out what the connector pane is from the vi reference....

Are you planning to release your XNode on the Lava CR ?

[John Lokanis]

QUOTE(LV Punk @ Dec 14 2007, 04:09 AM)

launching 100s every minute from several instances of a reentrant vi ?

Care to describe your system a bit more?

Sure.

I am building a test executive that is capable of testing N units at the same time. Each unit under test can have N tests, all of which (in theory) can be executed at the same time. So, for a given unit under test, the test executive will spawn a test engine that handles all testing operations (including a UI that is viewable on demand in a sub panel) and will also report back status to the test executive (overall progress, time, pass/fail/etc). Each engine will get the proper test suite from a database and then will start launching tests. It will launch tests until the unit under test returns a test blocking issue (shared resource needed). It will then wait for the block to go away and will resume launching tests. Meanwhile, another thread in the engine will monitor results coming back from completed tests. Since all the tests are plug-in VIs loaded from an external library of VIs, the engine launches a handler for each test that monitors it's status, performs watchdog checking and passes the results back to the engine before self terminating.

It is this handler that I am trying out the pass by queue call method.

The engine is currently a VIT. All of it's subvi's are reentrant, since I need to spawn N of these engines at any given time. The hander is also reentrant, obviously. And all the tests are called reentrantly.

The engine itself has 6 parallel threads (Incoming GUI, GUI Handler, Test Result Monitor, Test Launcher, Database Hander, Two independent Tree Control threads (I use two trees to display state and results)). Since the only limit to the number of tests running in parallel is the target hardware, there could be (practically) 10 tests running at once on each DUT. If I am testing 50 DUTs at the same time, that would be 50(duts)*(6(threads per engine)+(10 tests + 10 test handlers)) = 1300 parallel threads in a moderately loaded system.

That is why I have dual quad core Xeons and 4G RAM in the box.

Also, even though each step in the system is not super CPU intensive or time critical, the sum of all the VIs running at once is significant. So, I have needed to be vary careful about not blocking other threads and staying out of the UI thread as much as possible. For that reason, my Tree threads maintain tree state in a shift register and only update the actual tree control when the user requests to see the test details.

It has been interesting to see just how far you can push this parallel programming stuff and what pitfalls are out there. Maybe I'll write a book if I ever get this project done!

[Norm Kirchner]

Have you heard of TestStand?

The money/time saved in replication of all these abilities already present in TestStand would possibly make up for the money/time needed to get a copy/get trained up.

[TobyD]

QUOTE(jlokanis @ Dec 17 2007, 02:27 PM)

It has been interesting to see just how far you can push this parallel programming stuff and what pitfalls are out there. Maybe I'll write a book if I ever get this project done!

Forget the book, just post the project when you're all done!

[jdunham]

We use a simpler version of "option 2" with our spawned processes. I am a strong believer in leveraging LabVIEW's strong typing. Drop all of your parameters into a typedef cluster, and make a singleton queue (size=1) of those. That's so much easier than using a FOR loop and variants and parsing your own data.

If you are spawning different VIs from the same diagram, then I would still make one cluster for each VI, and make a singleton variant queue. The dynamic VI should know what kind of variant to expect and you can flag an error if it gets the wrong input.

A trick we use is that the caller passes parameter cluster into the queue, starts the dynamic VI, and then puts another item into the queue. Since it's a singleton queue, that function will be blocked until the dynamic VI is up and running. At that point the spawning VI can know it is safe to destroy the queue and run any other code which needs to be sure the dynamic VI is alive.

I think that kind of method way easier and much safer than using control references by name. You can change your front panel text and you won't introduce any hard-to-detect bugs.

Jason

[Norm Kirchner]

Once again, passing the data via a common top level LVOOP object through a call to a dynamic method in the dynamically loaded VI is very handy.

Check it out. You'll be surprised

http://forums.lavag.org/LVx-Exported-LV-Fu...lity-t9437.html

If the dynamic VI is registered for a user event w/ a common top level object data type as the data type, then if the user event can be fired remotely from a host and pass the more specific child data to the dynamic vi. Then once the event is fired, the dynamic vi will get the correct object type which can be passed to whatever and even be dynamically dispatched depending on the type of object.

~,~

[John Lokanis]

QUOTE(Norm Kirchner @ Dec 17 2007, 02:31 PM)

Have you heard of TestStand?The money/time saved in replication of all these abilities already present in TestStand would possibly make up for the money/time needed to get a copy/get trained up.

Yes, I taught myself LV back in 1993 by reverse engineering the original beta code for the first NI test executive.TestStand will not do what we want. It cannot run multiple parallel tests on multiple DUTs from the same machine simutaneously.It also cannot interface to a custom database with a specific schema for handling all test steps and data transactions.My test system is tightly integrated with our DB so if we want to know what is going on anywhere in our MFG site, the data is live in the DB.TestStand is fine for many test applications, but sometimes, you just need to roll you own. Besides, what would I do all day if I wasn't writting LV code? ;-)QUOTE(TobyD @ Dec 17 2007, 03:05 PM)

Forget the book, just post the project when you're all done!

I do plan to post many of the interesting bit to LAVA when I am done. Things like the .NET DB interface, XML interface, Tree data management, etc...Not sure if the final app would be of use to anyone else.Also, I plan to go back and refactor it using LVOOP once I find some time to learn LVOOP...

[Norm Kirchner]

QUOTE(jlokanis @ Dec 18 2007, 02:45 PM)

Yes, I taught myself LV back in 1993 by reverse engineering the original beta code for the first NI test executive.TestStand will not do what we want. It cannot run multiple parallel tests on multiple DUTs from the same machine simutaneously.It also cannot interface to a custom database with a specific schema for handling all test steps and data transactions.My test system is tightly integrated with our DB so if we want to know what is going on anywhere in our MFG site, the data is live in the DB.TestStand is fine for many test applications, but sometimes, you just need to roll you own. Besides, what would I do all day if I wasn't writting LV code? ;-)

Where are you getting your info about TestStand from? TestExecutive 1.0 spec sheets?

We use a very customized DB schema and use TS to log to it, albeit through LV code and not the native TS DB stuff

Multiple parallel tests on multiple DUTs.... have you heard of the batch process model or the parallel model?

Not trying to be a smart-ass, but after using TS for 2 years now I have a lot of faith in how much it can do. (first hand experience too)

TS is really an open environment and I would find it hard to think of something that you can't configure it to do. (except streamline memory usage easily)

~,~

PS You can execute code on remote PCs from TS too

[John Lokanis]

QUOTE(Norm Kirchner @ Dec 18 2007, 12:53 PM)

Where are you getting your info about TestStand from? TestExecutive 1.0 spec sheets?

We use a very customized DB schema and use TS to log to it, albeit through LV code and not the native TS DB stuff

Multiple parallel tests on multiple DUTs.... have you heard of the batch process model or the parallel model?

Not trying to be a smart-######, but after using TS for 2 years now I have a lot of faith in how much it can do. (first hand experience too)

TS is really an open environment and I would find it hard to think of something that you can't configure it to do. (except streamline memory usage easily)

~,~

PS You can execute code on remote PCs from TS too

I admit that I have not looked at test stand in the last 2 years, so things might have changed. But I just took another look at what is on the NI site about parallel testing and could not find a single reference to the ability to run multiple tests at the same time on the same DUT. They did show how you can run more than one DUT at once, but every example showed the tests for a given DUT running in sequence. Unfortunatly that does not meet our requirements. Also, I need to not only log test results to a database but also pull all test sequences from the database using a plan/suite concept and log the start and completion events of every test step as they happen. So, while they state their interface is customizable, I would be suprised if it could meet all those requirements.

But, I could be wrong. So, if you know of some documentation or white papers that address the parallel test issues, please post the links. Maybe I could use that on my next project.

[Norm Kirchner]

QUOTE

run multiple parallel tests on multiple DUTs from the same machine simutaneously

Within a specific sequence you can run as many things as you want in parallel, which covers the multiple tests on a single dut.

and as far as the multiple dut goes, use the parallel model.... or the batch. but some modification to the process model would be needed to ensure the right flow and such, but I doubt many advanced TS users have an unmodified process model.

So going back to the previous comment, some TS training sounds in order. You'll thank me in the long run, because for the next system you have you'll not have to re-invent your custom executive.

Albeit not as fun as creating something totally new but very more robust flexible and easier for someone else to maintain if necessary.