VI communication between "master/slave" architecture

[Greg Hupp]

I have an application in development that I need to have a main controlling application and several other dynamically loaded VIs ("slaves"). The main application acts as the master controller for communication and control over processes dealt to the respective dynamic VIs. Therefore, all these need to be parallel processes.

I think this application is perfect for using queues: 1 for the master VI and 1 created by the master for each of the dynamic VIs. I can then pass each dynamic VI the queue references for the master and the one created by the master for the slave VI for bidirection communication between the master and slaves. However, how is the best way to pass the queue references?

If I use a call by reference that allows inputs that can be the queues, the execution is halted for the master VI until the slave finishes. This defeats the parallel operation requirement. If I use the Run VI method and set the Wait for Finish to False, this works but I cannot pass the queue references into the slave VI.

Any thoughts/ideas?

[ragglefrock]

QUOTE (Greg Hupp @ Dec 9 2008, 11:12 AM)

If I use a call by reference that allows inputs that can be the queues, the execution is halted for the master VI until the slave finishes. This defeats the parallel operation requirement. If I use the Run VI method and set the Wait for Finish to False, this works but I cannot pass the queue references into the slave VI.

Any thoughts/ideas?

Before you call the Run method, call the Set Control Value (Variant) method on the VI and set the Queue reference inputs by name. As long as the slave VI has queue reference controls of the correct data type and name, this will work. One caveat is that if you build your system into an executable, you can't remove the Front Panels for the slave devices, because they need to be present for the Set Control Value method to work.

[Mark Smith]

Don't pass queue references at all - used named queues. The master creates the queues as named queues and then the slave can just get a reference to the queue using the Obtain Queue function. This completely removes any data flow dependency - that's good for asynchronous processes (threads) but one should always be careful because it's really easy to step on your ****. You can set the Create if Not Found in the slave to false if you want to make sure the Master has created the queue before the slave tries to access it. Also, make sure and close the queue reference in your slave when you exit or you will leak memory - the queue ref created here will remain in memory until all queue refs are closed unless you explicitly close it.

Mark

[ragglefrock]

QUOTE (mesmith @ Dec 9 2008, 11:47 AM)

Don't pass queue references at all - used named queues. The master creates the queues as named queues and then the slave can just get a reference to the queue using the Obtain Queue function. This completely removes any data flow dependency - that's good for asynchronous processes (threads) but one should always be careful because it's really easy to step on your ****. You can set the Create if Not Found in the slave to false if you want to make sure the Master has created the queue before the slave tries to access it. Also, make sure and close the queue reference in your slave when you exit or you will leak memory - the queue ref created here will remain in memory until all queue refs are closed unless you explicitly close it.

Mark

The downside here is that it is difficult to know what the proper name will be if the spawned slave VIs are reentrant clones of each other. You'd need to have some sort of ref count to know which reference to obtain. For instance, your queue name might be Slave Ref %d, where %d represents an incrementing number unique to each spawned slave. But then you need to pass the slave that unique number somehow, which invalidates the whole situation.

If the spawned slave VIs are all different VIs, then this is a perfectly good solution.

Just a thought...

[Mark Smith]

QUOTE (ragglefrock @ Dec 9 2008, 11:01 AM)

The downside here is that it is difficult to know what the proper name will be if the spawned slave VIs are reentrant clones of each other. You'd need to have some sort of ref count to know which reference to obtain. For instance, your queue name might be Slave Ref %d, where %d represents an incrementing number unique to each spawned slave. But then you need to pass the slave that unique number somehow, which invalidates the whole situation.

If the spawned slave VIs are all different VIs, then this is a perfectly good solution.

Just a thought...

This is a good point - if you launch multiple copies of a reentrant clone, you have to make sure you communicate with the correct one. But one way to do this is when you launch the reentrant VI, get the VI Clone name - this is available thru the VI properties using the ref you just opened. Use this to build the Queue Name string (like InputQueue_%s formatted into string, which becomes something like InputQueue_My_VI:1). Then have the spawned VI get its own Clone name thru VI Properties and open the same queue. Then you have a unique one-to-one association without ever actually passing anything on a wire to the spawned VI.

Mark

[Greg Hupp]

QUOTE (mesmith @ Dec 9 2008, 12:40 PM)

This is a good point - if you launch multiple copies of a reentrant clone, you have to make sure you communicate with the correct one. But one way to do this is when you launch the reentrant VI, get the VI Clone name - this is available thru the VI properties using the ref you just opened. Use this to build the Queue Name string (like InputQueue_%s formatted into string, which becomes something like InputQueue_My_VI:1). Then have the spawned VI get its own Clone name thru VI Properties and open the same queue. Then you have a unique one-to-one association without ever actually passing anything on a wire to the spawned VI.

Mark

Hmm...this is interesting. They slave VIs will indeed probably be different VIs not reentrant. Although, I still would like to be able to create the queue of the form Queue_%d (as you mentioned) in the master and then have the slave pick it up! To do this, I still need to pass the information. I like the named queue approach, but I am a little wary of having defined values in the program. This is kind of an open development system, so it needs to be able to function without knowing the specific target it is talking to...only that certain generic commands will be recognized.

[Mark Smith]

QUOTE (Greg Hupp @ Dec 9 2008, 12:07 PM)

Hmm...this is interesting. They slave VIs will indeed probably be different VIs not reentrant. Although, I still would like to be able to create the queue of the form Queue_%d (as you mentioned) in the master and then have the slave pick it up! To do this, I still need to pass the information. I like the named queue approach, but I am a little wary of having defined values in the program. This is kind of an open development system, so it needs to be able to function without knowing the specific target it is talking to...only that certain generic commands will be recognized.

Here's what I'm talking about (if you don't use reentrant VI's) - it's not specific to any target, it just shows how you might use a named queue to communicate with a dynamically launched VI. It has some advantages over using the Set Control Value methods since it's not sensitive to the control names on the dynamically launched VI. It does, however, have one drawback in that if the caller actually leaves memory before the callee gets the queue ref, the queue ref might get garbage collected before the callee can get a ref. If the dynamic launcher is, for instance, in a UI that stays in memory, that won't be a problem. Also, you could just leave the Create if not found True and avoid this problem. This also presumes that you're going to pass data to the slave from some VI other than launcher, but you could easily pass initial data from the launcher.

Mark

http://lavag.org/old_files/monthly_12_2008/post-1322-1228852101.png' target="_blank">

[Greg Hupp]

QUOTE (mesmith @ Dec 9 2008, 01:49 PM)

Here's what I'm talking about (if you don't use reentrant VI's) - it's not specific to any target, it just shows how you might use a named queue to communicate with a dynamically launched VI. It has some advantages over using the Set Control Value methods since it's not sensitive to the control names on the dynamically launched VI. It does, however, have one drawback in that if the caller actually leaves memory before the callee gets the queue ref, the queue ref might get garbage collected before the callee can get a ref. If the dynamic launcher is, for instance, in a UI that stays in memory, that won't be a problem. Also, you could just leave the Create if not found True and avoid this problem. This also presumes that you're going to pass data to the slave from some VI other than launcher, but you could easily pass initial data from the launcher.

Mark

http://lavag.org/old_files/monthly_12_2008/post-1322-1228852101.png' target="_blank">

Yeah...think this will work...may put some hooks in to handle re-entrant VIs, but I can prototype the base architecture using this concept I think...queue is already typdef'd....typdefs are "our friends"! :laugh:

[Norm Kirchner]

QUOTE

but I cannot pass the queue references into the slave VI.

This was your original problem.

Why not have a Q Reference manager (assuming that they are string based (as they should be)).

This would be a LV2 Style Global from which you could pass in an enum which would dictate which ref you passed out.

Inside of this you would register the slave and master Q and as the slaves go dead and are replaced w/ another one you just register the new ref.

That way everyone can access everyone elses q's as needed in the same manner.

Also then each plug-in has a standard method of registration of Q and access to the master.

Whadayathhink?

[LAVA 1.0 Content]

QUOTE (Norm Kirchner @ Dec 9 2008, 03:37 PM)

...

(assuming that they are string based (as they should be)).

...

Whadayathhink?

I think I missed that chapter. Why SHOULD they be strings?

Polymorphic queues is what changed my preference from LV2's to queues since because the overhead required to flatten and unflatten made the string versons slower than LV2s.

Ben

[Norm Kirchner]

Actually my feeling is that each choice has it's place and time. Some have more times than others. In this case though, where he's sending messages/commands to the slaves and standard messages back to the host, the most flexible solution for him I feel would be utilizing strings. I know that there is a mixture of typedef enum and string throughout the program he's working on so it would also depend on how he wants to use it.

So my 'should' stands with the realization that 'always' did not precede it in my comment

[Mark Smith]

QUOTE (Norm Kirchner @ Dec 9 2008, 01:37 PM)

This was your original problem.

Why not have a Q Reference manager (assuming that they are string based (as they should be)).

This would be a LV2 Style Global from which you could pass in an enum which would dictate which ref you passed out.

Inside of this you would register the slave and master Q and as the slaves go dead and are replaced w/ another one you just register the new ref.

That way everyone can access everyone elses q's as needed in the same manner.

Also then each plug-in has a standard method of registration of Q and access to the master.

Whadayathhink?

So maybe I didn't exactly answer the original question, but I think that named queues in LabVIEW are one of the more overlooked tools. I use these most anytime I need interthread communication inside LabVIEW. I even discovered that they make a reasonable way to create a functional global for sharing data among a single group of reentrant VIs - kind of like a private instance variable in a class so that methods of that instance can share data. But I digress...

Anyway, I have explored the idea of creating queue managers (or something like that) inside LabVIEW but I think appropriate use of named queues makes it kind of redundant. LabVIEW is internally keeping track of the named queues - if I need to know if a queue ref is dead I only need to try to obtain that queue. If I manage the queues correctly - for instance, force destroy if I know I want the queue ref to die - I can always just 1) get all VI's in memory and 2) try to open their queues (if I expect them to have one). Then I effectively have a queue manager and I don't have to exert any effort to make sure I keep it up to date - LabVIEW gives me that for free :thumbup: . So now, all I have to do is have a consistent naming convention for my VIs with queues - maybe something like Queued_DoSomething.vi and consistent names inside those VIs for the queues - something like Input_Queued_DoSomething.vi and Output_Queued_DoSomething.vi. Also, keeping to your suggestion to keep everything strings (which I usually do), then any VI that needs to communicate can try to open the queue and if the queue exists, put something on it or take something off and new VIs are easy to introduce.

Which brings us to the next issue with queues - queues don't broadcast to all registered listeners. Whoever gets there first consumes the data. This can be a real can of worms and kind of argues against a queue manager with unfettered access to anybody else's queue. It's all too easy to introduce problems when a new VI starts consuming data that you expected to have available for another VI.

At any rate, it's good we can have these discussions because I'm always learning something!

Mark

[Greg Hupp]

QUOTE (Norm Kirchner @ Dec 9 2008, 02:37 PM)

This would be a LV2 Style Global from which you could pass in an enum which would dictate which ref you passed out.

Inside of this you would register the slave and master Q and as the slaves go dead and are replaced w/ another one you just register the new ref.

This sounds reasonable but I am unsure of the enum to the LV2 style global. Unless you are referring to it to use as a command enum (add, find, etc)....how can I create an enum when I don't know all the queue references? I suppose you could have standard values: enum{queue1, queue2, queue3...) but you would need to fix it at a certain number. Whle currently, there probably would be a fixed number of queue references floating around, I would like to try and leave it open so other functionality could be introduced if and when needed. Am I just missing the point here!?

[Norm Kirchner]

Well in your system (if memory serves correct)you only have 3 primary portions of the program going (the main host, the device control and the data analysis) so it's rather fixed if you think about it like that.

[Greg Hupp]

QUOTE (Norm Kirchner @ Dec 11 2008, 04:25 PM)

Well in your system (if memory serves correct)you only have 3 primary portions of the program going (the main host, the device control and the data analysis) so it's rather fixed if you think about it like that.

Yeah and it is although I am think of future potential to allow more than more analysis tool at a time...we still have that now, you just have to switch back and forth. So I don't know if this is a tradeoff for the better or just more work!!