Best way to share data from different subpanels?

[Labview Junky]

Hi everyone I have a question about how I should approach the design of an application. Here it goes.

I am making a DAC which will be reading data from multiple sources and controlling servos/steppers based on some control inputs and then logging things like position strain gauge readings and rpm. All that is fine and I have made successful DACs which perform those tasks. Now I want to make each portion of a separate vi and integrate them together in one main gui using subpanels. I have seen this done using the AMC message queues, which worked but seemed overly complicated and messy. I know this could be done using shared variables but I am not sure what the drawbacks/challenges I would face are? Would this be a viable way to go about doing this? Is there another cleaner way via events or something I haven't mentioned? I have been try to use VI Server References but I don't know how to transfer that data between sub panels, can this be done?

I am using Labview 2010 and have access to if needed 2011.

Thanks in advance!

[Jon Kokott]

I prefer message queues. Shared variables are... slow. You'll have problems with race conditions using them as well. I don't suggest using user events unless it is actually a user event. They are very similar to queues in performance but don't give you the same level of control (they only size unlimited, message can queue up badly in rare cases.) Your other options are LV2 globals (ugh, I don't like these, but they can be fast to program atleast.) and DVRs. With DVRs/LV2 you'll be polling in all likelyhood if thats the only mechanism you have for synchronization. Queues/User events will be all event based processing.

This is a real challenge, and if you are new to these concepts, I would almost plan on writing your VI, taking a step back in a week or two, and re-doing the whole thing. Ultimately you'll understand how you're solving the problem better, and end up with something that doesn't look overly complicated to you anymore.

[PaulL]

What are your data rates?

We use networked shared variables for communication between components. We send a lot of data, but our fastest loops run at 62.5 Hz. Within these constraints this approach works quite well and is quite powerful and easy to work with after going around the learning curve. (The learning curve itself can be a bit steep.) We use shared variable events but this requires the DSC Module.

Maybe Jon can clarify his comment about shared variable race conditions. I don't understand that claim. I admit I don't share his concerns about user events either, but I expect that this is due to a different design approach.

We use networked shared variables since they offer the best out-of-the-box publish-subscribe solution. Is publish-subscribe something your application needs?

One more thing. Our views are separated from the controllers and models. The views send and display information.

[Labview Junky]

My attraction to using shared variables is that they are very easy to work with but as you said data rates are not extremely high. 60 Hz is okay but I would prefer to start with a architecture that can be a bit faster. I would not be communicating over a network though, it would just be between a few VI's in sub panels. Would the communication be faster then? The application that I want to rewrite uses AMC message queues and gets similar data rates but there is quite a bit of overhead which I think might be slowing the data rates down. What would be one of the faster architectures that I could use?

Also would a mixture of different methods be advisable? Such as DVR's for events and then Shared variables for data?

I am quite new to labview so I was wondering how you would get a reference from one VI to another? I have been trying to use the Static VI Reference but I don't know how to implement it or break out the separate references such as buttons and indicators. Is that even how it works?

[Jon Kokott]

Networked shared variables are certainly easier to work with, and if they "work" then use them. That said, compared to straight up TCP (which is what they are built on anyway) crushes them. Its WAY more involved, and you're completely changing the architecture at that point. I take back that all shared variables are slow, its really just network shared (reading and writing a shared variable is pretty wicked fast).

Publish-Subscribe shared variables is an approach that will definitely work. I really don't ever use them, but shared variables offer no mutexing (like a DVR or FG) if an atomic transaction is required, and need to be polled unlike a Queue. The publisher-subscriber approach with Shared Variables can be replaced by a Notifier. At the end of the day I prefer reference objects (like variables) to have their reference wired to them. Maybe its just personal preference.

Aside from that, if you are dealing with same-application synchronization, which is my assumption based on the passing data to subpanels, Queues and DVRs are generally my bread and butter. If everything is in the same application space there is no reason to use a network shared variable (or any TCPing in general) as it will be unnecessary.

Paul brings up a good point that Shared Variables are an effective way to do things and you yourself said you envision a solution using them. Sometimes its better to stick with what you know and get things done. If you are doing something simple like displaying a reading from a gauge, you'll be done in less time than it took to read this post! Without knowing the specifics of your application its difficult to know what the "best" solution is. I still like queues.

Use the static VI reference, but "strictly" type it (right click on the static VI ref) then you can use an invoke node to insert that reference into a subpanel, then call by reference node to actually call the VI. This will allow you to wire the connector pane references that you may/may not need.

[PaulL]

Jon brings up a good question. Will the pieces be on the same machine or networked? Will they be part of the same application? (We purposely break up the pieces into several applications, but that isn't always necessary. Moreover, our applications need to run anywhere on the network.)

I guess I would emphasize that I suggest thinking of this (and I think you do) in terms of messaging (sharing data) between separable (which can mean many things) components. My recommendations are to separate messaging from the application logic and to separate the message content from the messaging mechanism.

By the way, earlier I said our fastest loops are 62.5 Hz. I realized that's perhaps not the best way to say that. Our RT messaging/state control loops run that fast. We have loops in our FPGA VIs (on compactRIOs) that run extremely fast.

Edit: I thought about this on Friday and I realized I should explain further. Our application speed is limited by processor usage on the compactRIO-9074, only a couple percent of which is due to handling shared variables. In fact, a while back I tried to see how fast I could loop just doing a round-trip with single variable. I think I got to 2 kHz before seeing messages getting behind, but even that I decided was due to the processor, not shared variables themselves.

...shared variables offer no mutexing (like a DVR or FG) if an atomic transaction is required, and need to be polled unlike a Queue.

The DSC Module offers the ability to use shared variable events, and we have implemented a simpler version of this on RT with user events. We very rarely read a shared variable value without handling an event, and then only when we want to read a pre-existing value (e.g., on start-up on RT).

OK, this is more controversial, but we avoid mutexing and transactions in our communication paradigm because we make sure we are doing messaging only. (Or maybe I am missing something and we need these capabilites for messaging for some reason?) Anyway, my point is that we use shared variables for messaging only. The application logic itself handles any state changes. The messaging system never does. I actually think this is quite important, which is why I am bringing it to attention. [OK, this is the most important reason why we stopped using "action engines" ("functional global variables" with an emphasis on the functional) years ago.] We think messaging and what the application does with those messages should be separable concerns. You will, however, find many people on this forum who will disagree. :-) These are just my (reasoned, I think) recommendations. :-)

On a simpler note, we have found it is much easier to have "unidirectional" writing. In other words, the recipient of data never writes to that data. (That isn't quite the same thing as saying we have only one writer for a message. For instance, a "Start" command can come from a view for the component or from another component at a higher level in the hierarchy.)

OK, that's a lot to chew on for a while....

[0_o]

Hi,

I use a shared DVR that contains a class with the DB that your subpanel with all the dynamically loaded vis use.

I can add another class that contains messaging.

Since each of the dynamically loaded vis use an in place structure while manipulating the DB/MSG class and since a DVR is a reference I can share and break the data flow, I get a parallel architecture that provides a singleton over the database and no race condition inside a single action.

I assume no importance of race condition between plugins (in my real project the dynamically loaded vis are LVLibPs) actions and I have to pay attention to bottle necks:

1. If a single DB/MSG is used among a very large number of plugins then each singleton action of one of them will halt the rest, thus, I need to break down the DB into several more specified classes if speed is an issue.
   
2. Property nodes are slow since they work in the gui thread and turn the entire vi they are in into a vi running in the gui thread. Besides that, property nodes are operating through a single black box behind the scene, thus, they might turn into some kind of a bottle neck even if you make them reentrant. There is also the problem of debugging property nodes and dynamically loaded vis (moreover dynamically loaded vis from an LVLibP).
   

However, I think this is the most clean and beautiful way to work around plugin-plugin communication that will give you the benefit of a clear design any programmer following you could understand and manipulate easily.

Dror.

P.S. - I attached an example.

db and dynamic loading.zip

Edited May 2, 2012 by 0_o

[asbo]

Property nodes are slow since they work in the gui thread and turn the entire vi they are in into a vi running in the gui thread. Besides that, property nodes are operating through a single black box behind the scene, thus, they might turn into some kind of a bottle neck even if you make them reentrant. There is also the problem of debugging property nodes and dynamically loaded vis (moreover dynamically loaded vis from an LVLibP).

I agree that property nodes force everything into the UI thread, but I hadn't heard about the "black box" bit before. Is there substantiation to that and, I guess, is it even relevant since everything will be in a single thread anyway?

[K-node]

Regarding Property Nodes and the UI thread, we are referring to the property node of a front panel control and not class private data members that were 'made available through property nodes', right?

[JamesMc86]

I'm pretty sure only front panel item property nodes require a switch into the UI thread. This foes not force the whole VI into the UI thread (but they are pretty slow, a value property node is about 1300x slower than a local ore global variable.

I am surprised to hear do much discussion of DVRs for data sharing. They are certainly useful but provide no synchronisation if you need it. I personally use them more for memory management.

I also would say FGVs/LV2 globals are just as prone to race conditions! Any application where you have multiple data writers risk data loss with variables and anywhere with a non atomic read-modify-write risk race conditions (this can be achieved with an FGVS/AE but is not inherent.

But this all detracts. What you need is a form of inter process communication. The exact data depends on the type of data you are transferring. Queues are the best for streaming continuous data and pretty good for commands. Shared variables/FGVs are good for latest value transfers but are prone to losing data if the reader gets out of sync with the writer

(null)

[0_o]

I'm glad to hear that the property nodes transfer to the UI thread only in the case of front panel property nodes.

I based my knowledge on posts here in Lavag, probably the wrong ones.

I didn't know that it was 1300x slower than local variables.

Do you know what is the difference between bundle/unbundle from a class control and a local variable?

The problem with using bundle/unbundle is that is won't work outside of the class or even in it's children.

I wish NI made a fast basic class control data access which is OO, takes syncing into account, allows singleton and avoids races.

Yet, even before that day comes, I still think that in this case, for basic common DB parameter changing for a GUI system that might send HW requests, the example I sent is perfect.

I hate having many different parameters methodologies running across my vis with wires every where.

In the option I gave you'll have a single db class dvr wire running in the middle of your vi, as a life line, giving data to everyone along the data flow.

As for the DVR, it allows an easier to implement singleton than FGV that has many wrong interpretations and holds the operation far away from the object that uses them and if that wasn't enough, it holds the operation and data in one level depth for all the levels of the hierarchy with no code reuse.

In my option I don't have to predefine the operations I am about to use, I simply put each operation inside an in line structure inside the exact relevant place in the code. Nor do I need to manage variables spread out in my project with no way to organize them into a single logical framework.

If I don't need the operations to be singleton I can use property nodes directly on the class's dvr.

You just have to pay attention to the reentrancy (using several property nodes from the same class control might not work if they are all not reentrant - this is where I take the idea of a black box) and lock down problems (if you place a wait for queue element inside a dvr, for example, the dvr locks the rest of the code and you might wait forever).

Dror.

Edited May 3, 2012 by 0_o

[JamesMc86]

Do you know what is the difference between bundle/unbundle from a class control and a local variable?

The problem with using bundle/unbundle is that is won't work outside of the class or even in it's children.

I think that is the key difference to be honest but that is critical to an OO implementation. It enforces encapsulation which allows us to produce code with low coupling between components.

I wish NI made a fast basic class control data access which is OO, takes syncing into account, allows singleton and avoids races

Why does it need to be OO? OO is a way of designing/organising your application. The standard data transfer mechanisms in LabVIEW can work with OO or not. You can put objects through local variables, global variables, FGVs, DVRs, Queues, Notifiers whatever you want. Just because you want to work in an OO way doesn't mean that you are restricted by communication methods. What you really need to decide is what access/sync/buffering you need and these would decide which of these methods are needed.

As for your implementation it looks good if you need a variable/tag style access (no sync, latest value). By putting sequences of access in the in place structure you can avoid race conditions. You just have to consider if you have lots of access very fast in different parts of your application it could become a bottleneck as a shared resource.

[Labview Junky]

I haven't installed LV2011 yet so I can't see 0_o's db and dynamic loading example but I tried using queues and it actually was quite clean looking and super fast so I think that would be one of the best ways to go. Thanks for everyone's input it seems like there are many different ways of accomplishing what I want to do. One thing that I found that was really annoying when working with sub panels is that I couldn't probe my sub vi's because when the main GUI wanted to open then it would throw error 1144 if I already had the vi that the sub panel was trying to open open, I can't open them while running either. How do I go about debugging if I can't probe stuff.

[drjdpowell]

How do I go about debugging if I can't probe stuff.

I usually set my programs with the ability switch between using the sub panel, and opening separate windows (this is very easy to do). Then I debug with separate windows.

[PaulL]

I usually set my programs with the ability switch between using the sub panel, and opening separate windows (this is very easy to do). Then I debug with separate windows.

Exactly. You can also add breakpoints.

[asbo]

Exactly. You can also add breakpoints.

If you don't always want to debug, add a context menu item or an off-panel button with a hotkey set. Bury your breakpoint in the event handler of that object, and tada! On-demand block diagram!

[0_o]

I attached an LV2010 version of my example.

JamesMc86, if the operation is protected as a singleton you'll have the same bottleneck issues with every method you use.

Using get/set version of pseudo FGV does not implement the singleton design pattern and has only a little advantage over a simple global variable.

drjdpowell, great idea! I'll add it to the switch between using a vi from lvlib or using the version from lvlibp.

Can the sub panel property node "allow user to open block diagram" help here along the "pause vi" method?

Besides that, JamesMc86, pay attention to the vi properties being changed once it is inside a sub panel like the "prefered execution system" and to the issues of debugging reentrant vis.

If you need to debug executables you can open the "debug applicatio or shared library option" or even log events into the trace toolkit.

lv2010.zip

[todd]

Ignore error 1144. Then if the VI is already open, it will still run. While running, right click inside the subpanel to see "Open Block Diagram". Wait, that's in the subpanel help, right?

[JamesMc86]

JamesMc86, if the operation is protected as a singleton you'll have the same bottleneck issues with every method you use.

Using get/set version of pseudo FGV does not implement the singleton design pattern and has only a little advantage over a simple global variable.

Absolutely, That a get/set FGV is far superior to global variables is one of the most oversubscribed myths around at the minute. As you described anything which is a central access becomes a shared resource unless there is some buffering to reduce the affect) and this means parallel access will have to be arbitrated between. I am not saying that this is a bad method, the point I am trying to make is that there is no silver bullet or any one technique that fits every situation and as Ryman suggested in one of his posts a mix of techniques is normally required.

Your method would be described as a tag mechanism. It is good for when you need the latest value available on demand to many areas and you don't care about what the value used to be.

However if you need to guarantee every value is recieved then a queue (or network stream between targets) is far more appropriate. There are different care-abouts and needs.

Section 1 of the cRIO dev guide actually discusses the types of communications quite well although probably isn't a extensive discussion of all the methods available. Its at http://www.ni.com/compactriodevguide/sec1.htm

[ShaunR]

Using get/set version of pseudo FGV does not implement the singleton design pattern and has only a little advantage over a simple global variable.

The singleton pattern is a method to resolve a common problem associated with OOP when a single object is required to be instantiated and referenced rather than a new instance of an object (the default OOP instantiation-Grandma loves sucking eggs). Native LV does the opposite by default (a VI is a singleton - a single object referenced wherever it is placed - system wide). No design pattern is required as it is implicit to the language. If you don't want this behaviour, then it can be set to "re-entrant". This aspect, however is a side-show when talking about FGVs vs Globals.

Where the differences between FGVs, globals really lie is in "State" not "Data".

In non-data flow languages state has to be managed and an answer to the icky state management problem was OOP. In the dataflow paradigm, state is implicit in the language. However. Sometimes state managed by the [LabVIEW] language is not sufficient or appropriate. So, when it is advantageous to do so, we specifically design an object to store state (a FGV). The "get/set version of pseudo FGV" [sic] or "Action Engine" is the native labview method of replicating OOP style objects where you encapsulate the state of the data and manipulate it with methods.

Global variables cannot maintain state (only data). Neither can they be "sequenced" to maintain state via dataflow. This is the advantage of FGV over globals. Singleton behaviour is just the language specific being taken advantage of.

[Aristos Queue]

> Where the differences between FGVs, globals really lie is in "State" not "Data".

I'm still not seeing any difference between a get/set FGV and a global VI --- other than that the global VI is more performant. The get/set FGV is NOT an action engine. The original comment was that if you're only using them only for get and set, you should be using a global VI. That I agree with. Of course, we also say that you should not be using a global VI. By the transitive property, you should not be using a get/set FGV. :-)

[0_o]

Aristos Queue, oh how I wish your post appeared on page 1 instead of 2.

Whoever reads only page 1 will get the wrong impression.

In my job I found it really hard to make people understand what you just wrote.

I spent months arguing back and forth to no avail.

Pseudo FGV really turned into some kind of a religion.

However, I wish NI came up with a way to speed up class control property nodes.

Indexing the class controls into a red black tree over a defragged class space should make it possible to speed up the access to O(bundle/unbundle*hierarchy depth).

JamesMc86, thanks for the great link. I'm using an in house hardware so I never checked out the cRio documents.

This is going to change soon since our PLL project is delayed and our hardware is not fast enough without it, thus, hopefully, cRio here I come

ShaunR, if you write a calculator with +-*/ that saves the state of X and Y through a pseudo FGV with no action engine around the +-*/ you'll get into race conditions.

If you try to calculate in parallel both a + operand and the * operand it will depend which action checked the state of X and Y first and if each action's data flow managed to change the state before the other started. I hope I'm not trying to teach grandma to suck eggs.

Edited May 6, 2012 by 0_o

[ShaunR]

ShaunR, if you write a calculator with +-*/ that saves the state of X and Y through a pseudo FGV with no action engine around the +-*/ you'll get into race conditions.

If you try to calculate in parallel both a + operand and the * operand it will depend which action checked the state of X and Y first and if each action's data flow managed to change the state before the other started. I hope I'm not trying to teach grandma to suck eggs.

I'm not quite with you on this one.The storage would be a part of the calculator AE and there would be no "pesudo FGV". FGV stands for "Functional Global Variable and the functions would be +/- which is why I don't really discriminate between a "Get/Set FGV" and an AE. As long as the functions are atomic, then the +/- FGV would complete its operation (read the value, operate on it, then output) and the other operation could not read the stored value whilst this is happening (as it can with Globals). The race condition that FGVs address is that the value cannot be read in other parts of the code until the current function has completed. So in your example, the result of 2+2-1 (with two VIs in parallel) will always be 3 once both have been executed. With a global it could be 5.

Your example, however, uses functions which are commutative. So when you lay down your Calc VIs and do an add and subtract in parallel. You cannot guarantee which of the VIs will be executed first by LabVIEW, but, after they both have executed, the answer will be correct. If the order is important (i.e. the operations are not commutative) then you have a different type of race condition that has nothing to do with access timing to the underlying stored value (which is what globals suffer from). However. unlike globals, FGVs have an error terminal, so if it sequence is important, they can be sequenced via the terminals (e.g add then subtract).

[0_o]

I agree!

If you are using FGV the right way they can work just as well as in my example and I have nothingto say.

The thing that bugged me was "get/set FGV" you talked about.

To me it sounded as if you were using the FGV to get/set the X/Y values since you don't get/set an operation.

Your corrected version is not talking about get/set a value but rather do +-*/.

An entire operation over the DB should be atomic, not just the get/set.

Otherwise, there is no much difference between a global and FGV as Aristos Queue just said.

Anyhow, this is not related to the topic and there are many posts about this issue.

Dror.

P.S. -

1. I think that with a global 2+2-1 could be 1 or even -1 but not 5.

2. If you parallel the operations then you can't guarantee the order of +-*/ in FGV too.