OOP byVal vs byRef

[Samapico]

This might be a first of many topics / posts I'll make in the future, since we're starting to prepare a complete rebuild of our software platform... The current one is in LabVIEW, but it's pretty ugly... Nested .lvlib's, hundreds of global variables without any control, a lot of duplicated code... Now that we acquired more skills and experience with LabVIEW, and that our requirements are clearer, we can/want to build something that is upgradeable, modular, flexible, etc.

The beast is pretty huge. It's an inspection system that uses an industrial robot. So we have cameras, IO modules that communicate with the robot and other stuff, inspection algorithms and filters, threads that control the overall sequence with the robot and inspection threads, etc.

I read a bunch of topics in these forums about OOP and architecture in general, and from what I understand, LVOOP byRef should be avoided, unless necessary, since it's kind of opposed to the nature of LabVIEW. However, I can't imagine our platform without a lot of parallelism, and parallelism doesn't make sense to me with byVal objects...

I'll try to save the architecture-related questions for later, but how would you build a large application without (or with as little as possible) parallelism, or in such a way that byRef objects don't end up being used everywhere?

Ideally, we want (mostly) everything to be as modular as possible (possibly dynamically loading stuff as plugins, but that's beside the point) so that a module responds to events, and generates events that the other modules "catch". However, modules will often require data from other modules, and if many modules require data from one module, how can you get that data by value? My guts tell me that byRef classes would work wonderfully, but at the same time, it's probably going to lead to some bad programming habits, or bad design.

This is probably pretty vague, and some parts might not make sense, but I don't really know where to start myself. I'm hoping that your guidance will at least make me ask the right questions

Anyway, thanks in advance

[Fire]

Hi

your project sound be huge enough to think about a (Mobile) Agent System.

I would like to bring the HGFBaseClassLibrary to your attention. This class library provides (base-) classes to deal with LabVIEW Object as entities in a distributed computing environment respecting strictly the LabVIEW dataflow paradigm.

The first PowerPoint presentation gives you an introduction in english language. You can also read the publication in the GSI Scientific Report 2008. All other publications are in german language. You can also download a snapshot of the ongoing development from this page. The zip-file are splitted into packages containing the (base-) classes and corresponding examples. The software is published under the terms of the GPL v3.

Maybe you like this approach.

Regards Holger

[Aristos Queue]

I was just about to post a link to HGF. Holger beat me to it. :-)

The other alternate architecture to look into is the NI Week 2010 Actor Framework:

http://forums.ni.com/t5/LabVIEW/Actor-Framework-from-NI-Week-2010-presentation/td-p/1211309

SHORT STORY:

A) Use of by-value classes does not mean you never use references. It means you think before you use references.

B) You should avoid the trap of thinking of an object as a living process. That thought model works in C# and JAVA. Instead, think of a top-level VI as a living process and model communications among those running VIs. The state of each VI is an object. The messages passed among the VIs are objects. If you can hold that separation in your mind, whatever design you choose will be much much much stronger.

[Samapico]

Thanks for the links... I'll look into that......... and probably ask more questions then

[Daklu]

I'll try to save the architecture-related questions for later, but how would you build a large application without (or with as little as possible) parallelism, or in such a way that byRef objects don't end up being used everywhere?

Parallelism isn't inherently bad, nor does it necessarily require by ref objects. You can have lots of parallel loops executing without using any by-ref data. A recent small project I did had... I dunno... a dozen or so(?) loops running in parallel depending on the state of the application without a single bit of by-ref data. (I used queues to pass the data around instead of accessing it directly.) I think I remember Ben mentioning having hundreds of loops running in parallel. By-ref data structures are either a) a shortcut to sharing data because you don't know how or don't want to take the time to implement a by-value solution, or b) a necessary sacrifice to meet performance constraints.

You mentioned you want a high degree of modularity. Modularity --> Loose Coupling --> Abstraction --> Additional Complexity. How much decoupling do you want? Using some sort of messaging system as glue for parallel components probably produces some of the least coupled code. I've found it works well for desktop applications when I don't really know what the final design will look like. It may not be a good solution for high performance data collection.

...modules will often require data from other modules, and if many modules require data from one module, how can you get that data by value?

If one module is producing data for several other modules to consume, I'd look into using some form of publish-subscribe or observer pattern. One solution is to have the producer send a data message to a mediator loop, which then resends the message to each consumer. (Or to the next mediator loop in the chain on the way to a consumer.) Another solution is to require each consumer to provide a queue when subscribing so the producer can pipe data directly to it.

If this is the first big OOP design your group is undertaking I can almost guarantee you'll get the design wrong. Unless you have time and budget for your group to work through the OOP design learning curve, I'd recommend spending a little money to get an architectual consultant involved in the initial phases. They probably won't be able to give you a complete solution you can run off and implement, but they'll help prevent you from going down the wrong path and save you time and money in the long run. I hesitate to mention names for fear of leaving someone out, but I'd feel comfortable hiring Chris Relf at VI Engineering, Tomi Maila at JKI, or Stephen Mercer at NI. (I'm sure there are others at all three of those companies who could help as well... those are just the three who I preceive as having the most OOP experience there.)

You should avoid the trap of thinking of an object as a living process.

Spot on. That line of thought tends to put some limitations on how you use objects.

Instead, think of a top-level VI as a living process and model communications among those running VIs.

Doh! May I propose a slight variation?

I view the loop as LV's fundamental living process. (To be precise I should say the "parallel execution path" is the fundamental living process, but loops are the most common manifestation of PEPs.) We choose to wrap a loop (or group of loops) in a vi, class, or library to simplify the interface to its functionality and provide some level of encapsulation.

The state of each VI loop is an object. The messages passed among the VIs loops are objects.

[Black Pearl]

Sharing data between parallel process requires some kind of reference. You can either use a by-ref class implementation or use a LV-native reference communication (queue, user event, notifier).

Using a by-ref object is similar to the LV2-style global/ActionEngine (with the bonus that you have inheritance as well as multiple instances). A situation where you won't be able to avoid by-ref is recursive data structures. Another situation where you might use by-ref is when translating code from other OOP languages or using their design patterns/architecture ('cause they are by-ref).

Using a messenging system and by-val OOP as Daklu mentioned will give you an abstraction between the data (object) and the communication. If you use uml to architect the system, this is really nice as you code the class diagrams into the by-val classes and the sequence diagrams into the communications. Also getting into troubles due to race-conditions is much less likely when using queues&friends. Also, clusters (especially type def'ed) are straight foreward converted into by-val classes (just adding the OOP super-powers).

Felix

[jdunham]

I'll try to save the architecture-related questions for later, but how would you build a large application without (or with as little as possible) parallelism, or in such a way that byRef objects don't end up being used everywhere?

My first few LabVIEW apps were not parallel. I used plenty of subvis, so it's wasn't bad spaghetti code, but I very quickly ran up against difficulties making the apps easy to develop and maintain. I don't think that a big software system that accomplishes many things and is not frustrating for humans to use can be mapped into a straight dataflow model containing nested loops and cases. It would be nice, but I don't think it's possible.

As Daklu said, attempting a monolithic program like this will increase the coupling between your modules and makes the app harder to build and maintain.

[Aristos Queue]

Doh! May I propose a slight variation?

Yes, you're right... for now. Does it tell you anything about some of the design proposals I've been working on lately that I've kind of stopped differentiating "structure node" and "VI"?

[Daklu]

Does it tell you anything about some of the design proposals I've been working on lately that I've kind of stopped differentiating "structure node" and "VI"?

Well it certainly doesn't tell me enough...

If I were to take a shot in the dark I'd guess some kind of feature that wraps structure nodes in a vi-ish type icon, perhaps without requiring it to be saved as a separate file. Kind of an "embedded" (or maybe "parasitic") sub-vi. Double clicking on it might open it in a separate window for viewing and editing with the changes being saved back to the host vi. Maybe in the project window the host and embedded vis will be displayed as a tree, complete with twist-outs so we can expand and collapse the view. Possibly there are different kinds of embeds for different structure node. A case structure embed could have a project window twist out that reveals all the cases. Same idea for an event structure embed. For and while loop embed twist-outs could show the number of iterations and/or stop conditions. Stacked sequence embeds will automatically torch all ethernet and wifi hardware, and crunch the usb host processor to prevent it from escaping from that computer. For good measure I think it ought to expose the user to electroshock therapy.

Embeds are an interesting idea, but I haven't convinced myself they're a *good* idea. I can't figure out what problem it solves. Block diagrams are too big? Use sub vis. Too many files on disk? I know a lot of people complain about that (I did when I first started using LV) but there are real benefits to having lots of sub vis when it comes to multi-developer projects, source control, and testing. Not enough conpane terminals on sub vis? Use clusters or classes. I suppose one benefit would be the ability to use local variables and control references in the embeds directly instead of passing refnums through. Is that enough? I dunno... part of me thinks embeds would be encouraging bad programming habits.

I considered a few other things too:

-Eliminating sub vis completely and forcing us to write applications using a single block diagram. (Yeah, right.)

-Eliminating structure nodes and allowing unhindered data flow. Nodes have a nasty habit of holding on to data at exit boundaries when it could be released to the next data consumer, but that would fundamentally change the way Labview behaves and make programming harder I think.

But, you know... I'd only throw that out there if I were to take a shot in the dark.

(How far off target am I?)

[Black Pearl]

Yes, you're right... for now. Does it tell you anything about some of the design proposals I've been working on lately that I've kind of stopped differentiating "structure node" and "VI"?

If I climb a mountain and view everything from such a high abstraction level, that's clear.

You propably need to do software in graphical IDEs such as LabVIEW or uml to see it. But actually it boils down to software-developement basics. You have a box around something (the SubVI boundary, a loop, encapsulation in OOP, virtual folder). Formulated it the wrong (graphical) way. Encapsulation is the abstract concept, the box around is the graphical (language-specific) implementation. But it's hen and egg (the ancient greeks and geometry, that's long before we invented tubes and transistors).

Second step, we do something with information/data. So let's put this data inside the box. Call it state information, variable, invariant, whatever. Put everything (data) we have in a container. Place containers inside container. Recurse or iterate that process. This way, we just get order in the chaos, information out of the entropy, the ultimate process of creation.

All we still need is a way of interaction: queues, tunnels, controls, interfaces, acess scope, a door to open the box ....

But: is this the nature of being? Or is this just a reflection of or mind (we live in rooms that have doors that only go to specific other rooms, no wormholes possible).

Felix

Edited June 29, 2011 by Black Pearl

[Omar Mussa]

Embeds are an interesting idea, but I haven't convinced myself they're a *good* idea.

I agree. I saw this product called Flowstone recently (don't take this as any kind of endorsement) ... and it looks like they 'embed' sub-nodes (they don't have VIs of course ). At around the 6:40 mark of this

, it shows the sub-node embedded in the main app node. Its pretty interesting in that you can 'reuse' a subcomponents UI but at the same time customize it - similar to subpanelling a VI in LabVIEW only you can edit the position of subpanelled controls and indicators at edit time. Anyways, it looks interesting conceptually.

[Aristos Queue]

Both Daklu and Black Pearl have some gist of it. But it isn't perhaps about eliminating one or the other. If I take Black Pearl's analogy and extend it, having a bathroom separate from a kitchen is healthy, and having one room that tried to support both activities would end up serving neither goal well. But notice that both the bathroom and the kitchen have lights. They both have running water. Air conditioning and heating ducts. Walls. Electrical sockets. Doors. Indeed, there's a whole set of "room" attributes that both things have. Bringing this analogy back to LabVIEW...

• We have a "conpane" for subVIs but we have "tunnels" for structures. Do we need two separate concepts? Maybe yes, maybe no.
• Both have a description and tip. Are there other settings that apply equally well to both? Could a structure node be meaningfully marked as password-protected? How about marking a structure as "UI Thread"? The execution settings page of VI Properties is the most intriguing point of congruence between VI and structure.
• What about a subVI that is marked as "inline" -- can we just show its contents directly? Could a user looking at the caller VI be able to edit the inlined subVI right there without having to switch windows? How bad would that be? How common are subVIs small enough for that to be even slightly useful?
• What if I had a subVI that took a single scalar input and produced a single scalar output... could I have "autoindexing" on the subVI and just wire an array to the input and get an array out?
• How much commonality is there between marking a subVI as "reentrant" and marking a for loop as "parallel"?

How far could we -- in theory -- go with unifying these two aspects of LabVIEW? These are two features that essentially developed in isolation from each other, and although I have no idea whether anything will come of it, it's been interesting to explore the design space around the commonalities. This is the part of my job they call "research". Just sitting around drawing pictures... :-)

[Black Pearl]

AQ, thanks for this explanation.

More than once (when VI scripting) I stumbeled on the fact, that unrelated objects actually have a common set of properties.

Are your considerations already down to the SW level (e.g. about an alternative class ancestry or other means that group these properties together, e.g. seperate objects that will be included in the 'old' obejct)?

On the abstract level, structures and SubVIs have a lot of similarities. Most obvious: both own a diagram (sometimes more than one, e.g. Case Structure). This is important, as we now can distinguish between 'inside' and 'outside'. Having an inside and an outside, we can have directions (input and output). Also controls/indicators, tunnels, shift registers, case selector, n-terminal (for loop) have an inner terminal and an outer terminal.

• What about a subVI that is marked as "inline" -- can we just show its contents directly? Could a user looking at the caller VI be able to edit the inlined subVI right there without having to switch windows? How bad would that be? How common are subVIs small enough for that to be even slightly useful?
• What if I had a subVI that took a single scalar input and produced a single scalar output... could I have "autoindexing" on the subVI and just wire an array to the input and get an array out?

Continue that route! One feature I'd like in LVOOP would be, if I could write a VI that operates on an array of objects and perform the same type propagation as the array primitives do.

I need to make a bigger explanation of my thoughts. I was studying the concept of multiplicities in uml. This is a feature not covered by LV at all. So at first I would need a palette for set theroy operation like union for multiplicities that require uniqueness of the elements. An example of this is the bag collection posted by Daklu.

So my own try was to use an array of objects and add a new object wasn't found, could be performed in a for loop for an union operation. If the code was manually inlined, I got the type propagation I wanted. Making it a SubVI, I always had an array of parent class, even if all elements where of child class. This requires a for loop with type casts outside the SubVI (so actually more code than contained in the SubVI).

Well, this was just leading to a very big set of ideas that would get maybe 20 votes on IE, as most users will just don't care about this abstraction layer.

Felix