Composite Pattern help

[jr_BobDobbs]

Hi all, first post here, hopefully it's the right place. I posted over on the dark side in the AF forum but was lead into the direction of the composite pattern rather than to use the AF for this portion of my application. 

 

The reasoning seems sound, so I didn't want to continue my post in the AF forum and it seems like there's a little more OO talk over here than there. 

 

So I read the wiki (http://en.wikipedia.org/wiki/Composite_pattern), and looked at the Graphics example in the OO shipping examples in LabVIEW, as well as the OODesign.com page for it (http://www.oodesign.com/composite-pattern.html) but I'm still a little lost on how to implement it in my application. I'm trying to create a sequence engine, the basic system looks like this;

 

Protocol - the definition of the "sequence". 

Group - The highest level of heirarchy, contains an array of children called "Step". This iterates between one and N times. Every time a group iterates it increments a variable used in another part of the application.

Step - Next level down, has an array of children called "Functions". A step can have anywhere from one to N repetitions, essentially a For loop of "Functions"

Function - Next level down, contains an array of "Actions"

Action - The base component. This is what actually executes.

 

The composite pattern says that there's an interface object and than a leaf object and a composition object which both inherit from the interface. The leaf is where the action happens, the "Execute" as it were. So I think my Action class is a leaf. But if so than the Function, Step, Group, and Protocol, are all variations on a composition and that seems to not make sense to me. 

 

I feel like I'm not looking at this the right way, or if I am, than there's some piece of the puzzle that I'm missing to make this all fall into place in my head. 

 

Any help would be appreciated to try and point me in the right direction.

[drjdpowell]

The usual pattern that I (and others, i think) have used is analogous to a file directory: directories contain a mix of different files and also subdirectories (which can contain sub subdirectories, etc., etc.).   The classes are a Parent, with children File and Directory, the later containing an array of Parent.  File can have subclasses for different file types, and there can also be different subclasses of Directory.  It’s a recursive tree-like structure

 

But your hierarchy of different levels is not the same as a tree-like structure.  Steps can only contain an array of Functions, and so on.  You would have a common parent (which defines your “Execute†or “Do†main method) but you need each subclass to contain an array of a specific other subclass.  So Step contains an array of Functions.

 

But, are you sure you need a specific hierarchy like this?   Seems more work and unnecessarily rigid.   Why can’t the User call a Function inside another Function?  If a User wants to execute an Action as part of a Group, why require the extra work of creating a Step with one Action?

[jr_BobDobbs]

This is the pattern I've followed before when writing sequencers (without using OOP, this is my first), but they were just as you described. There were sequences and frames, frames did an action and sequences consisted of frames or more sequences. So that at least is familiar to me. 

 

The problem with this particular application is that historically (I'm re-writing existing unmanageable code) they've used this rigid hierarchy and I'm having trouble generalizing it down into a basic frame and sequence problem. The problem I'm solving is not test sequencing, but it's close enough to it that for the benefit of explanation I'll explain it that way since I think we can all understand that (and since I can't explain what we're really doing due to NDA concerns).

 

So, consider that there's 8 DUTs. These DUT's are more or less alike and will be tested by the same piece of hardware. This piece of hardware applies a voltage to each DUT, but can only apply that voltage to one at a time. The amount of voltage that's applied is specific to a sequence that is specific to each DUT. So DUT₁ may get [5v, 3.3v, 5v, 1v, 3.3v] and DUT₂ may get [3.3v, 1v, 5v, 3.3v]. Which step of each DUTs sequence that we're on is determined by the group. So, there's a Group0 which does some pre-work, but applies no voltage, and starting with Group1 we apply the first voltage. Also, each Group may apply the voltage in a little different way, maybe we pulse the voltage 5 times, maybe we apply a voltage and wait, than apply the same voltage again. This is defined by the Steps. So, all of the DUTs have a Step performed on them, than we go to the next Step. When we run out of Steps we either increment the cycle count of the Group, or move to the next Group if we've reached the last cycle of this particular Group. Steps contain Functions, and each Function is performed fully on a DUT before you move on to the next DUT. So DUT₁ has the first Function performed, (which is something like, move the DUT to the voltage source, apply the voltage, wait, stop the voltage), than DUT₂ and so on until we've done all DUTs and than we go back to DUT₁ and do the next Function. Each Function consists of the specific Actions that define how to apply the voltage, the move, voltage on, voltage off, etc... There's also those random requirements that are thrown in (that always caught me up on my CLD and CLA test) like, the last voltage that we apply always has to be applied at the same time for all DUTs, even if some DUTs have shorter sequences than others (so in the above example the 3.3v of DUT₂ would get saved to run with the 3.3v of DUT₁) and some certain aspects of a function (repetitions for example) are specific to what voltage we're applying that cycle. 

 

Sorry for the wall of text, hopefully it makes sense. But this is the reason I'm a little stumped. I agree that the basic premise of this structure is composite, but I don't know how to generalize this and still capture some of the gotcha's. My first iteration of code had me building a specific object for the Group, Step, and Function, but in reality they basically do almost the same thing. Get the current child, pass it to the sequence engine, increment an iterator, etc... and I had a ton of repeated code, so I knew that was wrong. I guess a specific ancestor for each of these objects fixes the problem of duplicate code, but again than I fall into mis-using the pattern. A client can not handle a Group with a similar interface that it uses for a Step or a Function, because of the small differences I'm seeing in them. Or perhaps I'm looking to hard at the problem and not seeing the whole, which is where the solution really is.

Edited May 13, 2015 by jr_BobDobbs

[ShaunR]

Lots of contending terminology here. So I will adopt Test Stand terminology since this is what we are trying to emulate. Sequences, Steps and Actions.

 

One way forward is to consider Sequences and Steps as an ordered list (I posted an example on here somewhere). The "List Object" has add, remove, delete insert etc. You may consider an Action as a single element list or you might wish to break it out into a discrete item. It doesn't matter.

 

If you derive Sequence and Step from a List object. That gives you all the management items. Through composition (see the vehicle example shipped with LabVIEW) you can, add  an index as a current execution pointer and a string name so we can display a label for our Sequence/Step.

 

That's the easy bit. It gives you management functions and a currently executing marker with a display name that you can show the user. It also gives you a list of items to iterate over. The hard part is how much do we put in the list objects and how much do we put in the contained items within the list object......

 

So this is great. We steam ahead and add skip, move up and move down for our lists and pretty soon we have the whole UI mapped out and working where the user can add, remove and modify sequences and steps.

 

Then we get to actions. This is the point when I abandon POOP so the others will have to jump in here.

Edited May 13, 2015 by ShaunR

[jr_BobDobbs]

Through composition (see the vehicle example shipped with LabVIEW) ...

 

 

Can you point me to this example? Searching the example finder yields nothing, and the only vehicle example I find in the //Object-Oriented Programming folder of examples deals with custom probes as doesn't seem like an example for composition.

[ShaunR]

Can you point me to this example? Searching the example finder yields nothing, and the only vehicle example I find in the //Object-Oriented Programming folder of examples deals with custom probes as doesn't seem like an example for composition.

Well. They have screwed that up in later versions.

 

Here's the old 2009 version

.old example.zip

 

You will see it adds "Bed cargo capacity" to "Vehicle" through composition for the "Truck" object.

 

I think the Graphics example also uses composition unless they changed that too.

Edited May 13, 2015 by ShaunR

[shoneill]

So I read the wiki (http://en.wikipedia.org/wiki/Composite_pattern), and looked at the Graphics example in the OO shipping examples in LabVIEW, as well as the OODesign.com page for it (http://www.oodesign.com/composite-pattern.html) but I'm still a little lost on how to implement it in my application. I'm trying to create a sequence engine, the basic system looks like this;

 

This post will not directly address your specific problem but rather concentrates on clearing up some confusion which cost me personally many headaches and which I have not seen addressed before (I never actually looked that hard).

 

I have had lots of trouble finding the link between theoretical discussions on the Composite Pattern and actual implementations (Composition) in LabVIEW for years.  A short time ago, the penny dropped (figuratively).

 

The Composite Pattern refers to a single hierarchy tree where you have a number of child classes.  You define one class of this hierarchy Y (for example a single step above the base class X) to actually contain an array of the base class.  It is simply an object which allows a group of objects of base Type X to be operated on as if the object containing them all was also a simple Object X. You can therefore easily picture a hierarchy of objects by simply having an array of Objects X where some of them are actually Object Y containing other objects, some of which again may be X and some Y and so on.  By abstracting the "many X" into Y (which due to inheritance is also X) you have implemented the Composite Pattern.

 

The Composite Pattern (as the GoF describe) is certainly not what NI propagates as Composition (a solution where many objects of DIFFERENT hierarchies are collated into a single object in order to implement different functionalities).

 

The key sentences in the links you provide regarding the Composite Pattern are:

Implementing the composite pattern lets clients treat individual objects and compositions uniformly (Y containing two or more X is itself an object of Type X)

and

There are times when a program needs to manipulate a tree data structure and it is necessary to treat both Branches as well as Leaf Nodes uniformly (Whatever operations you can perform on X must also be doable with Y - they have the SAME interface)

 

Composite Pattern != Composition

 

Composition is a general term and can be used for defining a class A which contains classes B,C and D all of which have different interfaces and hierarchy trees.  This is a grouping, a Composite object.  The word Composite here does NOT infer application of the Composite Pattern.  The Composite Pattern is a small sub-set of possible object groupings or Compositions because it has much different requirements and is much more restrictive.

 

 

WAY back in college, I had a really hard time understanding inheritance in OOP.  I couldn't figure out how the mapping from child to parent could possibly be implemented in a way that was stable and robust.  It was only after LVOOP came to LabVIEW that the proverbial penny dropped (I'm not wealthy, the time it takes for pennys to drop for me means I'm actually really really poor).  Inheritance is an EDIT-TIME declaration, not a RUN-TIME declaration.  To anyone who understands OOP this is a no-brainer.  But my brain interpreted it differently and because nobody cared to make this distinction when teaching it to us at college I lost out on nearly 20 years of understanding OOP.

[shoneill]

So, consider that there's 8 DUTs. These DUT's are more or less alike and will be tested by the same piece of hardware. This piece of hardware applies a voltage to each DUT, but can only apply that voltage to one at a time. The amount of voltage that's applied is specific to a sequence that is specific to each DUT. So DUT₁ may get [5v, 3.3v, 5v, 1v, 3.3v] and DUT₂ may get [3.3v, 1v, 5v, 3.3v]. Which step of each DUTs sequence that we're on is determined by the group. So, there's a Group0 which does some pre-work, but applies no voltage, and starting with Group1 we apply the first voltage. Also, each Group may apply the voltage in a little different way, maybe we pulse the voltage 5 times, maybe we apply a voltage and wait, than apply the same voltage again. This is defined by the Steps. So, all of the DUTs have a Step performed on them, than we go to the next Step. When we run out of Steps we either increment the cycle count of the Group, or move to the next Group if we've reached the last cycle of this particular Group. Steps contain Functions, and each Function is performed fully on a DUT before you move on to the next DUT. So DUT₁ has the first Function performed, (which is something like, move the DUT to the voltage source, apply the voltage, wait, stop the voltage), than DUT₂ and so on until we've done all DUTs and than we go back to DUT₁ and do the next Function. Each Function consists of the specific Actions that define how to apply the voltage, the move, voltage on, voltage off, etc... There's also those random requirements that are thrown in (that always caught me up on my CLD and CLA test) like, the last voltage that we apply always has to be applied at the same time for all DUTs, even if some DUTs have shorter sequences than others (so in the above example the 3.3v of DUT₂ would get saved to run with the 3.3v of DUT₁) and some certain aspects of a function (repetitions for example) are specific to what voltage we're applying that cycle. 

 

The composite pattern is not really what you need here since you admit yourself that the different levels of the hierarchy require different interfaces.

 

What you need is simple object composition (a grouping of objects to increase functionality) which is NOT the same as the Composite Pattern.

 

I think your different object hierarchies are relatively clear.  Your confusion arises perhaps from the false expectation that all levels of your object composition need to share interfaces.

 

So a Group is a single step for ALL DUTs, right?  Group 0 is Initialise essentially, Group 1 is Step 1, Group 2 is Step 2?  This sounds like it may actually be a candidate for the Composite Pattern.  Why not implement a Group as a Composite of Step with the same interface?  This will allow you to treat groups and steps transparently.  I don't know if that functionality would actually aid your development, I'm too far away to be able to judge that but it would allow you to have "Virtual DUTs" which are actually comprised of several individual DUTs.

 

Your functions are a completely different beast, as are your Actions.  Again you could theoretically define Functions and Actions so that the Composite Pattern could apply.  Again this would allow the generation of an Action which actually contains several other actions.

 

Part 1:

 

Base Class : Step

Child Class : Group (contains 1..N Steps)

A Step is an object which represents a stage of your testing procedure for a single DUT.  A group represents the same thing but for 1..N DUTs.  A group may contain other groups.  A group and a Step can be used interchangeably within the program.  The interface you would code to is that of the Step (Parent class).

 

Part 2:

 

Base Class : Action

Child Class : Function (contains 1..N Actions)

An Action is a single action (Set Voltage, Raise Alarm, whatever). A Function is also an action but contains 1..N other Actions.  A Function can also contain other Functions (Hierarchical definition of test procedures).

 

So I could imagine (without in-depth knowledge of EXACTLY what you're trying to do) that you could use a combination of Object Composition  (Action&Function or Step&Group) and the Composite Pattern (Step contains 1..N Actions) to achieve what you want.

[jr_BobDobbs]

 

I have had lots of trouble finding the link between theoretical discussions on the Composite Pattern and actual implementations (Composition) in LabVIEW for years.  A short time ago, the penny dropped (figuratively).

 

 

Thank you, I was starting to go a little crazy. It seems over at ni.com depending on the white paper you're reading or which example you're looking at the terms composite and composition are thrown around to describe both rather wily-nilly. 

 

I hadn't considered your implementation that you put in your second post, but I'm going to give it a shot. Last night I came up with a base object called Frame, which had an Execute method. Everything inherits from Frame and on creation just passes a JSON message down to each child, cleaving off what it's childrens definitions look like. In the Execute of a Group, Step, and Function it pops the next child from a stack of children, as well as increments the appropriate counter (Cycle, Repetition, etc...) Then each Action is built upon the base class, i.e. a Move action which calls the appropriate drivers to move in it's execute, etc... That way my client just calls Execute on the top-level Frame and it recursively executes all of the actions. I also pass a reference to my Client down the chain so that I can check if we're paused or stopped, etc... This is obviously not the Composite Pattern, more like just inheritance. 

 

I've got a working system currently, but my biggest concern is how difficult is it going to be in six months when my boss comes to me and says he wants to put another level between Step and Function. Or, like this morning when they come to me and say that it would be really nice if the Actions could be optimized, like, if DUT₁ is lined up to get 3.3V and that in turn lines up some other DUTs in place to get the voltage they require, go ahead and put the voltage on all of them, you know, non-sequentially, which now means my Function object has to evaluate all of it's Actions first and re-arrange if necessary. 

Edited May 14, 2015 by jr_BobDobbs

[jr_BobDobbs]

 

Here's what I think I'm going to go with. The base class is a frame, which consists of a name and the enqueuer for the Client. It has two child classes, Function and Collection. A collection is composed of Functions or Collections and iterates a number of times. So, in my hierarchy, a collection covers the Group and Step requirements, with one caveat. It has to know if it's a group, because if it is it does something special each time it completes an iteration by incrementing a cycle counter managed the Client Actor. The function consists of an Array of actions, or in the case of the Optimized Action, the array of actors and the algorithm for how to optimize those Actions. The optimize algorithm will be a plugin, so it's kind of like a mix of Composite and Decorator. It all breaks down to JSON and seems to work relatively well in the prototype stage tonight. 

 

Any thoughts from anyone on possible pitfalls here? 

 

For the record, the Client is an Actor to integrate with the rest of the applications architecture which is actor based, also, just passing it's enqueuer to each Frame allows me to maintain the Frame object in the Client, and not have to worry about how each Frame talks back to the Client (since you can't have place an object in the Class Data of a class that belongs to a Class holding the first object in it's Class Data, although I think there's an easier way to say that). 

[ShaunR]

So, in my hierarchy, a collection covers the Group and Step requirements,

Uhuh. Like the List object I described earlier

 

with one caveat. It has to know if it's a group, because if it is it does something special each time it completes an iteration by incrementing a cycle counter managed the Client Actor.

Uhuh. One list with an execution pointer  and one without Or, as you have decided,  A single list with a switch (purists won't like that)

 

The function consists of an Array of actions, or in the case of the Optimized Action, the array of actors and the algorithm for how to optimize those Actions.

Uhuh. A list. 

All sounds very familiar to me.

 

The optimize algorithm will be a plugin, so it's kind of like a mix of Composite and Decorator. It all breaks down to JSON and seems to work relatively well in the prototype stage tonight. 

 

Any thoughts from anyone on possible pitfalls here?

No pitfalls on the surface that I can see (the devil is in the detail). Just that your "Frame" can also be considered a List/collection of one.

Edited May 15, 2015 by ShaunR

[jr_BobDobbs]

Uhuh. Like the List object I described earlier

 

But I called it a collection. 

[ShaunR]

But I called it a collection. 

 

I say liken [sic] you say litchen [sic] lets call the whole thing moss.

[smithd]

I'm not sure if it fits your needs but have you looked at this?

http://www.ptpartners.co.uk/ptp-sequencer/

It looks pretty cool (haven't used it but theres a video series on that page). It seems to cover all the basics you're developing here and it's got a 'run next step' function you could call from anywhere including an actor. Seems like it might fit your needs.

Also on the tools network: http://sine.ni.com/nips/cds/view/p/lang/en/nid/212277

[shoneill]

Uhuh. Like the List object I described earlier

 

 

Almost.  There's one major difference you're missing out on here Shaun.

 

When designing the "list" according to the Composite Pattern the List itself is an instance of the objects it contains.  You cannot do that with your List class.  If I make a List of "Frame", the List is a List, not a Frame.  In the Composite pattern, the List is also a Frame, allowing it to be placed in another List and so on recursively.  It's the interchangeability of the List and the object it contains which is the essential part of the Composite pattern.

 

In BobDobbs code shown above the user can program with a Frame Collection or Frame interchangeably.  In your case, all of the objects are inheriting from the List class so you're going about things the opposite way.  By making EVERYTHING a List you also have a single interface but the downside is that when you DO want to have a single object rather than a collection or list, you still have a list (and associated now non-functioning interface elements).  This is a variant of the problem where the "single interface" is the sum of all possible implementable interfaces within the problem space.  Why, if the user program doesn't need to know, is it presented with a List Interface for the objects being processed?

[ShaunR]

Almost.  There's one major difference you're missing out on here Shaun.

 

When designing the "list" according to the Composite Pattern the List itself is an instance of the objects it contains.  You cannot do that with your List class.  If I make a List of "Frame", the List is a List, not a Frame.  In the Composite pattern, the List is also a Frame, allowing it to be placed in another List and so on recursively.  It's the interchangeability of the List and the object it contains which is the essential part of the Composite pattern.

In BobDobbs code shown above the user can program with a Frame Collection or Frame interchangeably.  In your case, all of the objects are inheriting from the List class so you're going about things the opposite way.  By making EVERYTHING a List you also have a single interface but the downside is that when you DO want to have a single object rather than a collection or list, you still have a list (and associated now non-functioning interface elements).  This is a variant of the problem where the "single interface" is the sum of all possible implementable interfaces within the problem space.  Why, if the user program doesn't need to know, is it presented with a List Interface for the objects being processed?

I never said that I was using a pattern of any kind. All I am doing is using common denominator implementation for a very specific part of the problem. Frames and steps are both lists and either can contain any object type; it is just a semantic difference.

 

I know what you are getting at, though. This is a good example of the tradesmans viewpoint dilemma or if you go back a few years the very similar Bottom Up Vs Top down design argument. I have stated on previous occasions that I use both in every project (diamond design)

I suggest we get into that on a new thread if you want to explore it further as it could go on quite a bit.

Edited May 19, 2015 by ShaunR

[shoneill]

I never said that I was using a pattern of any kind. All I am doing is using common denominator implementation for a very specific part of the problem.

 

I also made no claim whatsoever that you were using any kind of pattern.  I was comparing with a pattern.

 

The obvious drawback of your idea is the static linkage to the List class.  While the List class is most likely a lovely piece of work, if you want to implement a Frame as a different kind of grouping (don't ask me, I'm just theorizing, perhaps a conditional looping of individual elements until a condition is met) then either you build that into your List class or you implement it separately in your child class.  By moving the List functionality into the child class, each child is free to implement whatever method of aggregation they see fit without requiring interface bloat.  Each item implements whatever it needs.  You could have several different flavours of collections (lists) which do things different based on some corner cases.

 

You also cannot hide the interface to the List aspect of your objects which is something you CAN do when the child implements it (in fact it almost requires it).  Depending on your taste you might prefer one solution over the other.  If I want to do something with a Frame, why does the interface for that object insist on telling me it's a list when that's not something I need or want to be confronted with.  It creates noise in the interface which is seldom a good thing.

 

So imagine a "collection" which internally manages a List (your class).  When dealing with a collection (strictly typed child of Frame), you have the exposed methods for the list, when working with Frames (when you don't need to know whether it's a list or not) you have a much cleaner interface.  By moving your same List class up two levels of the hierarchy you gain a lot of flexibility and remove static linkage to the functionality of your list.  What are the downsides you see with this approach?  The only downside I see is the creation of some proxy VIs which, depending on your taste might be enough to sway you.

 

 

I know what you are getting at, though. This is a good example of the tradesmans viewpoint dilemma or if you go back a few years the very similar Bottom Up Vs Top down design argument. I have stated on previous occasions that I use both in every project (diamond design)

 

I don't want to weigh this relatively focussed topic down with previous discussions which may or may not be applicable and to which I was not a party.  I also don't mean to be nasty but your usage of both has no relevance to the discussion.

 

 

I suggest we get into that on a new thread if you want to explore it further as it could go on quite a bit.

 

That's fine by me if you think it requires it.

[ShaunR]

That's fine by me if you think it requires it.

I think it does,

Edited May 19, 2015 by ShaunR

[shoneill]

Continued HERE.