Jump to content

Refactoring the ReferenceObject example in LV 8.2


Recommended Posts

Hello All,

I have posted an article on my blog called Refactoring the ReferenceObject example in LV 8.2, which is an brief analysis of the by reference object-oriented programming example that ships with LabVIEW 8.2. I have created three new examples, which (IMO) greatly improve the usability of this pattern. I look forward to your feedback (please feel free to discuss it here) and any ideas that come out of it that might improve the designs.

I have attached the downloadable examples to this post. (They are also available for dowload on the article page.)

[update 8/19/2006: I have updated example 3, after learning about a new LabVIEW feature that allows a queue reference to be linked to a type definition of the element type. Thank you, Philippe Guerit, for telling me about that. :) ]

Cheers,

Download File:post-17-1156035929.zip

Download File:post-17-1156035941.zip

Download File:post-17-1156055411.zip

Link to comment

Jim,

I only looked at project 3, and I like it.

One of the great features of storing by queue reference is the ability to cut loose from the dataflow paradigm. Therefore I added a modified example to project 3 to emphasize the different nature of ReferenceClass.lvclass: its value is static and there doesn't require dataflow. The thick green wire suggests a real object, but it doesn't require dataflow wiring like ordinary objects do. One does need to be careful to prevent race conditions, though.

My 2 cents..

Aart-JanDownload File:post-1175-1156077738.zip

Link to comment

Aart-Jan,

> I only looked at project 3, and I like it.

I REALLY like #3, too. I think that it is the way to go, moving forward.

> One of the great features of storing by queue reference is the ability to cut loose from the dataflow paradigm. Therefore I added a modified example to project 3 to emphasize the different nature of ReferenceClass.lvclass: its value is static and there doesn't require dataflow. The thick green wire suggests a real object, but it doesn't require dataflow wiring like ordinary objects do. One does need to be careful to prevent race conditions, though.

My 2 cents..

Yes, I took it for granted that most people here (on LAVA) would already understand the implications of by reference objects -- this is the GOOP, forum after all :) . The implications being the following:

  • data does not flow. only a reference flows, unchanged, which points to the data
  • forking a wire copies the reference but not the data
  • race conditions and locking must be considered, since calls may be made in parallel

I think that your additional example drives these points home. Another example that I like is a dual loop, where each loop has a reference to the object and one loop modifies the data (calls SetValue) while the other loop reads the data (calls GetValue). On that note, there is a very nice (IMO) overview of by reference GOOP in appendix D of LabVIEW for Everyone, 3rd edition (this appendix is 20 pages long).

Cheers,

Link to comment

Attached is another version derivated from Jim version 3.

Download File:post-121-1156222854.zip

It has the following new functionalities:

- GetData is fully asynchroneous (it will not be locked by the GetDataToModify and it will always return data).

- Access to named object from asynchroneous process through the create.vi.

Drawbacks

- To be able to have a fully asynchroneous GetData another queue is used with a copy of the data (if someone can figure out a way to get this without making a copy...)

- The create is more complicated (the GetDataToModify, SetModifiedData and GetData are still very simple).

Any comments and/or improvements are welcome.

PJM

Link to comment

The example #3 works for simple classes but it still lacks the ability to use the class with external algorithms that would modify the class content. (Read the post scriptum first...)

To enable this we need some further modifications

- addition of new public lock method

- this would lock the object and return a "lock reference"

- if class user intends to modify a class object based on a result of a class method he needs to acquire a lock

- only one lock for the object is allowed simultaneously

- during lock all method calls that would alter the object are prohibited

- during lock all get_only methods that do not modify the object are allowed (see limitations below)

- addition of new public release method

- releases object lock

- all class methods modified

- all methods have a new connector for a "lock reference in" and "lock reference out"

- get only methods and methods that do not modify the object

- can be accessed when object is locked and lock reference is not wired

- cannot be accessed when object is locked and lock reference is wired

- all methods that modify the object

- cannot be accessed when object is locked unless correct lock reference is wired to lock reference input

- all methods wait until the lock is released or timeout is reached unless correct lock reference is provided

The programming model for algorithms that are external to the class is following:

- acquire lock

- get some data from the object

- process data algorithm internally

- modify the object trough class methods based on the processed data

- release lock

As an example of an algorithm consider the following

An object that is container for some time series data, let's call it DataObject

A algorithm that takes DataObject as an input and reduces noice and returns DataObject, let's call it Denoise

- Denoise acquires a lock for the DataObject

- Denoise gets the time series by calling DataObject.getTimeseries

- Denoise reduces noise from the acquired time series

- Denoise writes new denoised time series by calling DataObject.setTimeseries

- Denoise releases the lock

POST SCRIPTUM

This kind of programming model closely resembles of functional programming and is often preferred over the programming model where all algorithms are defined as class methods. The advatage is generality and code re-usability, a single algorithm can be applied to a number of different classes as long as classes fullfill certain preconditions.

I however started thinking while writing if this kind of functional paradigm can even be used with LabVIEW. In C++ algorithms are implemented using templates to overcome strict typing. Haskell naturally supports algorithms by polymorphic typing. Java algorithms can be defined trough defining common interfaces for the classes. None of these methods is available in LabVIEW. Polymorphism is only available via XNodes, variants, flattened strings and polymorphic VIs. XNodes are not available, variants and flattened strings don't seem to do the thing, and polymorphic VIs require adding a separate VI for each new supported class. I've used scripting to create polymorphic VIs with normal data types but I'm not sure this concept extends to creating class methods; LabVIEW 7.1.1 ,which I use for scripting, doesn't have the concept of classes. NI is propably not going to implement interfaces or templates in a long time. So we are left with manually adding new VIs to polymorphic algorithms. Perhaps the idea of algorithms just don't fit into the world of LabVIEW in it's present stage. Wouldn't be the first programming concept I'm missing... :)

Link to comment

Man! do I Love this stuff.

Ok here is something I've been working on since the Beta version but never got there until NI WEEK gave me an Idea.

What if we create a class That uses both the by ref model and the by Value model and then inherit the class so any class can be run as a by reference.

What I came up with was a class called Flip-Flop. Its purpose is to move the data to either a Variant stored in a Que (By Ref) or the Child class wire itself (By Value).

In the Main Vis the class wire data is stored in the Que and when a public function receives the wire it transfers the data to the wire so it can process it. The power of this Idea is it takes full advantage of the new inheritance.

Please check this new Idea Out. I'm calling it FLOOP for Flip-Flop OOP.

Download File:post-584-1156262742.zip

Link to comment
Man! do I Love this stuff.

Ok here is something I've been working on since the Beta version but never got there until NI WEEK gave me an Idea.

What if we create a class That uses both the by ref model and the by Value model and then inherit the class so any class can be run as a by reference.

What I came up with was a class called Flip-Flop. Its purpose is to move the data to either a Variant stored in a Que (By Ref) or the Child class wire itself (By Value).

In the Main Vis the class wire data is stored in the Que and when a public function receives the wire is transfers the data to the wire so it can process it. The power of this Idea is it takes full advantage of the new inheritance.

Please check this new Idea Out. I'm calling it FLOOP for Flip-Flop OOP.

Download File:post-584-1156262742.zip

Mark: Actually I was working on the same thing, for the past couple days :-) I've got some ideas too. I think that we're going to come up with something great, here!

Link to comment
Man! do I Love this stuff.

Ok here is something I've been working on since the Beta version but never got there until NI WEEK gave me an Idea.

What if we create a class That uses both the by ref model and the by Value model and then inherit the class so any class can be run as a by reference.

What I came up with was a class called Flip-Flop. Its purpose is to move the data to either a Variant stored in a Que (By Ref) or the Child class wire itself (By Value).

In the Main Vis the class wire data is stored in the Que and when a public function receives the wire it transfers the data to the wire so it can process it. The power of this Idea is it takes full advantage of the new inheritance.

Please check this new Idea Out. I'm calling it FLOOP for Flip-Flop OOP.

Download File:post-584-1156262742.zip

And FLOOP, You got to love this name!

PJM

Link to comment

(Jim, what follows is mostly stuff I already sent to you in e-mail, but it seems useful to post here.)

I am thrilled at all the different implementation mechanisms that Jim has proposed. But there's an underlying assumption that he makes: that we are searching for the One True Way to implement a reference class. I disagree. I think we're searching for all the good ways to implement references, and that there is no One True Way. Different implementations serve different use cases, and I think in time LV users will standardize around a set of reference models, not one single model.

Example: Jim argued that the programmer should make "Check In.vi" and "Check Out.vi" private. That's true if and only if the programmer has created wrapper VIs for all the member VIs of the original class and replicated them on the reference class and if there is no performance gained from doing a Check Out, doing multiple function calls, and then doing a Check In.

  • Making the check in/out functions public is a model where the data is mostly used by value and the reference is created only occassionally for a particular value. It is similar to "Index array, do a bunch of modifications, replace element in array."
  • Making the check in/out functions private is a model where the reference model is the primary way the class is used. That model is like "Tell Array Y to do the following modifications to index X."

Both implementations are valid concepts, and both are useful depending upon the architecture being implemented. Many customers have argued that NI should put forth a standard by-reference implementation so that everyone uses the same system, but I've got half a dozen "standards" that I can envision, each serving a different design goal. I'm more comfortable letting conversations like this one run for a while before NI starts putting an official standard together -- I think we need to see the full range of what develops in the world before anyone can definitively say what the standard(s) should be.

Keep the ideas coming ... most folks (both inside NI and outside) don't agree with me, but I truly think that every LV developer who moves beyond a set of Express VIs on a single diagram will eventually consider classes as normal and standard as VIs. We've got to get it right in these early years.

PS: Jim described his case #3 as the hands down winner. In my eyes, the "con" would be:

* Reference API has to supply all possible permutations of operations for efficiency. If I have a reference to a Numeric, I need my reference class to have methods for Add and Multiply. I also need to have "Square and add another square then take the square root". Otherwise I'll have to do a check in and check out between every step of the mathematics (very inefficient). With the Check Out as a public function and the a class as the queue's type, I can check out and do a whole stream of operations, still with my class shield in place to protect the data from arbitrary changes, and only do a single check in.

Link to comment

To add material to the discussion I post another design pattern. If you want to separate the by-reference semantics from the class, this is one way to do it.

I have implemented a separate container class: ObjectRepository. It has the ability to store any LabVIEW Object. The idea is that if you want to use a class by value you don't need to do anything. If you want to store the class by reference you create an instance of the ObjectRepository.

On the up side:

- Separetes the class from the way it is stored or handled.

- The ObjectRepository can be used for any native class without adaptation

On the downside:

- Explicit downcast is needed when you read the object from the ObjectRepository since it returns LabVIEW Object. It has a potential for run-time errors. Without templates we cannot create generic type safe containers.

- Code readablity; to pass the object by reference you actually wire an instance of the ObjectRepository.

- Probably more, haven't spend too much thougth on it.

Disclaimer: There may be some flaw in the implementation, it was a quick hack to show an idea.

Jan

Download File:post-2808-1156330697.zip

Link to comment

I made two examples yesterday that mimics a chunk of memory in a computer in a single LV2 style global with an array inside. It is a ref making system and makes real references for any kind of variable using variants and another version using LVOOP. It is very efficient, particularly the LVOOP, and compared with using queues it is fully asynchronous.

The other thread is here:

LV2 style ref

Link to comment

Hi,

I wrote a Counter class, in which each instance of the counter acts like a by-reference object. So I decided to post it here. The counter class is based on storing the counter value in shift register. When an object is created with constuctor, a VI reference is obtained for storing the values counter in the shift register of that VI. Forked wires refer to same VI and therefore represent the same counter object. There is also a copy-constructor to create a separate copy of the wire that doesn't refer to same object. By-reference objects need to be closes with destructor method to close the used VI-references.

In principle the class can represent any countable object for which +1 operation, reset, set and get can be defined, although now it uses only I32.

Download File:post-4014-1156596876.zip

Link to comment

Edit: Completely rewritten to clarify the message.

Hi,

I wrote a by reference objects template using two separate queues; one for the private class data and the other for locking mechanism. The private object data is defined in a private typedef instead of the Class Private Data control.

The class template also contains a constructor, a destructor and a copy-constructor. The consturcor should be renamed to Create Class Name. If there are multiple constructors, all the constructor names should begin with Create Class Name. The copy-constructor and destructor are named Copy and Close respectively. The responsibility of constructors are to initialize the queues, initialize private data members if needed and call parent constructor if class is a child class.

There are five private methods: _Init, _Get Data, _Set Data, _Get Data to Modify and _Set Modified Data. All private and protected methods start with "_" to indicate that they are not public. Also red color should be used in private method icons instead of black to indicate that the methods are private. The _Init method is used to initialize the queues and is called by the constructors and the copy-constructor. _Get Data and _Set Data are used to get and set the private data without locking. _Get Data can be used even when the object is locked; it returns the current state of the private data. Also _Set Data can be used even when the object is locked, but it's primary use is to store intermediate values to locked object when the lock cannot be yet released. There is however no guarantee that the data written with _Set Data will not be overwritten before the lock is released. _Set Modified Data can be used to release lock and to simultaneously store modified data retrieved with _Get Data to Modify.

All the private methods are reentrant to ensure that there is no VI level locking; that is private methods can be simultaneously called from different sources even though the object is locked and there are private methods currently waiting for the lock to be released. There is however no way I can think of to guarantee that VI level locking doesn't happen to public dynamically dispatched methods. First only single instace of each private methods is used in each of the public methods. That is if there is a private method waiting for lock to be released and this private method has been called from a public method, then this public method is locked for all the objects since no object can access this private VI simultaneously. There is therefore VI level locking which I don't think can be avoided easily. The same problem is propably with all the other concurrency control methods proposed here.

A ZIP file containing a LV 8.2 project is attached. The project contains two libraries. The first library contains the template and the second library contains an example how to use the template. I think also inheritance can be worked out, but I haven't tried it yet. I'll investiage it later either tomorrow or next week.

I hope this helps people to get forward with by-reference objects and also clarifies why I have very strongly requested the addition of built-in constructors, copy-constructors and destructors. This example wouldn't work without all these.

Download File:post-4014-1156620139.zip

Edit 2: There are a few items that still needs to be worked out

  • Inheritance
  • Clean up of member objects at time out and at object close, propably separate private _CleanUp routine should be used in all methods
  • If class Close is called during simultaneous call to _Set Modified Data the data queue may be empty. How to avoid this situation?
  • Ability to create named objects, the same object can be referred in multiple places without joint wire source. Named queues will be used instead of unnamed ones. This would make it very easy to create Producer - Consumer design pattern.
  • How to avoid VI level locking, is there any workaround?
  • Can the template internal methods be somehow encapsulated into another class so that the implementation of the by-reference objects in general is not tied to the implementation of each individual class. This would allow updating the by-reference system all the time without need to update all the classes using the by-reference system as long as the interface to the by-reference system stays the same. VI level locking may be a difficult problem with this kind of approach.

I appreciate suggestions to the above mentioned questions.

Link to comment
Edit: Completely rewritten to clarify the message.

Hi,

I wrote a by reference objects template using two separate queues; one for the private class data and the other for locking mechanism. The private object data is defined in a private typedef instead of the Class Private Data control.

The class template also contains a constructor, a destructor and a copy-constructor. The consturcor should be renamed to Create Class Name. If there are multiple constructors, all the constructor names should begin with Create Class Name. The copy-constructor and destructor are named Copy and Close respectively. The responsibility of constructors are to initialize the queues, initialize private data members if needed and call parent constructor if class is a child class.

There are five private methods: _Init, _Get Data, _Set Data, _Get Data to Modify and _Set Modified Data. All private and protected methods start with "_" to indicate that they are not public. Also red color should be used in private method icons instead of black to indicate that the methods are private. The _Init method is used to initialize the queues and is called by the constructors and the copy-constructor. _Get Data and _Set Data are used to get and set the private data without locking. _Get Data can be used even when the object is locked; it returns the current state of the private data. Also _Set Data can be used even when the object is locked, but it's primary use is to store intermediate values to locked object when the lock cannot be yet released. There is however no guarantee that the data written with _Set Data will not be overwritten before the lock is released. _Set Modified Data can be used to release lock and to simultaneously store modified data retrieved with _Get Data to Modify.

All the private methods are reentrant to ensure that there is no VI level locking; that is private methods can be simultaneously called from different sources even though the object is locked and there are private methods currently waiting for the lock to be released. There is however no way I can think of to guarantee that VI level locking doesn't happen to public dynamically dispatched methods. First only single instace of each private methods is used in each of the public methods. That is if there is a private method waiting for lock to be released and this private method has been called from a public method, then this public method is locked for all the objects since no object can access this private VI simultaneously. There is therefore VI level locking which I don't think can be avoided easily. The same problem is propably with all the other concurrency control methods proposed here.

A ZIP file containing a LV 8.2 project is attached. The project contains two libraries. The first library contains the template and the second library contains an example how to use the template. I think also inheritance can be worked out, but I haven't tried it yet. I'll investiage it later either tomorrow or next week.

I hope this helps people to get forward with by-reference objects and also clarifies why I have very strongly requested the addition of built-in constructors, copy-constructors and destructors. This example wouldn't work without all these.

Download File:post-4014-1156620139.zip

Edit 2: There are a few items that still needs to be worked out

  • Inheritance
  • Clean up of member objects at time out and at object close, propably separate private _CleanUp routine should be used in all methods
  • If class Close is called during simultaneous call to _Set Modified Data the data queue may be empty. How to avoid this situation?
  • Ability to create named objects, the same object can be referred in multiple places without joint wire source. Named queues will be used instead of unnamed ones. This would make it very easy to create Producer - Consumer design pattern.
  • How to avoid VI level locking, is there any workaround?
  • Can the template internal methods be somehow encapsulated into another class so that the implementation of the by-reference objects in general is not tied to the implementation of each individual class. This would allow updating the by-reference system all the time without need to update all the classes using the by-reference system as long as the interface to the by-reference system stays the same. VI level locking may be a difficult problem with this kind of approach.

I appreciate suggestions to the above mentioned questions.

Hi

I like your implementation (I especially like the now obvious way you got over the data copy issue I previously had).

Attached is a revision of this ObjectByRef class.

Download File:post-121-1156716022.zip

It has the following major modifications (see code for details):

  • Add support for name objects.
  • Disable the set data (I dont think this should be allowed, too much potential for major screw ups).
  • Change behavior of SetModifiedData so it always release the lock.
  • Minor changes in several other VIs (most ot them regarding the timeout input).

I have not decided yet if the copy method is really usefull, but I left it there.

In regards to this point:

  • "Clean up of member objects at time out and at object close, propably separate private _CleanUp routine should be used in all methods"

I gave some though about this, and I think the timeout should return an error from within the private method whenever possible (meaning whenever we know what is the only possible situation that may have arised to return a timeout). For an example of this see SetDataToModify.

About this point:

  • "If class Close is called during simultaneous call to _Set Modified Data the data queue may be empty. How to avoid this situation?"

I think this is a non isue since in that situation while you may get default data you will get an error because the queue refnum has become invalid therefore you can recover if you handle this error somewhere.

I will have to give some more though to your other points.

PJM

Note: any comments are of course encouraged :P

Link to comment
  • 4 weeks later...
Have you had any luck with the inheritance?

Well, I've been busy. But I decided to work trough it this morning... upon your request. The picture below is the front panel of the attached example (edit 2: was block diagram).

post-4014-1158838861.png?width=400

There is a parent class and a child class. Both have their own private data. Create Child methods calls Create Parent method. Copy method is dynamically dispatched to the correct one, and this method then calls it's parent method. Close method is also dynamically dispatched to correct one and close also calls parent method. This way the class hierarchy is correctly transversed for object creation, copying and disposal. Inheritance is implemented using LVOOP built-in inheritance.

You can download my reference implementation below.

Download File:post-4014-1158838778.zip

Edit 2: old one is here Download File:post-4014-1158829705.zip

EDIT: One more thing. All private parent and child methods need to have different name due to the fact that LVOOP doesn't allow overriding private methods either. So private methods are not really private but they also affect the decendent and ancestor classes by narrowing the namespace that can be used to name methods. This may cause problems since especially the developer of the ancestor class doesn't know in general what methods there are in all possible decendent classes developed by other people. So if the ancestor class developer decides to change the implementation and creates a new private method, the decendent classes may break even though the class interface stays the same.

EDIT 2: I modified the wire color of the child class so it can be distinguished from the parent class.

The create child method looks following:

post-4014-1158838872.png?width=400

The close child method looks following:

post-4014-1158838880.png?width=400

The copy child method looks following:

post-4014-1158838887.png?width=400

Link to comment

Hi everyone,

I want to point out that any "retrieve - modify - store" system has a problem: when/where do you retrieve and store the data when calling an other method of the current class ?

I see two possibilities.

1. You do that in a wrapper VI. This has the disadvantage that each method needs a wrapper VI, so you effectively double the number of VIs. Code replication. If you want to call a method of the current class you don't call the wrapper VI but the method VI directly, and you hand it the object data directly.

2. You do it in the method VI. Code replication over there. Before you call an other method of the current class you need to store the current object (if you have made any modifications) , bacause the other method retrieves the object again.

Neither way supports calling two methods in parallel. Because they would not modify the same object but two different objects and on return of the methods, both data sets would be saved to the same object in the repository so one set of modifications would be lost.

The way to prevent this would be a system that immediately stores any change into the object. Like you would handle an object in any other language. Much more intuitive, less prone to error and much much less code replication. Think outside the GOOP box !

Joris

Link to comment
Hi everyone,

I want to point out that any "retrieve - modify - store" system has a problem: when/where do you retrieve and store the data when calling an other method of the current class ?

I see two possibilities.

1. You do that in a wrapper VI. This has the disadvantage that each method needs a wrapper VI, so you effectively double the number of VIs. Code replication. If you want to call a method of the current class you don't call the wrapper VI but the method VI directly, and you hand it the object data directly.

2. You do it in the method VI. Code replication over there. Before you call an other method of the current class you need to store the current object (if you have made any modifications) , bacause the other method retrieves the object again.

Neither way supports calling two methods in parallel. Because they would not modify the same object but two different objects and on return of the methods, both data sets would be saved to the same object in the repository so one set of modifications would be lost.

The way to prevent this would be a system that immediately stores any change into the object. Like you would handle an object in any other language. Much more intuitive, less prone to error and much much less code replication. Think outside the GOOP box !

Joris

This problem will allways be here due to the call by value nature of LabVIEW. The only practical way of storing data by ref is to put it in a cluster and store that cluster (an LVOOP object is more or less an advanced cluster). If you want to modify any data in that cluster, you have to get the cluster before you can do anything with the individual data. One alternative is to store each individual data in a LV2 global instead of a cluster. This will probably work, but will be very unpractical because you have to make get/set actions for every single element + wrappers because the LV2 global would have to be reentrant and would have to be called by reference node. Besides, you cant use LVOOP objects in such a configuration.

Anyway, I have still not seen any satisfactory reason why LabVIEW just have to be call by value and not by ref. The parallelism reason just does not hold water because the main (often the only) reason for doing parallel runs is to be able to read/write the same data different places. This can only be done by using some kind of by ref system.

Link to comment
Anyway, I have still not seen any satisfactory reason why LabVIEW just have to be call by value and not by ref. The parallelism reason just does not hold water because the main (often the only) reason for doing parallel runs is to be able to read/write the same data different places. This can only be done by using some kind of by ref system.

Performance is the reason. LVOOP can perform much better being by-value rather than by-ref. They haven't yet even taken advantage of all possible optimizations. So we'll see better performance in the future. By-ref in a dataflow language cannot take advantage of dataflow optimization techniques and therefore performs much poorer than dataflow by-value counterpart. Therefore I think it was correct decission to base the LVOOP on by-value system, so that when refs are not needed, full performance advantage of dataflow can be gained. Of course by-ref should still be a parallel alternative to by-value and I really hope we'll see this in LV 9 or what ever it'll be called.

Link to comment

This thread, as well as others, is showing that users are forced to make their own by reference implementation on top of the LVOOP to make use of it. The performance gained is then more or less lost?

If the performance using LVOOP with a by reference implementation like yours (looks promising by the way...) is higher compared to maybe dqGOOP. Then performance can not be the answer, since a NI native reference version could do even better in terms of performance.

So I agree with bsvingen that I haven

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.