Polymorphism of inputs in SubVIs-similar to Queues

[ktkumar]

Hi,

I would like to implement the polymorphic behavior exhibited by the Queues, in the VIs that we develop.

To explain this more, I have attached 2 files along with this email. In the "queue1.jpg file", when we wire a datatype to the input of the "Obtain Queue" VI we can see that all the queue VIs wired to the same reference exhibit the same datatype. If we do a create control/indicator on the enqueue/dequeue VIs, the control/indicator created will be the same as that of the datatype wired in the "Obtain Queue" VI.

As in "Queue2.jpg", if we replace the simple cluster in the "obtain queue" VI with a complex cluster. We can see the datatype of enqueue/dequeue VIs change as well.

I would like to implement this feature in the VIs we develop as it will make the VIs highly flexible and reusable in many situations. Is this possible?

I would gladly appreciate any help.

Thanks

KT

(Kumar.K.T)

[ktkumar]

Hi,

I guess one way in which we can achieve this is using Variant. But if we use variant, we need to use the datatype descriptor to get the data back.

It would really be helpful if the user need not have to worry about the datatype just like in queues, where the datatype automatically changes.

I can also realize that we can create a polymorphic vi to make this happen. But the overhead here is that we need to create a VI for every datatype that we want to handle.

Apart from these, are there any other way for us to achieve using the polymorphic behavior?

Thanks

KT

[Mike Ashe]

If you use a variant in the Queue, one way to avoid the need to use the datatype descriptor is not to wire the variant out directly, but to use the Set Control - Variant method to input the data to a control on your target VI. This requires a little more work to achieve an elegant implementation, but is generic.

Another option is to pass straight variant to each of your target VIs and make it their responsibility to decode their own data.

[i2dx]

What you a searching for is somethins that simply does not exist in "pure form" in LabVIEW and will never exist in LabVIEW due to its structure.

I call this "the dynamic cluster", which is a construct of a cluster that can chance its components at run time. Of course you can never create a control/indicator/constant and so on. And because there are no Pointers in LabVIEW (all Work is done Call by Value ...) a "dynamic Cluster" can not be created in "the LabVIEW way" but you can use brute force

And after a lot of heavy brain-work i realized, this construct is allmost useless. If you need reusable VIs with changing datatypes is different projects, use Typedefs ...

If you want to see, why its useless, take a look at the example. The dynamic cluster is built in the while loop ...

best regards

CB

Download File:post-885-1104397595.zip

[JackHamilton]

I rarely used "naked Queue" calls in my applications. I've very successufully used the following architecture for queue messaging:

1. The queue message is a cluster consisting of the following elements:

A. Enum (Strict Type Def) this is the "Command" of the message. Typically this refers to the actual state of the state machine the queue is communicating to.

B. A variant: This is the catch all datatype handler.

C. *optional* U8 numeric that signifies the 'priority' of the message or the sender ID.

D. *optional* Message timestamp (timestamp datatype)

C. *optional* Error cluster.

I ALWAYS write a wrapper around the Queue which serves as the "SEND MESSAGE" function.

The Enum is the "Function Call" the variant is the data. The error output can actually be the error returned from the sub-code commanded by the queue. (if you code the sub-VI to return any errors after a queue command is received.

It's just too easy to forget what the queue prototypes are after a few weeks away from the code. Also, the concept of applying Object oriented programming describes the concept of "Public" and "Private" Data. Once you've created the queue - do you really need make it's reference avaliable throughout all the code?

Having ONE and ONLY ONE Send function - allows you to debug your code much easier. I have dropped File I/O in them at times to log all the messages and who's sending them. This has also given me the added benifit of creating command user/tracing and macros!.

The Send Message function uses USRs to hold the queue reference. It's also a great place to retain documentation about what commands do what - and what eactly the datatypes are for each particular message. I've even in some cases had mulitp inputs of specific datatypes for specific for each command.

What you want to create is a Sub-VI module that receives and process the queue commands - and a message sender that is 'mate' for that module. The two routines work as a match pair. This way you only have to make these two function 'know' exactly what the variant contains for each command. I often have very different datatypes in the variant based on a paticular command, it can be a string, cluster or array. I only have to assure that the sending module & receive module match.

Typically, I create 2 communication queues, one sends commands and the other receives responses. This allows my "Send Message" function to also support getting data back from the module. Like errors, current configuration, data, etc.

Although it's more complex - it handles nearly all instances of messaging.

Regards

Jack Hamilton

[Albert-Jan Brouwer]

It would really be helpful if the user need not have to worry about the datatype just like in queues, where the datatype automatically changes.

3175[/snapback]

Using variants or flattened strings you can make the transport type-agnostic, but, as has been pointed out, you always need the type definition on the production and consumption end of your queue: G is a statically typed language.

One thing that is easy to overlook is that you can go a little further using variants: you cannot change the elements of a cluster at run time, but you can add/remove named attributes to/from variants. See the variant palette.

The attribute lookup algorithm scales better than O(n) with the number of attributes. Likely it is O(log(n)) though the scaling is so weak that it might even be O(1), with the degradation being due to cache misses. So it is suitable for dictionary-alike purposes as well.

Albert-Jan

[ktkumar]

Hi,

I agree totally. Currently I am using an enum and a string cluster together to serve as a datatype (similar to what Jack Hamilton had mentioned) for the functions that I write so that any data can be represented and can be used by my functions. One such function is a circular buffer. I am planning to learn more GOOP as i heard that LV8 would support GOOP. (Is this true !!??)

If we tried to write a function called circular buffer in goop and we want it to function by optimized performance. I feel using variant is, in a way, short-coming. (Think of using an enum and a string to represent U8 !! )

Assuming that we use this GOOP based function in more than one place in the project. If we use typedef, I guess we would have to make the VIs specific for the datatype.

Regards

KT

[Mike Ashe]

I rarely used "naked Queue" calls in my applications. ...

1. The queue message is a cluster consisting of the following elements:

    A. Enum (Strict Type Def) this is the "Command" of the message. ...

    B. A variant: This is the catch all datatype handler.

I find that for me a string is more flexible as the command than an enum. I also use this often as the input to a case, so the string gives me the readability in the case structure. However, I often put a plugin interface call in the default of my CASE structure. That way I can load up extension plugins at startup (or on the fly later) and if a command comes in that the basic state-machine does not recognize it goes to the default frame. There I search a list of the plugin VI names (sometimes stripped or pre/postfixes and/or .vi extensions) and if found then I do a VIServer call to that plugin, passing the variant array as parameters. This lets me add extension commands easily.

  C. *optional* U8 numeric that signifies the 'priority' of the message or the sender ID.

  D. *optional* Message timestamp (timestamp datatype)

  C. *optional* Error cluster.

I ALWAYS write a wrapper around the Queue which serves as the "SEND MESSAGE" function.

I do these as named variables in the variant array. I search the variant array by name. If the optional variables are there I process, if not, move on. I agree totally on the wrapper. I also put a wrapper on the queue creation and sometimes create queue pairs that normally just pass one to the other. This allows the option of "hooking" in-between the queues and filter/processing of the command/data stream simular to subclassing in Windows OS.

Typically, I create 2 communication queues, one sends commands and the other receives responses. This allows my "Send Message" function to also support getting data back from the module. Like errors, current configuration, data, etc. ...

3317[/snapback]

Another good architectural technique. I sometimes prepend <replyXXXXXXXX> (not the brackets) to the beginning of Send Messages. On the receive side I look for "reply" anchored at the beginning of the command string. If I find it, then I know that XXXXXXX is the input queue name of the module to reply to. If not found then no reply is expected.

Using variants or flattened strings you can make the transport type-agnostic, but, as has been pointed out, you always need the type definition on the production and consumption end of your queue: G is a statically typed language.

Actually, with a little more work this is not quite true any more. It is easy to flatten anything to string or variant and you can even do it without wiring by using VIServer "Get Control - Variant" method. The rub comes on the "decoding end", but again, you can either use VIServer to write the data, as variant at the decoding end (this is assuming you are using some type of plugin architecture) or you can do a parse of the variant data tree.

For a great example, (and code!) see Jim Kring's Universal Probe or the Variant Config File example VIs.

One thing that is easy to overlook is that you can go a little further using variants: you cannot change the elements of a cluster at run time, but you can add/remove named attributes to/from variants. See the variant palette.

3318[/snapback]

Again, with a little work....

You can add a named attribute to a cluster in a variant, then use the variant config cluster VIs to write to your cluster. Anything that doesn't get written gets reported back as an error, but the data variables that do match (by name and type) will get written successfully. I didn't believe this either til I saw it work a few months back. Its pretty nifty.

We've come an extremely long way from when all we had was Flatten to/From String. For all practical purposes we can now implement polymorphism if we want to put in the work. The decision is, is my current project worth the work to make it that generic? or is it overkill for this application?

As the OpenG set of tools expands, we will have more examples and code snippets that implement the above techniques and make it easier too.

[Albert-Jan Brouwer]

It is easy to flatten anything to string or variant and you can even do it without wiring by using VIServer "Get Control - Variant" method.

Indeed, but in both cases there is a specific type associated with the data. In the first case it is the wire that carries the type. In the second case it is the control. I agree that the diagram code that writes to the queue need not necessarily concern itself with the type. But setting the value of the data, which is what is usually meant by "production", always requires the type.

Well..., OK, not always. There a rather sick way to write polymorphic production code: programatically generate both an I16 type definition array and a corresponding flattened string. Unfortunately, the format of the type definition array is complex and incompletely documented.

Recursively parsing the type definition array and flattened data for polymorphic consumption code is much harder still. Not impossible though: LuaVIEW does so when converting arbitrary compound LabVIEW data into Lua tables.

Albert-Jan

[Mike Ashe]

Indeed, but in both cases there is a specific type associated with the data. ... Unfortunately, the format of the type definition array is complex and incompletely documented. 

Recursively parsing the type definition array and flattened data for polymorphic consumption code is much harder still. Not impossible though: LuaVIEW does so when converting arbitrary compound LabVIEW data into Lua tables.

3368[/snapback]

The Type Descriptor is explained in NI's application note AN-154c.pdf, you can get a copy at:

LabVIEW Data Storage

I agree that recursively parsing the TD array/data is hard, (the first time) but it has already been done and the code is available for free in the examples I referenced before, therefore it is no longer hard and is less a matter of new engineering than one of scholarship.

The OpenG Toolkit also has lots of tools/VIs for doing this type of thing and I seem to recall a couple of LabVIEW Technical Resource (LTR) articles about these techniques. This wheel has been invented. Now someone needs to give it whitewalls, raised lettering and some decent hubcaps :thumbup:

[Albert-Jan Brouwer]

The Type Descriptor is explained in NI's application note AN-154c.pdf, you can get a copy at:

LabVIEW Data Storage

But that documentation is incomplete. The type descriptors of some types are not documented beyond the typecode. Even when you're not going to decode or process the type that can be a problem: to get at the name of a cluster element you need to know the precise length of the type data (which for some types is of variable length) since the name string is placed beyond it. Indeed, the fact that the names of cluster elements are placed there is not even documented. Though it is possible to skip beyond a type, there is no unambiguous way to work backwards from there to retrieve the preceding Pascal string of the cluster element name.

I agree that recursively parsing the TD array/data is hard, (the first time) but it has already been done and the code is available for free in the examples I referenced before, therefore it is no longer hard and is less a matter of new engineering than one of scholarship.   

Yes, I concede that it possible to dynamically generate and parse flattened data. I was just trying to question the practicality for general purpose use. You have to look hard to find use cases where doing so makes sense. There is not much type-agnostic code that can be written in LabVIEW, it is a strictly typed language. And the number of use cases for type-driven code is limited.

The only type-driven code I ever did in LabVIEW served to automatically convert a cluster into an SQL insert statement based on the cluster's type (that was based on type parsing code written by Rolf Kalbermatter). In retrospect, simply constructing some string formatting statements for the various record types would have saved a lot of time.

Contrast that with dynamically typed languages. These bundle type information with data, so none of this packaging and unpackaging to pass variable type data around. And a means of checking a type at run time is built in so writing type-driven code is trivial. Also, the composition of compound data does not change the type. For example, a Python list can hold elements of multiple types. This means that there is lots more scope for writing type-agnostic code.

Albert-Jan

[Mike Ashe]

But that documentation is incomplete. ...

I was just trying to question the practicality for general purpose use. You have to look hard to find use cases where doing so makes sense. There is not much type-agnostic code that can be written in LabVIEW, it is a strictly typed language. And the number of use cases for type-driven code is limited.

Sadly, I have to agree on the incompletemess of the code. Perhaps I should have put in a disclaimer on the Applicatin Note reference. Thanks for holding my feet to the fire on that. But I must respectfully disagree on the type-agnostic code and the use-cases. There are lots of use cases, there just has not be a good way to do them in LabVIEW until recently. The OpenG toolkit and variant data tools in it have changed that. There are a lot of tools/VIs there that make writting type-agnostic code possible now. Notice I did not say easy, not yet. The OpenG VIs need some additions (and I'm happy to say that I know that work is getting ready to rev up on that again ).

The only type-driven code I ever did in LabVIEW served to automatically convert a cluster into an SQL insert statement based on the cluster's type

Jim Kring also did some stuff for autocreation of SQL INSERTs from a cluster. The Universal Probe I mentioned above has a parser that does a pretty good job, but still needs a little work. The parsers in the Variant-Config-File example VIs are better, almost bulletproof (not quite, but close, and they'll be perfect in another couple months as we are going to make heavy use of them on an application)

Contrast that with dynamically typed languages. These bundle type information with data, so none of this packaging and unpackaging to pass variable type data around.

3388[/snapback]

Yes, but variants also include that type data, and if you are careful with insuring your variants have a good naming convention and use a good parser you can get pretty generic code. Not perfect yet, but improving and will get much more complete this year.

:thumbup:

[aledain]

I am planning to learn more GOOP as i heard that LV8 would support GOOP. (Is this true !!??)

3358[/snapback]

Actually GOOP has been hidden in LabVIEW since version 6. Furthermore, many of the native LabVIEW VIs or function nodes are objects (such as queues and the lower file IO). However what we really want is to be able to click on a GOOP wire and have a probe describe the data for that instance of an object. Maybe that will appear with 8!

cheers, Alex.

[Jim Kring]

Actually GOOP has been hidden in LabVIEW since version 6. Furthermore, many of the native LabVIEW VIs or function nodes are objects (such as queues and the lower file IO). However what we really want is to be able to click on a GOOP wire and have a probe describe the data for that instance of an object. Maybe that will appear with 8!

cheers, Alex.

3436[/snapback]

There's no reason that a GOOP class developer cannot create a custom probe for a GOOP class. The hard part is integrating these into the custom probe's framework. Right now the mechanism for integrating custom probes is a little awkward. Ideally, custom probes would be declared in a GOOP class's manifest (if such a thing existed) and LabVIEW would automatically know how to make it available to users.

Finally, for those of you who love the Universal Probe, here is another little gem called "Config File Probe". Drop it into your <My Documents\LabVIEW Data\Probes> folder and then probe a Config File wire.

Download File:post-17-1105376809.vi

-Jim

[Randall Thurman]

Finally, for those of you who love the Universal Probe, here is another little gem called "Config File Probe". Drop it into your <My Documents\LabVIEW Data\Probes> folder and then probe a Config File wire.

Thanks Jim for another great tool.

[Mike Ashe]

Finally, for those of you who love the Universal Probe, here is another little gem called "Config File Probe".  Drop it into your <My Documents\LabVIEW Data\Probes> folder and then probe a Config File wire.

3445[/snapback]

Very nice. I noticed that it displays data backwards from how it is listed in the file. You might want to reverse the listing and sub the new version in here. Again, great little tool. :worship: