LVOOP and DLLs

[orko]

Hello all,

I've been working now with LVOOP for some time and have developed a class hierarchy that seems to work well to interface with test instruments and other hardware *inside LV built code*. Our team has used this hierarchy to quickly develop our test tools *in LV* while maintaining a common (re)usable code base.

Now we've attracted the attention of others and we're being asked to provide this functionality to external code (C++/C#) calls inside other test tools. My initial thought was, "Great! I've wrapped LV up into dlls before, should be cake!"... Now I'm having difficulty in seeing the path to success here.

Although I was dissapointed to see that you can't just point the App builder to a class and say "Build me a DLL", I can understand why it would be difficult for it to do such a thing. In LV code everything is by-value, and relies on a class wire to hold the instance in memory to work from. When you are calling exports from the outside however, we would expect to initialize an instance and have all subsequent calls use it (as a pointer to the object) with parameters to perform our functions.

I think I see some ways of doing this, but I feel I may be off-track. So I'm asking the forum if you have dealt with this and if so, what did you find worked for you? I am shooting for some method that I can re-use on all of my LVOOP projects I want to expose as shared libraries to other languages.

How about:

Wrapping up all the methods into exportable non-class members? If so, we would need to hold the class data somewhere... so perhaps wrapping up an "initialize/destroy" that holds an LV object in a shift register to be used by subsequent VIs?

Is it possible to do this without the other team having to know anything about LV? Providing just the dll/header file and defining the instances to be just C data types such as I32 pointers? Basically, I (they) would prefer that they could build their projects without having a bunch of NI stuff installed.

I would appreciate any and all ideas before I start on this grand adventure!

Joe

[ragglefrock]

QUOTE (orko @ Apr 20 2009, 02:21 PM)

Is it possible to do this without the other team having to know anything about LV? Providing just the dll/header file and defining the instances to be just C data types such as I32 pointers? Basically, I (they) would prefer that they could build their projects without having a bunch of NI stuff installed.

I would appreciate any and all ideas before I start on this grand adventure!

Joe

I'm not really sure how much work this would end up being, but you can definitely use LabVIEW queues to store your data, and pass the queue references in and out of DLL calls much like pointers. It's a 32-bit value that is in essence a pointer to your data. You can even just Type Cast the queue reference to an I32 for your public interface. Just create non-class wrapper functions that dequeue the class data, call the class member, and then enqueue the data back into the queue.

[Aristos Queue]

You're not going to like these two solutions but I'll mention them anyway...

1. Any VI that has a class on its FP (as either an input or an output) that you want to expose through the DLL should be wrapped by a VI that takes a cluster instead (as input or output) and creates the class from that cluster (input) or creates the cluster from that class (output). Doesn't work if you were hoping for something that uses different object types for different calls into the API, and you may have to expose more public methods than you would want to. But to get through the DLL layer, you have to shed the class sheild.

2. Create either

• an LV2-style global of an "array of object" type that can do get/set on individual indicies of the array
  
• or some sort of reference architecture (a la the GOOP Toolkit)
  

and any time you would pass an object out to the DLL layer, pass the index into the LV2-global or the refnum out instead. When you make a call back into the API with an object input, pass the previous index/refnum as input. You can use -1 or not a refnum as a sentinel for "create a new default object."

Those are the best solutions I can offer. It's a hard to manage problem.

[ShaunR]

Am I right in thinking you would still have to have all the "NI stuff" anyway? (i.e the run-time engine).

Oh for the days of standalone executables !

And what is this obsession with queues on this site?

[Gabi1]

QUOTE (ShaunR @ Apr 20 2009, 11:23 PM)

And what is this obsession with queues on this site?

i'm in love with queues...cant get enough of them!

i have now transformed my design template with queues all around. have only two loops on the main diagram, Event and data display. all the rest is passed trough subs using queue. very satisfied!

But still cant get around the LVOOP: to me, nothings yet beats a good old cluster with LV2 types constructors/destructors (or directly references). maybe because i'm still on 8.2 tough.

[jdunham]

QUOTE (ShaunR @ Apr 20 2009, 02:23 PM)

Am I right in thinking you would still have to have all the "NI stuff" anyway? (i.e the run-time engine).

Oh for the days of standalone executables !

QUOTE (ShaunR)

And what is this obsession with queues on this site?

Yeah the runtime would definitely be needed. Even Microsoft products have a runtime, but theirs is usually preinstalled with Windows or IE or something like that.

Queues help me because I find that it's easier to write pure dataflow on a small scale. Each device or object gets its own while loop to handle messages on its own schedule. If you try to put a whole complex app in one huge while loop, it's just too hard to program it so everything works nicely.

Once you have a bunch of parallel while loops in your application, queues are the best way to share data between them.

Also, LabVIEW dataflow is inherently by-value, and sometimes you really need a by-reference object, like if the dataset is huge, or needs to be protected by a mutex. Queues can do all that and more.

Now that there are lossy queues, you also get a free circular buffer implementation.

[Mark Yedinak]

QUOTE (ShaunR @ Apr 20 2009, 04:23 PM)

Am I right in thinking you would still have to have all the "NI stuff" anyway? (i.e the run-time engine).

Oh for the days of standalone executables !

Didn't most versions of LabVIEW executables require the run-time engine? At least this is what I recall from as far back as LV 4.0.

QUOTE (ShaunR @ Apr 20 2009, 04:23 PM)

And what is this obsession with queues on this site?

Since LabVIEW is inherently parallel in nature you always need some mechanism to pass data between parallel tasks and to synchronize them as well. Queues naturally do this. Unlike a global variable queues provide synchronization as well. In addition they are event drvien so you can avoid polling local or LV2 style globals. They are also very efficient with respect to performance.

[ShaunR]

Not wisihing to hijack this thread (perhapse start a new one? This could go on a while...lol)....but!

QUOTE (Mark Yedinak @ Apr 20 2009, 11:20 PM)

Didn't most versions of LabVIEW executables require the run-time engine? At least this is what I recall from as far back as LV 4.0.

Well. It was a while ago (I weened on version 2.0...it came on floppies...lol). But I seem to remember creating monolithic executables. Deployment was a doddle

QUOTE

Since LabVIEW is inherently parallel in nature you always need some mechanism to pass data between parallel tasks and to synchronize them as well. Queues naturally do this. Unlike a global variable queues provide synchronization as well. In addition they are event drvien so you can avoid polling local or LV2 style globals. They are also very efficient with respect to performance.

Actually Labview is inherently serial in nature (left to right). That is why you have to resort to other techniques to manage parallelism. I also fail to see how they are event driven given that you put something in the queue and take things out...nothing more. It is a serial buffer with access to only the first and/or last entry in the buffer.

I don't tend to use queues much, but I do extensively use notifiers for passing data and synchronising, which aren't really event driven either in so much you can wait for a notifier and it will get triggered. The only truely event driven features in Labview are the Event Structure and VISA Event. But linking events programmatically accross disembodied vi's (which I use far too often); I've always found troublesome.

[Mark Yedinak]

QUOTE (ShaunR @ Apr 20 2009, 06:21 PM)

Not wisihing to hijack this thread (perhapse start a new one? This could go on a while...lol)....but!

Well. It was a while ago (I weened on version 2.0...it came on floppies...lol). But I seem to remember creating monolithic executables. Deployment was a doddle

Perhaps this should be moved to a new thread.

QUOTE (ShaunR @ Apr 20 2009, 06:21 PM)

Actually Labview is inherently serial in nature (left to right). That is why you have to resort to other techniques to manage parallelism. I also fail to see how they are event driven given that you put something in the queue and take things out...nothing more. It is a serial buffer with access to only the first and/or last entry in the buffer.

LabVIEW is inherently parallel in that to program parallel tasks all you need to do is drop two nodes next to each other. The underlying scheduler and task switcher will automatically run these pieces of code in parallel. Now with multicore CPUs LabVIEW can truly run them in parallel by running code segments on separate cores. The only "serial left to right" nature of LabView is the dataflow nature of the language (which also leads to parallelism) and programming style. There is nothing in LV that forces code to be executed from left to right if there is no dataflow. This is actually a common race condition bug that people make. They will put some node down on the right side of the diagram and think to themselves that since it is right of everything it will execute after the code to the left. In reality though once all of its inputs have been satifisied it will execute.

Creating parallel tasks in C or C++ is not a trivial task. You have to manually create your processes and threads. In LabVIEW this is all done for you. Drop two loops next to each other and you have parallel code. It is as simple as that.

No, queues are event driven. While waiting on a queue (or a notifier) your code does nothing. It is waiting for an event to trigger indicating that either an element has been queued or you have a timeout. Event driven means that your code is doing nothing until some event occurs. The code will effectively be using zero CPU cycles. (LabVIEW's scheduler will consume a few CPU cycles but the higher level code in your application will do nothing.)

QUOTE (ShaunR @ Apr 20 2009, 06:21 PM)

I don't tend to use queues much, but I do extensively use notifiers for passing data and synchronising, which aren't really event driven either in so much you can wait for a notifier and it will get triggered. The only truely event driven features in Labview are the Event Structure and VISA Event. But linking events programmatically accross disembodied vi's (which I use far too often); I've always found troublesome.

Notifiers are nothing more than a queue of size one. A notifier can only have one element "queued" at a time. Queues simply allow more elements to be placed on the stack. Event structures and VISA events are just different event types. Regardless, when using queues or notifiers you are writing event driven code. As for your disembodied code I suggest you try http://forums.lavag.org/Network-Queue-t11048.html&p=46805' target="_blank">network based queues. Since you already use notifiers it shouldn't be too difficult for you to use queues.

[jdunham]

QUOTE (ShaunR @ Apr 20 2009, 04:21 PM)

Not wisihing to hijack this thread

C'mon, you know you want to!

QUOTE (ShaunR @ Apr 20 2009, 04:21 PM)

Actually Labview is inherently serial in nature (left to right). That is why you have to resort to other techniques to manage parallelism. I also fail to see how they are event driven given that you put something in the queue and take things out...nothing more. It is a serial buffer with access to only the first and/or last entry in the buffer.

Well you're right 'inherently' was a poor word choice. Let me try again: In LabVIEW, parallelism is implicit when any two loops are not sequenced by a dataflow connection. In plain english 'parallel execution is easy in LabVIEW'.

Queues are event driven because a loop can be put to sleep until new data arrives. You actually have access to all the data in the buffer (Get Queue Status and Flush Queue). When you say 'nothing more', you are totally off the mark. Check out the Singleton design pattern in the LabVIEW examples for one cool application, and I already mentioned circular buffers.

QUOTE

I don't tend to use queues much, but I do extensively use notifiers for passing data and synchronising, which aren't really event driven either in so much you can wait for a notifier and it will get triggered. The only truely event driven features in Labview are the Event Structure and VISA Event.

You can use queues as a notifier with guaranteed delivery. Queues are sort of like TCP and Notifiers like UDP. It turns out that TCP is used a bit more often, because in many situations, missing a message is not acceptable.

OK, I give up. What more would it need to be event driven? Just because the event is generated programmatically doesn't make it any less eventful.

QUOTE

But linking events programmatically accross disembodied vi's (which I use far too often); I've always found troublesome.

Maybe if you used more queues, it would be less troublesome . I don't see anything wrong with disembodied VIs, especially if there is no direct dataflow (only events) connecting them.

Anyway Shaun, I don't mean to be jumping all over you. I hope you are enjoying the back-and-forth. (Looks like Mark Y beat me to it anyway).

QUOTE ( @ Apr 20 2009, 04:45 PM)

Notifiers are nothing more than a queue of size one.

Well, notifiers are a lossy queue of size one, with multiple subscribers allowed, and some bonus features, like wait on notifier from multiple, and notification history.

[Mark Yedinak]

QUOTE (jdunham @ Apr 20 2009, 06:56 PM)

Well, notifiers are a lossy queue of size one, with multiple subscribers allowed, and some bonus features, like wait on notifier from multiple, and notification history.

True, my "queue of one" for a notifier was an oversimplification but they are very similar in nature, especially with respect to events. BTW, queues can have multiple listeners too however the outcome is not deterministic. One listener will get the event but you don't really have control over which one if all listeners are waiting on the queue at the same time. Notifiers as you stated will allow each listener to receive a notification.

[Mark Smith]

QUOTE (orko @ Apr 20 2009, 01:21 PM)

Hello all,

I've been working now with LVOOP for some time and have developed a class hierarchy that seems to work well to interface with test instruments and other hardware *inside LV built code*. Our team has used this hierarchy to quickly develop our test tools *in LV* while maintaining a common (re)usable code base.

Now we've attracted the attention of others and we're being asked to provide this functionality to external code (C++/C#) calls inside other test tools. My initial thought was, "Great! I've wrapped LV up into dlls before, should be cake!"... Now I'm having difficulty in seeing the path to success here.

Although I was dissapointed to see that you can't just point the App builder to a class and say "Build me a DLL", I can understand why it would be difficult for it to do such a thing. In LV code everything is by-value, and relies on a class wire to hold the instance in memory to work from. When you are calling exports from the outside however, we would expect to initialize an instance and have all subsequent calls use it (as a pointer to the object) with parameters to perform our functions.

I think I see some ways of doing this, but I feel I may be off-track. So I'm asking the forum if you have dealt with this and if so, what did you find worked for you? I am shooting for some method that I can re-use on all of my LVOOP projects I want to expose as shared libraries to other languages.

How about:

Wrapping up all the methods into exportable non-class members? If so, we would need to hold the class data somewhere... so perhaps wrapping up an "initialize/destroy" that holds an LV object in a shift register to be used by subsequent VIs?

Is it possible to do this without the other team having to know anything about LV? Providing just the dll/header file and defining the instances to be just C data types such as I32 pointers? Basically, I (they) would prefer that they could build their projects without having a bunch of NI stuff installed.

I would appreciate any and all ideas before I start on this grand adventure!

Joe

OK - let's get back on topic

I have similar but not quite the same design problem I'm addressing. I'm using LVOOP and LuaVIEW in a new tester framework that I'm implementing. My issue is that I'm using the object/method paradigm in Lua to instantiate script objects and then call methods on those objects. To do that, I'm using the functionality that the Lua authors at CIT designed in where the class name gets loaded into a Lua table (here, the class name means a naming convention of the Lua callable VI) and then subsequent calls to that object use that class name to get the path to the correct function. Here's how it might look in Lua

-- create the object

myInst=Instrument.new("myInst")

-- call a method on that object

myInst:configure("some config string")

This would call into two VIs named Instrument.new_lua.vi and Instrument.configure_lua.vi. myInst is a Lua object and can contain a 32 bit refnum. To create a persistent "call by reference style" object, the VI called by the new method is an action engine (smart LV2 global) and it uses the argument "myInst" to new to create an instance of the class and store it in a array. This "instance engine" also has the ability to get (return) an existing object, set an existing object (get the object, expose it for some data modification, and then place it back into the array), and delete an object - it does this by name lookup. This is more than acceptably fast when the number of objects of any given class is expected to number in the tens. The Instrument.new() method returns a LV string refnum that contains a string that looks like <LVClass name>_<methodName>, or in this example, Instrument.lvclass_myInst. The LuaVIEW calling convention passes the object back in the myInst:configure() call so I can get the class type and instance name. Now I can retrieve that object from the action engine and operate on it. This is all working well.

This also sounds similar to AQ's suggestions. My concern here is how to keep all of the LV components of the scheme alive between calls to the DLL. In my situation, LuaVIEW works because all of the Lua callable VIs I use get "registered" at run time and that means the action engine gets loaded and stays loaded during the lifetime of the script so I can be sure that any objects the action engine creates don't get lost because the action engine went out of scope. With a LV DLL, it would seem that the first call to the DLL to create a "new" object would have to start a background thread that stays running even after the call returns - otherwise, the context of that DLL call will disappear as soon as the call returns. Is my understanding correct? Here, I can see a call to the DLL that creates a new object of a given type and returns some ID to that object. Now, subsequent calls on that object pass in the ID and the LV DLL gets the ID and gets the object from the action engine and does whatever. How you get all the object data set may be more problematic since you can't just create a LV object and pass it from the C/C++ code (or whatever) - you'll have to expose accessors and then call those with the correct object ID - or something like that.

Mark

[orko]

QUOTE (ragglefrock @ Apr 20 2009, 01:48 PM)

I'm not really sure how much work this would end up being, but you can definitely use LabVIEW queues to store your data, and pass the queue references in and out of DLL calls much like pointers. It's a 32-bit value that is in essence a pointer to your data. You can even just Type Cast the queue reference to an I32 for your public interface. Just create non-class wrapper functions that dequeue the class data, call the class member, and then enqueue the data back into the queue.

This is an interesting concept (and I may be able to use this on other applications), but seems a little overkill for this situation. I'm also a little leery on creating another level of abstraction with queues when this will require the same or more "wrapper" VIs to be created anyway. I can see doing this though if you wish to pass complex structures back and forth from the calling (C++/C#) code and the underlying LabVIEW.

QUOTE (Aristos Queue @ Apr 20 2009, 02:13 PM)

You're not going to like these two solutions but I'll mention them anyway...

I always cringe when I hear this kind of preamble...

QUOTE (Aristos Queue @ Apr 20 2009, 02:13 PM)

1. Any VI that has a class on its FP (as either an input or an output) that you want to expose through the DLL should be wrapped by a VI that takes a cluster instead (as input or output) and creates the class from that cluster (input) or creates the cluster from that class (output). Doesn't work if you were hoping for something that uses different object types for different calls into the API, and you may have to expose more public methods than you would want to. But to get through the DLL layer, you have to shed the class sheild.

ACK!

Even though this would work, it seems that it would be a maintenance nightmare... Probably valid on small onesy-twosy type calls into a DLL, but I think I'll pass on this for now.

QUOTE (Aristos Queue @ Apr 20 2009, 02:13 PM)

2. Create either

• an LV2-style global of an "array of object" type that can do get/set on individual indicies of the array
  
  
• or some sort of reference architecture (a la the GOOP Toolkit)
  
  

and any time you would pass an object out to the DLL layer, pass the index into the LV2-global or the refnum out instead. When you make a call back into the API with an object input, pass the previous index/refnum as input. You can use -1 or not a refnum as a sentinel for "create a new default object."

Now this...this has me thinking. I can see how this method could be very useful in keeping the LabVIEW architects on their side of the fence and the C guys happily on the other. The main goal I had was to keep using the

LV

OOP code base in LabVIEW applications, while exposing the functionality to external calling code. This seems to fit that bill!

It would seem that I could make generic "LabVIEW object" setObject/getObject/deleteObJect methods (init/get/close) that perform the indexing you talk about. Then the non-class member VIs that are going to be used in the DLL could all have the getObject VI as the precursor to get the right object out of the array. Hmm...Wouldn't there also have to be another VI to put that object back into the array (to maintain state and avoid copies)?

[Daklu]

QUOTE

It would seem that I could make generic "LabVIEW object" setObject/getObject/deleteObJect methods (init/get/close) that perform the indexing you talk about. Then the non-class member VIs that are going to be used in the DLL could all have the getObject VI as the precursor to get the right object out of the array. Hmm...Wouldn't there also have to be another VI to put that object back into the array (to maintain state and avoid copies)?

Sounds like a Collection object. I did some prototyping of LabVIEW collection objects in my current project and it seemed pretty straightforward. I didn't get too far into the nitty gritty details though.

[Mark Smith]

QUOTE (orko @ Apr 21 2009, 12:18 PM)

Exactly. I'm not clear yet how the non-class member VIs would be able to call class member VIs without an actual object wire going in and out of them to preserve state and avoid copies.

AQ?

Use class member VIs to call into the class, but don't include class controls or indicators on those VIs - then you can call the action engine. The only control(s) you need to expose from the caller is the unique ID so the action engine can look it up and a control you'll use to set the class data, if you want to set a class value. If you want to return a class value, just include whatever indicator you need. Now, use the ID to call into the get method of the action engine, perform whatever on the object, and then use the set method to return that object to the action engine collection. No class types were exposed to the outside world. It really doesn't matter if the object pulled from the collection is the same one that gets returned as long as the state of the new object is correct and it carries the same unique ID. In fact, I don't really know if the object I pulled from the array is the same one as the one I return - anybody care to educate me? - The operation is to wire the object retrieved to a class method, bundle something, then pass the wire back to the action engine and put that element back in the array - is this still the same object?

Of course, this presumes that class member VIs that don't include class types as arguments can be used as interfaces to DLL's - I don't see why not, but I haven't tried.

Mark

[orko]

Thanks Mark. That solidifies the concept in my mind. Now that I understand...I'll try this out and see how much time is required.

Is it me or doesn't this seem like something that would be able to be done programmatically within the App builder? The whole "Build me a DLL from this class" concept.

[Aristos Queue]

QUOTE (orko @ Apr 21 2009, 01:18 PM)

Exactly. I'm not clear yet how the non-class member VIs would be able to call class member VIs without an actual object wire going in and out of them to preserve state and avoid copies.

AQ?

Suppose I have X.lvclass which as a member VI Y.vi. Normally Y.vi takes the class in and does something.

I now create WrapperY.vi which takes as input an unsigned 32-bit integer (uInt32). On the block diagram of WrapperY.vi, I either

• give that uInt32 to an LV2-style global that indexes its contained array and returns an object to me [the obvious way of coding this -- using Index Array -- will create the overhead of a copy of the object out of the array, but that can be avoided by swapping a default value object into the array... see online help for the Inplace Elt Struct in combination with the Swap primitive.]
  OR
  
• type cast that uInt32 as a refnum of some type -- perhaps a queue refnum, perhaps a GOOP refnum, or something else entirely -- that is the data storage mechanism that I've chosen to hold onto my object, and get the object out of that data storage mechanism.
  

With the object now in hand, I call Y.vi. If Y.vi passes the modified object out, I store the output back into my LV2-style global or my refnum architecture.

QUOTE (mesmith @ Apr 21 2009, 02:25 PM)

Of course, this presumes that class member VIs that don't include class types as arguments can be used as interfaces to DLL's - I don't see why not, but I haven't tried.

Yes, that works just fine.

QUOTE (orko @ Apr 21 2009, 03:01 PM)

Is it me or doesn't this seem like something that would be able to be done programmatically within the App builder? The whole "Build me a DLL from this class" concept.

If you'll notice, the architectures required to do this vary wildly depending upon requirements and desires. There is not a cannonical solution to this problem. Indeed, the only thing that R&D can assert is what we already assert -- you can't build a DLL with a class directly in the interface. You have to be inventive...

[orko]

Thank you for the example, AQ. I am going to be delving into this in these next few days, so clarifying the process helps tremendously.

QUOTE (Aristos Queue @ Apr 21 2009, 01:58 PM)

If you'll notice, the architectures required to do this vary wildly depending upon requirements and desires. There is not a cannonical solution to this problem. Indeed, the only thing that R&D can assert is what we already assert -- you can't build a DLL with a class directly in the interface. You have to be inventive...

I think I see what you mean. Interfaces have always been tricky...and there are many different ways to implement them. However (at least for my purposes) I think the LV2 Object array method serves many if not all of the wants/desires for LVOOP built instrument drivers. Similar to the problems that IVI faced, there are just too many ways to do this, so it seems best to just pick one way that works and run with it...