How to call a plug-in object from an executable?

[PaulL]

OK.

We have a message broker (compiled in C) that we use to send and receive messages to external systems.

We wrote a messaging component (call it Alpha) that connects to the broker (via a dll) to read and write messages. Now the messages are in XML format, and the messaging component delegates the task of translating the messages to another component (call it Beta). Let's say for simplicity that Alpha invokes Beta.readFromXML and Beta.formatIntoXML methods.

This works. Now we want to use this for other subsystems, so we need to include some version of Beta, where the implementation of Beta is specific to the subsystem. Hence we chose to make Beta an interface, and implemented Gamma and Delta classes to handle messages for Subsystem1 and Subsystem2, respectively. Gamma and Delta each include overrides of the Beta.readFromXML and Beta. formatIntoXML methods.

OK, but we only want to release Alpha once (as v1.0). We don't want to rebuild it every time we add a subsystem. (For one thing, the builds take an absurdly long time for reasons under investigation on a support issue with NI. More importantly, we don't want to keep rereleasing a piece of software if it already works, because our project team members will think we never have finished it.) What we want is to run a single application (Alpha v1.0) as an executable and dynamically load the appropriate instance of Gamma or Delta (or maybe a future Epsilon class) that we build separately.

OK, so we (my colleague, actually) have tried the following:

Building Gamma and Delta into executables, exposing the readFromXML and formatIntoXML methods. This hasn't worked to date, although I'm not sure if there is or is not some strange way to make this work somehow. I'm not sure it will since we really want the Beta or Gamma object, not call a top-level VI.

Building Gamma and Delta into packed project libraries. Despite our horrible experiences with lvlibps, we decided to see if they would work here. (We have found packed project libraries are just awful in LabVIEW 2010 but we look forward to using some version of them when they reach a releasable implementation. I'm actually quite surprised they were released in their current state.) Alas, they don't, since when we build Gamma and Delta into a packed project library the build also includes Beta (since it is in the dependencies) and we have no way to reference Gamma or Delta properly from Alpha, then.

Building Gamma and Delta into a source distribution. This works, my colleague tells me, but there are serious drawbacks, I think. First, this is not a proper build with a version number, for instance. So it is not proper to release it. (Yes, it is still possible, but it's a super-messy maintenance option.) Second, the build on disk contains a complex hierarchy with all the dependencies visible, which is highly undesirable. Third, since it is a source distribution it is unclear to me what the performance will be, and performance is important in this application. (Maybe the performance will be just fine. Can someone clarify this for me?)

My colleague has done a variation on this by creating a zip file, and then only temporarily unzipping the file as long as necessary before deleting the unzipped files. This is a really clever approach, but it masks the fact that we are still doing a source distribution instead of a proper releasable build, and the question of run-time performance lingers.

So... has anyone found a good way to do what we want to do?

Thanks!

Paul

[mje]

Hmm, I've read through this a few times now, and I must be missing something. Why can't beta be a distribution which alpha, gamma, and delta all depend on? Is there a need for alpha to be aware of gamma/delta (I assume that since beta is an interface, the answer is "No," but I thought I'd ask.

[PaulL]

Hmm, I've read through this a few times now, and I must be missing something. Why can't beta be a distribution which alpha, gamma, and delta all depend on? Is there a need for alpha to be aware of gamma/delta (I assume that since beta is an interface, the answer is "No," but I thought I'd ask.

OK, I've been going over your response and every time I think I understand it, I realize I admit I don't.

I will start with the last question first, since it's the easiest! Alpha (the top-level application) indeed does not know about Gamma or Delta, except that it thinks they are instances of Beta.

So... can you clarify what you mean by making Beta a distribution? I took this to mean that I make Beta a packed project library, build Alpha talking to this, and then somehow call Gamma and Delta at run-time (from Beta). Now I'm thinking that is not what you meant, though.

Hmm.... Maybe you mean we build Alpha with Beta (in a packed project library)? Then we build Gamma and Delta, where the parent is Beta inside the first packed project library. I still don't see how Gamma and Delta end up in the application at run-time (without rebuilding Alpha).

Will you be willing to clarify?

Paul

[Michael Aivaliotis]

I would think that source distribution is your best bet. If you don't like the detailed hierarchy then you may use llb as your destination. You can put each class into separate llbs.

If you want to use the EXE method that I think you would need to run the EXEs and communicate to them with an application reference mechanism and VI server calls. Not sure what you did exactly in the EXE implementation.

[mje]

Last time I did this was back in LabVIEW 8 (?). It was actually out of necessity then, since VIs namespacing was different (if it existed at all?) and if you had dynamic dispatches, you ended up with a hierarchy of identically named VIs which couldn't be compiled into your executable. The end result is all dynamic dispatches had to be saved external to the executable...that is on disk. Ultimately this proved a good thing for me because I was able to use the external classes to develop further extending classes outside of the executable. In the end I had a plug-in architecture that was designed entirely serendipitously due to limitations of the IDE at the time. I wish I had access to the code, but it's several jobs ago...

What ultimately materialized was an interface class gets defined. In your case I believe this is beta. This interface will need to be part of some distribution so other projects outside the scope of your application can use the class to extend as required. This "distribution" can simply be a literal copy of the source code you pass from project to project, or it can be a proper "Source Distribution" defined as a LabVIEW Build Specification where block diagrams and front panels are removed (if you want to lock down your code, for example).

Take home point is you end up with your .lvclass file and all supporting files, which you may or may not be able to directly edit depending on how it's distributed. You use this as you develop all your other components.

Now consider the gamma project. I use my copy of the beta distribution, add it to my gamma project, and go ahead coding everything as I normally would, extending the class, etc. When I'm done, how do I generate a working copy of gamma that I can use in my alpha? A proper "Source Distribution" is the best choice since you can add your gamma class to the build specification, but also explicitly exclude beta from the spec. After all, the plug-in will ultimately be used with alpha, which already knows what a beta is. Why should I include beta a second time? Think if you have 20 plugins, do you really want to have 20 copies of beta laying around?

Then go ahead and create alpha, using the same beta distribution as you've done before. When it comes time to make your alpha.exe, make sure your beta.lvclass comes along for the ride (usually does automatically since you'll probably have a constant of it somewhere on your block diagram).

There you have it, gamma was just developped completely independently of alpha (or delta, or any other plug-in). And alpha is completely unaware of what a gamma is, the only thing it needs to know is the path to load a gamma from.

See this attachment: Dependency.zip (LV2010 SP1)

If you go the Builds directory and launch the executable, click the "Look For Plugins" button and point the dialog to the folder containing the exe. Then select the plugin you want to load (Gamma/Delta). When you click the Load Plugin! button, the class will be loaded on demand. If you look at the code for Alpha, you'll see the original exe has no clue what either of the plugins are ahead of time, and neither plug-in knows about the other.

You can extend your beta as many times as you want, the only thing you need to worry about is some sort of contract such that your executable knows where to look for these extra classes at run time. In the case of the executable I half cheated by having you pick the path, and the exe then looks for a *.lvclass file with the same name as each folder contained in the folder you point to. I'm sure you can dream up something much more sophisticated, but I hope the example gets the point across?

Have fun!

-michael

[PaulL]

I would think that source distribution is your best bet. If you don't like the detailed hierarchy then you may use llb as your destination. You can put each class into separate llbs.

If you want to use the EXE method that I think you would need to run the EXEs and communicate to them with an application reference mechanism and VI server calls. Not sure what you did exactly in the EXE implementation.

Michael,

Thank you for the helpful hints.

I'm on the fence about whether a source distribution is really the best answer. It may be we find this is the best answer currently available (I'm still evaluating that), but even if that's true it seems that there should be a better option. I will say what I know about the pros and cons in a moment.

Anyway, I actually had already asked my colleague to try putting Gamma and Delta into LLBs. Note that these are not actually single classes, but hierarchies of classes in LVLIBs. My colleague did this but found that to avoid namespace collisions LabVIEW put some of the files outside the LLBs so the distribution is still quite messy (yuck!).

I think you are quite right about having to use some sort of communication between EXEs if we went with that approach (and I'm intrigued by the method you suggested--we haven't tried that yet). Since we really want one application, I'm thinking this is not the way we want to go*.

*But if we don't find something else...

OK, more about source distributions in this context:

The pros:

1) Works.

2) Can exclude Beta from the distribution (see mje's example), which I will reply to next.

3) Relatively easy to build.

My concerns about doing a source distribution:

1) Performance: I know (especially if we strip the diagrams), the code is compiled, but I'm not sure there isn't a cost. (Actually, I expect there is a load cost, but I'm guessing there isn't a performance penalty in execution once loaded. I'm guessing, though.)

2) Ontological: Even if we strip the diagrams, we are still delivering "source" code in some way, and this just seems undesirable on an existential level. When I install something to run with an executable and the RTE, it just doesn't seem right somehow to make this source. Maybe the distinction isn't all that great in practice....

3) Complexity of distribution: At least in some cases (like ours), it doesn't seem possible to package the source distribution neatly. We always end up with multiple files. (Yuck!) I consider this the biggest issue at the moment....

[PaulL]

You can extend your beta as many times as you want, the only thing you need to worry about is some sort of contract such that your executable knows where to look for these extra classes at run time. In the case of the executable I half cheated by having you pick the path, and the exe then looks for a *.lvclass file with the same name as each folder contained in the folder you point to. I'm sure you can dream up something much more sophisticated, but I hope the example gets the point across?

Have fun!

-michael

Michael,

Thank you for the comments, and for taking the time to put together a wonderful example!!!

As I just noted in my comments to Michael A., I especially like that you can exclude Beta from the plug-in builds.

My colleague and I, in the meantime, were focusing on what we could do with LVLIBPs. He sent me the attached file with the following instructions:

pluginArchitecture.zip

"Instructions:

1. Unzip attachment

2. For each lvproj, in the order specified below: open it, run the build, then close it PluginInterface.lvproj Plugin1.lvproj Plugin2.lvproj testApp.lvproj

3. Run pluginArchitecture/applicationBuild/testApp.exe

Notes:

Post-Build-Action VI's remove the unnecessarily created lvlibp files from the Support Directoy.

The build process automatically places them there but the application doesn't actually link to it, instead it seems to keep a relative link from the exe to the original lvlibp."

(Note that he actually created this example to report the LVLIBP linking bug--yes, yet another issue with LVLIBPs--to NI.)

Anyway, my colleague independently arrived at something that looks remarkably like the example you created in most essential ways, except that it uses LVLIBPs instead of source distributions.

What I like about this solution:

1) We can build into a single LVLIBP file (this is a big plus).

2) An LVLIBP is truly compiled and we can distribute it nicely as a plug-in....

What I don't like:

1) We are stuck with all the existing LVLIBP issues (until they are fixed).

2) I'm not sure that some of these issues aren't just bugs, but that LVLIBPs may have ontological issues of their own....

3) The Plug-in Interface is part of the Plug-in build, I think. This isn't actually a big deal in our situation, but it certainly does confuse the situation, if nothing else.

I'm curious to know what you think of this solution....

Paul

[mje]

Following the instructions I get a non-executable VI when the exe is created, but I see where the example is going.

Ultimately, it's the same architecture, only using a packed library instead of loose files. Two things spring to mind which would prevent me from using it:

1. As we've both pointed out, copies of the interface are created in every packed library. For a simple class, this is not a big deal, but if the interface is part of a larger hierarchy, this can get wild pretty darn quickly.
2. It's a packed library, there's no way I'm going anywhere near them until the implementation has matured way more than it is now. I created one out of my most common reuse library as soon as I got LV2010, and trying to use the packed library resulted in LabVIEW rendering so poorly the IDE was not useable. I share your thoughts, I have no idea how lvlibp's got out the door in the condition they're in. They seem like a really good idea, they should have been left in the oven for another year.

I completely understand your desire to have every plug-in as a single file: it's clean and simple as can be to distribute.

[mike_spacex]

The pluginArchitecture.zip posted by Paul earlier in this thread belongs to me; and I admit, I botched it. I recently fixed it up and created a step-by-step presentation on how it is done. Find it all here: https://decibel.ni.com/content/docs/DOC-19176

[Tim_S]

The pluginArchitecture.zip posted by Paul earlier in this thread belongs to me; and I admit, I botched it. I recently fixed it up and created a step-by-step presentation on how it is done. Find it all here: https://decibel.ni.c.../docs/DOC-19176

There is an issue with using packed libraries in this manner. The issue is when the child class uses VIs from another library the Get LV Class Default Value returns error code 7 (file not found) when the packed library is used. NI R&D through Tech Support has recommended to me to use source distributions instead, which I'm in the middle of trying. See http://lavag.org/topic/14925-code-breaking-after-one-execution/page__view__findpost__p__90248 for the test code I used.

[mike_spacex]

Unfortunately, NI Tech Support was dead wrong when they said, "Unfortunately you cannot use file paths that are inside build specifications (e.g. exes, dll, lvlibps)". I'm surprised you didn't jump back at them, since you had just proven that you can, in fact, use file paths inside lvlibp; You did it for Child 1 and Child 3 classes.

It will actually work for Child 2 in your code if you form the path to the lvclass like this "Loading Plugins\Child Library\Child 2.lvlibp\Child 2\Child 2.lvclass"

The reason for this is that Child 2 depends on your CommonLibrary.lvlib. When LabVIEW builds this into a packed project library, it retains the disk hierarchy inside the lvlibp so relative paths between your code and any of it's dependencies remain intact. I'm not sure where it puts vi.lib stuff though, you'd have to ask NI about that.

Thankfully, LabVIEW helps us out with this issue! Drill down in the tools pallette to 'File I/O>>Advanced File Functions>>Packed Library' and you'll find some helpful VI's (two of them in labview 2010, and one extra new one in 2011). The new function 'Get Exported File Path' in LabVIEW 2011 is precisely what you need in this situation. Wire the qualified name to it (like "Child 2.lvlibp:Child 2.lvclass") and it outputs the path to that file within the lvlibp. In 2010 you'd have to get the whole list of exported files and paths and search for the one you want. If you wire this path to 'Get LV Class Default Value' you won't get error code 7 anymore.

I plan on revising the code and presentation I published in NI Community/Large App Development to explain all this.

I find this method far superior to source distribution for a plug-in architecture because source dist. inevitable results in nasty name-conflicts plus gives you a big messy load of source files that must be carried around with your build application... yuk!

Edited November 3, 2011 by mike_nrao

[Tim_S]

Unfortunately, NI Tech Support was dead wrong when they said, "Unfortunately you cannot use file paths that are inside build specifications (e.g. exes, dll, lvlibps)"...

I directed the tech support person to this thread and she has been reading it. I didn't have the R&D person handy to boggle at.

It will actually work for Child 2 in your code if you form the path to the lvclass like this "Loading Plugins\Child Library\Child 2.lvlibp\Child 2\Child 2.lvclass"

[snip]

I will be trying this when I come up for air again (perhaps this year... ). Thanks.

I find this method far superior to source distribution for a plug-in architecture because source dist. inevitable results in nasty name-conflicts plus gives you a big messy load of source files that must be carried around with your build application... yuk!

I managed to get the source code version working during gasps of air. It's not pretty. The class file had to be in the directory and all of the other files were in a llb with stripped block diagrams. I had to prepend to the file names in the source distribution and include lvanlys.dll in the project outside of the dependencies so I could make sure it didn't get renamed. (Yuk! ) Packed libraries are definitely a preferred and more professional looking method.

Tim

[Tim_S]

It will actually work for Child 2 in your code if you form the path to the lvclass like this "Loading Plugins\Child Library\Child 2.lvlibp\Child 2\Child 2.lvclass"

Okay, I was able to try this out without success. The issue, as I understand it, is the file not found is related to the CommonLibrary:Display Message.vi and not the Child 2.lvplib:Child 2:Child 2.lvclass; this is the distinction shown between Child 1, 2, and 3 where 1 has no subVI, 3 has a subVI that is not part of a library and 2 has a subVI that is part of a library. I noticed this does not happen with VIs that are part of the vi.lib.

EDIT -

Okay my brain kicked in on something you mentioned about file paths. The Child 1 and Child 3 have a similar root directory where Child 2 has a different root directory one level up as that is where CommonLibrary is located. Making the path .\Child 2.lvplib\Child 2\Child 2\Child 2.lvclass instead loaded it correctly where as Child 1 only needed .\Child 1.lvplib\Child 1\Child 1.lvplib.

Tim

Edited November 4, 2011 by Tim_S