RFC: How to get AppBuild to automagically know what to include on host for a particular RT/remote?

[Aristos Queue]

I've been asked the same question multiple times in the last month from several different groups and I not only do not have a good answer in today's LabVIEW, the solutions I have thought of to propose for tomorrow's LabVIEW all feel clunky to me. So, I figure I'll ask the community for ideas.

 

The situation:

Alice creates a collection of VIs that together form two halves of a client-server architecture. The client side sends in requests, the server side sends back responses. Each response includes a name of a dynamically loaded VI or a class (doesn't matter which for the purposes of this problem) that the client has to load in order to process the response. (If we want to get really crafty, the server might have to do similar dynamic loading in response to the information coming from the client, but I don't think that added complexity adds anything to the overall problem to be solved, so let's leave it aside for now.)

 

Bob now builds an application using Alice's VIs. He has his client-side application and his server side application. Everything works in the development environment because all of Alice's VIs are on disk. Now he builds his two applications using AppBuilder. His EXEs do not work because his client application does not include the dynamically loaded VIs/classes (assume Alice did all the right path magic to make it work had the build spec been told to include those files).

 

Problems:

1. The dynamically loaded VIs/classes need to be a part of Bob's built applications. But Bob doesn't even know there's any dynamic loading going on -- that's all private inside Alice's collection of VIs. Is there a way for LabVIEW to magically know "oh, these VIs/classes need to be included in the EXE?" even though there is no static reference to them anywhere in either Alice or Bob's code?

 

2. Assume there is no way for LabVIEW to magically know this information. Is there something Alice could have done to her code such that Bob is made aware of the need to manually add some files to his build spec as dynamic dependencies (other than just documenting it in some file somewhere and praying that Bob notices)? The answer here may include new features of LabVIEW as needed, but please specify whether you are giving an answer that can be accomplished with LV 2013 vs LV Someday.

 

3. The whole point of using dynamic loading is so that you don't bring all the VIs/classes into memory at launch. But there may be some VIs/classes that are *never* used in Bob's application. If the answer given in #2 is that Alice specifies all the things she could dynamically load, then they'll all get bundled into Bob's EXE. That may bloat the EXE. Is there any way for Alice to write her code such that Bob can figure out which dynamic loaded VIs/classes are actually possibly used by his application (bonus points if it's a solution that lets LabVIEW can figure it out on Bob's behalf)? 

 

4. Assuming you came up with answers to #2 and #3, do those answers still work if the set of dynamically loaded VIs/classes is open-ended (i.e., a third party could have installed yet more plug-ins for Alice's library to use, and thus Alice cannot enumerate all the possible things that need to be loaded)?

[hooovahh]

Okay so in the past a way to force LabVIEW to include some VI was to add it to a part of the code that is never called.  That however breaks some of your requirements (like dynamically loading) but can be accomplished today.

 

What about this solution which is also "clunky" and I don't have an example of it working.  Alice makes a Pre/Post Build VI that runs which scans the application hierarchy.  Alice can know the interdependence that LabVIEW can't, and can write Pre/Post Build VI to know the link without loading the VIs, and include them in the build somehow.  Possibly by just copying them to a subfolder that the EXE is in, or by modifying the Build Specifications to have them be Always Included.  This can also be done today but sounds like a pain, and again Bob will have to know to use this Pre/Post Build VI, but hopefully he got the project file as a template from Alice which already calls this VI.

 

EDIT:  To get number 4 there could be a tag of some kind, maybe a #AlwaysInclude so that any 3rd party could add more assuming they adhere to the standards Alice added to her Pre/Post VI.

 

For the future...I guess there could be some kind of new object similar to the Static VI Reference called Dynamic VI Reference (too confusing?), which would be some kind of flag to the compiler/builder that makes it known that this VI needs to be included, because of a non static dependency, but should not be loaded into memory because of this.

Edited October 3, 2013 by hooovahh

[ShaunR]

1. The dynamically loaded VIs/classes need to be a part of Bob's built applications. But Bob doesn't even know there's any dynamic loading going on -- that's all private inside Alice's collection of VIs. Is there a way for LabVIEW to magically know "oh, these VIs/classes need to be included in the EXE?" even though there is no static reference to them anywhere in either Alice or Bob's code?

Not that I'm aware.

2. Assume there is no way for LabVIEW to magically know this information. Is there something Alice could have done to her code such that Bob is made aware of the need to manually add some files to his build spec as dynamic dependencies (other than just documenting it in some file somewhere and praying that Bob notices)? The answer here may include new features of LabVIEW as needed, but please specify whether you are giving an answer that can be accomplished with LV 2013 vs LV Someday.

Sort of. See next.

 

3. The whole point of using dynamic loading is so that you don't bring all the VIs/classes into memory at launch. But there may be some VIs/classes that are *never* used in Bob's application. If the answer given in #2 is that Alice specifies all the things she could dynamically load, then they'll all get bundled into Bob's EXE. That may bloat the EXE. Is there any way for Alice to write her code such that Bob can figure out which dynamic loaded VIs/classes are actually possibly used by his application (bonus points if it's a solution that lets LabVIEW can figure it out on Bob's behalf)?

 

The classic solution is to make access to the dynamic VIs loaded by a relative path. Then you can have a directory that you just add dynamic VIs/Classes to. This is slightly different to what you are describing as they are not bundled or built with the exe. Instead, they are usually supplied and managed as a separate installation. But they will not break the exe if not present (just that feature/function will not be available) and are flexible to add/remove functionality.

The issues with this approach are that the "modules" have to have diagrams attached otherwise they cannot be loaded into different versions of LabVIEW. So stripping the diagrams creates version and maintenance problems but as the password protection of some LabVIEW versions was compromised (the best solution); it is unsavoury for commercial products.

Distribution of the total solution is problematic from a NI only stand-point too. It can easily be achieved via a VIPM package, but NI don't own that. So Alice would distribute a VIPM package with a dependency on the modules that would get pulled in and installed to the right place alongside the exe. Using NI builder solutions are much less elegant requiring a source distribution for the modules and an exe build for the main VI that must be modified to get to install in the right locations on another machine or labview version (because of absolute paths) and requires work from Bob to realise.

 

 4. Assuming you came up with answers to #2 and #3, do those answers still work if the set of dynamically loaded VIs/classes is open-ended (i.e., a third party could have installed yet more plug-ins for Alice's library to use, and thus Alice cannot enumerate all the possible things that need to be loaded)?

Yes.

Edited October 3, 2013 by ShaunR

[shoneill]

I've gone through this though process a few times int he past looking at properly scalable architectures.

 

Given point 4 (That Alice cannot herself know the full scope of the possible classes involved) then we essentially have a range of undefined classes on the server side.  This makes it essentially impossible to configure Bob's software pre-compile in isolation because we can only link to a given state of Alice's software which seems flaky.  You would almost need to be able to link to the build spec for a given version of Alice's software (impractical) or link to an actually running version of the software where we could actually retrieve the entire source code for ALL included classes within Alice's software after which Bob can then build against that and include all of these classes.  This of course requires Alice's compilation to be made with a feature to make the class source (preferable password protected) available on request.  A method to NOT expose the classes outside of the build process would of course be a big addition to the security of such a system.

 

I have thought it would be really nice to have the ability to send the object code WITH the object over the interface (perhaps only on request from Bob's software) and then instantiate the object dynamically on Bob's computer.  Of course this raises a lot of questions such as OS differences between Alice's and Bob's software and such but it seems to be the ultimate solution to the problem at hand.

 

PS My considerations were actually between Host and RT Systems, but the problems are exactly the same.

 

Shane.

 

PS A mini-compiler for Alice's software where the classes can be custom compiled for Bob's machine before making them available for Bob's software would solve the problem if and only if it had the ability to compile classes for any given target (RT, Win, Mac, Linux etc.)

Edited October 4, 2013 by shoneill

[shoneill]

The thread seems to have gone very quiet.

 

I personally think that the server actually needs to do the managing here.  The client is a client not only for the messages received but also for the CODE to process those messages.  We're actually sending code to execute over the network.  How to deal with different OSes, I dunno.  How to deal with some serious security issues this could raise I also dunno.  Going this route also allows changes in Alice's software AT RUN TIME to be mirrored on Bob's client.

 

Seems to me at least like the solution is a way to piggyback code onto the messages themselves.  Kind of sounds like a virus to be honest......