User Library Management

[crossrulz]

I was recently tasked with setting up a system for managing our LabVIEW User Library. The library hasn't been created yet, but it is coming. I want to use VIPM, but my company is very hesitant about spending money on software. What other options has anybody tried? I'm looking for wide array of opinions supporting and criticizing any option.

[crelf]

I was recently tasked with setting up a system for managing our LabVIEW User Library. The library hasn't been created yet, but it is coming. I want to use VIPM, but my company is very hesitant about spending money on software. What other options has anybody tried? I'm looking for wide array of opinions supporting and criticizing any option.

I've worked at several companies who have tried reuse, and IMHO VIPM is the only solution that has really worked. To apply what I just said in another thread, if you don't use VIPM you'll either abandon your reuse efforts or you'll end up with a BBOM. Now all I want is for someone to come out with TSPM (TestStand Package Manager)

[Aristos Queue]

VPM is so good that NI is looking into using it for a couple of its own toolkits (we may already be doing so, I'm not sure, but I know it was being investigated).

[Daklu]

I was recently tasked with setting up a system for managing our LabVIEW User Library. The library hasn't been created yet, but it is coming. I want to use VIPM, but my company is very hesitant about spending money on software. What other options has anybody tried? I'm looking for wide array of opinions supporting and criticizing any option.

You don't manage user.lib -- you manage your reuse library. NI has moved away from advising people to put their reuse code in user.lib and instead recommends vi.lib. (See here.)

I agree with Chris, your reuse library is best managed by VIPM. Note that you only need to purchase licenses for developers who are creating packages. You can use the freeware edition to manage the packages on each computer. (Though the Pro edition does have the added benefit of being able to create package configurations, something that is useful for all developers.)

If management is still unwilling to spend money on VIPM Pro, you can use OpenG Package Builder to build packages instead of VIPM. I guarantee* it will cost you more figuring out how to do everything using OGPB than it would cost to purchase a $1k VIPM Pro package. OGPB is more flexible than VIPM however, so if VIPM doesn't create the packages you need AND you have time to burn, OGPB will likely provide you with a solution.

*To put some rough numbers to it, over the past 1.5 years I've spent well over 200 hours figuring out how to build my packages using OGPB. (Granted, some of the things I've been trying to do fall outside the norm.) If your management values your time at less than $5/hr, OGPB makes perfect economic sense.

[jgcode]

If management is still unwilling to spend money on VIPM Pro, you can use OpenG Package Builder to build packages instead of VIPM. I guarantee* it will cost you more figuring out how to do everything using OGPB than it would cost to purchase a $1k VIPM Pro package. OGPB is more flexible than VIPM however, so if VIPM doesn't create the packages you need AND you have time to burn, OGPB will likely provide you with a solution.

*To put some rough numbers to it, over the past 1.5 years I've spent well over 200 hours figuring out how to build my packages using OGPB. (Granted, some of the things I've been trying to do fall outside the norm.) If your management values your time at less than $5/hr, OGPB makes perfect economic sense.

I personally don't think OGPB is that complicated.

Plus, in most cases you will have to (currently) use it with VIPM to create your top level palette menu.

I think more emphasis should be directed to the .vipc files too.

These will become an important part of the project under SCC.

Its like having all the benefits of not copying your reuse library dependencies to your project, but the piece of mind that you are

[crelf]

I agree with Chris...

Of course you do

Note that you only need to purchase licenses for developers who are creating packages. You can use the freeware edition to manage the packages on each computer. (Though the Pro edition does have the added benefit of being able to create package configurations, something that is useful for all developers.)

...and the "Enterprise" level gives you real control of the administration of your library - read more here: http://jkisoft.com/vipm/

If your management values your time at less than $5/hr, OGPB makes perfect economic sense.

VIE used to make a lot of tools internally (eg: the VISTA suite) because there just weren't any really professional software engineering tools available in the marketplace. Thankfully, with products like VIPM, VI Analyzer, Desktop Execution Trace Toolkit, the Project Explorer, etc, we can limit the number of in-house untilities and rely on COTS products <- someone else designs, develops, maintains (what happens when NI comes out with a new version of LabVIEW that somehow unintentionally blocks the hooks that you were using in your custom tool?), etc, which means we can concentrate more on what we do best: test. I've seen far too many companies get stuck developing internal tools rather than use COTS products to try and save a few bucks (how many of use have tried, and failed, to write our own TestStand-Lite? ) - in summary: in almost every case, going with the COTS product is the smart way to go, and using VIPM for resue is a great example of that.

[Aristos Queue]

Chris: What is "COTS"?

[Grampa_of_Oliva_n_Eden]

Chris: What is "COTS"?

Commercial Off The Shelf

Ben

[Jim Kring]

Chris: What is "COTS"?

It's a FLA

Hi All,

Thanks, everyone, for your recommendations of VIPM. As you know, JKI has put a lot of energy into the tool and we plan to put a lot more effort into it in the future. In fact, we're actively working on VIPM 2010, right now, which will provide a lot of powerful and flexible packaging features that make it the best choice for creating distributions of LabVIEW reuse libraries and add-ons. We'll will be blogging about some of the new features we're working on soon, too.

If you have specific questions for JKI or want to evaluate VIPM Professional/Enterprise, please contact us. We're more than happy to chat with you.

Thanks,

-Jim

[Daklu]

I personally don't think OGPB is that complicated.

Yeah, I'm a little slow...

That does include time spent figuring out OpenG Builder too. VIPM encapsulates the build/packaging process while OpenG has separate tools for them. A good chunk of the time was spent fighting with NI's mnu files. VIPM's palette building tool is my favorite part of the software. (If only I could yank that out and use it in my own workflow.)

Plus, in most cases you will have to (currently) use it with VIPM to create your top level palette menu.

Are you referring to a top level dynamic palette?

[jzoller]

Commercial Off The Shelf

Ben

Side note:

One of the recent ISO standards transforms it to SOUP: Software Of Unknown Provenance.

Joe Z.

[crelf]

One of the recent ISO standards transforms it to SOUP: Software Of Unknown Provenance.

Mmmm soup... "COTS" is also interchangable with "OTS" <- off the shelf.

[Aristos Queue]

One of the recent ISO standards transforms it to SOUP

COTS for SOUP... I guess that makes sense... food can frequently be substituted for sleep.

[Ton Plomp]

COTS for SOUP... I guess that makes sense... food can frequently be substituted for sleep.

Getting way off topic, but COTS sounds like Vomit in Dutch...

On topic, User Library Management is doable, however you need good discipline. And anything that helps you lowers the need for discipline.

Ton

[crelf]

...anything that helps you lowers the need for discipline.

That's a great saying, but I don't think it's appropriate here. That's like saying you can program most things in a 3GL - people that use 4GLs are weak Besides, my business is test, not reuse, so I'd much prefer to use a tool that helps me with reuse so I can get on with my business.

[jgcode]

Yeah, I'm a little slow...

That does include time spent figuring out OpenG Builder too. VIPM encapsulates the build/packaging process while OpenG has separate tools for them. A good chunk of the time was spent fighting with NI's mnu files. VIPM's palette building tool is my favorite part of the software. (If only I could yank that out and use it in my own workflow.)

Ok, I use NI builder API when I use OPPB, so I haven't spent time trying to tackle the OpenG Builder yet!

What are your experiences with it?

What are the advantages/disadvantages in using it over NI API?

So far, NI builder API is great for what I do (especially in 2009, not so much 8.2)

Are you referring to a top level dynamic palette?

Yer, I get the organisation I want by doing the following:

All .mnu files get stored in a "dynamic_palette" folder (under company name folder in vi.lib/addons), then a dynamic menu file (which is the what I call the top level palette menu) is created and points to that "dynamic_palette" folder.

This menu goes in the Categories and Controls folder under the vi.lib\menu folder.

I just copied JKI after talking to Jim about it.

Do you do something similar/different?

[Cat]

I'd much prefer to use a tool that helps me with reuse so I can get on with my business.

I need a tool that slaps me severely if I try to alter code I've used in some other application. I used to do just fine keeping track of all that, and then LabVIEW switched me to the Project Paradigm. I've had more than one really big boo-boo since then.

Or maybe it's just that I've just finally hit the critical threshold of brain cell loss from too much .

[Daklu]

Ok, I use NI builder API when I use OPPB, so I haven't spent time trying to tackle the OpenG Builder yet!

What are your experiences with it?

What are the advantages/disadvantages in using it over NI API?

OGB vs. LVSD (Labview Source Distribution) is very similar to OGPB vs. VIPM. The former offer more flexibility but can be quirky. The latter are much easier to use and have better fit and finish. I did a user group presentation a year ago titled Using Freeware Tools to Manage Reuse Code. (You can get it here*.) Slide 11 summarizes the main differences between OGB v2 and LVSD 8.6. (See attached pdf.)

*Some of the information in the Code Management Best (Pretty Good) Practices section is no longer valid and needs to be updated.

I just copied JKI after talking to Jim about it.

Do you do something similar/different?

I do the same thing with the minor exception that I put them in vi.lib rather than vi.lib\addons. Jim was kind enough to post an example of how they do it in response to a bug I posted here. Looking back on that thread I think I set up my relative paths incorrectly--classic example of a user ID-10t error. I've been having a fairly in-depth discussion with Bob Des Rosier of NI about the Palette API. Getting the mnu files set up correctly represents a pretty good chunk of that 200+ hours.

I need a tool that slaps me severely if I try to alter code I've used in some other application.

I used to have the problem of unintentionally modifying the code in my reuse library... now I make all my reuse code read-only while building it. Problem solved.

(You do keep your reuse source code separate from the reuse code you use in your applications don't you?)

OGB v2 vs LVSD 8-6.pdf

[Cat]

I used to have the problem of unintentionally modifying the code in my reuse library... now I make all my reuse code read-only while building it. Problem solved.

Hmm. Interesting solution. I could apply that to my generic reuse code...

(You do keep your reuse source code separate from the reuse code you use in your applications don't you?)

I keep generic reuse code in my user.lib (which I guess is passe now, too). I have a few hundred functions I've written over the years in there (many of which were eventually supplanted by OpenG functions ).

An example of my problem is this: I have the Big Project (BP) that stores data is certain file formats. A New Project (NP) comes along and wants to be able to use the tools that I've written for BP to analyze its data, so it needs to have its data stored in the BP file format. Not wanting to rewrite code for the NP, I reuse BP-specific functions/typedefs/etc. Or, I come up with a better way to do something in NP and want to use it in BP. Or when I started using queues instead of GVs to get data from my buffer to my analyzer in BP and it broke various NPs.

So how do I organize this? Does code that gets used over and over in BP, and one time in NP, get moved to a "reuse" directory? Am I just being old-fashioned in thinking of things as going in "directories" in the first place? I've spent some time on the NI website reading documents about the Project Paradigm, but I have not found anything that really answers how to best utilize Projects when you've got multiple large overlapping applications. That, BTW, all have to be turned into executables at some point, so I really don't have the option to just ignore Projects and manage it all how I used to.

Ehh. Sorry about the rant. This is just something that I feel is very important to get under control before I do some damage that it takes some real time and effort to recover from.

[jgcode]

...now I make all my reuse code read-only while building it. Problem solved.

Do you create a random password each time you build? Or do you use the same password for all releases?

If random, how to you manage all the passwords? Given that the clients that you distribute your code to (internal or otherwise) may require the password (from time to time)?

[Cat]

This is just something that I feel is very important to get under control before I do some damage that it takes some real time and effort to recover from.

So, I ranted yet again to my boss about this topic and (finally!) should have VIPM on order momentarily.

[crelf]

So, I ranted yet again to my boss about this topic and (finally!) should have VIPM on order momentarily.

Hazzaah! Welcome to the next level of professional software engineering

[Grampa_of_Oliva_n_Eden]

Hazzaah! Welcome to the next level of professional software engineering

?

Where you don't have anyone else left to blame?

?

Ben

[Daklu]

So how do I organize this? Does code that gets used over and over in BP, and one time in NP, get moved to a "reuse" directory?

Short answer: Yes.

Long answer: My reuse code covers relatively large units of functionality. I don't deploy anything smaller than a class and most of my reuse modules are multiple classes contained in an lvlib. I don't bother keeping a set of independent utility vis as part of my reuse library. If there's a handy utility vi I could use from a different project, I'll create a new vi in the current project and copy the code. I never have source vis from a separate, independent project show up anywhere in my current project's project window.

Your problem is a little bit of a special case in that the two projects aren't completely independent. I have a similar situation right now where I have been writing a helper app to create calibration data that will be used in our main application. In this case I created a folder for my sub-project in the scc tree of our main app, started a new project there, and including the new project in the main project. My CalFileEditor links directly to the other code, but since it's all in the same scc node the chances of conflicting changes is lessened. I've also tried to insert abstraction layers to protect the CalFileEditor from changes to the main app source code. (See below for one example.)

If the code I need somewhere else is expected to be generally reusable, I'll extract it, generalize it, and deploy it to my reuse library. More specific reusable code stays in the projects.

I've spent some time on the NI website reading documents about the Project Paradigm, but I have not found anything that really answers how to best utilize Projects when you've got multiple large overlapping applications.

If you're going to have applications with overlapping functionality and you want to reuse source code, the single most important thing is to make sure you manage the dependencies of your reusable code. Nothing's worse than dropping in a class you think is reusable, only to see it import another 2000 vis from an application written last year.

Assuming you've done that correctly and you expect there to be more apps to follow, I would probably create a single parent directory containing directories for each of the individual applications that make up the system. I'd also add a Shared directory for code used by multiple apps. When you find something in one app you want to use in another app, extract it from that project directory and put it in your Shared directory. (Be careful not to break the links in your original app!) Then, once you've moved the code to the shared directory, remove it from your original app's project to indicate that app no longer "owns" that code.

This is just something that I feel is very important to get under control before I do some damage that it takes some real time and effort to recover from.

Better to recognize the potential trouble now than after the damage is done.

Do you create a random password each time you build?

Nope, I don't use passwords at all. I do lock the libraries, but no passwords. Nothing I distribute to users contains confidential IP they are not allowed to see. If a user wants to get in and muck around with the code... that's on them. I can't protect them from their own stupidity.

If the time comes when I do need to use passwords, I would likely start with random passwords. If someone needs to see a block diagram I can always rebuild that module with a specific password for that user.

So, I ranted yet again to my boss about this topic and (finally!) should have VIPM on order momentarily.

Lucky dog cat!

[Cat]

Where you don't have anyone else left to blame?

Oh. Crud. Why didn't you mention that before?!?