Best way to handle versioning classes and libraries?

[Daklu]

Does anyone have a simple and effective way to manage versioning labview classes and libraries? I've finally figured out something that seems to work (although it took me ~30 hours of fighting broken links, missing classes, etc.) but it is fairly cumbersome.

The basic idea is that I include a version number in the name of module (class/lvlib). In my toaster hierarchy example I might have an "I2C Slave Base v0.90.lvclass" contained in a folder with the same name and version number. If I'm going to branch code to try something new I copy the module folder and rename the module (and folder) with a new version number. ("I2C Slave Base v0.91.lvclass") I've found this works well--I can add the new module to a project without any cross-linking issues. There are a couple scenarios that still present some difficulty.

My toaster child classes all use Slave Base v0.90 as a private data member. If I want to upgrade Toast Magic to use Slave Base v0.91 obviously I have to replace the class cube in the private data. What's less obvious is that I also have to go through all the Toast Magic vis and replace any exposed Slave Base controls/typedefs/constants Toast Magic uses. This also applies to having a child class inherit from a new base class version. I wish Labview were smart enough to realize that class controls/typedefs/constants used in the client vi should be taken from the actual class being used.

This process is much more cumbersome if I'm upgrading a lvlib instead of a class. Since the library vis don't have a dynamic input like class vis, I have to go through and replace every single control/typedef and library vi with the correlating item from the new library version. (I've uttered many curses for Labview not having a more customizable dev environment. Would it be that hard to enable more hotkey options? Replace With...?)

Even doing something as simple as renaming a module creates all sorts of linking problems. If I'm working in Toast Magic and rename I2C Slave Base to I2C Slave Base v0.90, the next time I open Super Toast 2000 or 3000 the entire project is corrupted as it can't find the class name it is looking for and, contrary to the behavior when looking for missing vis, you must select a class with the same name. (This has bit me countless times... you think I'd learn not to do that.) This issue is the primary reason I started using the above convention.

Are there better/easier ways to deal with module versioning?

(I do use SCC but still haven't figured out the best way to manage projects with it.)

[Aristos Queue]

a) Is there a reason that you need to put the version number in the file name instead of just using the version number in the Library Properties Dialog that gets recorded with the libary (or class or xcontrol)? If you copy the files on disk, you can load both the old version and the new version into memory at the same time, just not in the same project. So if you duplicate your entire hierarchy off, you can change all you want without messing with your original project.

b) On the assumption that you do want to actually change the file name, let's address the project "corruption." The links between files are based on name. If you rename something without its caller knowing about that rename then of course the caller cannot find it the next time the caller loads. That's not corrupt, that's just broken. To avoid this, create a project that has all the stuff that you want to use the new name and then rename the library from inside LabVIEW. Every VI will change its reference to point to the new name. Doing the rename in isolation leads to the problems you've been experiencing.

[LAVA 1.0 Content]

QUOTE (Daklu @ Sep 3 2008, 06:28 PM)

Are there better/easier ways to deal with module versioning?

I'm absolutely against putting versions into any filename. You should use the build in versioning capabilities of classes, xcontrols and libraries.

If you put the version into the library name you have to relink upon upgrade. To beat cross-linking it is better to have a building process in which the source is named different from the code you use in other projects. There are several ways to do so. One of them is VIPM or OpenG builder.

Ton

[Daklu]

QUOTE (Aristos Queue @ Sep 3 2008, 09:52 AM)

a) Is there a reason that you need to put the version number in the file name instead of just using the version number in the Library Properties Dialog that gets recorded with the libary (or class or xcontrol)? If you copy the files on disk, you can load both the old version and the new version into memory at the same time, just not in the same project. So if you duplicate your entire hierarchy off, you can change all you want without messing with your original project.

b) On the assumption that you do want to actually change the file name, let's address the project "corruption." The links between files are based on name. If you rename something without its caller knowing about that rename then of course the caller cannot find it the next time the caller loads. That's not corrupt, that's just broken. To avoid this, create a project that has all the stuff that you want to use the new name and then rename the library from inside LabVIEW. Every VI will change its reference to point to the new name. Doing the rename in isolation leads to the problems you've been experiencing.

Let me respond to your points in reverse...

b) ".detpurroc" naht esu ot drow retteb a ylbaborp si "nekorB"

Wait, that's not what I meant...

b) "Broken" is probably a better word to use than "corrupted." Although when I can't remove classes or vis from a project because LV can't find them I'd call it "really, really broken." (In the past I've had mixed results resolving this by creating an empty class of the correct name just so LV would be able to find something to load. Then I'd remove it from the project.)

I realize renaming or relocating code modules is generally a bad idea. All too frequently I find myself in situations where I need to do one or both of those to maintain code organization. Chalk it up to my inexperience and trying to develop reasonable universal directory structures without fully understanding what the final code hierarchy will look like. For example, part of my initial directory structure (which is mirrored in SCC) was \v1.0\Toasters\Base Class\..., \v1.0\Toasters\Toast Master\..., etc. Eventually I decided it made more sense to use a \Toasters\Base Class\v1.0\..., etc directory structure.

Undoubtedly the majority of the 30 hrs I spent was fixing issues I created when trying to move or rename code using methods that ultimately didn't work, although some of it is due to untimely crashes. (LV 8.5 crashes *every time* I try to move a folder using the "Move on Disk..." context menu.) The other situation I run across frequently is when I am refactoring code during development. When I have a vi I am no longer using I'll prefix the name with "DEP" to indicate it is depreciated.

Opening a project with all the code that uses the vi works, it's not necessarily easy to do. If I'm working on my Toaster hierarchy and need to change something, I need to dig through all our SCC to find everything that uses those classes. I've missed things more than once and grind my teeth while spending the next several hours fixing broken links.

a) There are a couple reasons I put version number in the name...

1. The primary reason is I don't know any better. I used the shotgun approach to figuring out a solution and this is the first one I found.
   
2. It allows me to load multiple versions in the same project which was key for me to avoid cross-linking and broken links. (i.e. Different vis in a project can use different version of the code.)
   
3. It makes it easier for me to tell which version I'm using when coding client vis. Context help shows the version as part of the name.
   
4. I decided on this convention after development had already started and I can't decrement the version number of a user-defined class.
   

That said, I'm not locked into keeping version numbers as part of the name. If there are better ways to handle it I'm all ears.

Regarding copying the entire hierarchy and working on the copy, this raises a question on the SCC model to use. (Keep in mind my local directory structure mirrors the SCC directory structure.) As far as I know there are a couple SCC models that are commonly used:

• Single Trunk - Released code is always stored in the same location. Dev code is branched off the trunk and merged when completed.
  
• Parallel Trunks - Each version gets its own trunk. When a new version is released the previous trunk is killed and a new trunk created.
  

I migrated towards a parallel model because it seemed to ease my problems with broken links at the cost of lots of replacing when upgrading a client vi to a new module version. You seem to be suggesting the single trunk model is more appropriate?

In general, do I only need to create copies of the code module I'm changing or should I create a copy of the entire hierarchy? For example, if I'm refactoring my Toast Master class, do best practices dictate copying the entire Toaster hierarchy or is making a copy of just the Toast Master class adequate? (Please don't say I should make a copy of all the vis in the project.)

How do you go about integrating the new test code into your application? You can't load the new Toast Master class into the project without manually replacing all instances of the old Toaster Master class members in the main application.

QUOTE (Ton @ Sep 3 2008, 10:24 AM)

I'm absolutely against putting versions into any filename. You should use the build in versioning capabilities of classes, xcontrols and libraries.

If you put the version into the library name you have to relink upon upgrade. To beat cross-linking it is better to have a building process in which the source is named different from the code you use in other projects. There are several ways to do so. One of them is

VIPM

or

OpenG

builder.

Ton

My cross-linking issues occur in the dev environment. Those solutions seem to be geared towards building executables. Am I missing something?

[eaolson]

QUOTE (Daklu @ Sep 3 2008, 11:28 AM)

Are there better/easier ways to deal with module versioning?

I was doing something similar lately (though I was using directories tagged with the version number, not individual files) and it's not a good solution. It doesn't scale well and changing versions is a nuisance.

What I've been doing lately* is making sure I get some reusable component into decent shape (adequately tested, debugged, etc.) then tag it in SVN with some unique tag name. Then when I have a project that needs to use that component, that tagged version of the component gets checked out into the project's directory using svn-externals. This means that future revisions to the component do not affect the project, since I've checked out a particular version of the component.

* Note: I'm a single developer, am still working the details out, and may not know what I'm doing. Use at your own risk.

[Jim Kring]

I would use VIPM to create a package of your class. This way, you can easily get your class in the palettes and also achieve version control and configuration management in your projects.

[LAVA 1.0 Content]

QUOTE (Daklu @ Sep 3 2008, 09:45 PM)

My cross-linking issues occur in the dev environment. Those solutions seem to be geared towards building executables. Am I missing something?

Yes, you miss something.

You should consider your own used classes as products. So you have a development source which is source code controlled.

For usage in other projects you make source distributions. However you need to rename those to prevent cross linking between development code and production code. This can be done in various ways, if you use VIPM you should change the custom postfix to make use of hierarchies of classes (I have a serious doubt if VIPM can deal with hierarchies of classes). The only thing you have to do in your production project is build the class hierarchy. Now if you need a new version of your class you can build your class, install it and the next time you open up your production project the new version will be loaded. (Make sure the production class isn't loaded during this action).

Here's your definition of relinking:

QUOTE

This process is

much

more cumbersome if I'm upgrading a lvlib instead of a class. Since the library vis don't have a dynamic input like class vis, I have to go through and replace every single control/typedef and library vi with the correlating item from the new library version. (I've uttered many curses for Labview not having a more customizable dev environment. Would it be that hard to enable more hotkey options? Replace With...?)

If you had used the exact same name for the library and the VIs you would have no relinking issues.

Ton

PS Fixing classes inside OpenG builder would fix VIPM, so if you have time join us.

[Daklu]

QUOTE (eaolson)

It doesn't scale well and changing versions is a nuisance.

Grr... you can say that again.

QUOTE (eaolson)

...using svn-externals...

We don't use svn so I'm not familiar with the terminology. What are svn-externals?

QUOTE (eaolson)

I'm a single developer, am still working the details out, and may not know what I'm doing.

Me too, except change "may not" to "certainly does not."

QUOTE (Ton)

If you had used the exact same name for the library and the VIs you would have no relinking issues.

I used to use the same name, but that's what led to all my headaches in the first place. However, based on your comments it looks like the root of my problem is not separating "dev code" and "production code."

I'll apologize now for the length of my posts. I need to make sure I fully understand how all this fits together.

Here's the workflow I've been using:

• Open a project that contains my top level application. (In my case it is a .lvlib I will use with Test Stand rather than an executable.) The Toaster, I2C Interface, and I2C Slave class hierarchies I am developing in parallel are listed under Dependencies. All the code is located in my dev folders. Dev folders and classes have version numbers in the names.
  
• When I find something that needs to be changed in my class hierarchy, I check out the code, make a change, and check it back in. In most cases I have all the calling vis loaded so unexpected broken links isn't an issue.
  

I do this partly because I (again) don't know a better way and partly because it seems like it will be easier to move the source code to other computers. For this application I'm developing on one computer and running the code on a second computer. When I need to test the code I just open up Beyond Compare and copy it over to the test station.

Here's the workflow I think you're describing. I'm thinking through this as I type it up so tell me where I go wrong:

• Develop each of my classes independently, perhaps by creating a single project for each hierarchy. Dev folders and classes do NOT have version numbers as part of the name.
  
• Build my hierarchies to some location other than my dev folders, such as user.lib. If I use a tool that appends a suffix to the class member names, the vis I'm using in my top level application won't interfere with the class members in my dev folders.
  
• When I need to make a change to my classes, I do so in the dev folder. I close my top level project and rebuild the class hierarchy to user.lib. Any application that uses those classes automatically uses the most recent version.
  

If I need to make a change to a class that breaks backwards compatibility, but I also need to use the previous version in projects...

• Use the same dev folder for v2.0 that I did for v1.x. This allows all the links to remain intact during dev.
  
• In my Source Dist build specs, use a different suffix for v2.0 vis than I did for v1.x vis. This allows both versions to be on the palette and used in applications at the same time.
  
• (Optional) Send the Source Dist to a separate folder in user.lib. The versions will have different names so it's not required, but nice to keep the code separated.
  
• I can only have one version of the class in my dev folder at a time, so if I need to modify v1.x I check in v2.0 and check out v1.x.
  

Assuming I have all that correct, I have a couple questions. (Big surprise.)

1. Is it best to wrap a single class hierarchy in a project and distribute it as a whole rather than have each class in a project and distribute them individually? My gut says yes, but I need to ask anyway.
   
2. When developing my classes, should any vis that are not part of my class hierarchy be taken from user.lib? I'm specifically thinking about my Toaster child classes, which include (as opposed to inherits) the I2C Interface base class. I assume I should be using those versions from the palette as they will remain updated whenever I build the class source code.
   
3. If I want to update the Toaster child classes to use I2C Interface v2.0, I need manually update all the I2C Interface v1.x vis/controls/typedefs the Toaster child class uses, since the v2.0 vis have different filenames, correct?
   
4. On the other hand, the Toast Master class inherits from (as opposed to includes) the Toaster base class, meaning I should inherit from and use those Toaster vis in my dev folder rather than user.lib while developing the Toast Master class? Otherwise I'm developing a child class using a previous build of the Toaster class instead of the Toaster class currently being developed.
   
5. If another developer (hypothetical, as I'm currently the only developer here) needed to do some work on my top level application, what's the best way to make sure he also gets the correct versions of the classes in vi.lib?
   
6. What's the best way to change my class names and vi locations (to remove the version number from the directory name) without completely trashing the work I've done so far? (AQ did address changing the name below...)
   

QUOTE (Ton)

PS Fixing classes inside OpenG builder would fix VIPM, so if you have time join us.

Uh... don't you want developers who actually know what they're doing?

[eaolson]

QUOTE (Daklu @ Sep 4 2008, 02:18 PM)

We don't use svn so I'm not familiar with the terminology. What are svn-externals?

Think of it as a reference to a different repository. So when I check out BigProject, it sees a reference to LittleComponent v1.0, and checks out a copy of that, too. It's nice because it means the BigProject repository just needs a reference to LittleComponent, and so it doesn't actually contain duplicate copies of the LittleComponent files. And because I specify a particular version of LittleComponent, if I check out a copy of BigProject a year from now, after LittleComponent has gone to version 2.0, SVN will check out the version of LittleComponent I know is compatible.

[Daklu]

Just to follow up and record information for future readers who may have the same problems I did... (Thanks to Ton for pointing me in the right direction.)

I don't have the Professional version of VIPM and the limitations of the Community version made it inadequate for my requirements. I ended up spending several hours with the OpenG Builder and figured out how to use it to distribute "released" code to my user.lib. It appears to do everything I need quite well. There are many ways to customize the distribution. If anyone is not using OGB, I highly recommend looking into it. It is much better than Labview's Application Builder. (Though OGB does require Application Builder to compile executables.)

I decided to create a single project file for each class inheritance hierarchy. When I make changes to any vis within that project the entire hierarchy is re-released to my user.lib. On the dev side each project file and hierarchy is wholely contained within a directory with a version number for a name. (i.e. \MyDevProjects\Toaster Classes\v1.00\Toasters.lvproj ...) I do NOT include version numbers in the name of any source code vis. When I need to make a minor rev I simply copy the folder and rename it with a new version number. (i.e. \MyDevProjects\Toaster Classes\v1.01\Toasters.lvproj ...) This makes it easier to keep track of what version I'm working on at any given time and allows me to view multiple minor versions simultaneously during dev, as long as the minor versions are in separate projects. Since all the files are wholely contained within that directory structure OR reference vis from user.lib, there are no broken links in the new version.

As Ton mentioned, OGB has the ability to append suffixes ("namespaces") to filenames during the build to guarantee uniqueness while at the same time maintaining all the links. I decided on a namespace that includes major version number in the code released to user.lib. (i.e. "MyToasterVI__toasters_v1.vi") If I make a change to a class that breaks backwards compatibility I'll increment the major version. This allows me to use multiple released major versions of the same hierarchy at the same time. Minor versions and bug fixes are automatically integrated into any vis that use that class hierarchy. Minor versions *must* be backwards compatible so I don't anticipate needing to use more than one released minor version at any time.

I have 4 class hierarchies and one lvlib that were all linked within my dev directory. Once I figured out how to use OGB it only took me ~6 hours to break them apart, create build files, and relink to the correct files in user.lib. Much less than the 30 hours the previous exercise took.

There are a couple things to look out for though:

1. Take the time to figure out where the leaves of your calling tree are and work on those first. When a vi uses a vi/ctl from another hierarchy or library, that hierarchy or library already needs to be distributed to your user.lib. This is obvious once you run into the problem, but may not be obvious when you start.
   
2. Take the time to think about the long term consequences of your distribution scheme. I decided to distribute my classes to user.lib by maintaining my subdirectory structure instead of as an llb. This would get me up and running quickly while maintaining some level of palette organization. I realized after the fact that once I release a public vi in a certain location I can't move it without creating a maze of destination exceptions.
   Since I frequently change the locations of source files during dev I'm going to have to go back and change the distribution to an llb and relink all my dev code. I believe this will allow me to arbitrarily change source file locations without affecting the released code in user.lib. I'm also going to release it to \user.lib\_MyTools\... and create custom palette menus for each class. More work up front... easier long term maintenance and usability.
   

Hope this helps someone else through the pain of figuring out how to organize projects.

[Daklu]

Follow up to the follow up... (Follow up^2?)

OpenG doesn't appear to be able to handle putting class hierarchies in .llb files due to name collisions. I tried a build with each class in its own llb and various other schemes but none of them worked. For the time being I'm stuck with using a directory structure build.

[LAVA 1.0 Content]

QUOTE (Daklu @ Sep 8 2008, 04:59 PM)

Follow up to the follow up... (Follow up^2?)

OpenG doesn't appear to be able to handle putting class hierarchies in .llb files due to name collisions. I tried a build with each class in its own llb and various other schemes but none of them worked. For the time being I'm stuck with using a directory structure build.

Yes, OpenG builder suffers from the same issue as NI builder.

To counteract that I have placed every class in a separate folder. (something you most likely already have done)

In OpenG builder I have setup source (and target) directory for each folder. (so every class in it's own llb)

http://lavag.org/old_files/monthly_09_2008/post-2399-1220897435.png' target="_blank">

Ton