Using Subversion for distribution and versioning

[Onno]

Hi! I'm currently pondering a way to distribute and version a software package that's used within my scientific research group for data acquisition and instrument control. I'm considering Subversion as a solution, but the idea seems a bit weird. So I was wondering if someone here would have some feedback on this? Or pointers to other, better ways to solve the issue?

The issue:

• Basic versioning: the application should know its own version (for display in an About dialog, etc.)
  
• Update notifications: the application should alert users if they're using an outdated version. (It happens quite a lot that people are using outdated versions without realizing it).
  
• Easy installation/upgrade. (The only installation step is extracting a directory tree with VIs, really, so I don't really need a sophisticated installer, per se).
  

Now it seemed to me that having everything as a SVN working copy makes it all easy:

• Installing? Just do a checkout.
  
• Current version? Just check the URL of the working copy. (If it's, e.g., svn://url/to/repo/my-project/tags/1.2.0, it's version 1.2.0).
  
• Update notifications? Just do an "svn list svn://url/to/repo/my-project/tags" to list all the tags, and compare the highest one with the current version.
  
• You can easily check for local modifications to the source, as well (just do an "svn status").
  
• Upgrading is as easy as doing an "svn switch" or an "svn update".
  

What do you think? Silly idea, or good solution?

Thanks a lot for your thoughts on this!

[Tim_S]

Are you distributing source code or executables?

[jcarmody]

VIPM is made for this.

[asbo]

The largest problem with using SVN is if you commit partially-working code (and you should, versioning isn't just for major, functional versions of code) and someone pulls the partially-working code thinking that it's an update. I suppose you could make the trunk your release version and then always branch to do development.

[jgcode]

Are you distributing source code or executables?

From what I understand of the OP, they are distributing an application as a bunch of VIs.

If that is the case I recommended building an exe using the LabVIEW Builder (assuming you have a llicense for it), and read back that versioning for your About screen. That way no one can edit your code. Also the code could check a central DB for the latest version and notify the user etc...

If are distributing code then VIPM would be the best - and the community version is free!

[xavier30]

Hi Onno,

I found myself in the same situation as you a while back. (this was before the VIPM came out, if it had existed at the time we probably would have used it)

We (a team of researchers within a larger organization) where and are doing dedicated, institute specific modules in labview for integrating NI equipment in an existing infrastructure.

This has after several projects and years of development turned into a framework that we now (try to ) maintain (and update).

To find a easy way of distributing this framework to all the "users" we also thought that svn could get the job done in an easy way even dough it is kind of slow and has strange behavior from time to time (i guess the perks we get are more user related than by the versioning system itself)

What we did was the following:

First for the framework developers we agreed on an naming syntax and structure on the repository (using the typical "trunk", "tags" and "branch") where everything stored in the "tags" folder/repository should be considered a release and never modified. If one finds a bug, the tag is copied into branch, fixed, and then re-published as a bug/minor fix (plus the fix if applicable is fixed in the trunk). Sometimes we fix things directly in the trunk but it might have advanced beyond the released version that needs fixing so branching is necessary. The tags released libraries/tools would have the folder structure, menu files and resources needed for direct integration into the labview file/folder structure. - This "solves" the problem of someone committing partially working code.

Second we created a simple svn installer (at first by simply using the system exec in labview to execute svn commands in the terminal but later by making a light weight svn client in c++/c and using the labview external code node) that would do a svn export and not a svn checkout. We did this to avoid users having all the .svn synch stuff in their labview development environment (remember these guys are researchers, meaning if they can they will fiddle with absolutely everything and of course break it at some point ).

The Installer stores all needed info in a config file with the modules in user.lib or wherever it is installed. In this file we keep track of where/when/what is installed, versions, vis/files etc. and every time the users launch labview we run the installer in background doing a quick check if there are any new versions of the modules they have installed and if so give them a prompt to install the latest release. if the file is deleted we do a "recursive list folder" in the most common install locations and if we find some "naming conflicts" we try to sort these out (we try to use .lvlib and .llb "containers" to avoid this but it still doesn't work 100% )

The installer checks various things and tries to cope with it (local modifications, custom installation location, changes in palette .mnu file etc) and there are surely many perks to sort out but in general i would say it does what we expect it to do and ease the job of distributing the framework.

- I guess what i am trying to say here is that in essence a simple installer can easily be realized using svn, and if the environment you are targeting is well known you might pull it off with just a few hours of work. On the other hand if you have several users and have to cope with all the unforeseen/unique things in their environments, you might be better off selecting a "off the shelve" installer (like the VIPM) that has all ready solved many of these problems (dough i do find fiddling with these problems fun from time to time )

- X

[Ton Plomp]

If you are going for using a version control system (VCS) as a deployment server I would advise Mercurial or Git (distributed VCS or DVCS).

You would setup a deployment server that only the author of the tool has write access.

The author would setup a development server for his hour-to-hour checkins/commits (Save early, save often).

When a release is ready you'd push the changes to the deployment server.*

On the end user's side the toolkit is installed with 'hg clone http://mercurial-deployment-rep'.

Periodically, you can use 'hg incoming', if there are incoming changesets you'd use 'hg pull -u' so that the working copy (visible files) are updated.

If you setup the deployment server as a HTTP-server you can also allow the server to have a zip/tar as a single download of the current state of the project (so the client doesn't need Mercurial installed).

Now for user engagement a DVCS is particular practical, the end user would pick up bugs and improvements and save them. If they want them to be fixed on the deployment server, they would commit them to there local repo, and publish them somehow (via http, file) to the developer. The developer would look at the changes and accept them (or not). That is a superb way to have your users committed. Such services are provided by commercial hosters like Bitbucket of Github)

Ton

*If you don't want your users to know the history between the releases you could setup the rebase extension, that pushes the different changesets as a singel changeset.

[Onno]

Thanks a lot for all your replies!

Are you distributing source code or executables?

I'm currently distributing the application as source code, since that seems to be the easiest. I spent a lot of time getting the executable version to work, but kept hitting snags here and there (especially with the application's LVOOP-based plugin system; but that would be food for a different forum topic). Unless I'm overlooking major advantages of an executable, I'll probably go with a source distribution for now.

VIPM is made for this.

Hm, I considered that. I'll have another look at it, thanks!

(Side note: it might also help me with properly integrating my application into the LabVIEW directory structure — so far, I've avoided dealing with all the 'user.lib' folders, pallettes, etc., and just put everything in one .lvproj and a clear directory structure).

The largest problem with using SVN is if you commit partially-working code (and you should, versioning isn't just for major, functional versions of code) and someone pulls the partially-working code thinking that it's an update. I suppose you could make the trunk your release version and then always branch to do development.

Ah, I'm afraid I wasn't clear. People only ever use "tagged" versions/releasese, which are guaranteed to be working; they should never check out the trunk, for exactly the reason you mention.

If you are going for using a version control system (VCS) as a deployment server I would advise Mercurial or Git (distributed VCS or DVCS).

That's a great idea, Ton. I'd love to switch to a DVCS, but that'd also mean re-building an infrastructure I have already invested quite some time in. But I should try this in the near future! You seem to have good experiences with it?

I found myself in the same situation as you a while back...

Thanks so much, Xavier, for sharing your experiences. Given the relative homogeneity of the environments I'm targeting, I might actually end up choosing a route similar to yours. But I'll first give the VIPM a try, see if that's a better option for me now.

[Rob Calhoun]

We use subversion for both source code control and storing build products. (The cool kids are all using git or mercurial these days, but subversion is better in my opinion for binary files like LabView.) As long as subversion is using FSFS (the default) it works fine for multii-GB repositories with tens of thousands of revisions.

Set "separate object code" flag on all VIs, and change the environment to make new VIs follow this. (It is annoying that LabView doesn't provide an easy way to do this; there are some workarounds posted on lava.) Otherwise, you run into "conflicts" that are really just recompilations.

There are those who feel committing build products to a source code system is an abuse. To me subversion is just a distributed file system that provides a traceable history of what was put in when. It works very well for this. Putting builds in a different repository is one answer to complaints about repo bloat.

-Rob

[hooovahh]

There are those who feel committing build products to a source code system is an abuse. To me subversion is just a distributed file system that provides a traceable history of what was put in when. It works very well for this.

-Rob

I agree but it can be slippery slope. Why stop at a built EXE? Should you commit the installer for your application?

What about if the installer contains all the NI products that your application uses? Then every build and commit will potentially add GB's of data to the repository, and pulling that from a remote location will be very time consuming.

Lets say you agree to commit the installer with all the tools, what about making a hard drive image with the system on it, maybe sysprep'ed for deploying to duplicate machines. Now we're talking about 10's of GB's of data every build and commit. And again someone may just do an update to get the latest and then unexpectedly be waiting for several hours while source, EXE, Installer, Tools, and hard drive images are downloaded.

I generally stop after the installer (without the NI tools), but there have been instances where it made sense to commit the installer with all the NI tools.