Re-use and hierarchy strategy

[InsaneObject]

DropBox, Perforce, and Re-use

Anyone feeling like speaking about the best way to develop re-use code? I'm taking some time to do this now that I have a bit of time to develop some code for a project that is has a concept that is begging for re-use. I'm about to develop it, it will be quite a bit more work for me to develop it for re-use and visibility this time, but won't much more in the future if I get used to the process. In particular, in this case it is for massaging data before it goes into a graph, and keeping a circular data buffer. You know, the really frequent cases where neither the graph nor the chart quite solves the problem. Do I want to display a particular channel, do I want to do a moving average on it, do I want to do some limit testing?

So, I'm not asking about how to code that, but rather how you go about naming folders, where you put them, how you go about checking them into source code control.

I have been using DropBox recently, and it's friggin great. Simple, maps right to Windows folders, as well as my phone (which I haven't found a use for yet), but I don't know if it really helps my work.

Code duplication is something I really really hate, two copies of something that are similar but not the same. I looked at the DBox support page, but they mentioned something awful, some type of deep Windows NTFS folder linking done through the lowest interface. I'm not going to do that.

What I'm really looking for is a single folder, let's say 8.6 is my foundation, and where I'll develop re-use code. I want a single folder where I can upload it to my SCC, use it in whatever later version of LV I'm using, and make sure it's available in the source code of my latest project.

I know you've run into this before. Strategies you use?

I appreciate the replies in advance

[MikaelH]

This is the way I share common libraries and tools (e.g. RCF plugins, GOOP Development Suite).

I basically have the whole LabVIEW xxxx folder under version control.

I don’t check in everything under the LabVIEW folder, only the files and folders I like to make sure everybody has the same version of.

This screen shot shows how the repository for LabVIEW 8.6 looks like.

We then of cause have a dedicated repository for every project specific application, so when I build an executable I make a tag of the Project Specific Repository and ALSO the LabVIEW xxx repository.

That way I can always reproduce my code when needed.

[jdunham]

Well I think you've figured out that you want to keep your library code in a separate folder and keep it under managed development with the help of a source code control system. I use Subversion, but I don't think the choice of system matters all that much.

Where the real problem arises is making sure that improvements to your library don't break your old projects. Even if you think it won't happen, it certainly will, and Murphy's law dictates that it will happen in some dangerous or at least embarrassing way.

So what is important is a workflow where you link your active projects to your main library "trunk" code, and then when your projects are feature-complete and headed into a predominantly testing phase, you freeze a version the library code used in that project into its own branch.

Then if you do find a problem in the frozen branch and you fix it there, you have to remember to merge it back to the trunk and any other project which might be vulnerable, and don't forget that merging really sucks in any language and is truly awful with LabVIEW. Or else you have to have the self-control to fix it in the library and push new versions of just those libraries you fixed into the otherwise frozen code. If you don't have granularity in your library versioning, then you're back into the risk of fixing one thing and breaking two more.

Of course as you push a fix into your frozen code, you should check the check-in logs and make sure you understand all the other changes you've made to the library's trunk and decide whether the other changes are putting your code at risk. Obviously the amount of effort you put into all of this is dependent on the risk tolerance of your project, but you are better off enforcing good discipline in your workflow before you start programming the reactor core control rods than after.

If you're unlucky enough to be a freelance consultant, you'll probably end up having active projects stuck in different versions of LabVIEW, which will just make the problem worse. Either you stick with developing library code in the oldest extant version (which is a bummer) or else you get even more careful about handling your changes.

So I think VIPM (google it!) is supposed to help quite a bit with this, and I'm sure the JKI guys will pipe up with more insight. Sheepishly I admit that I don't yet use it for my own reusable code. However I would say that using any tool is not as important as understanding the basic problems around versions and reliability and backwards compatibility and unintended consequences. The point is acknowledging that copies of your library code will proliferate and managing that, rather than pretending it won't happen because you're such an awesome coder and your designs come out so well the first time.

Jason

[InsaneObject]

Sorry, just trying to not write too much. I'm well aware of, and use, the OpenG tools, and use the JKI packager to get them, for each LV version.

Your replies were useful, though, thanks.

Well I think you've figured out that you want to keep your library code in a separate folder and keep it under managed development with the help of a source code control system. I use Subversion, but I don't think the choice of system matters all that much.

Where the real problem arises is making sure that improvements to your library don't break your old projects. Even if you think it won't happen, it certainly will, and Murphy's law dictates that it will happen in some dangerous or at least embarrassing way.

So what is important is a workflow where you link your active projects to your main library "trunk" code, and then when your projects are feature-complete and headed into a predominantly testing phase, you freeze a version the library code used in that project into its own branch.

Then if you do find a problem in the frozen branch and you fix it there, you have to remember to merge it back to the trunk and any other project which might be vulnerable, and don't forget that merging really sucks in any language and is truly awful with LabVIEW. Or else you have to have the self-control to fix it in the library and push new versions of just those libraries you fixed into the otherwise frozen code. If you don't have granularity in your library versioning, then you're back into the risk of fixing one thing and breaking two more.

Of course as you push a fix into your frozen code, you should check the check-in logs and make sure you understand all the other changes you've made to the library's trunk and decide whether the other changes are putting your code at risk. Obviously the amount of effort you put into all of this is dependent on the risk tolerance of your project, but you are better off enforcing good discipline in your workflow before you start programming the reactor core control rods than after.

If you're unlucky enough to be a freelance consultant, you'll probably end up having active projects stuck in different versions of LabVIEW, which will just make the problem worse. Either you stick with developing library code in the oldest extant version (which is a bummer) or else you get even more careful about handling your changes.

So I think VIPM (google it!) is supposed to help quite a bit with this, and I'm sure the JKI guys will pipe up with more insight. Sheepishly I admit that I don't yet use it for my own reusable code. However I would say that using any tool is not as important as understanding the basic problems around versions and reliability and backwards compatibility and unintended consequences. The point is acknowledging that copies of your library code will proliferate and managing that, rather than pretending it won't happen because you're such an awesome coder and your designs come out so well the first time.

Jason

PS What is wrong with being a freelance consultant? I don't mind negative feedback, but curious. If you're really good, it's not such a bad thing.

[jdunham]

Sorry, just trying to not write too much.

Whoops, I screwed that up as well!

I'm well aware of, and use, the OpenG tools, and use the JKI packager to get them, for each LV version.

Right, but you should at least consider using the packager to distribute your own tools between your projects, or be ready to come up with an equivalent workflow.

PS What is wrong with being a freelance consultant? I don't mind negative feedback, but curious. If you're really good, it's not such a bad thing.

It's OK, I'm just being snarky. Have fun!

[jgcode]

Hi InsameObject

I use Perforce, VIPM and OGPB to manage reuse.

For me, I don't want to be branching & merging (yuk!), the dist code is the snapshot (as a package format).

If I ever want to install an older version (package) or upgrade then VIPM lets me do that quickly and easily.

VIPC files are the secret key here, they allow a dev to manage reuse code versions between projects.

You should definitely check it out more IMHO.

Cheers

-JG

[Daklu]

I'll second what Jason and JG said. Package your reusable code in VIPM packages and distribute it to yourself. You'll save yourself headaches in the future.

A couple other thoughts...

-I put all my reusable code in lvlibs. It makes the whole process much easier.

-Keep the public api simple. Avoid the urge to include every feature someone might want to use. That makes the api harder to understand and use. Instead, try to give users the ability to create the functionality they might need, within the scope of what the library addresses. It's much easier to add to an api than it is to subtract from it.

-Figure out how you're going to manage changes that break backwards compatibility. (Yes, the golden rule is "never break backwards compatibility," but that rule ignores other real world issues.) If you give the breaking changes a new namespace (for instance, renaming the containing lvlib as v2) then upgrading an app from v1 to v2 is kind of a pain, but you have the advantage of being able to have both versions installed at the same time. If you don't give it a new namespace you have to choose one or the other and run the risk of accidentally editing a project that is supposed to use v1 when you actually have v2 installed in your dev environment. Each has it's advantages, the choice is yours.

-Once you release a library you're committed to supporting it. I give my reuse libraries "trial runs" via copy and paste in a couple different projects before I create packages for it. This gives me an opportunity to see how well it works under slightly different user scenarios. I almost never get the api right the first time and the trial runs highlight problems with the initial api design and gives me the opportunity to improve it. Nothing is worse than issuing a release then discovering a use case you can't address without breaking backwards compatibility.

[Black Pearl]

Take a look at the last 2 posts in this thread:

http://lavag.org/topic/13492-reuse-packages-and-scc/page__st__20

The first (by me) is showing my reuse-process using OpenG + VIPM community and the second from Justin Goeres (JKI) shows the same strategy with VIPM.

I think this could be a good starting point to modell your process with the tools of your choice.

Felix

[dannyt]

This is the way I share common libraries and tools (e.g. RCF plugins, GOOP Development Suite).

I basically have the whole LabVIEW xxxx folder under version control.

I don’t check in everything under the LabVIEW folder, only the files and folders I like to make sure everybody has the same version of.

This screen shot shows how the repository for LabVIEW 8.6 looks like.

We then of cause have a dedicated repository for every project specific application, so when I build an executable I make a tag of the Project Specific Repository and ALSO the LabVIEW xxx repository.

That way I can always reproduce my code when needed.

Hi MikaelH

I know I am a bit behind with this thread, I just want to ask you a question about how your solution works, in terms of the LabVIEW xxx folders in your repository's.

Do you use the LabVIEW options paths to then run LabVIEW from within your repository on the network or do you locally map your repository ?

cheers

Dannyt

[MikaelH]

Hi Danny

I have the repository in our version control server, and every developer runs LabVIEW locally in the normal place. The benefit I find with this is that when someone adds or updates an instrument driver (instr.lib) , I just quickly do an update on my LabVIEW XXX folder to get the latest version.

Cheers,

Mike