SCC, binary LV files, granularity, specific tools

[ensegre]

Maybe a blurb of noob faqs, so apologies if I'm asking the obvious.

 

I have used different SCCs for various text-based projects in the past, like cvs, svn, git, and I'm familiar with the idea.

Now I decided to adopt git (namely hosting at gitlab.com this time, for convenience) for a big LV project which screamed for it since its beginning, and I'm tucking fine with that.

.

However, git itself is agnostic in treating all my vis, llbs and so on like binary files. That means to me that my repository grows in time at a quite fast rate, and that diffs are completely obscure.  I am not able to tell for instance if an llb changed because a single vi was recompiled or because of more extensive changes. That may ultimately constrain my strategy of committing, for example avoiding llbs at all in favor of subdirectories in order to gain in granularity, or reducing the frequency of updates to limit size blowup.

 

I'm also missing specific tools for, say diffing and blaming files in a LV meaningful way. This in contrast to text-based languages. In the present project my only text files are .lvproj, .lvlib, some xml configuration files and lyx documentation, not the core code, so that doesn't bring me very far. To compare two snapshots of the project I presume that I would have to create two copies of it on disk, rename some relevant VIs, load both copies in LV and compare them... not inspiring as a workflow. Is it all one can get, or am missing the obvious?

 

I mean, after all what is this Tools/Options/Source Control/Provider name thing, and why does it display "<None>" in the pulldown? Oh, help: "On non-Windows platforms, LabVIEW runs a query to determine if Perforce is installed". So what, I can do SCC only if I depend on a single commercial service provider? Can't believe...

[dannyt]

You have basically got it right, the huge leverage you get from using SCC's does not come when using LabVIEW and SCC's.... however that is not a reason or excuse NOT to use a SCC tool as there are still significant befits.

 

You just have to work in a different way when using LabVIEW, the same is true is you use a SCC for your word documents, image files etc.

 

The LabVIEW integration you get using the "integrated" Tools/Options/Source Control/Provider option is really nothing more than you can get using an SCC not supported and putting your own tools in place.

 

So you will want to sort out VI diff's and merging, they exist and can be done. For myself I have never really got them functioning to a really useful extent, but I have not spent a great deal of time playing with them.  I preferred to go down a route of decoupling peoples work loads to avoid there need.

 

I have used LabVIEW with ClearCase and now work with Mercurial.

 

If you look around there is a lot of people using LabVIEW with SubVersion, Mercurial (hg) and a growing number with git.

 

 

  for git  https://decibel.ni.com/content/groups/git-user-group/blog/2013/02/20/some-basic-git-info and https://decibel.ni.com/content/blogs/Matthew.Kelton/2011/09/30/labview-and-versionsource-code-control--introduction

 

   for subversion https://decibel.ni.com/content/groups/large-labview-application-development/blog/2010/03/29/using-subversion-svn-with-labview

[odoylerules]

So i use git as my version control and the thing you will find is that the issues you mention are inherent to all source control options within Labview.  It just isn't built very well for this sort of thing.  I would highly recommend continuing to use source control with it, but it does take some getting used to and can be a pain compared to text based languages.  

 

A couple of things i would recommend.  Make sure you check the options to "Separate Compiled Code" within labview.  This will save you a lot of headaches on commits due to basic recompiling of the source.  

 

Also, labview is able to compare and merge some of its binary files such as Vi's and controls using LVCompare and LVmerge.  However you have to set up your git tool or git itself to invoke these executables when performing a diff.

 

You are correct that Labview does not have a good method of comparing "container structures" such as classes, libraries and projects.  As far as the .lvclass and  project files go, you are correct there really isn't a great way to compare diffs on them.  One trick that can help in certain situations is that if you open these up in a text editor, most of the information is not in binary format and these "container" files are mostly pointers to the location of the files they contain.  So you can compare them at the text based level but it doesn't get you super far.  

 

As far as comparing snapshots of the entire project, you are correct that you would need to load a new copy on disk and open it up.  You are also correct that if your dependencies aren't structured correctly you will need to rename some items b/c both project will point to the same dependency.  One thing i do to try and get around this is make sure that all my project specific items are lower in the file structure hierarchy than my main project file.  Therefore when labview looks for the dependencies it always looks down the file structure tree of the project file location and not elsewhere.

This allows you to then check out the entire project to a separate folder on your hard drive and when you open it, there shouldn't be any project specific dependencies that overlap and you shouldn't have to rename any items.

 

Its a pain but its one way of doing it.  

[PaulL]

You can find out how to set up Git to work with LabVIEW's diff and merge tools here: https://lavag.org/topic/17934-configuring-git-to-work-with-lvcompare-and-lvmerge/#entry107740.

[ensegre]

Thanks everybody. Will dip into the threads suggested. Good to hear that I'm on the right track.

Edited January 31, 2015 by ensegre

[ensegre]

A couple of things i would recommend.  Make sure you check the options to "Separate Compiled Code" within labview.  This will save you a lot of headaches on commits due to basic recompiling of the source. 

which brings me to rush to do it at this point, like this:

but what should I do for *.lvclass (and other types which I possibly forgot)?

[JKSH]

The easiest way to mark everything (including LVClasses) is to load the LVProj file and go to Project -> Properties -> Project -> Mark Existing Items

[ensegre]

doh! you're right. [excluding Dependencies/vi.lib, I assume]

[odoylerules]

doh! you're right. [excluding Dependencies/vi.lib, I assume]

Bah, sorry i should have mentioned that...

 

Also, i don't know if you are in windows or not.  I really like using TortoiseGit for explorer integration for git.  A lot of the folks around here also use source tree.

Edited February 1, 2015 by odoylerules

[ThomasGutzler]

The easiest way to mark everything (including LVClasses) is to load the LVProj file and go to Project -> Properties -> Project -> Mark Existing Items

 

This doesn't always work for me - also, it takes forever for larger projects.

Here's my solution, which handles .vi, .ctl, .lvclass and .lvlib files:

RemoveCompiledCode.viLcArray_Join_STR.vi

I run it every now and then just to make sure I haven't missed anything in my projects.

Edited February 2, 2015 by ThomasGutzler

[JKSH]

This doesn't always work for me - also, it takes forever for larger projects.

What happens? Sounds like a bug that needs squashing.

 

 

also, it takes forever for larger projects.

Here's my solution, which handles .vi, .ctl, .lvclass and .lvlib files:

RemoveCompiledCode.viLcArray_Join_STR.vi

I run it every now and then just to make sure I haven't missed anything in my projects.

NI would do well to accept contributions from the community!

[ensegre]

Also, i don't know if you are in windows or not.  I really like using TortoiseGit for explorer integration for git.  A lot of the folks around here also use source tree.

I'm on ubuntu with qgit (my comments about integrating LVCompare). I have tortoise for mercurial and am not aware of plugins for git, maybe they exist but I haven't researched too much.

[ThomasGutzler]

What happens? Sounds like a bug that needs squashing.

 

I haven't tried removing compiled code via menu options for a while because I just use my own script now. From memory it just froze or didn't actually process all vis in the list.

Also, sometimes I don't want to remove compiled code from the entire project but only from the files in my projects folder (not the ones in user.lib etc). The dialog you get through the menu options doesn't support sorting by file name, which makes that rather painful.

[Rolf Kalbermatter]

However, git itself is agnostic in treating all my vis, llbs and so on like binary files. That means to me that my repository grows in time at a quite fast rate, and that diffs are completely obscure.  I am not able to tell for instance if an llb changed because a single vi was recompiled or because of more extensive changes. That may ultimately constrain my strategy of committing, for example avoiding llbs at all in favor of subdirectories in order to gain in granularity, or reducing the frequency of updates to limit size blowup.

 

One more advice: Don't use LLBs in your code development. It may be ok to do so for the final executable where you want to have certain libraries as container in an external file as some sort of plugin, but more likely you will want to use packed libraries for that anyways. The LLB fails badly for such solutions as soon as any of your dependent VIs are part of an lvlib or lvclass since LLBs can not contain these files so the VI inside an LLB is then broken as it misses its lvlib or lvclass file.

 

Learned this the hard way when upgrading an older application that used LLBs as plugin. Suddenly most of the plugins were broken. Turned out that it failed as soon as a plugin called an AAL function as these got lvlibed in newer versions.

[ensegre]

Note taken, thanks.

In defense of llbs I can say: they are fit for me to unclutter small scale projects (up to a few tens of vis, SCC secondary), which are to be deployed as main.vi + ancillaries packed in a single place, when the development environment is anyway present on target. That is our common use case in the academia. We have a blanket license and don't mind installing the extra copy, lifetime of software may be very short (three days till the student realizes that he wanted something else, or that the measurement has to be performed differently), need for maintenance and adjustments onsite are a constant, and instruction is often part of the game. And plain llbs don't need a project.

[Rolf Kalbermatter]

Note taken, thanks.

In defense of llbs I can say: they are fit for me to unclutter small scale projects (up to a few tens of vis, SCC secondary), which are to be deployed as main.vi + ancillaries packed in a single place, when the development environment is anyway present on target. That is our common use case in the academia. We have a blanket license and don't mind installing the extra copy, lifetime of software may be very short (three days till the student realizes that he wanted something else, or that the measurement has to be performed differently), need for maintenance and adjustments onsite are a constant, and instruction is often part of the game. And plain llbs don't need a project.

 

As I said, they may be working for your situation. But plugins tend to be a problem sooner or later. You can't usually just include your own VIs as the vi.lib directory is not present in an application. So you need the entire hierarchy but then you can't use any standard LabVIEW function that exists or uses a VI inside an lvlib or lvclass. With many of the LabVIEW VI libraries being turned into lvlibs recently, this is a serious problem.