Unit Testing - Preferences?

[ak_nz]

I'm looking at recommending a Unit Testing tool at our workplace for LabVIEW code. The majority of the code will be OO-based with relatively simple APIs. There are appear to be three options:

• UTF from NI. This has the integration into the IDE, test vectors, code coverage. We also have a license to use it. Unfortunately it also seems quite slow (second or more per test case). My main concern is that the tests will take too long to run under our larger projects (we will have hundreds or more cases), meaning they won't be run frequently. This goes against the common thought that unit tests should be quick to execute.
• VI Tester from JKI. This custom framework does support management of cases and suites but also requires test customisation and does not provide any mechanism for analysing coverage. It also appears to not be updated to support LabVIEW 2012 and 2013 and thus has a few gotchas (eg. incorrect template copying).
• Roll our own. Not the best solution with the obvious down-sides.
  

What are the community's thoughts on this? Are there more alternatives?

[hooovahh]

I really think given NI's support, it would make a lot of sense to stick with what they have available.  Rolling your own sounds like a big undertaking and while I really enjoy owning my own tools, the time and effort needed in building, deploying and supporting them just doesn't seem like a good idea.  Especially on a thing that involves testing code components.

 

Sorry to hear the NI solution is slow but because you have a license you should be able to get some support from NI to investigate this, or at least get some feedback from them.

 

JKI's timing for their VI Tester was unfortunate.  I felt bad for them when I heard the Unit Test Framework was coming out in what felt like a few months after the JKI one.  As a result I get the feeling JKI is focusing their attention on other tools that they have a competitive advantage with.

[Jim Kring]

Hey Guys,

I just want to mention that JKI is committed to making sure that VI Tester works well in LabVIEW. We can't live without it at JKI and it's going to continue to be improved.  We've been really busy lately, so that's why there hasn't been a new version pushed out.

 

We see the LV UTF as addressing a totally different market need than VI Tester.  As I see it: UTF is great for people who want to show 100% test coverage through its static analysis tools -- NI created it to address the needs of people developing for regulated industries.  VI Tester is great for people who want to do better and faster software engineering in LabVIEW -- JKI created it to help test object-oriented (and other) LabVIEW applications using the industry-standard xunit architecture.

I can't comment on time-frames, but please stay tuned and don't give up on us.  BTW, VIPM 2013 SP1 (one of the areas where we've been busy) was just released (do a check for updates)!  We couldn't have done that without VI Tester.

[ak_nz]

Hey Guys,

I just want to mention that JKI is committed to making sure that VI Tester works well in LabVIEW. We can't live without it at JKI and it's going to continue to be improved.  We've been really busy lately, so that's why there hasn't been a new version pushed out.

 

We see the LV UTF as addressing a totally different market need than VI Tester.  As I see it: UTF is great for people who want to show 100% test coverage through its static analysis tools -- NI created it to address the needs of people developing for regulated industries.  VI Tester is great for people who want to do better and faster software engineering in LabVIEW -- JKI created it to help test object-oriented (and other) LabVIEW applications using the industry-standard xunit architecture.

I can't comment on time-frames, but please stay tuned and don't give up on us.  BTW, VIPM 2013 SP1 (one of the areas where we've been busy) was just released (do a check for updates)!  We couldn't have done that without VI Tester.

 

Thanks for responding to this! Personally I quite like the VI Tester tool as running unit tests quickly is a great advantage to me.

 

As some feedback I operate in a regulated industry (medical devices); however operating in a regulated industry does not mean that standard architecture and unit testing techniques (such as xUnit) are ignored. They are an obvious advantage and, perhaps compared to other industries, are virtually compulsory for our projects that are not LabVIEW based since they provide a proven mechanism for test infrastructure. And everyone wants to design and test software engineering better and faster - especially those of us in regulated industries .

 

However I concede that the reporting features and code coverage of the UTF is a distinct advantage and adds to our ability to prove sufficient test coverage. I can also understand that it is not desirable to tread on the UTF toes, so to speak.

 

I look forward to hearing more from VI Tester in the future.