What unit test framework for LabVIEW do you use?

[Omar Mussa]

Just wondering what kind of unit testing frameworks LAVA members are using.

[Eugen Graf]

I know what JKI VI Tester is, but I never use that. Why I should if there is the LabVIEW Probe and Highlighting Tools? May be I didn't good understand the idea, but I tryed it, in really.

I read the documentation/videotutorials and tryed the templates and examples. I don't know why I should use that.

Would you explain it better? What are the really advantages of it?

Thank You, eg

P.S. the really problem what I have is to extract a testing unit from a whole project for testing. I use many hardware and parallel loops with QSM, so...

[Omar Mussa]

Would you explain it better?

Here's a reference on what unit testing is all about:

http://en.wikipedia.org/wiki/Unit_testing

My summary is this: unit testing is a software engineering process for validating that your code functions as expected. You basically write tests to validate that your code works as expected, if your tests fail then you update your tests or your code until they pass. If all of your unit tests pass, then you have a level of assurance that your code works as you expect (as defined by the unit tests).

What is useful about unit testing is that if you need to change your code (say, to make a sequence into a state machine), and you have created unit tests, you can validate that none of your changes have caused you to 'break' your previous logic simply by re-running your unit tests.

P.S. the really problem what I have is to extract a testing unit from a whole project for testing. I use many hardware and parallel loops with QSM, so...

In order to make you application logic more easily testable from your hardware interfaces, you may need to consider how you will test your code when you design your code. For example, you may want to create a simulation mode or abstraction layer that lets you test your application logic with unit tests, independently from your hardware interface code.

[Daklu]

Just wondering what kind of unit testing frameworks LAVA members are using.

I've used VI Tester for various side projects--mostly to learn how to use it and what it can do. I haven't applied it in a structured way to any real world problems though.

Why I should if there is the LabVIEW Probe and Highlighting Tools?

Because the Probe and Highlighter are very slow debugging tools with large complex applications. If you have a good suite of unit tests it will help you quickly find bugs.

[dannyt]

I know what JKI VI Tester is, but I never use that. Why I should if there is the LabVIEW Probe and Highlighting Tools? May be I didn't good understand the idea, but I tryed it, in really.

Would you explain it better? What are the really advantages of it?

There is a whole world of difference between the LabVIEW Probe and Highlight Tools and Unit Testing. The former are aids to help you find and debug problems, the later is the basis of a whole software methodology.

I have not yet used and Unit Testing in LabVIEW but I have see extensive benefit's of its use to other projects.

One key benefit is Regression Testing, how do you check that what you change today does not break something that was working yesterday.

I once worked on a very large C/C++ project (100+ developers) that was was run under the Agile Software Methodology. Before any code was written the Unit Test Software for that code had to be written. When a developer finished some code and checked it into their source control system we used Curise Control to automatically do a build and run the full test suite on the software, if any tests failed automatic emails were fired off. With this system we know within an hour that something that used to work had been broken, and it was easy to find out what changes cause that break.

So as I said at the start there is a whole world of differnce, between these tools.

Dannyt

[crelf]

Why I should if there is the LabVIEW Probe and Highlighting Tools?

Because that's not unit testing, and you can't get traceable results from that style of adhoc testing. What you've described is debugging, not unit testing.

[Black Pearl]

I was experimenting with Unit Tests without any framework yet, just to understand how it works. I did it on simple vi's from my reusable library, so it is not giving the advantages of 'regression test' (which is a really good point to get such a framework). So far it just gives me a documented behaviour (even if it doesn't work as I want it under every situation, these exceptions are documented, very valuable).

I would like to see some designs/implementations that allow for hardware simulation for automated Unit Testing. Appart from the hardware issue, I wonder how to deal with UIs.

Also, I would like to know more about Test Driven Developement specific for LV.

Maybe we should have a forum specific to software engineering here on LAVA? If we had more debate on such issues (and I see that need for it), we will go more for these tools or come up with other solutions.

Felix

[Daklu]

I haven't applied [VI Tester] in a structured way to any real world problems though.

Omar, one of the reasons I haven't pushed VI Tester more internally is because it's unclear what JKI's future plans are with it. The way I read the license it appears that JKI can revoke permission to use the tool at any time and instead sell it as a commercial package. I certainly wouldn't fault JKI for doing this. VI Tester is very good software and JKI certainly deserves to be compensated for your work. However, building and maintaining a suite of unit tests is very time-intensive and I'm hesisant to invest that much time in a tool that might be yanked out from under me.

Can you share JKI's future plans for VI Tester?

I would like to see some designs/implementations that allow for hardware simulation for automated Unit Testing.

There are a couple ways I've dealt with testing modules that interface with hardware:

1. Do write/read combinations--send a command to the hardware and read back a value to confirm the command was received. Technically this violates the unit test paradigm as it requires the hardware to be connected during testing. It also doesn't work for all commands, such as commanding an LED to blink.
2. Use a dialog box to ask the user if the command worked. Obviously this also violates the unit testing paradigm. Not only does it require the hardware be connected, but it requires a user to be present. Still, it's better than no testing at all and doesn't need to be run every time.
3. Create dummy objects that represent the hardware. This is the preferred way of doing it. It does require a lot more time to implement though.

Maybe we should have a forum specific to software engineering here on LAVA? If we had more debate on such issues (and I see that need for it), we will go more for these tools or come up with other solutions.

I fully agree more discussion on software engineering is needed. I think that is too broad of a topic for a single forum though. SE includes Application Design and Architecture, OOP, User Interfaces, Source Code Control, and Code Distribution, just to name a few. Each of those topics rightly needs its own forum. Maybe if tags were used more it would be easier to find threads discussing SE principles?

[Omar Mussa]

I would like to see some designs/implementations that allow for hardware simulation for automated Unit Testing.

Part of test driven development will always come back to design. One of the key benefits of unit testing in general is that it forces you to think of your code in terms of APIs. Your unit tests are actually testing the API to your internal code. If you can't test your code without hardware, you don't have a good API to your application logic code. There are different ways to enable simulation: Adding a simulation input (or property), using LVOOP and inheritance to create a simulation implementation of an abstract class, etc. Creating unit tests forces you to think about your code's interface at an early stage (or even at a later stage when you may need to refactor code and want to validate that it isn't broken).

One thing that I think works particularly well in VI Tester is the ability to use the test harness (VIs that are called before and after each test) to do things like set a simulation flag so that your tests run your VIs in simulated mode.

Appart from the hardware issue, I wonder how to deal with UIs.

At JKI, we currently are able to do UI testing using VI Tester and are running these tests to validate certain VIPM features. Adding UI testing capabilities to VI Tester is on our roadmap.

I was experimenting with Unit Tests without any framework yet, just to understand how it works.

I really recommend that you investigate whether VI Tester would meet your needs, trying to create unit tests without a framework is a fairly arduous task and will ultimately cost you more time than investigating tools that already exist. You may as well leverage the years of effort that have gone into the existing tools rather than start from ground zero. Besides that, VI Tester is free, so there really isn't a barrier to entry

[Black Pearl]

Besides that, VI Tester is free, so there really isn't a barrier to entry

The barrier is my LV version.

But I'm trying to get an upgrade and even some extra tools.

Felix

[Jim Kring]

The way I read the license it appears that JKI can revoke permission to use the tool at any time and instead sell it as a commercial package. I certainly wouldn't fault JKI for doing this. VI Tester is very good software and JKI certainly deserves to be compensated for your work. However, building and maintaining a suite of unit tests is very time-intensive and I'm hesisant to invest that much time in a tool that might be yanked out from under me.

Can you share JKI's future plans for VI Tester?

Sure! Our plans for VI Tester (and other JKI Labs products) are to continue to improve the product and make it freely available, but with no promise to actively support and market it with the same rigor that we would a full-fledged JKI Software product. It is not our intention to take away your ability to freely use the JKI Labs version to assist in your development. You can rest assured that JKI believes strongly in the philosophy of "do no evil"

[Daklu]

Thanks Jim. I didn't figure JKI would pull the license but it's nice to have reassurances.

[jgcode]

At JKI, we currently are able to do UI testing using VI Tester and are running these tests to validate certain VIPM features. Adding UI testing capabilities to VI Tester is on our roadmap.

Hi Omar

This sounds extremely interesting! Are you able to elborate, give a sneek peek or show us a demo vid (jing)?

Cheers

JG

[Omar Mussa]

This sounds extremely interesting! Are you able to elborate, give a sneek peek or show us a demo vid (jing)?

Unfortunately, we don't have anything to show yet because mostly we have design patterns that still need to evolve a little before they can become 'tools'. Generally, what we have done is utilized Windows API calls for keyboard and mouse emulation and used control references to validate the properties of UIs that we care about.

[jgcode]

Unfortunately, we don't have anything to show yet because mostly we have design patterns that still need to evolve a little before they can become 'tools'. Generally, what we have done is utilized Windows API calls for keyboard and mouse emulation and used control references to validate the properties of UIs that we care about.

Very cool!

Thanks for replying

Edited July 26, 2009 by jgcode

[ASTDan]

I was wondering since NI has the unit test framework how does JKI's VI tester fit in? Are you planning to compete with it, or does your tool have some fundamental differences?

Could you guys comment on that.

Thanks

Dan