Unit Test Frameworks: NI vs JKI?

[Daklu]

I'm looking for information on the differences between JKI's and NI's unit test frameworks. What advantages does NI's UTF offer over JKI's? What disadvantages?

The BIG difference is one is free and the other costs $1500. I'm looking for other, not so obvious differences. I've used JKI's UTF enough to be comfortable with it; I've not used NI's UTF.

Edited September 30, 2009 by Daklu

[Black Pearl]

I did compare both by webcast only. I would use the JKI one, not only due to the price.

Using the NI UTF gives you support and a 'ripe' product. But it's not counting that much as JKI will also give you support thru LAVA and as it is based on xUnit, the core is rock solid. And work will continue...

Why I prefer the JKI one is actually that it seems more light-weight. If I do test driven developement, I should (my opinion) code/design for easy testing (which theoretically will make 'good' code). And I want little effort to define the tests.

Turning the coin, NI UTF seems to give you more features for comparison of actual and required output, as well as to check code coverage.

Felix

[crelf]

IMHO the JKI product would be useful for test driven development, whereas the NI Unit Test Framework extends that to include traceable artifacts.

[Black Pearl]

crelf: can you please explain the term 'tracable artifacts'. I don't get your message.

Felix

[Jim Kring]

Here's my heavily-biased weigh-in (disclaimer: I'm on the team that created VI Tester):

I haven't used NI's UTF (but I've seen it demo'ed). It seems to have great features for knowing whether you are testing 100% of your code and need reports to prove it.

Of course, I have used JKI's framework, which is great if you're interested in using a proven software engineering architecture (xUnit) for software testing that's implemented in LabVIEW, for LabVIEW developers, by LabVIEW experts who use it themselves to write and test commercial software products written in LabVIEW. With VI Tester, writing unit tests is fun, since all your tests are written in LabVIEW. This also means that you can reuse code within your tests, and employ object-oriented techniques in the design and implementation of your tests.

JKI uses VI Tester for testing all the software it writes and we're going to keep making improvements over time, including possibly integrating it with other products like VIPM's package builder (JKI currently uses VI Tester to automatically run unit tests on all our VI Packages [software reuse libraries] during the build process).

Update: I'll also add that there is a wealth of information (book, websites, etc.) on xUnit, including: test architectures, design patterns, best practices, tutorials, etc. This means that there's a wealth of training materials that apply almost directly to how to use VI Tester.

Thanks,

-Jim

[jgcode]

I'm looking for information on the differences between JKI's and NI's unit test frameworks. What advantages does NI's UTF offer over JKI's? What disadvantages?

The BIG difference is one is free and the other costs $1500. I'm looking for other, not so obvious differences. I've used JKI's UTF enough to be comfortable with it; I've not used NI's UTF.

This is a great topic and I would love to hear everyones experiences with both toolkits.

We get the UTF for free so price isn't an issue. I have been playing with the UTF a lot lately as well.

Is someone from NI able to comment too?

Keep this thread going...

Edited October 1, 2009 by jgcode

[dannyt]

With VI Tester, writing unit tests is fun,

Jim,

I sometimes wonder if you should get out a little more

Just to tie in a previous thread covering this topic.

Edited October 1, 2009 by dannyt

[Daklu]

Thanks for all the feedback. It sounds like NI's UTF is geared more towards regulated industries where traceability is required. I'll recommend we adopt JKI's UTF and spend the money elsewhere.

[Jim Kring]

I sometimes wonder if you should get out a little more

I thought that everyone uses LabVIEW because it's fun

Thanks for all the feedback. It sounds like NI's UTF is geared more towards regulated industries where traceability is required. I'll recommend we adopt JKI's UTF and spend the money elsewhere.

Saving your money to spend elsewhere is a great idea (BTW, VIPM Enterprise is going to be released very soon )

Cheers,

[crelf]

It sounds like NI's UTF is geared more towards regulated industries where traceability is required.

Well, yes and no - the unit test framework paradigm is a little different to the xUnit paradigm, but the NI toolkit can help with both test-driven development and it can produce regulated industry artifacts. Also, the NI toolkit doesn't require you to write code (except for setup and teardown, which, essentially, almost every test needs).

I'll recommend we adopt JKI's UTF and spend the money elsewhere.

I don't know your situation fully, but that sounds wise.

[Dan DeFriese]

Maybe a little off topic. Does anybody just use TestStand for there unit testing? If so, do you have any recommendation or caveats.

[Omar Mussa]

Maybe a little off topic. Does anybody just use TestStand for there unit testing? If so, do you have any recommendation or caveats.

Ben Hysell has written some awesome articles related to unit testing and LabVIEW. One of the articles talks about using TestStand for unit testing:

http://www.viewpointusa.com/newsletter/2009_january/newsletter_2009_jan2.php

He also wrote some great articles about using JKI VI Tester in the same article series here:

http://www.viewpointusa.com/newsletter/2009_february/newsletter_2009_febtdd.php

http://www.viewpointusa.com/newsletter/2009%20March/newsletter_2009_Mar.php

I highly recommend these articles!

[crelf]

Maybe a little off topic. Does anybody just use TestStand for there unit testing?

We used to, but binned that when the NI product came along.

[Dan DeFriese]

Ben Hysell has written some awesome articles related to unit testing and LabVIEW. One of the articles talks about using TestStand for unit testing:

http://www.viewpoint...r_2009_jan2.php

He also wrote some great articles about using JKI VI Tester in the same article series here:

http://www.viewpoint...2009_febtdd.php

http://www.viewpoint...er_2009_Mar.php

I highly recommend these articles!

Excellent, thank you for these links!

[crelf]

A great example of how JKI's unit test framework is excellent for test-driven development - you can compress/group common/like tests. Not appropriate for regulatory industries artifacts under these circumstances, but excellent for on-the-fly testing while you're developing.