LabVIEW vs. Measurement Studio for TestStand?

[SPM]

Hi, all,

Could some of the gurus here help me out? I need to advise a customer tomorrow about whether they should chose LabVIEW or Measurement Studio as an ADE for TestStand. The application will be engineering system test of a life-critical networked system, especially at large scales.

LabVIEW looks like a nice stable product, but I'm concerned about the level of difficulty of using it in this context. This may be my newness or something intrinsic. But the lack of something as basic as a Map VI or class is worrisome (as are the gyrations that the people who developed the prototype had to go through in a different thread.) Even something as basic as getting a VI to retain a value (memory) seems difficult.

For instance, configuration. The customer wants to load configuration info dynamically to keep it synchronized with their own hardware configurations. In the context of LabVIEW's seeming aversion to memory, does this mean that every test step in TestStand is going to have to reconfigure the system?

I'm not crazy about locking into .NET technology (having read the "Choosing the Right ADE" document,) but I just wonder if the dataflow paradigm is going to create a lot of trouble needlessly.

Any help appreciated.

Thanks,

- Steve.

[BrokenArrow]

SPM,

You are inferring that there is something intrinsically wrong with LabVIEW insofar as how it handles memory. If this is what you truly believe, then you've answered your own question.

You might need to explain your complaint more specifically in order to get help. For example, can you give an example where a VI had a hard time retaining a value? Also, how is TestStand going to tell LabVIEW to dump memory?

Richard

[Tim_S]

QUOTE (SPM @ Apr 27 2008, 01:40 PM)

Could some of the gurus here help me out? I need to advise a customer tomorrow about whether they should chose LabVIEW or Measurement Studio as an ADE for TestStand. The application will be engineering system test of a life-critical networked system, especially at large scales.

LabVIEW is a programming language; Measurement Studio is a group of controls. You would have to pick a programming language (C/C++, VB, etc.) to use Measurement Studio in to create a front end. This point may resolve your issue.

I've not used TestStand, but I have talked with a company who has created LabVIEW front ends for TestStand before; they did not seem to think it was difficult to do so and my little tests indicate that the two play nice together (though there is a learning curve there!). The dataflow paradigm is not an issue any more than the object-oriented nature of C++ is an issue compared to C. There are differences in thinking and programming and each has advantages and disadvantages; neither is better or worse.

LabVIEW has an advantage for what you're talking about in that it is (in my opinion) easier to develop complex programs in and it is a NI product so there is one place to go for support and the finger-pointing that can occur is minimized. That being said, I worked with a man about 15 years ago who could whip out a complex Fortran program with GUI in a half-hour that suited his needs; to learn a new programming language and convert his libraries over was cost and time prohibitive.

[SPM]

BrokenArrow,

Like I said, I'm a newbie to this stuff but have a long history with conventional OOP. And I am willing to learn. So here's an example of what I'm talking about.

The system under test (SUT), in the case I'm considering, is a network. Configuring the SUT involves at least enumerating the devices in a network and specifying how they are connected to digital I/Os. This could be hardwired, but the customer wants to use simulators as well as physical devices without changing the test scripts.

The configuration can be established using a Bridge pattern. (See my post in the LVOOP forum -- I've since hacked the stopwatch demo and verified that it can be done.)

Can the configuration be made persistent so that it doesn't disappear between TestStand test steps?

Thanks,

- Steve.

[BrokenArrow]

QUOTE (SPM @ Apr 28 2008, 10:49 AM)

Can the configuration be made persistent so that it doesn't disappear between TestStand test steps?

Certainly. Load the system parameters from a text file into the the VI's that need the info. If VI's share information dynamically, a file is a easy sell-off to agencies because the data is "always there". As TestStand calls the VI, the VI will load what it needs from the file. It only takes a few mS to read several hundered lines of configuration data. When each VI exits, it reliquishes its memory (or at least it should - close your references), but that's OK because we were reading the parameters on each call. Would this scenario work? How are you doing it now?

edit: .... also ask yourself.. do you really need TestStand? What is it doing in this case that LabVIEW couldn't do by itself?

[Michael Aivaliotis]

This is a great question.

---

QUOTE (SPM @ Apr 27 2008, 10:40 AM)

Could some of the gurus here help me out? I need to advise a customer tomorrow about whether they should chose LabVIEW or Measurement Studio as an ADE for TestStand.

QUOTE (SPM @ Apr 28 2008, 07:49 AM)

BrokenArrow,

Like I said, I'm a newbie to this stuff but have a long history with conventional OOP. And I am willing to learn.

---

I'm not trying to be condescending, but based on your content from these posts, I don't think you're in the position to advise the customer either way. It's apparent that you don't have working knowledge of LabVIEW or TestStand. This is not a bad thing in and of itself. However it can become deadly and very time consuming (re: spinning your wheels on customer time) trying to figure out how the pieces go together.

Having developed systems with the TestStand and LabVIEW combination, I would say that I'd rather be pulling my teeth out. Full disclosure, I'm a LabVIEW addict and don't program in any flavor of C. From that point of view, TestStand is just another text based "language". It seems like you are very good in the C world. If that's truly the case, and you will be supporting this customer, I suggest you stick with C since TestStand works well with any language.

On the other hand, if you do decide to forge ahead with LabVIEW and TestStand then be advised that you will curse and swear the day you made the decision. If, however, you make it through the fiery pits of hell, then you will be rewarded with a lot of knowledge and the confidence to do it again. You might even decide to start programming (heaven forbid) exclusively in LabVIEW and rid yourself of the shackles of TestStand and the other text like archaic languages.

Whatever you decide, make sure you stick with it until completion. Above all, the test system must be completed and be successful regardless of the good or bad choices we make at the start.

[BrokenArrow]

Michael,

Why do you refer to TestStand as a text-based language? I have months of mouse-induced carpal pain from "programming" in TestStand, and other than being in pull-down list hell, I don't recall "typing" a whole lot.

[crelf]

QUOTE (Michael_Aivaliotis @ Apr 28 2008, 02:24 PM)

Having developed systems with the TestStand and LabVIEW combination, I would say that I'd rather be pulling my teeth out... On the other hand, if you do decide to forge ahead with LabVIEW and TestStand then be advised that you will curse and swear the day you made the decision.

I dunno - I've never had an issue using TestStand. I figure something quite specific has bitten you on the arse Mike?

[jhoskins]

I use LabVIEW and teststand together all of the time. I keep it a simple (yeah right) concept though. I keep LV as the user interface to teststand and use TS as just a sequencing engine.

If you keep this in mind then they play very nice together. I have had a few problems in both LV and TS but it was nothing major and was fixed rather easily. I love the two of them together.

Keep LV what it is: a programming language (but in this case a user interface)

Keep TS what it is: A TEST ENGINE.

I would agree with Michael in the fact that you do have to know som C syntax to get the things that you want from teststand.

[Michael Aivaliotis]

I would trade a nice LabVIEW state machine for a TestStand sequence any day.

[jhoskins]

QUOTE (Michael_Aivaliotis @ Apr 28 2008, 04:12 PM)

I would trade a nice LabVIEW state machine for a TestStand sequence any day.

I would'nt. We have a lot of legacy dlls that LabVIEW just will not use correctly. I love the fact that I can code my tests in just about any language. I prefer LV but the other people around here prefer C or C++ and even .NET, using TS allows me to reuse code that they already have written.

Also the debug capabilities are great to. Try debugging a C dll in a LV statemachine! I am not knocking statemachines or anything else, I am just stating that for me and my situation the LV-TS combo works great and I would not trade it. Although sometime TS is an overkill.

[BrokenArrow]

QUOTE (Michael_Aivaliotis @ Apr 28 2008, 05:12 PM)

I would trade a nice LabVIEW state machine for a TestStand sequence any day.

Dido.

There have been many a test engineer manager that have been sold on the idea that you NEED TestStand in order to sequence your individual LabVIEW "tests". That, and TestStand can provide "reports". This is insipid. LabVIEW can do all of that, espaically with Toolkits. Why not just develop a LabVIEW interface that sequences the VI's? Some people think that without TestStand, you can't (for example) skip test 2 and 3, do test 13 twice, then go back and do test 1 again. Rubbish. Yes, TestStand has it's place, but it's not worth the effort JUST for sequencing already developed LabVIEW VI's.

QUOTE (jhoskins @ Apr 28 2008, 05:28 PM)

...... I love the fact that I can code my tests in just about any language...... ....

.....Try debugging a C dll in a

LV

statemachine! .....

Perfect reasons to use TS!

QUOTE (Tim_S @ Apr 28 2008, 10:30 AM)

LabVIEW is a programming language; Measurement Studio is a group of controls...........

This is true, but I think we can infer that he meant Visual Studio, and was using Measurement Studio for the controls?

QUOTE (jhoskins @ Apr 28 2008, 04:59 PM)

I would agree with Michael in the fact that you do have to know som C syntax to get the things that you want from teststand.

I still don't buy this, simply because if I can use it (with my limited C skills), anybody can! :laugh:

[Rolf Kalbermatter]

QUOTE (jhoskins @ Apr 28 2008, 04:28 PM)

I would'nt. We have a lot of legacy dlls that LabVIEW just will not use correctly. I love the fact that I can code my tests in just about any language. I prefer LV but the other people around here prefer C or C++ and even .NET, using TS allows me to reuse code that they already have written.

Also the debug capabilities are great to. Try debugging a C dll in a LV statemachine! I am not knocking statemachines or anything else, I am just stating that for me and my situation the LV-TS combo works great and I would not trade it. Although sometime TS is an overkill.

I would say that there is very little about DLLs that LabVIEW can't deal with and nothing which couldn't be solved with some wrapper DLL sometimes. That said you do want to understand some serious C coding if you are going to interface to complex DLL situations.

As to debugging DLLs: Of course that is not done in a LabVIEW state machine. You do not debug DLLs in real applications but in test frameworks. In my case that are sometimes (not very often) simple C programs, but mostly simple LabVIEW programs that call a number of VIs that interface to the DLL in the necessary order and with the right parameters. Once you got that right and the DLL interfacing (and the DLL itself of course) seems to work you can go and integrate those neatly programmed VIs that you created to interface to the DLL(s) into your real applications. This way you seldom have to revisit the interface code and if you do get across something strange during application development, try to reproduce these things in your DLL test framework and fix and test it there. If done right your test framework will grow and with a very simple framework manager you can simply run every single test of it after each modification to the DLL or it's interface VIs, to make reasonably sure you did not introduce a regression.

Rolf Kalbermatter

[jhoskins]

Rolf makes some great points, but I still prefer teststand. I have written dlls that Labview just will not interface with (even with a wrapper) it just does not see the inputs or sometimes will not produce an output. I have sent the dlls to NI and they cannot figure out why they do not work. I have also had dlls that will work just fine in one VI that will not work in another VI. LV has problems with dlls. Not only that all the dlls for LV must be written in C. TestStand can use dlls form: LV, CVI, C++, .NET. I also love the fact that you can set a breakpoint in your sequnce and if you have both developement environments on your pc then you can enable tracing and debug C code. I am not saying that TS is always the best way to go, sometimes it is overkill. I personally think that TS is a great tool and is worth every penny I put in to getting it. it serves my purpose very well.

[Rolf Kalbermatter]

QUOTE (jhoskins @ Apr 29 2008, 04:40 PM)

Rolf makes some great points, but I still prefer teststand. I have written dlls that Labview just will not interface with (even with a wrapper) it just does not see the inputs or sometimes will not produce an output. I have sent the dlls to NI and they cannot figure out why they do not work. I have also had dlls that will work just fine in one VI that will not work in another VI. LV has problems with dlls. Not only that all the dlls for LV must be written in C. TestStand can use dlls form: LV, CVI, C++, .NET. I also love the fact that you can set a breakpoint in your sequnce and if you have both developement environments on your pc then you can enable tracing and debug C code. I am not saying that TS is always the best way to go, sometimes it is overkill. I personally think that TS is a great tool and is worth every penny I put in to getting it. it serves my purpose very well.

Hmm, I only write DLLs in C (MSVC, CVI, GCC) but that on Linux, Mac and Windows. And I never had serious problems like what you describe that were not caused by my own stupidity. Also LabVIEW except in 6.0/6.01 does not have trouble to call LabVIEW DLLs. It does get a bit tricky for DLLs that are not compiled in the same LabVIEW version as the caller since you can't use native LabVIEW datatypes (handles) then. But if you intend to call that DLL with other environments than LabVIEW you do not want to export functions that use those types anyhow but use normal C datatypes instead.

On the other hand I have interfaced LabVIEW to third party DLLs such as from Delphi and Borland C and even some GNU made ones and there never was a problem that couldn't be solved with a wrapper and this was normally caused by things like callbacks or complex structure parameters where trying to do it directly in LabVIEW was simply to much of troubles (although even that can be done without going to write a wrapper if one insists on it).

You mention that you interface to C++. Does that mean that you can directly access C++ interfaces in Teststand. It would seem somehow unlikely to me since each C++ compiler uses its own binary object interface format so it could only relatively easily be compatible to the object layout of the compiler that was used to create TestStand. Unless you can interface directly to C++ interfaces it would not seem to be very different to what one can do in LabVIEW.

And .Net oh well, you can do that in LabVIEW too. Not that I have done that much with it, as it would seem to me one more way to tie myself exclusively to Windows.

Rolf Kalbermatter

[jhoskins]

This conversation can go on and on. Simply put we know that LV can lots of things just as good as TS. I prefer the LV-TS combination when it comes to complex testing that contain multiple developement environments. ie C, C++, LV, CVI.

[crelf]

QUOTE (jhoskins @ Apr 29 2008, 05:19 PM)

Simply put we know that LV can lots of things just as good as TS.

It sure can, but the reason I like to use TestStand is so that I don't have to write all the functionality in LabVIEW - if it already exists and works well (for my use cases) then there's no point in me creating requirements, designs, wiriting code, testing code, updating on new versions of the dev env, etc for the components that TestStand already does. I'm sure that if I pushed it really really really hard and tried to make it do something obscure then I could get to a point when it would have been better to write the functionality from scratch, but I'm not there. In short: I use TestStand, I program in LabVIEW.

[Rolf Kalbermatter]

QUOTE (jhoskins @ Apr 29 2008, 05:19 PM)

This conversation can go on and on. Simply put we know that LV can lots of things just as good as TS. I prefer the LV-TS combination when it comes to complex testing that contain multiple developement environments. ie C, C++, LV, CVI.

No argument about that. I just am not convinced that calling DLLs or external code in general is a strong point of TS in comparison to LabVIEW. If you do testing however then there is no doubt that TestStand gives you a nice framework that lets you concentrate on the actual tests and what data you want to store instead of having to bother about the test sequencer itself and how and where you want to get the test data into some database.

I could be wrong however. Never used TestStand for serious work as I'm not in general writing typical test applications.

Rolf Kalbermatter

[jhoskins]

QUOTE (crelf @ Apr 29 2008, 04:30 PM)

In short: I use TestStand, I program in LabVIEW.

My point exactly. i could not have put it any other way. I wonder, within all of this rambling, did we answer this guys question? Thanks Rolf for all of your insight I always enjoy your posts

[Rolf Kalbermatter]

QUOTE (jhoskins @ Apr 30 2008, 07:51 AM)

My point exactly. i could not have put it any other way. I wonder, within all of this rambling, did we answer this guys question?

Well I guess we did and did not at the same time. I got the impression that the OP was not really proficeint in both LabVIEW and TS. And to have to advice someone else from this position is a tough call in any case. I do know for myself that if I would have to advice AND do the programming, my choice would be LabVIEW in most cases since my experience is there and I can be a lot more effective in it. Also we have several LabVIEW test frameworks around here which we have been programming for and with various customers for their specific needs so there is a lot of knowledge about how to do it in LabVIEW too. If someone has more experience in TS then I'm sure he will be a lot more productive for a typical test application in using that.

Without some experience in either one it's sort of an open call.

TS is likely to be easier to pick up for typical final test scenarious. On the other hand for more lab like test scenarious it may be beneficial to go with LabVIEW instead since you have a lot more control about how the entire test is going to be implemented. But going the LabVIEW route all the way is likely to take up significantly more time in getting something up and running as well as learning about the various topics that will look around the corner as you go about it.

But independant of which way you go if you have not a good knowledge about either of these applications I would recommend anyone wanting to be productive fast to go and get a training class for the software in question. The courses may not seem cheap but they pay for themselves more than once in the time spent to get your application working in a timely manner and working good too.

Rolf Kalbermatter