Automation Testing Tools for LabVIEW

[eberaud]

Hi everyone, my apologies if there is already a topic about this, I couldn't find one. My company is looking into acquiring an automation testing tool to troubleshoot and validate our LabVIEW-built application (and maybe one day perform some CI but we're not there yet). I did some research and found a lot (Ranorex, TestArchitect, TestComplete, TestProject, Katalon, Telerik...) but as I read, I think none of them will work with LabVIEW. I need the tool to be able to click buttons in our application, fill text fields, and analyze what it sees on our application windows. Do you know if any of those tools would be able to do that?

The only one that seems to be based on picture recogniction is Sikuli, apparently now renamed Sikulix. Could anyone confirm? And is there any way other than picture recognition to interface that 3rd party tool with our Labview application? (I guess we could add a bunch of LabVIEW code to have some TCP/IP communication, but the goal is not to have to modify our application...)

Thanks a lot for any advice!

[Bryan]

AutoIT is the first tool that comes to mind.  I'm not familiar with it personally, but know some people who have used it in the past.

I don't know if there are any issues using it with LabVIEW, but I know it has been used to automate control of other types of Windows applications.

Edited April 15, 2021 by Bryan

[Bob W Edwards]

I used AutoIT extensively to automate test reports in Word, cutting and pasting tables and graphs from the EMC test software used to control our emc lab. However, I suspect automating Labview controls and indicators requires a more special technique, see this article. This comes from the book Practical Laboratory Automation published by Wiley.

This recent article reviews 10 automation tools, the key feature you need seems to be support of Windows UIA. If the Automation system comes with a Spy tool, then that makes identifying LabView controls and indicators easier.

Bob, retired EMC engineer, ham radio callsign G4BBY

Edited April 16, 2021 by Bob W Edwards

[eberaud]

Thanks Bob and Bryan, I'll try to dig into this.

[thols]

I used to use EyeStudio some years back. It worked quite well for finding buttons, clicking, entering text, evaluating visual results. https://eyeautomate.com/eyestudio/. At the time, it was free, but now its not, and quite expensive. So I have no idea if it is worth it now or how the product has evolved. Perhaps its great now or perhaps its an old unmaintained product they are trying to sell. But I wish I had this type of product now.

[joerghampel]

Have you seen Sam Sharp's presentation on Test Complete?

 

[FixedWire]

I've automated https://www.gurock.com/testrail/ (owned by the same company as Ranorex). It was a few years ago so a few things might be better now. Wasn't a bad experience with the api and allowed the integration to jira. The prime area of interest for the client was the ability to combine different specifications for a product.

... there are a lot of different automation tool concepts.

[eberaud]

On 4/23/2021 at 9:36 AM, joerghampel said:

Have you seen Sam Sharp's presentation on Test Complete?

I haven't. Thanks for the posting it, will definitely check it out!

[Rolf Kalbermatter]

On 4/15/2021 at 10:05 PM, Bob W Edwards said:

I used AutoIT extensively to automate test reports in Word, cutting and pasting tables and graphs from the EMC test software used to control our emc lab. However, I suspect automating Labview controls and indicators requires a more special technique, see this article. This comes from the book Practical Laboratory Automation published by Wiley.

This recent article reviews 10 automation tools, the key feature you need seems to be support of Windows UIA. If the Automation system comes with a Spy tool, then that makes identifying LabView controls and indicators easier.

Bob, retired EMC engineer, ham radio callsign G4BBY

LabVIEW controls are not Windows widgets but fully created and maintained by LabVIEW itself. As such you can not locate LabVIEW controls with tools like AutoIT that assume that a user interface is based on controls that are implemented as Windows child windows. I'm not familiar with UIA, but the claim that it can identify and control LabVIEW controls does sound a bit strong to me. From what I know, LabVIEW controls are fully owner drawn and implemented using a LabVIEW internal object oriented system originally even implemented in standard C, but since almost certainly ported to a C++ system. As far as any external system is concerned a LabVIEW front panel does simply contain lines, texts and maybe some alpha shading but no controls whatsoever. The only entity that can access this object hierarchy for external applications is the VI server interface, but that is highly LabVIEW specific. So I would suppose one could develop an AutoIT plugin that goes over VI Server to control the UI of a LabVIEW VI.

[ShaunR]

1 hour ago, Rolf Kalbermatter said:

So I would suppose one could develop an AutoIT plugin that goes over VI Server to control the UI of a LabVIEW VI.

This also goes for any other non-LabVIEW UI used to control LabVIEW. I came across this issue with the Websocket API (as you probably know, I ditched LV UI's a long time ago in favour of HTML).

Unlike other Websocket applications that send the HTML to be rendered, the implementation I chose was message based. So the VI would serve an image of the FP and the javascript would send back the click co-ordinates WRT the image. The Websocket API would then translate that into the LabVIEW FP co-ordinates and operate the control found there. It was a bit more involved than that but that was the basic principle. This also meant that you could script operations in JS  for testing (or anything else for that matter-as long as it could do websockets, like Python) but that wasn't the original intent.

One customer did write some Javascript (or maybe it was another language) that enabled them to record a series of operations and replay it back like the macro recording in Excel. I wasn't involved in that but thought it was neat.

[Omer Nachon]

Hello i would like to present TVI - Tester Automation Framework software has been implemented in more than 1000 different tests and it’s applied in the defense industry and leading high-tech companies.

Here are a number of the advantages we offer:

• Save 30% -50% of the funds and time of software development and engineering

• Full LabVIEW, Python, .net support using advanced software architectures

• Flexible reports viewer and generator (Word, pdf, csv, Html)

• Easy to use software and station configuration interface

• Built-in steps – run EXE, display text or picture message…

• Support MS-SQL and MySQL results server

• Set test parameters and test configuration

• Create multiple system profiles

 

Please contact: omer@testview.co.il

[Elena Lombrozo]

On 6/23/2021 at 5:20 PM, Omer Nachon said:

Hello i would like to present TVI - Tester Automation Framework software has been implemented in more than 1000 different tests and it’s applied in the defense industry and leading high-tech companies.

Here are a number of the advantages we offer:

• Save 30% -50% of the funds and time of software development and engineering

• Full LabVIEW, Python, .net support using advanced software architectures

• Flexible reports viewer and generator (Word, pdf, csv, Html)

• Easy to use software and station configuration interface

• Built-in steps – run EXE, display text or picture message…

• Support MS-SQL and MySQL results server

• Set test parameters and test configuration

• Create multiple system profiles

 

Please contact: omer@testview.co.il

Honestly, this software is really great as far as automation testing tools go. I'd whole heartedly recommend it.

[Bryan]

It's interesting that you just joined the forum yesterday, and all of the posts for your account so far have been with regard to TVI. 

Additionally, the OP was asking about tools to automate testing of LabVIEW itself, not a TestStand/Test Sequencer alternative. 

[Rolf Kalbermatter]

24 minutes ago, Bryan said:

It's interesting that you just joined the forum yesterday, and all of the posts for your account so far have been with regard to TVI. 

Additionally, the OP was asking about tools to automate testing of LabVIEW itself, not a TestStand/Test Sequencer alternative. 

Not so interesting! Both Elena and Omer have so far only posted to boost about the TVI framework. Omer supposedly as an employee of the company that sells TVI and with Elena I'm not so sure, but it could be also from their marketing department. 😀

While technically related it is bordering spam based on the repeated marketing style postings.

[JKSH]

26 minutes ago, Rolf Kalbermatter said:

While technically related it is bordering spam based on the repeated marketing style postings.

It's more than "bordering spam"; it's a common tactic used by spammers in forums across the Internet: One account posts a link and another extols its virtues while speaking as if they are a 3rd party.

[Bryan]

50 minutes ago, Rolf Kalbermatter said:

Not so interesting! Both Elena and Omer have so far only posted to boost about the TVI framework. Omer supposedly as an employee of the company that sells TVI and with Elena I'm not so sure, but it could be also from their marketing department. 😀

While technically related it is bordering spam based on the repeated marketing style postings.

I agree.  I had originally written a pretty lengthy and fairly mean post because I have such personal distain for such things, but after I finished my first cup of coffee for the morning I calmed down and made it more civil.  

Personally, this type of thing actually deters me and people like me from patronizing such products/companies/organizations. 

Edited February 9, 2022 by Bryan

[ShaunR]

1 hour ago, Bryan said:

I agree.  I had originally written a pretty lengthy and fairly mean post because I have such personal distain for such things, but after I finished my first cup of coffee for the morning I calmed down and made it more civil.  

Personally, this type of thing actually deters me and people like me from patronizing such products/companies/organizations. 

It's 8 months apart. I don't think it's a concerted effort to spam. It is useful to highlight products from time to time. Maybe we should have a commercial product sub forum for these types of products.

[Jordan Kuehn]

I agree with ShaunR. Despite it being posted as replies to several posts all in one day, I don't think it's overly spammy and they were mostly relevant to the discussions at hand. It was enough to get me to poke around at the product page and file it away in my mind should the need arise.

[Bryan]

5 hours ago, ShaunR said:

It's 8 months apart. I don't think it's a concerted effort to spam. It is useful to highlight products from time to time. Maybe we should have a commercial product sub forum for these types of products.

I agree, a dedicated location for things like this would be a good idea.

[Rolf Kalbermatter]

On 2/9/2022 at 5:14 PM, Jordan Kuehn said:

I agree with ShaunR. Despite it being posted as replies to several posts all in one day, I don't think it's overly spammy and they were mostly relevant to the discussions at hand. It was enough to get me to poke around at the product page and file it away in my mind should the need arise.

It actually made me poke at both accounts here on LavaG and the only posts done by both of them were clearly promoting this product and nothing else. Despite being told in one of the threads that it is only very loosely related to the thread at hand as the original post was about an Open Source alternative, which this clearly isn't.

And in two threads the last reactions from both were within 24 hours. Agreed, it is as a follow up post from Omer to the post by Elena, but still all very marketing style.

As I said I consider it bordering spammy, not actually spam, otherwise I would have reported it.

[Gribo]

I asked Omer to stop the spam, if it doesn't work, I know their manager