4 years ago, I took my current job because the lone engineer supporting the manufacturing of my product suddenly left. I believe there were multiple reasons, but after I got involved I realized that I would have probably done the same if I was in his shoes.
There was 10 test stations running 5 test sequences for three specific products. The test system was based on an older version of CVI / TestStand with run time deployment that originated at a consulting company. The product under test has some diagnostics, but much is tested by configuring it as a customer would. The product software changed over time (as would be expected) and the tests stop working. Sometimes the changes demanded a change to the CVI code, but the previous engineer had no experience. Solution? Replace the functions that don't work with LabVIEW VIs!
Move forward a couple of years and introduce new test requirements, new web based interfaces and the test system turned into quite a mess of curl, perl, CVI, LabVIEW and TestStand RTE.
It got so bad that the product OS was locked down to a specific version of software and then upgraded at the end to the shipping software. CVI code would crap out when IT would install Windows updates (DLL hell). Test times per UUT went from an original 30 minutes were now almost two hours. The product ended up being tested twice just so we could install the latest software before shipping. First pass yield was 80% with the primary cause being test execution failure; timeouts exceeded and Selenium/Firefox step failures because of JAVA updates or an operator changing the screen resolution.
This is where I came in. I dubbed the existing system "The Jenga Pile". Touch anything and it all came crashing down. It's easy to say that this is an implementation issue. My manager and I decided that we needed to move to a current version of Windows / LabVIEW / TestStand, get rid of cludges like Selenium and curl and rewrite the tests from scratch based on the latest version of the product software.
I was 6 months and 75% or so into the project when the only early advance of the new product software was given to me. Many of my tests were based on a telnet session to the product. Telnet was now REMOVED from the product! (not disabled, but REMOVED - security issues ). I needed to change to SSH or use the serial console port. LabVIEW and SSH (don't get me started)?!?!
I was already done with my test coding and intended to spend the last two months tweaking the TestStand environment and creating my first RTE deployment. I used SSH.NET here and there where necessary to recover. After taking 4 weeks to rework the telnet related issues, I found I could not reliably create and install TestStand deployments. SSH.NET would not work. Oops! Did I mention I had never created an RTE deployment before? My previous employer used debug deployment licenses and I never once dealt had to deal with the RTE. Now I know why.
I told my manager to cancel the plans for upgrading our 10 deployment licenses and spent 4 weeks in overdrive creating my own crude sequencer and report tools. I saved the cost of NI upgrades, met the deadline and made my life easier down the road.
Changes now only require compiling a new EXE and dropping it on the test stations. I have a UI that I can change, deploy in minutes and I haven't had to think about Process Models or creating 100s of MB of deployment packages for simple changes. It is not actor based or even very modular at this point.
I love the promise of TestStand, but I'm not controlling a super-collider and certainly not doing rocket science. Give me TestStand Lite (google it) as an actor where I can register some LabVIEW UI elements to monitor and control execution.
Give me TestStand Lite, limit the engine to running LabVIEW VIs and include the engine in the Professional edition. The TestStand Lite sequence editor should integrate seamlessly into the LabVIEW project.