Jump to content
0_o

Integrate Mocks and Stubs into HAL - recording app actions for testings and development

Recommended Posts

Hi,

First, I think there should be a main topic called testing instead of TestStand or along TestStand.

I implemented a Hardware Abstraction Layer as described by NI and I think it is poorly designed for the following reasons:

The ASL (application separation layer - API for general HW access) is not OO and lacks logic, error handling and stubs/mocks for simulation.

I found that during my app development I waist a lot of time since my HAL has no good testing framework with code reuse and each test unit needs all the system to be configured and running.

Besides that, if I have no access to the hardware it is very hard to develop since I need to write code, compile it, move it to a test machine, run + debug, come back to LabVIEW and try to find the bug with no live input for LV debug tools and without being able to exactly reproduce the scenario from the test machine. This process will repeat itself until I'll build an ad hoc recording mechanism which is not reusable for other features I develop.

Thus, I wonder, do you guys have an agile solution to this bad methodology?

Is there a tool that will automatically create some Mocks/Stubs to my code (stub is a recording of the application actions and responses. It can be used to virtualize the environment while actually working on a stand alone application. Thus, testing and development time is reduces).

I wrote such a recording mechanism for some of my features but it is not general or integrated into the HAL design I copied from NI.

Thanks in advance,

GOOfy Wires.

Share this post


Link to post
Share on other sites

Thanks for the feedback. There are some things I'd like to see improved in the HAL decomposition we've put forth (http://zone.ni.com/devzone/cda/epd/p/id/6307), but some of your feedback is new.

At the risk of introducing new terminology, I prefer to talk about a "measurement abstraction layer" and a "hardware abstraction layer". The ASL is closer to the MAL, and the DSSP is closer to the HAL. The MAL can present a high-level measurement--e.g., a "stimulus/response test", or "filter characterization test"--and implement the test strategy--e.g., to use an RFSG/RFSA frequency sweep, or wideband pulse measurement. I would expect both the MAL and HAL layers to be OO.

One of the cool things about our approach is that you can choose to simulate (stub/mock) at either level of the abstraction, and it's just a matter of writing another implementation class. If you want to use simulation at the HAL layer, then we already have simulation built into our IVI and modular instrument drivers.

It sounds like you want to record real-world measurement data and then play it back. Having a HAL in place will facilitate the recording of the data, since you can connect to your actual test bench and capture the data to a file without touching the rest of your application. Then you can use a different HAL implementation class to play back the data from that file.

Share this post


Link to post
Share on other sites

Hi Brian,

I didn't expect the developer of HAL to answer my questions here. Cool! :)

I would love to see an integrated version of MAL over HAL with integrated OO testing and bug fixes.

However, my question regards a different subject:

A good simulation will have to record a sequence of high level UI actions and understand which low level MAL/HAL action was taken so it could record a sequence of actions, record the relevant inputs for each action and finally play the responses for virtual UI actions.

This process should be general and not feature specific. At the moment I need to write specific record and replay for each feature. I'm searching for a general solution that will listen to the UI and add record replay to the lowest level of the DSSP and thus simulate all the high level actions at once.

All the high level (no driver/communication access vis) should be the same in the simulation mode since I'm actually trying to debug them. The lowest level should get a flag with a link to the relevant recording it should run if this is a simulation mode for a specific action that was recorded. If I need to maintain a parallel simulation class I lose all the point of having this simulation mode.

I'm trying to understand what you think I should do according to your last paragraph and I still don't understand how you are going to match together the recording to the UI action. A Stub/Mock doesn't just replay a black box, it allows a simulation of UI actions while replaying the relevant response to each specific action. Thus, if I reorder the response to move up, move down, roll over and howl I can ask in the virtual mode to do each of those actions in a different order even though some use the same DSSP vis and each time those same vis were called a different recording was run according to the action we simulate.

Thanks for the replay!

P.S. - I would pay good money for a good NI platform that will build Mocks/Stubs while adding some CI testing capabilities to a MAL/HAL design along some automatic builder in LabVIEW.

Share this post


Link to post
Share on other sites

At the risk of causing heartburn and panic among my fellows in LV R&D, there is a feature in LabVIEW that will do what you just asked for as far as recording is concerned.

The reason it may cause panic is because it is so rarely used, so although the test suite keeps passing, it hasn't had any developers work on it in well over a decade. And yet it is still there. We kind of loathe this feature because keeping it working has required some extra complexity in some new features, and we've talked about killing it. Me advocating it as a solution runs the risk of breathing new life into it. :-) I give you this intro so that you understand: the UI is a bit rickety, but it works, at least for its intended original use case.

Right click on any subVI node. In the menu, you'll see "Enable Database". You probably have never used this menu item (I've polled huge crowds of LV users and I almost never get anyone who knows what it does unless there's a LV R&D teammate in the room). "Enable Database" will cause a "halo" to appear around the node. All the input terminals turn into output terminals, and the halo has additional terminals.

When this node is part of your application, any calls to the same subVI as the halo'd subVI get logged -- the inputs and the outputs. When the halo'd node executes, it takes as input an integer that is a call ID. This allows you to retrieve the conpane of that subVI as it was the Nth time it was executed.

I know what this feature does for static dispatch, non-reentrant subVIs. For anything else, well, I will bet that it won't crash, but I've got no idea what the defined behaviors would be. I'm 90% certain this feature does not work for dynamic dispatch VIs (I recall consciously disabling it, but someone else on my team may have hooked it up at some point). I have no idea what its behavior is for reentrant VIs.

Play around with it. See how it works. It may be slower than you want (I've got no idea what the data structure that it uses for a database looks like). It may have memory issues (I know it doesn't do any disk caching or anything like that). But perhaps it has value to you.

Share this post


Link to post
Share on other sites

Hi Aristos Queue,

Thanks for the risk you taken here!

While searching for a solution I went over most options I could put my hands on and this was one of the winners.

It gave me some trouble in freezing a vi since it tried to record something and I didn't take it into concideration and as for dynamic vis, it misses the point for HAL which is OO and in some points reentrant.

If this feature was working correctly it could have been a solution and a very nice one too with just a little bit of a framework to wrap it up.

P.S. - if we talk about unused/depricated features I came across some property nodes for code coverage and other cool stuff like convert to vhdl that I didn't know LV could handle. I understood they are volatile but now I see even some right click features are risky.

P.S.2- now that I think about it again, I can use it to record only the lowest level that uses driver blocks and match it with the call chain and the control combination. This might just work since those vis are static non reentrant (even singletonian) !!! Thanks!!!!!

I'll post back once it is tested.

Share this post


Link to post
Share on other sites
While searching for a solution I went over most options I could put my hands on and this was one of the winners.

It gave me some trouble in freezing a vi since it tried to record something and I didn't take it into concideration and as for dynamic vis, it misses the point for HAL which is OO and in some points reentrant.

Should be easy enough to wrap the interesting calls in a static VI wrapper.

I'm not even sure how I would define the behavior to work for dyn dispatch... should it log every call to just the one VI that is halo'ed? Every call? How about calls that happen through the Call Parent Node? If it is every call, what about calls that are explicitly to a higher level of inheritance? What about calls that are to a lower level of inheritance?

but now I see even some right click features are risky.

Not risky in the same sense. This feature works, and works well, exactly as designed. It hasn't been depricated or anything like that. It just hasn't been polished in a while and if you have questions, it means a lot of "oh, how did that go again?" research on the part of folks here at NI. :-)

Share this post


Link to post
Share on other sites

At the risk of causing heartburn and panic among my fellows in LV R&D, there is a feature in LabVIEW that will do what you just asked for as far as recording is concerned.

The reason it may cause panic is because it is so rarely used, so although the test suite keeps passing, it hasn't had any developers work on it in well over a decade. And yet it is still there. We kind of loathe this feature because keeping it working has required some extra complexity in some new features, and we've talked about killing it. Me advocating it as a solution runs the risk of breathing new life into it. :-) I give you this intro so that you understand: the UI is a bit rickety, but it works, at least for its intended original use case.

Right click on any subVI node. In the menu, you'll see "Enable Database". You probably have never used this menu item (I've polled huge crowds of LV users and I almost never get anyone who knows what it does unless there's a LV R&D teammate in the room). "Enable Database" will cause a "halo" to appear around the node. All the input terminals turn into output terminals, and the halo has additional terminals.

Well I have been using this in the past for debugging communication drivers, using it on the Read/Write function to log all the strings sent and received to see how it went bad at some point and try to understand protocol handling issues. As such it is helpful but not exactly user friendly and I have since usually added in such cases a direct debug logger that takes the interesting parameters and writes a string line to a text file on disk. It means writing specific code for each case but is a lot easier to parse and analyze later on than manually stepping through each call iteration. If there would be a method to hook a log VI to a VI instance (or its class, I wonder if the Class Operator might be possible for that and I'm not talking directly about LVOOP but the App method App->Get/Set Class Operator) then that might be more helpful. Such a logger VI would probably receive a data structure that contains a copy/reference to the actual conpane data and would have a pre and post execution call.

The Enable Database feature has been discussed in some LabVIEW classes back in the old days (15 to 20 years ago) but has gone into the land of forgettingness since.

Share this post


Link to post
Share on other sites

rolfk,

I just want to emphasize the different uses of a logger and a recorder.

A logger is used for debug. You log just the relevant information in order to understand what happened and for this you'll need to parse the log and follow it.

A recorder records all the input/output in the ASL level so you could simulate the operation of the system without having it at hand.

You don't need to parse a recorder. You use it to verify that the program you wrote still works as expected and develop new features assuming the system gives you a specific response you got in the past.

There is a place for logging and there is a place for recording.

I find loggers to be useful for parallel loops (if you write good parallel code there is not much use for them) while recorders are very useful since they shorten the development time and add a very easy to implement regression testing.

Aristos Queue,

thanks! I'll write back after I get the chance to test it.

Edited by 0_o

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Similar Content

    • By Christian Butcher
      Hi,
      I'm writing unit tests for some logging code (OK... so I suppose testing the actual logging class would be integration testing, but...) and have (at present) two disk logging classes and an in-memory test implementation.
      My test suite/case arrangement is currently a bit of a mess (understatement) and what I really want to do is run the same set of tests using each logging implementation. Obviously I don't want to code up the same tests repeatedly (and have to maintain a set of multiples).
      The solution as I understand it is to store a class object of the type to test in either the Suite or the Case (or both?) and then always read from private data when running a test. Writing the individual tests in this way is fine. My confusion lies in the best way to arrange the Suites and Cases to do this.
      In particular, if I have some DiskLoggerTestCase containing a collection of tests (for simplicity, all of my tests) and with the Logger object as private data of the Case, then I can create a Suite with code like below in New.vi (duplicatedCasePerSuite.png). Here, I build an array of each Logging object, then use a Write accessor and an autoindexing For loop to build the Suite. Problem: In the VI Tester UI, I see "DiskLoggerTestSuite:DiskLoggerTestCase, DiskLoggerTestSuite:DiskLoggerTestCase". There appears to be no way to distinguish or add names separately (if this is incorrect, I'd love to know how to change it).
      My second attempt (SuitePerImplementation.png) was to create a new Suite for the single TestCase, with each suite having a new name but the same TestCase (and using the write accessor, without a For loop). This works out, but means I now need to create a new suite for each additional implementation. It's not so terrible, but changes have to be reproduced in each suite if I want to modify something. Further, if I have more than one TestCase, it isn't possible to place them in a loop without reorganising them to inherit from a shared class providing the "Write Logging Object" VI as Dynamic Dispatch. Adding cases that don't use this require a new loop to discover their test methods. Duplicate code again rears its head.
      My next attempt will be to create one Suite of TestCases, in which the Suite has a Logging object, and it passes the object to the necessary cases, whilst allowing me to then create a set of owning Suites that each contain the first Suite. This is probably more or less the same as the second approach, but should at least compartmentalise some of the changes needed when adding Test Cases and separate adding Test Cases (change to the first Suite) from adding implementations (add a new Suite of the second type).
       
      Does anyone know the way this should really be done? I feel l'm missing a good solution and just creating difficult-to-maintain messes.


    • By ShaunR
      This is an experimental demo to investigate VIMs (vi macros). It was a bit of fun to see if VIMs could be used to encapsulate events in LabVIEW which was a bugbear of mine for quite some time. You can see the entire thread here. VIMs are a NI experimental technology similar to Xnodes but less mature.
       
      The purpose of this release is to clarify the previously unstated licence since other forum topics are building on the original demo so they need a permissive licence (MIT). This release serves as an unambiguous statement to that effect. There are a few differences from the original which I have decided to call version 0.1 but they are minor.
       
      Note:
      This may or may not work for you out of the box. If it doesn't then please do not post. The purpose is to clarify the licence for others to build upon; not to provide a working example. The VIM technology is itself  experimental and unsupported by NI so most issues you will encounter will be due to this and It is unlikely there will be another version posted here.
    • By ShaunR
      Name: VIM HAL Demo
      Submitter: ShaunR
      Submitted: 25 Sep 2015
      Category: *Uncertified*
      LabVIEW Version: 2009License Type: MIT



      This is an experimental demo to investigate VIMs (vi macros). It was a bit of fun to see if VIMs could be used to encapsulate events in LabVIEW which was a bugbear of mine for quite some time. You can see the entire thread here. VIMs are a NI experimental technology similar to Xnodes but less mature.
       
      The purpose of this release is to clarify the previously unstated licence since other forum topics are building on the original demo so they need a permissive licence (MIT). This release serves as an unambiguous statement to that effect. There are a few differences from the original which I have decided to call version 0.1 but they are minor.
       
      Note:
      This may or may not work for you out of the box. If it doesn't then please do not post. The purpose is to clarify the licence for others to build upon; not to provide a working example. The VIM technology is itself  experimental and unsupported by NI so most issues you will encounter will be due to this and It is unlikely there will be another version posted here.

      Click here to download this file
    • By dmurray
      I have a fundamental question on HALs and abstract methods that I'd like clarified, please. In my HAL's up to this point I've been using a parent with abstract methods, implementing concrete methods in the concrete product(s) (see here for example), and using dynamic dispatch as appropriate to execute the concrete method. I think this is a standard technique and it works fine for me.
       
      Now I have another layer of abstraction I want to implement, but in this case the concrete products share a lot of functionality and often I can get away with just executing the same method for each product. Should I just go ahead and implement this logic in the parent method? If I do, that means I no longer have abstract methods in my parent (right?), so am I breaking some 'good-practice' rules in OOP design? 
       
      Or am I just getting bogged down in semantics? 
       
      [by the way, I have loads of little niggling questions like these which I may ask as I'm going along in my OOP learning - hope you don't mind.] 
    • By dmurray
      Hi all.. I've been playing around with various HAL implementations for a while now, and I have a question on implementing some simple "state" functionality in the HAL. I've attached one such example - It's just an abstraction layer for a range of I2C Adapter modules that I use (or intend to use in the future). So my parent class is an abstract I2cAdapter, with overridable (dynamic dispatch) methods and exposed private data, effectively forming my interface to upper layers. Then at run-time I will just instantiate one concrete 'product'. (By the way, when I say "Adapter", it's nothing to do with the GOF pattern). 
       

       
      Now I also want to add some simple "state" functionality, and I'm hoping to get some pointers on the best way to approach this. The states I can identify in the HAL are:   1. Adapter physically disconnected (i.e. plugged out from the PC/laptop/whatever). I want upper layer(s) to know that this has happened. 2. Adapter connected, and enabled (normal operation). 3. Adapter connected, but disabled. Effectively I want the adapter to be in a high-Z state as far as the bus is concerned. (Of course this should happen automatically for a 'true' I2C adapter when its not accessing the bus, but unfortunately I can't guarantee that with the devices I'm using.)      I've looked at the GOF State pattern, but I think it's too elaborate for this situation. Also, I'm not sure it really applies here anyway.   As I look at it, the Enabled/Disabled states can be done simply - just add one (or two) abstract method(s) in my parent class and let each concrete adapter implement the necessary functionality. But what's the best way to handle the Disconnected state? Should the concrete adapter implement some functionality and inform the upper layer if a disconnect has occurred? Or should the upper layer have some sort of a periodic check of connectivity?    Maybe I should also add that one of the learning 'glitches' I'm having with LVOOP is the topic of collaboration between objects, and this post ties in with that to a certain extent. So any help would be appreciated.  
       
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.