Requirements Docs & Testing Tools

[Reds]

I'm curious to hear what tools you guys are using to write and track software requirements/specifications. I'm especially curious to know if any of you have a workflow that can not only track requirements, but also document pass/fail test results for a given requirement in a given software release.

 

For myself, I'm using Microsoft Word and FogBugz....i.e. I write the requirement  in Word and then if it doesn't work, log it as a defect in FogBugz. However, there is a really naïve underlying assumption to this workflow: namely, that I would never release software that doesn't meet a requirement or pass a test.  That's obviously a bogus assumption. A defect logged in FogBugz also doesn't have any sort of link back to the original requirements document for future reference. Most importantly, this workflow doesn't allow me to run a report which says something like: "show me all unmet requirements in release 3.2" or "show me which requirements have never actually been tested in release 3.2".

 

I think (know) that it's important to track which requirements are actually implemented and actually pass tests in a given software release.  This type of tracking would require some sort of relational database, and my existing workflow just isn't up for it.

 

So, how many of guys are more sophisticated than Microsoft Word?

[hooovahh]

In the NI world people usually talk about Requirements Gateway. (RG)  It was a tool made by another company that NI bought and modified to support NI software tools a little better.

 

I don't know that it can do all the things you asked, it basically links Word documents, and LabVIEW source, finding tags in the code, usually to signify a requirement is being fulfilled by that part of the code.

 

Imagine you had a detailed requirement document.  RG can pull out the text and tag information so it knows "R12-345" means "The software shall have a UI".  Then in LabVIEW you can put down a comment on the BD or FP stating you cover that requirement.  Then you can write a test document that proves that the requirement is in the build by having a user test that the UI exists and stating it covers that tag.  Then you could have test results logged showing that the test was performed, and what steps passed, meaning those requirements in those steps were tested and were implemented, and so they were coded.

 

When all this linking works well, it is beautiful.  RG can generate matrix tables showing start to finish where the requirements go, from creation, to implementation, to test document, to test passing.  This requirements traceability matrix can be stored for each build and then you could record coverage this way.  It's a lot of work, and honestly most of the time it isn't worth it.  But for medical devices, customers like to see the ducks in a row.

 

Bug logging for me is separate and I haven't seen a good integration.  Not that it hasn't worked, I just haven't seen it integrate with all the other tools for traceability.

[Reds]

Yes, I took a look at NI Requirements Gateway, but I thought it was weird that that tool was so focused on documenting *where* in the source code a requirement is implemented. That doesn't seem useful to me at all (maybe it is to the medical guys and others who are heavily regulated).  I think (?) what I'm really looking for is something that links requirements to test results, not requirements to source code locations.

 

I guess, I'm not really sure what I'm looking for, but I'd like to hear from people who have gotten more sophisticated than Microsoft Word and your standard Defect Tracking website/SaaS tool.

 

I feel like in the LabVIEW community, our development processes and systems are often behind the curve compared to the traditional, text-based dev world. For example, it seems like it took us a lot longer to get on board with source control than the rest of the programming world. Perhaps it's because so many of us have to play the dual roles of both "domain expert" and "software developer".

Edited September 23, 2015 by Reds

[hooovahh]

For example, it seems like it took us a lot longer to get on board with source control than the rest of the programming world. 

I've never really thought about it but you are totally right.  I suspect it has a lot more to do with the target audience of LabVIEW, being scientists and engineers, who may have no or little software discipline background.  Who needs SCC when it's just me in a lab making a single giant VI?  Why would I need requirements tracking?  Bug tracking?  Documentation?  Coding style?  Certification?

[ShaunR]

I have something that I use quite a bit for many things but I don't think I have anything as sophisticated as you would be requiring. It's like a trackable "completeness" application-how complete a project is. It checks descriptions are filled out, whether VIs are orphans, re-entrant and lots of other things for keeping a track of a projects progress and making sure certain standards are met. You can compare previous project scans and do diffs of the changes in issues. It has plugins that can access its database so you can extend its features pretty much indefinitely-I've been abusing it recently by adding scripting functions to set VI names, making them re-entrant,  and other bits and pieces which it shouldn't really be able to do for a passive analyzer..

 

It doesn't do testing as such but it supports plugins so you could create a plug in or two to populate its database with results or attach another database to do cross DB queries. It also allows in-place SQL queries so you could also define views of your test data combined with all the other VI information. There is already a plugin for requirements coverage ala Requirement Gateway. Its one of those tools you always use but would be a nightmare to productionse and could cause havoc in the wrong hands.There is an image on LavaG somewhere

Edited September 23, 2015 by ShaunR