NI Requirements Gateway

[mje]

Do any of you have experience with the Requirements Gateway offered by NI?

I first heard about this at the recent NI Week and it seems very promising. After getting home I downloaded it, started my trial period, and quickly realized I'm in way over my head. I'll probably need days worth of time to play with it before I can make any real judgement with regards to it's utility in our work environment. For now I don't have that kind of time so I'm shelving the idea and putting it on the list of things to look into after our current release cycle.

For those who have used it, I'm curious how well it helps tracking requirements and how much "overhead" it adds to your workload. Do you find it overall beneficial or is the effort involved not worth what you get back out of it? What other solutions have you tried?

[JLMommy]

I'm probably biased since I used to work at NI (former Michigan DSM), but I think it works great - IF you already have a requirements document.

The overhead to create/update requirements documents is what actually defers me from using it - the company I work with would rather I spend my time programming. Most of the projects I have worked on do not have a strict set of "written" requirements and things change regularly as the project progresses. My customer typically tells me what they want and then see that I met the requirements when they "test" my code. I know this is not the best way and I wish I could change it, but for now that's what I'm stuck with.

It is actually a really useful tool if the industries you work with are highly regulated. It is a quick and easy way to prove that your code has met every single requirement of a project after the intial learning curve. It also helps point out what requirements may be missing - if you have a lot of extra code that doesn't map to a requirement - was it really necessary? Or does your requirements document need updated?

One last note - make sure you are a good speller and/or double check when you type your tags. RG will pick up your tags from multiple places in LabVIEW (FP, BD, VI Prop documentation, etc), but if you mistype something - it will look like you didn't meet the requirement. On that note, if you decide to use it, do not wait until the end to run RG - run it every few days so you can find these problems early and often.

Hope that helps!

- Becky

[asbo]

We use it fairly regularly to provide traceability from our requirements to software (usually LabVIEW/TestStand) to V&V. It's extremely helpful during development to get a rough idea of what functionality is covered already. I have only used it on one or two projects, but you're absolutely right that it takes some set-up time before you can evaluate it.

Because all of our documents (requirements/V&V) a consistent template, we had to write parsers once and can just drop our own documents into a project. Sometimes, we have to write customer parsers to pull out of customer documents, but once you've had to do it once .

We have used it for highly regulated industries (medical/aerospace), but just as much for more lax industries. Once you get the hang of it, it's easy to make it a part of your workflow and you come to depend on it like any other tool - it helps provide a lot of peace of mind by the time you get to acceptance testing.

[PaulL]

I worked with National Instruments Requiremements Gateway (NIRG) a few years back and found it worked well. My sense was that I think I would have had to work with it more frequently to get really comfortable with it, but it had some good features for traceability and some helpful graphical views (although it takes a bit to become proficient with these). It worked as promised. (Well, I did find one bug but NI quickly provided a fix for that.)

Of course, as Becky says, you need your requirements somewhere else first. We had some requirements in DOORS (integrates well with NIRG) but this was an expensive tool and we didn't all have licenses. Putting requirements in Word (we had lots of these) or Excel (by themselves) is less desirable since these are not databases and it can be prohibitively difficult to track a requirement between revisions of the document.

Warning: Software Engineering Theory coming!

In the end, we abandoned the use of NIRG mostly because we abandoned DOORS and Word in favor of SysML. SysML offers two major advantages:

1) A requirement is a first-class entity (true also in DOORS, although depending on how you use DOORS headings can be as well). It is easy to see, edit, move a requirement on a diagram (or diagrams). It is easy to create and link, for instance, a test case (at least in the tool we use). Moreover, SysML allows us to specify the type of relationship. DOORS links were of only one type (although they do specify a direction).

2) Much more importantly, DOORS, Word, and so on are document-centric approaches. When a requirement or set of requirements from one document apply in another document, you either have to copy the requirements to the new document (so that you have to maintain two copies) or you have to create a reference to the original document, forcing the reader to open the additional document to read the requirements. (Karl E. Wiegers devotes a chapter to this here: http://www.amazon.com/More-About-Software-Requirements-Practical/dp/0735622671/ref=sr_1_1?ie=UTF8&qid=1345135697&sr=8-1&keywords=wiegers+more+about+software+requirements. He concludes that there is "no perfect solution" -- see p. 116 -- but prefers links to duplication.) Anyway, SysML is database-centric, like DOORS, but, unlike DOORS, in SysML the fundamental unit for version control is a package within a project file (where a project file functions like a document), so that we can define which packages we want to include in a project. Since we can include a package in multiple projects, we can include a particular set of requirements (or anything else) in multiple projects while maintaining only one version of the package. (We can also create instances of a particular requirement in multiple views.)

For these reasons (as well as the improved ability to work as a team on the projects), we have found the model-based approach for SysML far superior to the document-centric approach.

Note that we are not working in regulated industries, so I have not explored whether our tools or the use thereof would be compliant with the strictest standards, nor are we tracing to code modules as NIRG allows you to do.

Paul

[crelf]

Put me down for a "me too" on a lot of these posts - we use it a lot, but our document templates match it well (or vice versa) so using it from project to project is simple and fast, but that's only because we spent a significant amount of time setting it up right (like most tools, I guess).

As asbo said, even if you're not in a regulated industry, it's worthwhile and gives you peice of mind. Even if it's just to make sure you don'e have duplicate requirements codes or the like. Of course, using it to its full ability is awesome too: there's nothing like a traceability report that's generated by a 3rd party tool which was written by an ISO 9001 certified company to make your customer sleep better at night

[Tim_S]

I looked into it and tried the demo, but can't justify the overhead. My customers typically spec out delivery date, parts per year, perhaps what tests they want done (versus how to do the tests), and GR&R procedure.

[jzoller]

I recently went through a training with Requirements Gateway. It struck me as a vastly powerful and versatile engine (really amazing!)... hidden behind an incredibly clunky and dated VB6 style interface. Screen switching to set up the model in one place and view the effects in another. Very strange mouse behavior: click a button to enter some mode (like there was no mouseover event...?), and the mouse teleports to a different part of the screen. Entry fields were also... odd: some things you could click on, some you couldn't (but it seemed like you should). And clicking on entry fields tended to make the screen flash as it redrew everything so I could type.

Nothing came up in the training about web integration, either. That's something I more or less expect a modern, expensive tool to have boldly stamped on the front.

Honestly, RG had the overall feel of a tool that had been abandoned except for the document interface engine.

Sorry NI folks, I really wanted to like it.

Joe Z.

[hooovahh]

It struck me as a vastly powerful and versatile engine (really amazing!)... hidden behind an incredibly clunky and dated VB6 style interface. Screen switching to set up the model in one place and view the effects in another. Very strange mouse behavior: click a button to enter some mode (like there was no mouseover event...?), and the mouse teleports to a different part of the screen. Entry fields were also... odd: some things you could click on, some you couldn't (but it seemed like you should). And clicking on entry fields tended to make the screen flash as it redrew everything so I could type.

I remember there was some kind of odd-ness with a few tree controls as well where two tree controls looked the same where you could select items by checking boxes in one, but the other was an indicator with boxes that couldn't be checked.

Also don't let your self be impressed with the graphical view shown in demos. With more than 10 requirements that view is worthless. I remember a few projects with several hundred, and one with over a thousand requirements where that view was just a blur of lines.

[crelf]

I looked into it and tried the demo, but can't justify the overhead. My customers typically spec out delivery date, parts per year, perhaps what tests they want done (versus how to do the tests), and GR&R procedure.

Sounds like it doesn't fit your model at all. And that's okay - that doesn't mean Requirements Gateway is a bad product, it just means that it's not a good fit for you.

Also don't let your self be impressed with the graphical view shown in demos. With more than 10 requirements that view is worthless. I remember a few projects with several hundred, and one with over a thousand requirements where that view was just a blur of lines.

I can't agree more - we often open up the graphical view on even a small project for a few laughs.

That said, for what I do, IMHO the graphical view isn't high value anyway.

[mje]

Thanks everyone. Some very good insight in there. For what it's worth we're just getting our feet wet as far as formally tracking requirements. We're on our second release cycle of our first product we've had that has had any formal requirements at all. I'm trying to figure out a better way of doing things.

We're a DOORS workplace, but personally I have had very little exposure and training to it. I hesitate to say this in case the wrong people read it and take it out of context, but I find our existing requirements system to be a bit of a burden to development mostly due to the disparate nature of the various tools that we use: no tool we use plays well with any other from a LabVIEW context. I still think we're better off with a requirements system than without, I just know that the system should be doing way more for us than it's capable in the current form. There's really no way for me to quantify code coverage, what requirements are missing, what code we need to review if we find a requirement isn't being satisfied, etc, short of doing things manually. I'm really hoping to get some level of automation into this the next release cycle and was trying to determine if NIRG can get us down that road.

The concerns about the dated interface worry me somewhat, but in all honesty, if it's functional I'll gladly use it, regardless of how quirky it is. Some of the DOORS/Change interfaces I've used are pretty much the same: very functional, though looking at it from a developer perspective it's hard not to view them as kludgy and I'm left wondering how fragile the system really is. As an aside, the demo NI & IBM showed with integration into doors/change at one of the keynotes and booths seemed like a very solid start. I'm excited to see what happens with that over the next few years.