Books on Software Engineering

[Black Pearl]

I'm looking into some general aspects of software developement 'best practices'. So I would like to get some recommendations on books covering that topic which are useful for LV developers. We are already 'fluent' in LV, so we don't need (or have already) books on LV. But having pictures/drawings would be gentle for our minds and having cryptic C# class hierarchies would bore us to death.

We don't have any formal method yet for our design process. A good overview of concepts would be nice to have to narrow down the choices. But also I will go a bit more into detail, so you could hopefully hint us on a set of methods to look at.

We are the software guys in a scientific environment. So it is all a bit chaotic with ever changing requirenments. And we never ever get any specs on how to design our software. This leads to feature creep/scope creep both by internal featureitis and customers new ideas (they are scientists!). The later we are on schedule, to longer the list of features to implement.

I feel that we can't escape that scenario, so we need to go for some 'agile software developement' (what are similar concepts that might fit?). It is obvious that we need to educate us as well as the customer on that design process to get things done faster (which might be the main obstacle for too formal methods). But we also want to keep the line that we design/own the software and they don't.

Another issue might be our internal developements of reusable VIs/libraries. In that case it's all software guys that want to get things done, with at least some idea of what we want to have. Propably we could use a more formal approach there.

Apart from books, having software tools to help us in the design process/project management would be an issue, preferrable open source/free of charge (we use SVN already).

Felix

[Antoine Chalons]

I haven't read this article (yet) myself, but the title made me think it could be of interest for you.

Hope this helps

[Chris Davis]

I'm in a similar environment to you and your team. I have one other developer that I'm working with and he and I are working to make our environment more software engineered. I'm finding that spending time on our reuse libraries, and using VIPM to package and distribute them, even between two people, has saved us time (and thus money) already. SVN, which you already mentioned you have, was another huge time saver. Setting up standard templates for programs, to make it easier to read and understand other people's programs was also a big deal.

[jgcode]

QUOTE (Chris Davis @ May 5 2009, 07:25 PM)

I'm finding that spending time on our reuse libraries, and using VIPM to package and distribute them, even between two people, has saved us time (and thus money) already.

Amen to that and to OGPB.

[jdunham]

Maybe its for bigger groups, but I would start with

http://www.amazon.com/Software-Project-Sur...5705&sr=8-1

There's probably 10,000 books like it.

Also make sure you score decently on the Joel Test

[PaulL]

Hmm... lots of good engineering process books. I think a good quick overview covering the principles is Head First Software Development.

We use Subversion here with TortoiseSVN and these tools have been even more helpful than I expected. I'm also a huge fan of Enterprise Architect (Enterprise Architect) as a modeling tool--and it can do a whole lot more. EA isn't free but it's inexpensive and I've found it to be orders of magnitude better than tools that cost ten times as much. It is immensely usable and well-designed. It works well with Subversion, too. We have also found JIRA (JIRA) to be an exceptionally useful issue tracking tool. (I wish JIRA had another layer of hierarchy but its advantages far outweigh this inconvenience.)

Paul

[Mark Yedinak]

Since you mentioned the Agile process you may want to look at Integating Agile Development in the Real World. I would also recommend that you find some way to capture the requirements for your projects. If you go the Agile route this can be accomplished by writing good acceptance criteria for your stories. The Agile process also helps you to prioritize your development efforts and allows you to manage feature creep.

[Black Pearl]

Thanks, keep the posts comin'. I'm still missing the big guys....

As a general note: we as developers are pretty much free to go with any method we like, so to say I don't need a book to convince management to any method. But on the other hand, this means we software engineers will have to negotiate a 'common practice' among ourselfes. Always implying that someone isn't ok with it and any method will not be followed literally/in detail (one of the reasons to go with agile instead of formal methods).

So I really need a book that is written for engineers that do a lot of LV programming (and maybe never have heard of OOP!).

Any of us is in charge of some projects and pulling others in on demand for support, and the unifying conceopt is best described by 'autonomy'.

Books in general: Give some more review please. It is a good starting point to look for readers reviews and the authors blog, but I would like a LV'er statement on the books (otherwise I could just hit a amazon search, and other forum users will benefit as well...).

@Several Posters: OpenG and SVN rocks!

@Chris Davis: VIPM: Community or Pro? If Pro isn't an option, can we go with SVN for that?

@Paul: Enterprise Architect looks good. Any experience to share about uml and LV (propably we'd go for a new thread for this, but I'm eager on that one)? I'll sign up for a trial as time will permit.

To go further: After some reading, I see that there are several ways to 'do agile developement'. Can you share any experience of which concepts work well with LV and which don't? (so I don't see any reason to do 'nightly builts').

Felix

[Mark Yedinak]

Where I work we use the Agile process extensively. The product firmware is written in C and C++ and they use Agile to manage that process. We have also been using it for our LabVIEW development for our tools and automation. Agile itself is language independent so it can work just as easily with a LabVIEW development as it will with other languages. As for which flavor we use all I can say is that the Agile process that works best is the variety you customize to fit your environment. Many of our developers and managers attend the annual Agile conference and they have always returned saying that the folks who were most successful using it made the process their own. Pick and choose the parts from the different flavors that best meet your needs. The one thing that is good about Agile is that it really helps you to break your development down into small manageable tasks. It basically forces you to prioritize your efforts on small parts and do the minimal amountof work to meet the requirements of that task. It helps to keep you on focus for the task at hand and not try to create something that does everything all at once. The down side though is that sometimes it does lead you down the wrong path when a little more forethought would have helped to avoid that. Overall it works fairly well. In our environment we started using Wiki pages for our stories but have since switched to Sharepoint. Sharepoint is a bit more feature rich and makes maintaining the stories easier.

[Chris Davis]

QUOTE (Black Pearl @ May 5 2009, 02:17 PM)

@Chris Davis: VIPM: Community or Pro? If Pro isn't an option, can we go with SVN for that?

We get by on one copy of VIPM Pro and everyone else using community edition. If you really wanted to get it done for free, OpenG Package Builder is an option, although it isn't a very pretty option. I used it effectively for at least two years, although I always dreaded trying to get my new packages to show up in the palettes. VIPM has solved that problem and provided a VERY nice way to make your reuse libraries show up in LabVIEW like you want them to. I think JKI is asking a reasonable price for what they are providing, and I'm happy to pay it. It sounds like you may want to check out their VIPM Enterprise for your small group, if you haven't done so already.

You may also want to check out mantis as a bug tracker, it is free and open source bug tracking software built using PHP. I also use dokuwiki as a wiki for development efforts in my group.

[jdunham]

QUOTE (Black Pearl @ May 5 2009, 12:17 PM)

@Paul: Enterprise Architect looks good. Any experience to share about uml and LV (propably we'd go for a new thread for this, but I'm eager on that one)? I'll sign up for a trial as time will permit.

To go further: After some reading, I see that there are several ways to 'do agile developement'. Can you share any experience of which concepts work well with LV and which don't? (so I don't see any reason to do 'nightly builts').

It's my opinion that UML doesn't add much to LabVIEW development, but maybe I haven't given it a fair shake. While I realize LabVIEW and UML have different purposes, I feel I can design well enough in LabVIEW that I don't need to waste engineering resources (i.e. my time) on a second set of graphical program expressions.

You may be right that you don't need to build your labview nightly, since it compiles all the time. But the underlying tenet is that you try to make sure your code (at least the trunk version) is nearly always in a state where it could be released, perhaps with many features missing, rather than getting into a pit where it is broken for weeks at a time. On our project we build an EXE, so we do have a nightly process to build it if there have been any SVN checkins. Ideally you will have automated tests running against the nightly version while you sleep. (Still working on that!)

[Daklu]

QUOTE

We are the software guys in a scientific environment. So it is all a bit chaotic with ever changing requirenments. And we never ever get any specs on how to design our software.

I think you will find most software engineering books are geared towards businesses that develop software products that will be released at some future date. After release, there may or may not be bug fixes released or new versions developed, but that product is essentially done. Before you start down the path of adopting that latest software engineering process, ask yourself the question: Is the software you are developing a "product" or a "tool?" The processes and patterns used for software products may not work well for engineering tools.

How do you know if you are developing a product or a tool? Here are a couple rules of thumb that might be useful, roughly in order of my arbitrary sense of importance...

-How does your feature set change over time? If the feature set starts out big and decreases over time, it's a product. If it start small and increases over time, it's a tool.

-How solid are the software requirements? If you have a document that is mutually agreed upon and relatively difficult to change, it's a product. If the requirements exist only in someone's mind and changes depending on the lunch menu, it's a tool.

-What is the lifetime of your application? If it's used for more than a year without any major changes, it's a product. If it needs updating or new functionality less than 3 months after deployment, it's a tool.

-What is the tolerance for bugs in your software? If heads roll when the application crashes, it's a product. If bugs are "okay" as long as the customer can get the software to work, it's a tool.

-What is the retail purchase price of your software? >$0, it's a product. $0, it's a tool.

-What attitude does management have towards your software schedule? If they allow you the time to do things "right," it's a product. If they want you to just get it done as soon as possible, it's a tool.

At a recent position I had I designed and built a software "product" for testing a component during development of a consumer electronics device. I designed it to be very flexible, allowing for as many different system configurations as I could reasonably conceive. Near the end of the design cycle the design engineers started asking for things I couldn't have predicted. Needless to say, the architecture I had developed couldn't support their requests very easily. I had to make dirty hacks that compromised the overall product just to be able to perform the tasks they requested. It wasn't pretty. It wasn't effective. And it certainly wasn't efficient.

I've come to the conclusion that when building tools you must expect the next "one little change" to break your entire architecture and require major refactoring. The trick (one that I have not yet figured out) is how to minimize that down time.

[Dan DeFriese]

QUOTE (Black Pearl @ May 5 2009, 05:36 AM)

...So it is all a bit chaotic with ever changing requirenments.

This will always be the case. Doesn't matter if you're developing for Microsoft, NI, or XYZ. Invariably, your customers will expect a certain amount of flexability in your product... they call it SOFTware for a reason.

QUOTE

...we never ever get any specs on how to design our software...

Your the software domain expert - its your responsibility to spec out the design the software.

Though, I assume you meant requirements specification. Nobody, will ever hand you a complete set of requirements. You'll have to dig for them...then YOU put them in writing and review with the customer. Yes, things will change...but any changes after the initial commitment should be tracked using a defect tracking tool:

Serena Tracker

TeamTrack

DevTrack

Bugzilla, or whatever.(You could even do this with a spreadsheet if you want to be cheap... but, you get what you pay for.)

I've found that using defect tracking helps me keep current development controlled and keeps a history of common requests from my user base. Future projects benefit because I already know what questions to ask.

IMHO, UML is worthless for capturing requirments since the customer usually doesn't understand the diagrams. I find, simple flowcharts and screenshot prototypes much more effective.

Anyway, my favorite SE related handbook is "The Pragmatic Programmer: From Journeyman to Master" by Andrew Hunt, David Thomas

[Tim_S]

QUOTE (Black Pearl @ May 5 2009, 06:36 AM)

We are the software guys in a scientific environment. So it is all a bit chaotic with ever changing requirenments. And we never ever get any specs on how to design our software. This leads to feature creep/scope creep both by internal featureitis and customers new ideas (they are scientists!). The later we are on schedule, to longer the list of features to implement.

We've got a similar albatross with our customers... it's called simultaneous engineering (SE).

For anyone not familiar with SE, it's akin trying to build a cart that will sit perfectly level forevermore while the horse is still growing.

There are pieces we know we'll use... calibration, communication to a PLC or remote I/O, error management, etc. These become part of the library. Anything we create gets looked at and we try to see if it's a one-shot item or something we'll re-use. Sometimes what we reuse is surprising.

Getting an initial specification is tough. I like to mock up screens, create a little Powerpoint and write-up of how it'll work and get sign-off on that design. The signature is the critical part as anything after that is a change and is chargable. Paper trails of what changes were desired and approved provides documentation as to why things are more expensive, taking longer, etc. I've heard this refered to as "managing the customer".

Tim

[Black Pearl]

Thanks for all that info so far, feel free to add more...

One question:

QUOTE (Dan DeFriese @ May 7 2009, 03:08 PM)

Though, I assume you meant requirements specification. Nobody, will ever hand you a complete set of requirements. You'll have to dig for them...then YOU put them in writing and review with the customer. Yes, things will change...but any changes after the initial commitment should be tracked using a defect tracking tool:

Do I read correctly between the lines, that these tracking tools go further than just bug fixing?

Felix

[Mark Yedinak]

QUOTE (Black Pearl @ May 7 2009, 09:44 AM)

Do I read correctly between the lines, that these tracking tools go further than just bug fixing?

They can. They are useful for entering new feature requests, enhancements, bugs found during your testing as well as customer found bugs. Generally you can assign priority to your items which help you to scope the effort required and organize your tasks.

[jzoller]

QUOTE (Black Pearl @ May 7 2009, 08:44 AM)

Do I read correctly between the lines, that these tracking tools go further than just bug fixing?

Most bug tracking tools have a type to fill in when a bug is filed. It's usually pretty easy to subvert this value to something like "Feature request". Two benefits: central feature management, and the ability to assign new features to specific releases (or oblivion ).

I personally like the hosted Fogbugz version: http://www.fogcreek.com/FogBugz/. Free for one or two people.

Favorite book: Code Complete by Steve McConnell.

Favorite essay on Agile vs agile (warning: offensive to multiple religions, workplaces, Agile practitioners, and possibly humanity in general) Steve Yegge's [Aa]gile essay

Are you really looking for software engineering tools and disciplines (SCC, UML, bug tracking, project management, etc), or are you looking for ideas on how to organize a group of coders for greater efficiency and productivity? The two are, in my mind, related but separate questions.

Joe Z.

[Dan DeFriese]

QUOTE (Black Pearl @ May 7 2009, 09:44 AM)

Do I read correctly between the lines, that these tracking tools go further than just bug fixing?

Absolutely! Keep in mind that defects (bugs) are not limited to logical programming errors...

SE texts generally suggest that the vast majority of software defects are created during requirements phase (either through omission or misinterpretation). By comparison, logical errors are actually quite rare.

[PaulL]

QUOTE (jdunham @ May 6 2009, 11:19 PM)

It's my opinion that UML doesn't add much to LabVIEW development, but maybe I haven't given it a fair shake. While I realize LabVIEW and UML have different purposes, I feel I can design well enough in LabVIEW that I don't need to waste engineering resources (i.e. my time) on a second set of graphical program expressions.

I have found UML is quite helpful when we use it appropriately. In particular, it is much quicker to make changes in a model ahead of time than to make changes in the code. Also, we have a good description of what we are trying to do that nonsoftware engineers can understand (although creating the models effectively and efficiently does require considerable practice) and review. Finally, UML is a great way to capture data structures for interfaces. (Especially for object-oriented systems I haven't found any other reasonable way to do this.)

This doesn't mean one should model every tiny piece of software, let alone in final detail. (We don't autogenerate LabVIEW code, for instance.) We do have to know when creating a model is useful and when it is more efficient to write code directly. I find the models quite often worth the effort, especially now that I can create them quickly.

[Black Pearl]

QUOTE (jdunham @ May 7 2009, 08:19 AM)

You may be right that you don't need to build your labview nightly, since it compiles all the time. But the underlying tenet is that you try to make sure your code (at least the trunk version) is nearly always in a state where it could be released, perhaps with many features missing, rather than getting into a pit where it is broken for weeks at a time. On our project we build an EXE, so we do have a nightly process to build it if there have been any SVN checkins. Ideally you will have automated tests running against the nightly version while you sleep. (Still working on that!)

Can you hint me how to automate these nightly builds (and tests).

And furthermore, isn't that an option to just buy one single license ($$) of the application builder for the whole team, as only the developer 'responsible' for the nightly builds will need to install it?

@Joe Z: I'm mainly searching for software/tools, as we are settings up a network based development enviornment. Later on it will be moved in the grip of the IT-gods. But for now we are free to experiment around and and choose the software.

The books are meant to get some ideas of what we might need and for what purposes we could use the tools.

And I found a nice new wave for surfing the web, as book sometimes link to blogs covering similar topics...

Felix