25 replies to this topic

[John Lokanis]

I have an opportunity to take an existing large project and re-architect it from the ground up. All options are open to me. So, I want to apply all the latest and greatest tools and techniques to build an eco system for my code and my company that will make this project and all others after it easier to maintain. I would like your thoughts on what things work and what are not worth the effort. Some ideas I am considering are:

Change the code to use LVOOP design patterns.

Organize all my reusable code into distributable packages (VIPM?) that are available in the palettes.

Add unit tests to all (or most) of my code.

Integrate the Diff and Merge tools into my SCC system (Perforce).

Use the GOOP Dev Suite to generate UML from my code (and vice versa) for doc purposes.

Use Packed Project Libraries to distribute plugin code instead of LLBs.

And on a more technical note specific to my project: decouple the UI from the code. What I mean is instead of the UI being a FP of a VI, implement it as a web page (using the UI builder?) and have that interface to the code via web services. This might be too radical to do right now...

So, please let me know what tools and techniques you use that work well in large applications with multiple devs. I want to try to be as state of the art as possible since I doubt I will get another chance to do this again.

-John

[Black Pearl]

Just on the two (four) topics I have experience with.

LVOOP & uml:
I would take any chance to use LVOOP. I found it not really complicated using the simple equation 'type def cluster'(+bonus power-ups)=object.
Once you are using LVOOP, you will want to look into common OOP design patterns, also they not always translate into the by-val paradigm.
I think for any larger OOP app, UML is a must. Just reading OOP design patterns will confront you with this, and if you are already a graphical programming freak using LV, why for hell skip UML.
On GOOP I'm not really sure. It's the tool that integrates with LV. But if I understood it right, it's favoring by-ref classes. I find by-val more native. On free alternatives, I found eclipse/papyrus to be a very powerful uml modelling tool (coming from the java world).

Reuse, SCC and VIPM (+ Testing):
I had great fun of getting a good process in place to distribute my reuse code using VIPM. It's paying off in the 2nd project.
I think it's important to design the reuse-code deployment also with your SCC in mind. As I don't use perforce, I can't really say much about the specifics.
One of the places I really would like to have automated testing is my reuse code. As it's used frequently, this should very much be worth the effort.
To that complex I'd also put in a new tool: bug-trackers. You can misuse them as feature trackers as well. Integrate them with the SCC seems important. Sadly I gave up my own use of mantis (lack of time to maintain the IT infrastructure)

Felix

[John Lokanis]

To that complex I'd also put in a new tool: bug-trackers. You can misuse them as feature trackers as well. Integrate them with the SCC seems important.

I like this idea. Our company uses Bugzilla. I will have to look into how that can integrate with Perforce and LabVIEW. I am sure the text coders have already sorted this out so I will have to learn from them.

[jcarmody]

Here't the first thing that crossed my mind as I began reading your post - Things You Should Never Do

[John Lokanis]

Here't the first thing that crossed my mind as I began reading your post - Things You Should Never Do

Good point. But I don't plan to start from scratch. Just re-factor, reorganize and apply best practices. The original code works great for us but only one person really understand it and can maintain it. We started with best intentions but deadlines forced the usual shortcuts. And fixes and features were added over time but not in the most elegant way since they were not architect-ed in the first place. The goal here is to look at what we ended up with, convert it into a set of requirements and then alter the existing code (and rewrite parts as needed) using best practices and a team of developers. The end goal is a code base everyone understand and can help maintain.
Oh, and since the system is working now, we have no deadline, so we can take the appropriate amount of time to get it done right.

[jdunham]

Here't the first thing that crossed my mind as I began reading your post - Things You Should Never Do

Thought of that too!

Good point. But I don't plan to start from scratch. Just re-factor, reorganize and apply best practices.

Oh sure, that's what they all say...



Anyway, as long as you understand the potential for quagmire, you are slightly better prepared to avoid it. Good luck!

Jason

[Tim_S]

We started with best intentions but deadlines forced the usual shortcuts.

Sadly I quite understand this...

[PaulL]

Lots of great ideas in this thread!

I would recommend using project libraries, but be very careful if you want to use packed project libraries. I tried them when they came out last fall and regret that. They don't deliver on performance and they are a maintenance disaster since you can't replace them with a project library (currently one direction only), which you likely will need to do at some point. For us this was resulted in a very costly nightmare. Maybe packed project libraries will be good in a future version, but I think in LabVIEW 2010 they are best avoided. (On the other hand, if you want to help make them better by providing good feedback, that would be great.)

Of course, I highly recommend using objects (and I agree with using by-value objects generally) and design patterns. Also, I think it is important to learn how to create and use simple interfaces, since this can make sections of the code separable, helps to demarcate the limits of their design, and (because you can greatly reduce links between classes) makes building applications more robust and faster.

Good luck with the project!

Paul

[ned]

Integrate the Diff and Merge tools into my SCC system (Perforce).

I really like these tools, especially Diff. Merge is a bit hard to use because it involves a lot of windows (at least 7 - block diagram and front panel for each of the VIs being merged and the result of the merge, plus the list of differences) but it is nice to have. Every once in a while a Perforce command takes an excessively long time to complete - over 30s - but in general it works well. When using Diff, I recommend doing it from within LabVIEW when possible, because it sorts out the search paths for you and you don't get lots of warnings about subVIs not found. This does not happen properly when launching diff from the command line (within P4V for example).

[Daklu]

Hey John, are planning on starting a new project or doing a running refactor on your current code? My sense is that a running refactor is technically more challenging but is less risky overall as it offers continuous incremental improvements on existing production code instead of a starting a new dev effort on an incompatible parallel branch.

So, please let me know what tools and techniques you use that work well in large applications with multiple devs. I want to try to be as state of the art as possible since I doubt I will get another chance to do this again.

My thoughts and experiences, for what it's worth...

Things to avoid
1. Don't try to do a detailed design before you start coding. I have yet to produce a design on paper that didn't need to be altered--sometimes significantly--when implemented. I discovered I spent wasted a lot of time working out details on paper that never got implemented because of a change somewhere else.
2. Don't get bogged down in process. The goal is to develop good software, not develop good processes. Too much process is just as bad as no process. Finding the sweet spot for your group will take some time.
3. Don't be overeager to use inheritance. It is actually a fairly restrictive relationship. Delegating tasks via object composition is a much better general purpose way to reuse code and give an object additional abilities.

Things to do
1. Find a development methodology that works for you. There are lots of competing ideas about the "right" way to develop software: Test Driven Development, Agile, Waterfall, Unified Process, etc. In truth, the "right" way is the way that helps you deliver the product on time. There is no single best methodology. It depends a lot on the corporate culture, types of projects being developed, requirement maleability, timeline, etc. I've taken bits and pieces from several methodologies and combined them into an informal process that works well for me in my current environment. Your process will likely be different.
2. Architect from the top down; design and implement from the bottom up. (This point is debatable--every dev has their favorite way to do things.) When I'm architecting the app I break the requirements into functional components and figure out the api for each of those components. Each component then get broken into sub components if the complexity warrants it. At this time I'm just thinking about the component's public interface; I don't worry about implementation details. When I get to the point where I have components with a manageable size then I start implementing code, assembling the components into larger components as I work my way back up the architecture.
3. Decouple the UI. You mentioned using a web front end. It's possible (and not terribly difficult) to decouple your UI from your functional code in a regular Labview app. I get irritated with programs where a simple UI change propogates down into the lower levels of functional code. That's just poor (IMO) design. Plus, decoupling the UI makes it possible to create unit tests at the application level.
4. Dedicate time for refactoring and simplifying your code. Software development is a very organic process. (At least for me.) When I get to a point where a component's public interface works the way it is supposed to, it's not really done. I can pretty much guarantee the internal workings have some extra fat laying around. Take time to do some lipsuction--it'll improve the long term maintainability of your app.


Addressing some of the specific items you raised...

Design Patterns - One of the best bits of advice I read about design patterns is to "apply design patterns gently." In other words, don't carry around your design pattern solution looking for a problem to solve. Applying them where they are not needed adds a lot of unnecessary complexity. (It's a great way to learn the design pattern, but I have a lot of extra stuff in many of my early applications.)

Unit Testing - Contrary to common perception, unit testing is not free. In fact, it is quite expensive. Not only does it take time for initial development, but you have to go in and fix the unit tests when a design change breaks them. When a test run results in a bunch of failure, chances are at least some of those failures are due to errors in your test cases. Every minute you spend fixing your test cases so they result in a pass is a minute you're not spending improving your code. Don't get me wrong; I think unit testing can be extremely helpful and I'm still trying to figure out how to best use it as part of my dev process. But I think it's a mistake to try and create a comprehensive unit test suite.

Packed Project Libraries - Some users have reported problems with ppls. You might want to wait a year or two before including them as an integral part of a mission critical app.


-Dave

[Omar Mussa]

Unit Testing - Contrary to common perception, unit testing is not free. In fact, it is quite expensive. Not only does it take time for initial development, but you have to go in and fix the unit tests when a design change breaks them. When a test run results in a bunch of failure, chances are at least some of those failures are due to errors in your test cases. Every minute you spend fixing your test cases so they result in a pass is a minute you're not spending improving your code. Don't get me wrong; I think unit testing can be extremely helpful and I'm still trying to figure out how to best use it as part of my dev process. But I think it's a mistake to try and create a comprehensive unit test suite.

I agree with you that unit testing is not free, but am surprised to think it may be a common perception. Also, I would agree that its not valuable to create a comprehensive unit test suite (testing all VIs for all possible inputs). What's most valuable is to identify the core things you care about, and test those things for cases you care about. Unit tests can be a really useful investment if you plan on refactoring your existing code and you want to ensure that you haven't broken any critical functionality (granted that the tests themselves will most likely need to be refactored during the refactoring process). But they do add time to your project, so just make sure that you're testing the stuff that you really care about.

[ShaunR]

This is going to be painful . Not so much that you are re-factoring code (many of us do that all the time). But you are switching paradigms so it's going to be a complete rewrite and you won't be able to keep perfectly good, working and tested code modules (even the worst programs have some).

But the good news is. There will still only be 1 person that understands the code only it won't be the other guy

I usually find one of the hardest initial steps is where to start. I strongly recommend you don't do it all in one go, but rather use an iterative approach. Identify encapsulated functionality (e.g a driver) and rewrite that but maintain the same interface to the rest of the code (to begin with). This way you will be able leverage existing test harnesses and, once that module is complete; still be able to run the program for systems tests. Then move to the next.

At some point you will eventually run out of single nodal points and find that you need to modify the way the modules interact to enable to realise your new architecture. But by that point you have gotten over the initial learning curve and will be confident enough to make much riskier changes whilst still having a functioning application.

The big bonus of approaching it this way is that you can stop at virtually any point if you run into project constraints (run out of time/budget, another project gets higher priority, you contract a serious girlfriend etc) and still have a functioning piece of software that meets the original requirements. You can put on the shelf to complete later but still sell it or move it into production or whatever you do with your software.

Edited by ShaunR, 15 February 2011 - 07:21 PM.

[John Lokanis]

Thanks for all the feedback. So very good points in there to consider.
Regarding PPLs, I currently package my plugins into an LLB using the OpenG builder. I do this to make them distributable. Essentially they become a 'dll' at this point, allowing me to install them on my target machines. The advantage of this is all the VIs used by the plugins are included and name-spaced at the time of the build so that I can never have a conflict with a similarly named file from another LLB or part of the code. The other advantage is I can pull from the same pool of reuse code for all plugins and I only get a snapshot of the VIs at the time of build.
The disadvantage is I am using a non-standard build tool (OpenG builder) and I want to separate my source from OBJ (new LV2010 feature) and I don't think that will work with the OpenG builder. I was hoping the PPLs would give me the same functionality that I get from these LLBs. For those of you who have had issues with PPLs, can you give me more details or reference some CARs so I can see if the bugs will affect me?

Regarding unit testing, my goal was to apply this to my reuse code. And I have a lackey I plan to utilize to write these. :-) The hope is this will make the reuse libraries more robust and ensures they continue to work as expected since everyone will be using them in their projects.

As for re-write vs re-factor, I plan to branch my code and develop the new version by editing the existing code. But, I need to continue to maintain the existing code while this re-factoring takes place, so I will be pulling existing code over from the branch and using it in the new version if it fits. This is not a total rewrite from scratch, all the functionality that exists will remain. Just the methods used to achieve it will be upgraded to more modern and best practices. I don't plan to release any of the new code to production until the re-factor is complete. And just so I am clear, this is not just about changing the code to use OOP but rather about changing the DEV process to project better software in a team environment instead of the lone LabVIEW ranger (me) cranking out code as fast as possible. I want to be a real CLA, not just a CLD on steroids...

And finally, I hope the end result is code that my whole team can understand and maintain, not just me. Yes it hurts job security but it allows me to build an efficient software dev environment that I and my team will benefit from for years.

I think when I am done, this will make a great case study to present at NI Week 2012.

[Daklu]

And I have a lackey I plan to utilize to write these.

Lucky dog. I asked for one for Christmas but alas, Santa wasn't forthcoming.

(Funny how delegation is so useful both in business and in software.)

[John Lokanis]

Any opinion on the NI Unit Test Framework vs the JKI unit test package?
(I have one licence for the NI version but need to pick one before getting everyone licensed)

[jgcode]

Any opinion on the NI Unit Test Framework vs the JKI unit test package?
(I have one licence for the NI version but need to pick one before getting everyone licensed)

Hey John, here are a few chats on the subject plus a couple of others on the subject of testing you may find interesting:

Unit Test Frameworks: NI vs JKI?
What unit test framework for LabVIEW do you use?
Unit Testing Strategies with xUnit?
Your favorite unit test solution for LabVIEW
Unit Testing dynamic dispatch VIs
Testing private class vis

[Daklu]

Any opinion on the NI Unit Test Framework vs the JKI unit test package?

I've used both but am expert in neither. Both appear to be capable of doing the job. My impressions...

NI's UTF is easier to start using right out of the box and doesn't require any coding. It's more of an application than a framework. Select a vi to test, define the inputs, check the outputs, done. You'll probably need to write code for more advanced unit test scenarios though. Unfortunately I have had some issues that pretty much make NI's UTF unusable. (Link) ("Unusable" is probably too strong a word. "Not worth the effort" is better.)

JKI's UTF is a true framework with all the plusses and minuses you might expect. Based on LVOOP, it offers more flexibility in ways that are natural to an OO programmer. On the downside, it's taken me some time to figure out how to effectively design test cases and test suites and there aren't a lot of examples to learn from. (And I'm still not sure I'm doing it "right.") True to form, JKI (usually Omar) has always been quick to respond to questions and helpful.

One thing I've learned is unit testing is not a "I'll just quickly bang out what we need" task. It's a separate development effort all on its own, requiring careful planning and forethought along with it's own brand of expertise. I guess that's why software development houses have dedicated teams for testing. Give yourself (or your lacky) plenty of time to experiment with the framework, learn what kind and when the need to write supporting code, how to best reuse test code, etc.

Final thought: In the past I've naively viewed unit testing a bit like a magic bullet. Turns out... not so much. It's good at catching certain kinds of bugs, such as when an interface is implemented incorrectly. Ultimately it will only catch bugs you're testing for, and if you've thought to test for them chances are you wrote the code correctly in the first place. Unit testing is only one part of a good test process. User-driven scripted tests (a written list of steps for the tester to do) and exploratory testing are valuable techniques too.

[ShaunR]

Final thought: In the past I've naively viewed unit testing a bit like a magic bullet. Turns out... not so much. It's good at catching certain kinds of bugs, such as when an interface is implemented incorrectly. Ultimately it will only catch bugs you're testing for, and if you've thought to test for them chances are you wrote the code correctly in the first place. Unit testing is only one part of a good test process. User-driven scripted tests (a written list of steps for the tester to do) and exploratory testing are valuable techniques too.

Indeed. It is more risk management than a no-bugs solution. The mere fact that you are writing more code (for the purpose of test) means that even your test code will have bugs so you can consider that software, testing software, actually introduces the risk that you will expend effort to find a solution to a non-existent bug in the main code.

Unit testing (white-box and black-box) has it's place. But it is only one method of a number that should be employed. Each to a greater or lesser extent. We mustn't forget systems testing which tests the interaction between modules and the fitness for purpose, rather than that an individual module actually does what it s designed to do.

The main issue for any testing though is that the programmer that created the code under test "should" never be the person who tests, or writes any code that tests it. The programmer will always design a test with an emphasis on what the module is supposed to achieve, to prove that it meets the design criteria - that's his/her remit. Therefore the testing becomes weighted to proving the positive rather than the negative (relying on error guessing alone) whether it's a software testing solution or not. It's the negative (unanticipated) scenarios where the vast proportion of bugs lie and, to expect the programmer to reliably anticipate the exceptions when she/he is inherently focused on the operational aspects, is unrealistic and (probably) the biggest mistake most companies make.

Edited by ShaunR, 16 February 2011 - 03:35 PM.

[PaulL]

I was hoping the PPLs would give me the same functionality that I get from these LLBs. For those of you who have had issues with PPLs, can you give me more details or reference some CARs so I can see if the bugs will affect me?

See these threads for a start:
packed project libraries
Libraries, Packed Libraries, Source Code Distributions, and the End of the Universe

[Daklu]

The main issue for any testing though is that the programmer that created the code under test "should" never be the person who tests, or writes any code that tests it.

I disagree with the statement as it is written, but I suspect we agree on the bigger picture.

I think the developer should write unit tests for the code he has developed. (And I know this is common in some software dev environments.) As you said, it helps verify the 'positive' requirements have been met. Well written unit tests also help communicate the developer's intent to other programmers. The very act of writing tests puts me in a different mind set and helps me discover things I may have missed. Requiring at least a basic set of unit tests keeps the developer honest and can avoid wasting the test team's time on silly oversights.

However, that set of unit tests should not be blindly accepted as the complete set of unit tests that verifies all (unit testable) requirements have been met. When the component is complete and checked in, the test team takes ownership of the developer's unit tests and adds their own to create a complete unit test suite for that component. And of course, in a "good" software development process the developer never has the authority to approve the code for production. I'm pretty sure we agree on that.