Lean Software Engineering

We’re currently in the process of upgrading our unit test practice, so it seems like a good time to bring up an evaluation framework for design verification.

Unit test execution is one form of design verification. Other forms include manual inspection and static analysis, and it’s possible to apply all of these to the goal of verification. Weaker styles of unit testing are mostly about the verification part. Stronger forms of unit testing are also about design specification. But even stronger forms of verification do not require unit testing at all. How is that?

There are some ordering relationships for verification methods. Before-the-fact design specification is stronger than after-the-fact, which is stronger than ad-hoc or none-at-all. Formal is stronger than informal. Static is stronger than dynamic. Automated is stronger than manual. These aspects overlap, but you can still define a capability ladder to reflect methods used in practice.

No matter where you are on the capability ladder, it is reasonable to try to incrementally work your way up. If you are doing ad hoc unit testing today, then you could integrate a static analysis tool like FxCop into your build and also start measuring unit test code coverage. Setting a code coverage goal will tend to drive you towards consistent post hoc (after the fact) unit test development at the least, if not ante hoc (before the fact) unit test development. Besides promoting high code coverage, ante hoc unit test development facilitates design inspection by becoming part of the design specification itself. A code review of a unit test can identify design defects before a single line of production code has even been written.

The effectiveness of a unit test depends partly on the cleverness of its designer, and partly on the suitability of the language that the test is written in. Most programming languages have the basic tools necessary to write good design specifications (e.g. unit tests) because they contain some subset that can express first-order predicate logic, which is the basic language for specification. Some languages contain additional features to express more advanced logics, and some languages contain special constructs precisely for the purpose of embedded specification. The Eiffel programming language is the chief example of the latter. Eiffel embodies an approach to specification and testing called Design by Contract. Design by Contract guides the designer towards organizing a program as a logical proof of its own correctness. Eiffel evaluates these correctness conditions at runtime, which we might consider to be a dynamic contract. A traditional unit test supplies both the test cases and the verification of correctness. Design by contract embeds the verification conditions in the code, so an external test driver only needs to supply test cases while the code verifies itself.

Not all of us have the opportunity to use Eiffel on the job [...why?], but design-by-contract capability can be added to most popular languages in the form of an annotation language, or preprocessor extension. It’s even possible to implement full-blown DBC without any special language features, much in the way that it’s possible to implement object-orientation in a language like C (i.e. with effort, knowledge, discipline, and idiom).

If you are practicing ante hoc unit testing today, whether in the form of a V model or TDD, and you still have concerns about code quality, then moving to design-by-contract is another step up in capability, but not the last step…