Sometimes terminology can help clarify a point. The practice I described here as a review would be more accurately described as an inspection, the purpose of which is defect removal. The benefits you describe are usually conferred by a “walkthrough”. Unfortunately, both these types of meetings get lumped together under the ambiguous category of “review”. My workflows usually contain at least one walkthrough, in addition to an inspection. Efficient walkthroughs might be a good topic for another article!

* the visibility issue mentioned above — people might always catch the most obvious bugs, in the same way that trappers will catch the animals most likely to get trapped (gullible, less cautious, hungrier for the bait, etc.)

* as you say, “The trick to making the population estimate work is the statistical independence of the samples.” If two programmers have very similar approaches to smoking out bugs, the results will be akin to two biologists who tend to place traps in similar areas.

But I want to vigorously disagree with your final paragraph. Your thesis statement: “There really is only one legitimate purpose for a code review: to find defects.” is simply false. Finding defects is ONE benefit of performing code reviews, but it is one of the more minor benefits.

Bigger benefits include: (1) increased communication among team members: they actually talk about the code to each other, forcing people to describe and defend their commonly used idioms and practices, (2) mentoring opportunities, (3) increasing reuse due to awareness of different parts of the code: if you’ve never seen a piece of code, how likely are you to reuse it; conversely if you’ve code reviewed something you’ll likely remember it the next time you need that subroutine, (4) no piece of code ever languishes unmaintainable because only a single developer understands how it works. There are other benefits (mostly centered around communication and understanding), but this gives you the flavor.

I DO agree that beyond a certain point code reviews are no longer improving communication and they can become “expensive and bureaucratic”. The solution is the same one that fixes “carpenter’s thumb”: if it hurts (isn’t benefiting the code or the team), then *stop doing it*!

Paul.

Capture-recapture is a practice associated with SEI Team Software Process. Mature TSP teams have been documented to produce code with quality on a scale of 10 defects per million lines of code. Different practices tend to discover different types of defects, so TSP/PSP employs multiple, complementary practices, with feedback and statistical analysis to systematically eliminate the root causes of defect insertion. Inspections tend to find some kinds of things, static analysis tends to finds other kinds of things, etc. But then there also can be correlations between the results of different practices, and that’s where things get really interesting.

Then your goal is limited to the more manageable problem of:

Make your inspections good at finding the things that inspections are good at finding. And make them efficient.

One of the things we’re trying to do is to simplify some of the methods of TSP/PSP for the Agile audience, and provide a gentler learning curve. Of course, we think that Lean provides some of the right tools for that, and deep down, we know that TSP and Lean have Deming and SPC in common.

http://www.ddj.com/architect/184415470

Do you have any experience with or data about this phenonmenon? Also, have you checked the results of this heuristic against a more in depth inspection to see what its success rate is?