February 2008

Better estimates with Wideband Delphi

Dates and deadlines are an essential human element of project management. People work better if they have challenging but realistic schedules to work against.

The trick is “challenging but realistic.” In software, we know there is wide variation in our estimates, because we are almost always creating something unique (if it’s been done before, we just copy the bits). And we have a systemic underestimation bias, because there are lots of ways to cut scope or cut corners in software, and the intangible nature of it all makes anything seem possible.

Unfortunately, when schedules are no longer realistic, it will quickly destroy a project: causing cynicism, demotivation, short-cuts, bad decisions, unwillingness to respond to new information, loss of honesty and trust, and other problems which will fester. We could avoid these pitfalls if only we could estimate better. There are good books on this, including McConnell’s Software Estimation: Demystifying the Black Art and Wiegers’ Practical Project Initiation.

Out of all the techniques covered in those books, one widely-used technique is particularly effective for those critical early estimates of large, not-yet-well-understood projects. Estimates upon which we base our go/no-go decisions and early expectation-setting for upper management and customers.

It’s called Wideband Delphi, and here is a simple spreadsheet template and guide for the Wideband Delphi technique. Take a look, and let us know if this is useful to you and your groups.

Comments (5)

Print This Post Print This Post

Email This Post Email This Post


Not all activities are best handled by generalists

This is part 6 of 10 Pitfalls of Agility on Large Projects. In part 5, we talked about how hundreds of people can’t check into “main” every day, and what to do about it.

When it comes to generalists vs. specialists, there are huge benefits to maximizing our use of generalists for creative/design engineering — in contrast to manufacturing activities, where specialization is always the way to go.

But there is a limit on how far you can take generalists, and there is a pro-specialist argument for design, too — as Corey lays out his case in his thoughtful one-piece flow post series.

I may not buy the broad knocks against “craft production” in that post; or that systems designed around generalists are unlikely to qualify as software engineering. But there is clearly a cross-over point where adding specialists of various types starts becoming a win, and eventually a big win. A good way to analyze where specialization can help is being thoughtful about a lean production flow.

Hand-offs, error, and waste: first, the case against specialization

Specialization in design suffers from a catch-22 problem: Design specifications are not truly complete until they are rendered at full detail (in software, that means coded). But incomplete specification causes errors during hand-offs due to misunderstandings and disagreements. Those misunderstandings and the time to resolve them are magnified to a surprising extent by hand-offs between specialists who don’t share a common foundation of knowledge or perspective.

Boehm Sprial

And because design is a knowledge discovery and creation process, we can’t actually specify a design fully up-front (if we could, it would mean our design isn’t forging much new ground). The most effective solution is to spiral down into a design ala Boehm: breadth-first circling around all aspects of a project (these aspects are potential specialized roles) refining the design from high-level to low-level details, with depth-first spikes into areas with more unknowns or risk.

Value Stream Map

Every specialist involved in the design requires an additional step in our value-stream map, repeated for each loop we take around the spiral (with some opportunity for parallel processing). If we think about our value stream map for this process, we realize how this can explode our cycle time, including waiting time for people to get back with feedback.

And every specialist is a resource that now needs to be balanced, a potential bottleneck. In a generalist-dominated system, it is much easier to attack bottlenecks by shifting resources.

Perhaps most importantly, as we add specialists with different backgrounds, we have less project knowledge that can be implicit or informal: to reduce misunderstanding, we have to capture more of that knowledge explicitly on paper or via longer, bigger meetings. Achieving consensus on decisions takes much longer. None of this directly adds any value for the customer.

So when does specialization start making sense?

  1. When we can get the benefits of both generalization and specialization. We do this by hiring generalists, but having them rotate into specialized roles (like test, project management, architecture, etc.) for the duration of a project or two. This is an excellent strategy to gain the benefits of focus through specialization, while retaining a flexible, low-overhead organization. TSP is a well-known adopter of this strategy. This is also a variation of the recommendation here of division of labor in lean software development workflows.
  2. As our projects grow, we can bring in specialists that help the project without gating the inner design loop.
    • System test is one of the first and most common of these opportunities. Because at this point the system has been fully specified (that is, the implementation is “complete” if not error-free) there is a clear, specialized role to fulfill without impacting the design loop.
    • Back-end value adding steps like localization (assuming the core group of generalists knows how to create a localizable product).
    • Specialized roles which are orthogonal to the design loop, like project management. Note that some companies make the mistake of defining hybrid roles like “Program Manager” that encompass project management, requirements analysis, high level design, etc. (Microsoft being a poster child). But this causes all the problems of specialists in the design loop.
  3. Finally, specialization is unavoidable when we can no longer find or afford generalists who can handle the typical roles in your project (interfacing with the customer, design, implementation, effective testing, a degree of self-management, etc.). Some organizations (Google would be a poster child) scale to hundreds or thousands of people without having their hiring and organizational structure over-specialize along those lines. But we’re not all Google. When our generalists have had trouble over time covering some aspect of the project, it’s time to accept the other costs to get the benefits of specialists focused on those problems.

Comments (3)

Print This Post Print This Post

Email This Post Email This Post


The spectrum of novelty

I have already solved this problem.

I have solved this type of problem before.

My team has solved this type of problem before.

My company has solved this type of problem.

My profession or industry has solved this type of problem.

Science has solved this type of problem.

Science has not yet solved this type of problem.

The right kind of design process to use depends on the novelty of the problem at hand. The first case of “I have already solved…” is a manufacturing process, where you simply have to replicate an existing design. At the other end is pure research, which happens on an uncertain schedule. Most software development problems lie somewhere in between. A good development process is tuned to handle the typical novelty of problems given to the team. And not what you wish that it were, but what it is.

Comments (0)

Print This Post Print This Post

Email This Post Email This Post


Division of Labor in Lean Software Development Workflows

Imagine we have team of three people, each working as generalists in an agile-style process. They are all qualified and competent workers, and correctly execute the process that they intend to follow. They break their work up into customer-valued features that each take a few days to complete through integration.

One developer is a true generalist. It takes her a couple of days to produce a testable functional specification and high-level design. It takes her a couple of days to produce a detailed design and working code. And it takes her a couple of days to verify and validate everything, from code correctness to functional acceptance.

Another developer is basically competent at all of these things, but he is a more stereotypically geeky programmer who can crank out high-quality code for most product features in a day, on average. It takes him a bit longer than the others to do the customer-facing part, usually about 4 days for analysis and high-level design. He’s also a bit slower with the validation, because again, if it ain’t writing code, he’s just not that excited about it. And for all the time he spends on specs, they are still mediocre, which results in rework in spite of his good coding skills.

The third developer, by contrast, has a sharp eye for design and is very friendly and sympathetic to the business and the customers. He knows the business so well that most of his specs only take a day to write. He’s a competent coder, but a bit old-school in his style and it takes him a bit longer with the current technology. Plus, his heart isn’t quite in it the way it used to be. It takes him three days to develop good code that everybody will feel comfortable with. On the other hand, since his specs are so clean and thorough, and he has a good rapport with the business, the testing usually goes very smoothly in about two days, also (tied for) the best on the team.

The team, on average, produces features with a cycle time of 6.67 days per feature. Overall, each team member produces at a similar rate.

2d + 2d + 2d = 6d
4d + 1d + 3d = 8d
1d + 3d + 2d = 6d
               20 days / 3 features = 6.67 days/feature

It is a one-piece flow (per developer), and everybody is always busy with his or her feature. Nobody ever has to wait to start a new feature. Other than the personal slack built into the task times, capacity utilization is high.

But imagine if this team of generalists were allowed to focus only on the skills that they were best at:

1d + 1d + 2d = 4d
1d + 1d + 2d = 4d
1d + 1d + 2d = 4d
               12 days / 3 features = 4 days/feature

Same people, same features, 40% improvement in productivity…

…if only it were that simple, because there is also a cost here. If they organize themselves as a pipeline, then that pipeline becomes subject to the Theory of Constraints. If they apply Drum-Buffer-Rope, then the testing task sets the pace at 2 days. That means the total cycle time per feature is:

2d + 2d + 2d = 6d

…still an improvement over the generalists, but only by 10% (!). On the other hand, capacity utilization is now low, because two people now spend half of their time idle, waiting for the drum. Since they are the same people who were cross-trained enough to work as generalists in the first example, is there anything that they can do to speed up the testing process which has been otherwise unimproved? Surely the answer must be yes.

Suppose each of the first two developers spends an extra half of a day doing additional work to optimize the testing process, so that testing only takes 1.5 days to complete instead of two. Introducing a pipeline might also introduce a new communication cost, but imagine that the extra half-day spent by each of the first two developers is in collaboration on the two features they have in process, both communicating and optimizing testability.

The total labor expended is now 4.5 days per feature, but all of the idle time has been stripped out, so that the total cycle time per feature is also 4.5 days. That is a real 33% improvement in throughput. Same people, same features, same skills, 33% faster. It is only an example, but is it not a realistic example?

What about training?

An enthusiastic and observant Agilist might, by this point, object that we could improve the productivity of the team in the first example by improving their skills with training. That is indeed true. We could provide such training, and it may very well yield improvement.

We could also provide training to the team in the second example, which might also yield improvement. What sort of improvement might we expect in each example?

The generalist model suggests that we help each team member improve their weak skills to bring them up to par with the rest of the team. In any model, there is something to be said for cross-training because it facilitates communication and allows the business to adapt to change. But investing in training to overcome weaknesses is a classic management mistake.

In First, Break All the Rules, Buckingham and Coffman make a compelling and well-researched case that the best return on investment in training comes from enhancing a worker’s strengths, rather than overcoming his weaknesses. The geeky coder may have a lack of charm or graphic design sensibility that no amount of training can ever overcome, but picking up a new coding technique or web application framework might pay immediate dividends.

The more specialized team has another built-in advantage in training because they simply get more practice with their currently deployed skills. The analyst gets more training on analysis, which he already has an aptitude for *and* then gets to spend all of his time practicing.

Back to our first team, imagine that we invest in generalist training, so that our

2 + 2 + 2 = 6
1 + 3 + 2 = 6
4 + 1 + 3 = 8


2 + 2 + 1 = 5
1 + 2 + 2 = 5
3 + 1 + 3 = 7

…for an improved average cycle time of (5+5+7)/3 = 5.67 days per feature. A generous result in training, for a significant outcome.

What about our “invest in strengths” scenario for the first team?

2   + 2   + 1 = 5
0.5 + 3   + 2 = 5.5
4   + 0.5 + 3 = 7.5

…not as good! Even with a very generous 50% improvement for all, we only get 6 days per feature. But what about the specialized team? If our original:

1 + 1 + 2 = 4


0.5 + 0.5 + 1 = 2

…and then we add back some collaboration overhead:

1 + 1 + 1 = 3

…well, then we are just smoking the generalist team. These are contrived examples, but they should still illustrate some of the advantages that go to small lean teams over small craft teams. The most effective teams have complementary skills and personalities, not homogeneous ones. Otherwise, why organize into teams at all?

Still not that simple? Enter the kanban

A problem with both examples is that they deal with average features. But nobody ever actually works on an average feature. They work on real features that can be averaged over time. Those features have variation, and sometimes a lot of it.

For this reason (and others), we don’t organize our workflow by role. We organize it by task or process, and let team members apply themselves to the workflow in the most efficient manner. They may even hand off the work at different transitions depending on the feature or the state of the pipeline. Such a soft division of labor preserves the efficiency advantage of each worker, while also allowing for variation and changes in circumstance. What matters most is that the work is done in the right order by the best resource available at the time, not who does what.

Pooling work-in-process according to the kind of asynchronous kanban system we’ve been discussing smooths out the flow of variable-duration work items, so that some of the variation in cycle times between processes is traded off for variation in inventory. Such a pooling strategy works better with more people than our examples, and also more people than the current common practice for agile teams. For a pipelined kanban system, we think that about 20 people is the sweet spot.

Comments (10)

Print This Post Print This Post

Email This Post Email This Post


There are no a priori best practices

There are only the practices you are using now.

And practices that are better than the ones you are using now.

Comments (7)

Print This Post Print This Post

Email This Post Email This Post


E-mail It
Socialized through Gregarious 42