September 2007

Throughput, Lead Time, and Deming

In the last post, I described a quality-of-service goal that we use to manage the performance of our development process. The underlying goal of the process is throughput, by which we mean the rate of delivery of customer-valued work into production. The two major variables that regulate throughput are work-in-process and cycle time. We manage WIP very closely using limits and visual control.

Cycle time is a bit more difficult to manage, because this is where most of the variation manifests. Here, we have a number of capabilities that we can apply to reducing variation, including:

  • analysis practices that accurately identify work items that are too large
  • engineering practices that reduce rework due to preventable causes
  • high visibility into blocking issues and active management of those issues

I say we have a quality-of-service goal, but “goal” might be the wrong word. As Deming said:

Eliminate slogans, exhortations, and targets asking for zero defects or new levels of productivity. Such exhortations only create adversarial relationships, as the bulk of the causes of low quality and low productivity belong to the system and thus lie beyond the power of the work force.

Eliminate numerical goals, numerical quotas, and management by objectives. Substitute leadership.

So, the importance of the lead time SLA is not in its motivational value. What’s important is that its value is under control. Therefore, there are always two phases of setting a numerical limit like our SLA:

1. Bring the system into a state of control. Whatever the control limit is, does the system perform consistently with respect to that limit? If not, identify the sources of variation that cause divergence and find a way to manage them. When it does stabilize…

2. …change something about the system in order to improve its performance. This may throw the system out of control for a while, which will focus your attention back on (1).

If you only randomly make your target, or consistently miss the target, then the problem is that the target is wrong, not the process. Proper management action is to identify a new target and stabilize the performance of the system around it.

Process stability is a necessary condition for continuous improvement. This kind of thinking is where you begin to cross the line from software development into software engineering.

Comments (0)

Print This Post Print This Post

Email This Post Email This Post


Striking a different bargain with the business

Iteration planning is another core element of Scrum. Scrum divides the calendar up into fixed-duration time boxes. Traditionally, this is 4 weeks, though some teams make it shorter. Scrum aims to limit disruption of work-in-process by the business, so the 4 week iteration cycle includes a planning window where the business is allowed (expected, even) to adjust the content of a backlog of pending work requests. Of the prioritized backlog, the team estimates the scope of work they believe they can deliver in the iteration and sets a goal to do that.

The bargain with the business is that the stakeholders have a time to have their say, but otherwise are expected to leave the team alone to complete their goal. In a nutshell, Scrum’s expectation of the business is:

You may not interrupt work in process, and you may not adjust the work plan more frequently than once every n days.

Our teams strike a different bargain with the business:

You may not interrupt work in process, and you may not adjust the work plan less frequently than once every n days.

But…if the business can come in and change priorities any time they like, how can you commit to a goal?

A scope-driven goal is only one kind of goal. We don’t make that kind of goal. Another kind of goal is quality of service, and that is what we do. The team commits to deliver working features, on average, within a time limit that we consider a Service Level Agreement, or SLA. A typical SLA for us would be something on the order of 30 days. The engineering team’s promise is:

When we agree to take on a work request, we intend to deliver it within n days.

According to this criterion, it doesn’t really matter what the work is in the backlog. It only matters that the work has been appropriately sized and has been specified with a reasonable amount of clarity. This also greatly simplifies estimation by setting a single sizing criterion: not too big to meet the SLA. And if you don’t make a scope-driven goal, then there’s no such goal to meet and no need to package any such group of features together for delivery.

So we don’t do that either.

What we do instead is agree to release all of the features that have been completed since the prior release on a periodic interval. Because the amount of work that is in process is limited, that means that the rate of features exiting the process has to be the same as the rate of features entering the process. Some features are bigger, and some features are smaller, but on average, most releases will be of a similar size.

This kind of arrangement means that the planning interval and the release interval don’t have to be the same. Which means that there are no time boxes and no iterations. The planning interval should be often enough to keep the input queue from starving. The release interval should be set to the optimum point between the cost of deploying a new release vs the opportunity cost of carrying finished inventory. If the cost to release is high, you’ll want to release less often. If the cost to release is low, you’ll want to release more often. Cost of release is a worthy target to optimize and the ideal result is deployment on demand.

Comments (3)

Print This Post Print This Post

Email This Post Email This Post


After eleven years, my favorite software methodology book of all time

…is still Steve McConnell’s Rapid Development.

I picked up that title because I was disappointed with the state of practice that I could see around me at Microsoft at the time. That was 1996. There were some good ideas and practices around to be sure, but the heroic cowboy coder mentality was still the dominant culture. Objects and iterations and daily builds and triage were all good, but it didn’t seem to be enough. Looking for answers, I also read the SEI’s The Capability Maturity Model around the same time. That probably threw McConnell into even sharper relief for me. What I liked most about McConnell’s approach that seemed so different from everything else I had read on the subject was the attitude:

Here is a collection of principles and practices that people have found to work for them (or not). Maybe some of them will work for you.

…which could hardly be any different from the SEI’s authoritarian presentation.

Well, I was completely delighted with that approach, and this was massively reinforced by another delightful book that I read in 1996 called Design Patterns. The deep message I got from both books was: study what other people in the wide world have done and think critically and creatively about your situation every day. The ensuing avalanche of Design Patterns clones might indicate that some people got a different message.

But my interpretation sent me off on a great adventure looking for more big ideas out there in the world beyond the vast echo chamber of 1990′s software engineering. Of course, that set me up to be both highly aware of the emerging ideas of Agile development, and seriously offended by the evangelical hype that often accompanied those ideas. In recent years, I’ve seen a lot of worthy ideas get steamrolled by True Believers because they don’t fit into some cult ideology. It is disappointing to me that we find ourselves in such a dogmatic and tribal era in the profession.

I think what I’ve really learned after all these years is that ideas are tools, and the more you know, the better you will be able to solve interesting problems. I can draw another lesson from the flip side of that:

The more anybody tells you that they know the One True Path, the more you should feel inclined to tell them to fuck off.

Rapid Development. As relevant today as it was then, and still the best antidote to mindless bandwagons that I’ve seen yet.

Comments (0)

Print This Post Print This Post

Email This Post Email This Post


The conditions for lean development

We recently explored some of the possibilities for managing software development explicitly around the principles of value, waste, pull, flow, and perfection. Our hypothesis was:

If we can divide requirements into small and similarly-sized pieces, then one-piece flow and continuous improvement are possible.

Our conclusion is that both analysis and evidence show that it is possible. So the next question should be: under what conditions is it possible? This leads us to question the premise:

Under what conditions can we divide requirements into small and similarly-sized pieces?

In my mind, that is really the fundamental question of iterative product development. In order to address that question, we’re going to have to talk about design. Clarifying our understanding of the nature of design will lead us to discover what we need to know about the process of design.

Comments (1)

Print This Post Print This Post

Email This Post Email This Post


Lean scales differently than Agile

One question I like to ask myself regularly is: What would Lean software development look like if Agile development had never happened? That thought helps to clarify some of the more dubious ideas bouncing around the Agile camp.

A Scrum team, like most Agile teams, is meant to be around 5-9 people. A key practice of Scrum, common to most Agile processes, is the daily standup meeting. The daily scrum meeting enumerates each of the team members and asks them for a brief history, a brief plan, and any obstacles. The idea is to regulate productivity and identify special cause variation. The cost of the meeting is controlled by limiting it to 15 minutes (typically). This regular n-way communication helps control some of the problems that Fred Brooks described in The Mythical Man Month. In theory, Scrum looks like a closed-loop control system with two nested feedback controllers (the daily standup, and timeboxed iteration planning). That theoretical description is what got me interested in Scrum in the first place.

One of the development teams I’m working with has got upwards of 40 people all on the same kanban board. The kanban teams I’m working with also do daily standup meetings. A daily standup with 40 people could not possibly finish in 15 minutes if everybody got their turn at status reporting. But they do finish in 15 minutes with great reliability, so they must be doing something different. The kanban process revolves around workflow first, so a natural consequence is that the standup meeting enumerates work-in-process instead of headcount. Only some of the WIP requires any comment (for special causes), and then only one or two people who are involved with that special cause really need to say anything. The board itself continuously broadcasts the productivity status of the system. Upstream resources are always on the hook for delivery to their downstream customers. Productivity lapses show up on the board very quickly and stand out like a sore thumb. Lean thinking is relentlessly value driven.

A consequence of Scrum and the “scrum of scrums” is that such a small span of control will create a deep hierarchy as you scale up. That’s not very lean. A lean organization ought to be pretty flat, and a span of control of 40 is very flat indeed.

Comments (9)

Print This Post Print This Post

Email This Post Email This Post


E-mail It
Socialized through Gregarious 42