Lean Software Engineering

In the last two posts, we’ve discussed some useful properties of internal workflow queues:

• queue states between processes can provide an early warning of process breakdowns
• local work-in-process limits serve to slow down a malfunctioning workflow and free up resources to fix it
• queues can sometimes be combined to reduce the total work-in-process while still preserving their buffering function

I gave an example of workflow throttling, and suggested there was another configuration of those internal queues that could respond more smoothly and gracefully than the simple, independent queues given in the example.

In order to pull a work item, there has to be a place to pull it from, and there should be some way to distinguish work that is eligible to be pulled from work that is still in process. At the same time, there has to be a place to put completed work when you are done with it. A completion queue serves both these functions.

In this case, we can have up to 3 items in the “specify” state AND we can have up to 3 items waiting for the next state in the workflow. The team can pull new work into “specify” whenever there are fewer than 3 work items in process. If there are already 3 work items in process then the team will have to wait until something is moved into the completion queue. If there is some kind of blockage downstream, first the completion queue will fill up, THEN the specify queue will fill up, THEN the specify process will stall. And when it stalls, it stalls all at once. The flow is either on or off, there’s no middle speed, and it keeps going until it stalls.

In another example, we still have a busy state and a complete state, but the token limit is shared between them. In this case, we can have 4 items in process OR 4 waiting. Or we can have (3 busy + 1 waiting) OR (1 busy + 3 waiting).

In the ideal case of 3 busy and 1 waiting, this queue works just like the first example does. However, if work starts to accumulate in the “complete” state, then the “specify” state will incrementally throttle down. The effective WIP limit for “specify” goes from 4->3->2->1->0 as more items are completed ahead of the rate of downstream intake. So, the process slows before it stops, and it slows much sooner than it would have under the independent queues.

What’s more, even though it operates in the same way in the normal case, it does it with two fewer kanban in the system. Fewer kanban, with gradual throttling and smoother flow, should result in lower lead times.

With this in mind, let’s reconsider our scenario from the previous topic:

	1. Something is going wrong in the design process, but nobody knows it yet.
	2. The specify-complete queue starts to back up, thereby throttling down the WIP limit for specify. A resource is freed as a result, who should now inquire into the cause of the backup, which may only be random variation. The code process continues to complete work and pull from the existing backlog.
	3. Code state begins to starve and specify state throttles down another level. Two more people are released as a result. There’s more than enough free resources now to either fix the problem or shut down the process.
	4. The stall completes by flushing out the specify and code states.

It still takes a while for the system to stall completely. The difference is that it begins stalling immediately, and when it does stall, it stalls with less WIP. For equivalent throughput, this pipeline should operate with fewer kanban and less variation in WIP, and therefore should have smoother flow and shorter lead times. It should respond faster to problems and free up resources earlier to correct those problems.

These shared completion queues might be the most common type of workflow queue. There are a couple of other types that we use, and we’ll take a look at those in a future post.

Michael Kennedy did his readers a real service by calling his excellent book Product Development for the Lean Enterprise instead of the more obvious Lean Product Development. In that book, Kennedy describes the set-based principles of the Toyota Development System, and how they may be applied to any product development business. TDS is a fascinating system, equally as innovative and interesting as the Toyota Production System, but also very different in the mechanism of its operation.

On the other hand, my own journey down the road of lean development started with Donald Reinertsen’s Managing the Design Factory. Don’s description of Lean Product Development is more like the application of lean production principles to product development, breaking the work into small pieces, managing capacity, measuring flow, and delivering value incrementally. It is this view of lean that my friend David Anderson expounded upon in his book Agile Management for Software Engineering.

The difference between TDS and TPS creates a lot of confusion in discussions about Lean Product Development. Does LPD mean product development targeted to lean production systems? Or does it mean the principles of lean production applied to product development? These two definitions have very different consequences.

If it wasn’t already clear, I am decidedly in the camp of lean production principles applied to product development workflows. Even though I think TDS is ingenious, and I wholeheartedly endorse set-based thinking, I do not consider TDS to be Lean Product Development. To me, lean development is a means to an end. That end is evolutionary design, and lean production fits that end in a way that set-based development does not. Which is not to say that there is not a strong evolutionary interpretation of SBD, as large-scale open source development is exactly that. But I’m not really speaking to the open source audience here. I expect my readers are largely of the enterprise variety, building products or IT systems to spec and for hire. And evolution with such finite resources means flow.

Observant readers have probably noticed that I don’t talk much about Mary and Tom Poppendieck here. I feel a little bit bad about that, because I think they deserve credit and respect. I think they have done the software profession a tremendous service by popularizing the relevance of lean principles to software development. But they have also done a couple of things that bother me, including blurring the definition between set-based, time-boxed methods like TDS and continuous workflow methods like TPS. The consequence of this is that people can (and do) weasel out of lean principles by hiding behind the weakest interpretation. By conflating these definitions, the Poppendiecks, perhaps unwittingly, encourage their readers to claim to be lean while avoiding actually doing it. One of the greatest offenses I see again and again is the rationalization of “craftsmanship” under the auspices of Lean.

In contrast, Womack and Jones (and Roos) were not at all ambiguous about what is Lean and what is not. Jim Womack, who coined the expression, “lean production,” to describe the principles behind the Toyota Production System, explicitly defined lean production as neither craft production nor mass production. Not craft production is part of the definition of lean, and he dedicates whole chapters in his books to why craft production is inferior:

Our advice to any company practising “craftsmanship” of this sort in any manufacturing activity, automotive or otherwise, is simple and emphatic: Stamp it out.
- The Machine that Changed the World, The Strange Case of the “Craft” Producers

Now, I realize that development is not manufacturing. But using that as an excuse to rationalize craft methods is overwhelmingly contrary to philosophy of lean. I am not saying that you shouldn’t practice craft development if that’s what you want to do. But claiming that it is lean to do so is either ignorant or outrageously dishonest. Standard Work is not an optional practice. Craftsmanship means doing it your way, Standard Work means doing it our way. If you’re not doing Standard Work, you’re not doing Lean. Period.

Software development:

The system performs function A.

Software engineering:

The system performs function A under operating conditions B with operational performance parameters C with tolerances within the probability distribution D and reliability within the probability distribution E and we are legally responsible if it doesn’t.

As you can probably imagine, one of these problems is harder than the other. My interest is in explaining why it is still possible and even desirable to address the second problem with evolutionary design.

Agile methods can explain how to manage evolutionary design for the first problem. They have so little to say about the second problem that they almost appear to deny its existence. Software engineering methods are largely concerned with the second problem, but they usually apply a simple-minded mass production mentality to management. I guess “serious” software engineering researchers haven’t considered management to be a sufficiently interesting problem.

Maybe I’m a dreamer. I want both. In fact, I don’t think Software Engineering can be made to work without evolution. The best that a phase-gate system can hope to offer is to solve yesterday’s problem (i.e. the wrong problem) with great precision. I want the optimum solution to today’s problem, today.

…is the class invariant.

Class invariants and contracts are not about testing. They are about reliability. The purpose of a class invariant is to minimize the state space of the system, both at design time and at run time. A class invariant is an extension to your type system. You will get the most power from them if you think about them and use them in that way. I love dynamic languages, but the people who bitch the most about type systems probably don’t understand how to use them correctly. The compiler is your friend. Static analysis is your friend. Class invariants make types smarter.

Reliable systems fail at the earliest opportunity. Sounds counterintuitive?

The loom stopped instantly if any one of the warp or weft threads broke. Because a device that could distinguish between normal and abnormal conditions was built into the machine, defective products were not produced.

- Taiichi Ohno