The essence of Lean software development

Derivation of the essential development workflow by induction

1.  Imagine that you have produced a high-integrity, high-capability design that is suitable for your problem domain.  The design is of high quality in every way that matters to you:  reliability, usability, security, response time, power consumption, whatever.  You have reliable metrics to assess all of your relevant quality attributes.  It does not matter how you created this design. It could be a rigid phase/gate process, it could be a million monkeys.  It only matters that you are in possession of such a design.

2.  Starting with your finished and delivered system, add one feature or improve one performance specification in a way that preserves all of the quality attributes that you could measure in the original system, plus any new attributes that are relevant to your change. Deploy the newly enhanced system and validate.  This is your essential development workflow.

3.  Repeat step 1, but with one fewer initial feature, or one relaxed performance attribute.

Any sequential development workflow can be pipelined

1.  Take all of the steps from your essential development workflow and arrange them in dependency order.  Work serially through the steps until a result is produced.  If the result is not satisfactory, then repeat the process and apply what you have learned until a satisfactory result is produced.  It does not matter if this process is not efficient.

2.  Allocate sufficient resources so that two such design increments, following the essential workflow, can execute in parallel without overlapping resources.  Create a build/integration process that allows each feature to develop in isolation and integrate through to deployment and validation.  Integrations will necessarily be serialized, creating a pipeline.  This is your essential development process.

3.  Map the value stream of your development process.  Add sufficient capacity to each pipeline to realize flow. Add or remove stages to the pipeline to reduce backflows or otherwise reduce cycle time.  Substitute new practices or stages as the situation demands and capability allows.  Find bottlenecks and relieve them.  Share resources between pipelines to improve utilization.  Add or remove pipelines as necessary to match demand.

4.  Repeat step 3, forever.

Comments (4)

Print This Post Print This Post

Email This Post Email This Post


A swimlane for ad-hoc workflow

Swimlanes are used to track parallel work streams within a common resource.  The most typical use might be grouping composite work items with a parent.  Another typical use is an expedited lane for emergency work items that can skip queues and preempt other work.  Swimlanes generally indicate some kind of branching.  That can be either work item branching or workflow branching.

Kanban systems can be used when the work generally follows some kind of common workflow.  For many software development projects, this is an easy constraint.  Most software involves some kind of problem definition, some kind of design activity, and some kind of verification.  Most software development also involves the use of a limited set of technology applied to a limited set of systems within a limited problem domain, so that most of the work done falls into a few general types.

Most is not all, however, and sometimes work will appear that doesn’t fit neatly into any well-defined type.  Or maybe it does have a type that we haven’t defined yet.  If the work is non-value-added, we can chalk it up to overhead.  If the work is value-added, we will want to track it like all of the other value-added work.  A useful trick for managing that kind of work is to reserve a swimlane for ad-hoc workflow:


If you find yourself using the ad-hoc swimlane frequently, you might want to map the value stream of some of the work items to see if you can discover some latent or emerging workflow.

Comments (2)

Print This Post Print This Post

Email This Post Email This Post


CONWIP systems

Part 2 of Patterns of Software Engineering Workflow

The simplest kind of kanban system is the CONWIP system, for CONstant Work In Process.  The simplest kind of CONWIP system is no more than our fundamental kanban element:


The simplest CONWIP cardwall is a classic Agile cardwall with a limit on work-in-process:


An equally intuitive interpretation of CONWIP defines capacity simply as the number of people available to work, so that each person is the kanban:


CONWIP is a rule about work items, not a rule about workflow.  We are free to define any workflow we like as long as we observe the global limit.  This can be a helpful approach when we want to observe the flow of work, but expect a lot cycling between states, perhaps in an exploratory design mode:


The earliest Scrumban design was just such a CONWIP workflow, which we can represent directly in a simple cardwall.  Here we have no limits on any specific column, but the total number of work items is limited by the yellow kanban cards, which are returned to the “Free” box when the task they contained is complete:


Breaking out a workflow implies a sense of sequence.  If there is no real sequence, we could define a global “busy” state and reduce specific activities to a checklist.  Exposing the workflow to visual control might suggest a need to level resource utilization or that we value reducing backflows as an improvement goal.  Pooling the WIP limit suggests that the visual control is enough feedback to help the team achieve leveling.  My personal preference is to work in this way where possible.

If a person can be the kanban, then how about a pair? If we combine pairing and kanban with a workflow or checklist, then we get something like Arlo Belshee’s Naked Planning:

I think the most interesting kind of CONWIP system is the Bucket Brigade. Everybody is allowed one card at a time, which can either be moving upstream or downstream. People can either pair (at the cost of some queueing) or work solo as much as they prefer. Constant WIP, zero queueing, full utilization, soft specialization, balanced workflow…what’s not to love?

The model can be edited and simulated in PIPE2.

If we can assign tasks and limit work by pairs, then why not do this with whole teams? That is essentially the approach of Microsoft’s Feature Crews with their quality gates.

Some value streams are too long or too complex to effectively manage with a single WIP limit.  Many development teams exist within some larger organization that requires coordination between teams and competition for resources.

Where do features come from?  Smaller shops may interact directly with the customer, but larger shops may have a more complex process for dealing with a large number of customers or with highly sensitive customers. If we peer inside the black box of the Product Owner, we might discover a whole team of people working to understand customers and define requirements: business analysts, product managers, usability researchers, product designers. Work-in-process on the business analysis side can just as easily go off the rails as development work.  Have you ever seen an epic requirements specification or a bottomless product backlog and wondered where it came from?  Your product owner might represent another group of people who feel pressure to produce and look busy.  Value stream thinking encourages us to take an interest in what those people are up to and why.

Where do features go after we’ve built them? A large enterprise may have complex deployment requirements that involve integrating code into a manufacturing process or provisioning a datacenter. This work probably involves a different team than the development team, but they are still part of the value stream and their throughput affects everybody.  Operations teams often have to deal with long lead times and different natural batch sizes than the development teams that feed them.  Each group can benefit from understanding the status and availability of the other.

A small team may be able to self-regulate with visual control, but a long value stream may need more explicit control. Kanban gives us an easy solution by chaining together pooled segments. Within each segment, we manage by visual control, but between segments, we manage by kanban:


CONWIP systems allow us to regulate team workflow with a lighter touch.  Pooling kanban across closely related functions and zooming out by one level of scale makes it easier to think about using kanban to manage large systems.

Comments (3)

Print This Post Print This Post

Email This Post Email This Post


Guest articles at Shaping Software

J.D. Meier is hosting a couple of introductory articles on Lean software development on his Shaping Software blog.  I admire J.D. not only for his prodigious writing and encyclopedic knowledge, but also for his insatiable curiosity and practical wisdom.  A true methodologist.

Introduction to Lean Software Development

Patterns and Practices of Lean Software Development

Comments (2)

Print This Post Print This Post

Email This Post Email This Post


Daily nuggets

Domain knowledge (problem and solution) is the raw material from which information systems are constructed–the source of the value stream.  Well, that and computers.

The software development pull system representations we have seen so far are not the end.  There will be a good deal more evolution of that line of thinking.

Comments (0)

Print This Post Print This Post

Email This Post Email This Post


E-mail It
Socialized through Gregarious 42