(part 4 in the series including 1, 2, 3, 4)
The pipeline model shares a common problem with network model scheduling. Variation in product development activities is simply too hard to control. Pipelines and network models can be made to work by adding a lot of time padding, and indeed, we started to resort to Critical Chain methods to try to make the pipeline work.
We can do something to manage the variation in work orders by controlling coupling and by specifying requirements according to a well defined schema that limits complexity. Those things help, but they are more order-of-magnitude levels of control.
Even if we could control work order size to, say, a factor of two, we’d still have the problem of variation within the workflow. Some requirements may be easy to design, but devilishly difficult to test, perhaps for instrumental reasons. There might be a simple and elegant design that almost matches the requirement and a monstrously complex design that exactly matches it. How long something takes might depend on who gets the assignment. High value problems are uncertain. Uncertainty is risk. Sometimes risk doesn’t go your way.
So a viable solution will have to do its best to control variation while still operating within the reality-based paradigm. Given the challenge, it’s unsurprising that craft production is attractive to practitioners, but I am certain that we can do better.
If you take a horizontal slice across the cumulative flow diagram of a development process, you get the sequence of the workflow for a particular work order. If you take a vertical slice, you get a snapshot of all of the current work-in-process. Curiously, with our pipeline model, these two sequences are the same:
Not only are they the same, they’re always the same. A consequence of our pipeline design is that it strictly limits work-in-process according to the proportions of the workflow. That’s good! That’s what we want.
Is there a way to more directly control work-in-process that allows for more variation than clock synchronization?
WIP and Flow
Imagine that we make a pooled workcell, where each station represents one step in the logical workflow of our hypothetical feature design process.
Each station has a work-in-process limit that corresponds to the time proportion of that state in the workflow. The limit governs the maximum number of work items that can be in that state at any instant. In this case, the analyze and test states have limits of 2. The design state has a limit of 3, and build and deploy have limits of 1.
If a state is below its limit, then it may take possession of a work item from a preceding state upon its completion by the preceding state. If a state is at its limit, it must wait for one of its own items to complete and be pulled into a downstream state before it can pull an item from an upstream state.
Without any further intervention, this design is already far less susceptible to stalls than any of our synchronized designs. If 2 out of 3 work items in a workflow state exceed the control limit on completion time, the rest of the system will continue to function. Even if an entire state locks up, it will take time for the upstream states to back up and the downstream states to starve. In the meantime, management will have ample warning to intervene. Flow will resume naturally once capacity is added or the obstacle is cleared.
Buffers in space vs. time
Most future events in a project plan will occur according to the long-tailed probability distribution that you ought to have tattooed on your other eyeball. The fundamental strategy for managing uncertainty in any cause-and-effect process is the buffer. Buffers in network model schedules take the form of time padding. Thinking about a flow model might lead us to think about other kinds of buffers.
Every buffer is waste of some form or another, but some wastes are easier to control than others, and sometimes exchanging one waste for another is still an improvement to the whole. Schedule buffers represent the waste of delay. With a pull system, we might introduce small inventories to smooth out the flow between adjacent processes and reduce delays due to congestion.
Kanban buffers
A Kanban queue is a small inventory between two processes to create the appearance of instant availability to the downstream process. Think about the stock on the shelf at the grocery store. When the shelf space is empty, this signals the grocer to replenish it. The inventory on the shelf can be very small if the grocer replenishes frequently. In fact, this is where the kanban idea comes from in the first place.
In a production process, a kanban buffer signals an upstream state to produce work only when there is actual demand for it. The productivity of downstream processes regulates the productivity of upstream processes, and this kind of regulation is called a pull system. At the moment a downstream process consumes a component from an upstream process, the upstream process begins production of a replacement. The kanban queue itself has a limit, so that if the queue fills up, the upstream producer will halt. The simple case is a queue limit of one.
The pipeline model forced process states to produce at the same rate and to release work at the exact moment that the next state needed it. By removing the synchronization clock, adjacent states can get out of step. Rate control can still be managed by setting WIP limits. The timing of work transfers can be managed by kanban buffers.
One of the nice things about not being a manufacturing process is that you are not constrained by the limits of physical space. Our kanban container can be infinitely large and occupy no space. Items can enter and exit in any order. In a manufacturing process, the kanban buffer will most likely be a first-in-first-out queue. A development workflow need not observe any such ordering. It might be advantageous to pick the dequeuing order according to local conditions visible to the people on the line.
However, it is still very important to control the size of the buffers. The ideal buffer size is 0, because all buffers are waste. But if a buffer is necessary for synchronization, then the ideal size is one. If the buffer size grows much bigger than one, then you might consider adjusting downward the WIP limit of the upstream state instead. Remember that the one and only goal of the kanban buffer is to create the appearance of instant availability downstream.
Decoupling buffers
There’s another kind of inventory buffer that we might need to keep things flowing. Some resources may have batch sizes greater than one, or have less-than-immediate availability. We don’t want to burden the upstream state with managing such a resource, so we can add a feeding buffer in front of the resource that still looks like a kanban buffer to the upstream state. Goldratt has more to say about this kind of buffer and we may come back to it sometime.
Are we there yet?
We had to take a stroll from the real through the imaginary in order to get back to the real again. But we are, in fact, back to reality, because the ideas described in this post are being applied at Corbis every day. And it works!
A perfect state of flow may be very difficult, or at least uneconomical, to achieve in a robust product development process. But we can get pretty close with a well-tuned kanban pull system. We have managed to combine most of the flexibility of craft production with most of the control of a pipeline. Work-in-process is limited, and cycle time can be managed. Most importantly, it is a highly transparent and repeatable process with all of the right conditions for continuous improvement.
And continuous improvement is really what this is all about.
Jason Yip | 29-Aug-07 at 9:39 pm | Permalink
I still prefer and aim for single piece flow by developing poly-skilled team members but this pretty much matches what we’ve seen for a well-tuned pull system as well.
Corey | 29-Aug-07 at 9:44 pm | Permalink
Outstanding! I would love to hear more about your results.
I2E » Blog Archive » Sesiones Agile Open Space 2009 parte 1 | 02-Nov-09 at 1:12 am | Permalink
[...] A la conclusión que llegue es que kanban sobre todo encaja bien en procesos de mantenimiento o helpdesk, aunque también hay ejemplos en los que utilizan kanban para el desarrollo de software. [...]
Hasan Jaffal | 31-Mar-10 at 4:16 am | Permalink
In Agile projects, it’s a common practice to visualize and share project status using charts or diagrams on boards. While no single document delivers all of the ammunition agile teams need to get the rhythm, this set of visual materials offers an easy framework to help guide software development teams through the various agile cycles.
A variation of the Kanban, an easy way to represent these cards, with some enhancements to this representation and a synchronized use of the concept, ArabiaGIS introduced the Sync Kanban concept.
The Sync Kanban is composed of a board for each project where the team leader or the responsible developer will try to represent the project status and the features progress with pies.
for complete description: http://www.jaftalks.com/Home/S.....Management
Regrads,
Hasan Jaffal
Four Principles of Effective Software Teams « kingsbery.net | 21-Dec-10 at 7:05 am | Permalink
[...] they use to build it, and the process they use in building it. He noted that there are several processes out there, but all effective software processes include the following four [...]
Kanban Nightmares - Zsolt Fabók's Page | 05-Jan-11 at 1:56 am | Permalink
[...] content of the Queue, the prioritization, and keeping the basic rules of Kanban, required attention almost on a hourly [...]
When great communication is required | Rural Source IT | 01-Aug-11 at 8:00 am | Permalink
[...] the team and with the client organization. Many processes in the Agile space fit the bill: Scrum, Kanban, XP, [...]
When great communication is required | Rural Source IT | Communication Advancement | 01-Aug-11 at 9:11 am | Permalink
[...] the team and with the client organization. Many processes in the Agile space fit the bill: Scrum, Kanban, XP, [...]
Yuraisha Chavan | 04-Aug-11 at 1:59 am | Permalink
I would like to see which companies have you implemented this in and what was the outcome? Would like to try this only if it has worked. I dont want to have trial and error.
Less haste, more speed | 14-Oct-11 at 9:53 am | Permalink
[...] these naturally occurring issues (after all every company, every team, is different), I will use Kanban. Kanban was devised by people from a company to deal with a problem of fully carrying through [...]