Lean Software Engineering

(part 2 in the series including 1, 2, 3, 4)

In our ideal case, I set out a goal to partition incoming requirements into similarly-sized pieces and run them through a one piece flow process through to integration and deployment. I concluded with the question:

a team of experts will somehow have to work together and coordinate with one another to make all of this happen without tripping over one another’s feet…how, exactly are we going to do that?

There are a few ways to go about this, so let’s start with something simple, see where that falls short, and work our way up to a better solution. Let us also assume that there is a common workflow that will be applied to each work request.

Craft Production

Imagine a small team of generalists. The work-in-process limit is made equal to the size of the team. As new work orders appear in the incoming queue, each idle team member will take ownership of one work order until there are no pending work orders or no idle team members. Each assigned team member applies the workflow to one requirement, continuously, until the requirement is integrated and deployed. A team member may only own one work order at a time. Upon completion, the team member then returns to the idle pool for reassignment.

Pro:

• incoming work-in-process is controlled
• defined workflow is possible
• pull is possible
• one piece flow is possible
• variation in the size of work orders is buffered

Con:

• generalists are slower than specialists
• competent generalists are rare
• knowledge transfer is hindered
• standardized work is hindered
• quality is inconsistent
• accountability is limited
• process improvement feedback is limited

There are pros and cons to this way of working, but the bottom line is that craft production is not lean production. Software development under this model is unlikely to qualify as software engineering.

Feature Crew

If it is not possible to assemble enough generalists to implement effective craft production, then perhaps small multidisciplinary teams could work. A Feature Crew contains a small set of workers with complementary skills. Work orders are defined in such a way to engage the team for a few days or a few weeks. The work-in-process limit is made equal to the number of teams available. As new work orders appear in the incoming queue, each idle feature team will take ownership of one work order until there are no pending work orders or no idle teams. The assigned feature team applies the workflow to the requirement, continuously, until the requirement is integrated and deployed. The feature team may only own one requirement at a time. Upon completion, the feature team then returns to the idle pool to be reassigned or recombined.

Pro:

• incoming work-in-process is controlled
• defined workflow is possible
• pull is possible
• one piece flow is possible
• variation in the size of work orders is buffered
• division of labor

Con:

• specialists are hoarded
• resources are underutilized
• knowledge diffuses slowly
• standardized work is limited
• quality is inconsistent
• process improvement feedback is limited

Feature crews have most of the advantages of solitary craft production, but fewer disadvantages. Within the feature team, people will self-organize around the workflow. Resource utilization will be lower than the simple craft mode, but the productivity loss will be offset by specialization and division of labor. Knowledge transfer will be greater if teams are periodically recombined.

These craft production approaches are essentially the domain of Agile development. While we could continue to explore the possibilities by evaluating various Agile methods, we will still be in the domain of craft production, so let’s back up and try a different approach altogether.

Synchronized Workflow

The principle, Schedule is Orthogonal to Workflow, suggests that there are two fundamental approaches to partitioning work: by schedule or by workflow. Traditional project management schedules large work orders and aligns resources by workflow. Agile/craft methods schedule small work orders and align resources by schedule. Why doesn’t anybody try to schedule small work orders and align resources by workflow? I don’t know…so let’s try it!

Imagine that you have a small cross functional team. There is one specialist for each step in the workflow, a classic division of labor. Work in process is limited to the number of steps in the workflow, and hence to the number or team members. The work is synchronized according to a clock. In other words, this is a discrete pipeline. At the first clock tick, a work order is pulled from a queue and placed in the first processing state. At the second clock tick, the first work order is moved to the second processing state and a new work order is set in process. Once the pipeline is full, each clock tick completes one work order, begins a new work order, and advances work-in-process to the next step.

In order for this to work efficiently, there must be limited variation in the size of work orders, limited variation between the durations of different processing states to complete any work order, and limited variation within any specific processing state. None of those conditions will be probable in any kind of creative process. Synchronization will only be possible if you set the clock interval proportional to the worst case duration of any of the processing states. That would give you control at the cost of enormous waste of delay.

Pro:

• incoming work-in-process is controlled
• standardized work is possible
• pull is possible
• division of labor
• knowledge transfer
• transparency and accountability
• process improvement feedback

Con:

• variation in the size of incoming work orders is difficult to control
• process variation will cause delay and/or inventory between workflow steps
• resources will thrash between underutilization and overutilization
• does not accommodate design iteration
• pipeline stalls disrupt flow

The fundamental problem with synchronized workflow is that there is too much variation in product development work to be strictly synchronized. But the idea of aligning resources by workflow has much more potential than this simplistic case. We’ll explore those ideas further in the next post.