Finding the balance between harnessing creativity and value added standards is a delicate balancing act requiring skill, understanding, and feedback; tilting too far in either direction results in waste.

The mistake too many organizations make is drinking the Kool-Aid on the latest craze set of practices without taking the time to understand the underlying principles that made them work in the contexts in which they emerged. Adaptations are almost always needed. Pair programming, for example, can pay off big in terms of productivity and quality however, slavishly followed as a standard it can lead to waste for some programming activities.

For lean to be truly successful in an organization, it’s leaders need to grasp the underlying principles so that they can be applied to maximize value in that organization’s context (i.e. market goals, corporate culture, customers, geographic area, work-force distribution, types of systems, competition, etc.)

Even fixed duration iterations are not sacrosanct; some of the more lean development organizations are evolving toward a more constant single-piece feature flow model of development.

As a complete aside - the real breakthroughs in productivity possible in lean development I think will come more from optimizing the interactions between the different parts of the organization; sales, product management, marketing, customer support, development, delivery, IT, etc. Yeah, there’ll always be more to optimize in the dev house - but restructuring interactions to promote pull and flow throughout the organization will provide opportunities for huge pay offs. None of that’s going to happen without a fundamental understanding of lean.

I agree that it would be possible to get carried away with standardizing development work and suffocating creativity, but creative engineering does not have to mean poetic anarchy.

Not all parts of the development process are equally creative, even for the most novel problem. And not all software development problems are especially novel. Most of the work that most developers do most of the time is significantly less novel than designing a new algorithm or a new molecule. I tried to say as much with my spectrum of novelty post.

Even then, methods like TRIZ or Dijkstra’s method of Correctness by Construction aim to make even inventive work somewhat regular and predictable. Probabilistic methods like Pugh Concept Selection or Toyota’s set-based development reduce variation through resource buffering, allowing wide variation for individuals, but narrow variation for the process as a whole. Agile methods like user stories, planning poker, pair programming, and time-boxed iteration are also focused on reducing or controlling process variation.

One of the things we try to focus on is distinguishing things that have intrinsic variation from things that shouldn’t. For variation that is truly necessary, we do not seek to eliminate it, we only seek to control it. The variation that we seek to eliminate is only of the unnecessary variety.

There are all sorts of things that could be subject to standards that shouldn’t constrain meaningful creativity. Coding conventions, design rules, check-in procedures, test coverage targets, inspection procedures, and so on, are not the places to look for creative license. Further, the Lean idea of standard work is not the blind following of a rule for once and for all. Standards are the foundation of Kaizen, or continuous improvement, and you can’t have one without the other. A standard isn’t the best way to do something forever. It is only the best way that the team knows how to do something today.

We don’t see Lean as being exemplary of manufacturing. Lean and Kanban, as principles, apply to any work that has a natural sequence. And as you suggest, processor scheduling is indeed a good place to look for inspiration in general scheduling problems, especially when you know what it is that you are optimizing for. In our case that is almost always throughput.

The SEI’s Team Software Process and Personal Software Process are very helpful in understanding the role that kaizen-style process standards can play in software development.