2009 February | Lean Software Engineering

{ Monthly Archives }

February 2009

Backflows are self-regulating

Some iterative development enthusiasts seem to get very nervous about detailed visual controls, or indeed any kind of suggestion that something predictable happens to new software features while they are under development. I think the fear is that this is (or will become) some kind of prescriptive serialized process that inhibits iteration.

I worry that an important point has been lost in some of the discussion about the WIP-limited workflow systems that we promote. The purpose of the kanban limits is to inhibit forward flows, not backward flows. Pull workflow systems handle design iteration just fine. I would never have gone down this path if they didn’t, but don’t take that as an endorsement that you should iterate with wild abandon. Consider this advice from Ron Jeffries:

I prefer treating the acceptance criteria given at story insertion
time as definitive. When those tests run, the story is done.

If the customer decides that the story needs more work, s/he can
just make a new story with the changes. AND … there is an
opportunity to learn how to think a little more about what is
wanted, and how to express it.

The basic idea is quite simple: we don’t undertake to build anything
until we have an agreement on what constitutes done. We can go ahead
on a very firm notion, or a very soft one … but whatever that
notion is, we make it concrete and live with it.

Your goal is actually to minimize cycle time and minimize variation in normalized cycle time. Let me say that again. The fundamental measure of a Lean process is its cycle time. The reason you should care about all of this Lean business is because that is your goal. In-process iteration has consequences for cycle time, so please be aware.

The reason why backflows are not a problem is that you can’t really give the kanban back until a process is complete, and a process isn’t complete until the person that gave you the kanban decides that it’s complete. Until then, the kanban is still in process. This might be problematic if we used per-station takt time, but we don’t, so it isn’t. Remember, we’re not operating a factory assembly line, and we don’t pretend to.

Even though one kanban might be exchanged for another one at different stages in the workflow, the total number of kanban in the system never changes as a result of a backflow. Some consequences of backflow are different for information system development because of the physical nature of the facilities involved. Brains and computers are typically much more flexible and adaptive than machine tools. Which is not to say that iterative backflows are without consequence. On the contrary, the system will react strongly to their presence by freeing up resources to help them to move forward again. That kind of “swarming behavior” is fully expected and exactly what we want.

2009 02 26

Representation of structure

Structure in the design of systems is usually represented by some form of schematic or block diagram. Block diagrams describe logical relationships between design elements, without getting hung up on physical detail. UML class diagrams and electronic schematics are examples of such abstract structural models of “things that are made out of things.”

Graph diagrams have one big trick for managing scale by collapsing compositional hierarchy into subdiagrams. Most of the time, I don’t care how an integrated circuit works, I’m only interested in the inputs and outputs. I’d rather treat it as a unified thing, even if the components on the inside of the thing are of the same type as the components I connect to the outside.

If our biggest scaling trick is reserved for object composition, then other hierarchical relationships may be described by color, symbolism, or layout. Color can be tricky because of problems with reproduction and perception (colorblindness is common). Symbolism works well for simple type relationships, but it’s difficult to represent taxonomy with nothing but icons. Edge layout is the first and most expressive tool of block diagramming, but layout quickly runs into problems with scale. Block diagrams are not the panacea of design modeling.

Large diagrams can become incomprehensible as they scale beyond a few 10′s of nodes. Most block-diagram notations allow an ad-hoc layout. Imposing rules upon the layout of the diagram can preserve comprehension and communicate additional information about structure. Resistance to orderly layout is itself useful information about structure.

There are some wonderful algorithmic graph layout tools that we can apply to help us make sense of things, but there are also a few simple manual layout tricks we can apply to finding latent structural patterns. It’s useful to know a few tricks for discovering structure when you are given a set of entities, but you don’t yet understand the relationships between them. It’s even more useful if these tricks can be applied at a whiteboard or in a spreadsheet without the need for expensive and/or complicated software tools.

Radial layouts equalize the significance of nodes and emphasize patterns between edges. They also scale up nicely. These can be very useful for discovering relationship patterns in a system that is new to you. You can make a pretty big circular diagram on a whiteboard with sticky notes for nodes and ink for edges.

Linear layouts are easy to draw.

They scale up decently, equalize nodes, and emphasize patterns between edges. They can optionally encode sequence along the node axis, but they don’t have to, and we’ll see why not encoding sequence is also useful.

Linear layouts are especially useful when we make a rule about the direction of the edges.

Any leftward edge that we can’t remove by reordering the nodes is a dependency cycle. Generally, you want to avoid such cycles in your design structure. A useful consequence of such a linear layout is that it can easily be translated into a matrix of relationships between diagram nodes. The reason becomes more apparent when you orient the graph along a diagonal axis:

The graph

can be represented as the matrix

Such an adjacency matrix can be referred to as a design structure matrix, or dependency structure matrix. The tabular form of a design structure matrix makes it a useful tool for understanding the relationships of large interconnected systems. A block diagram with 26 nodes may be simple, or it may be visually unwieldy. The equivalent DSM always has the same layout:

dsm

With our matrix form, we can apply simple row and column reordering to maximize the number of edges on one side of the diagonal. The dependency matrix of a decoupled design can be made to have all of its edges under the diagonal. Any edges above the diagonal that can’t be removed by row/col reordering are dependency cycles.

In a coming article, we will expand on this matrix technique to further understand and manage the structure of large systems. We may also discover something in the patterns of relationships that will help us to manage the work of building and maintaining such systems.