~ OR ~

Adventures in workflow diagramming

We have talked a bit about workflow here.  This has mostly been analysis of existing, well-known workflow types.  We’ve also talked quite a bit about the pragmatics of operating those workflows as pull systems.  We have spent less time trying to describe the organization of workflows designed specifically for pull.  Understanding something about how to organize workflow might help us to understand something about how to organize people, which is ultimately the hard part.

Suppose I am building a product and I ask you to build a component for me so that I may complete the product. That simple transaction already creates three phases:  1) I start making the product;  2) You make the component while I continue working;  3) I receive the finished component and integrate it into the product. We can represent such a process by a simple branch/merge workflow.

If we know we have a parent/child relationship between the branches, we might simplify the diagram by collapsing some of the parent process:

…which is a convenient representation, for reasons that we’ll see shortly.

We can easily transform our branching workflow into a pull system by adding kanban, flowing in the opposite direction to the product.

There is more than one way to break down a process into steps.  We might decompose a product into components, or we might decompose a workflow into processes. In practice, we may have a blend of these.  An evolutionary design process should have more of a workflow orientation, as the “product” should be a small cross-cutting enhancement to the previous instance of the system.  For a design product, the value added is almost entirely in the application of the knowledge and judgement of the participants.  A design workflow should be somewhat consistent between each design iteration, while the structural decomposition won’t be consistent at all. For the purpose of example, I’m going to assume an evolutionary design process.

Remember that the states in our workflow map to processes and not people. People will organize themselves to operate the processes according to their judgement.  There may be some correspondence between people and process, but that is a another concern.  There will be constraints on reasonable roles for team members.  You should be cautious about multitasking.  If you are serializing activities in the service of “one-man flow” then you are off in the weeds.

That said, let’s explore an example of how workflow might be structured for evolutionary design.

Ideal design of a pull development workflow

1. A customer comes to the team with wallet in hand and says, “I want a new feature.”  Now, the customer might say she wants a new feature, but what she really means is that she has a problem to solve and she’s willing to pay for a solution.  Maybe that solution will be the deployment of a new feature, and maybe not.  Maybe we can provide her with an off-the-shelf product that already does what she wants.  Maybe we can supply her with instruction on how to use existing features to solve her problem.  We don’t know yet, and neither does she, so we should start by working with her to describe the problem to be solved.  This description of the problem to be solved is a requirements specification, and it’s the first thing we’ll do in our workflow.

For the sake of example, let’s assume that the solution really is to develop a new feature.  In order to satisfy the requirement, we will have to build and deploy something, which will require time and resources.  So the next thing we do is…

2. Request a release package that will eventually include whatever we need to build and deploy a new feature.

Organizing our workflow in this way makes a statement about the meaning of done.  We’re not done when we throw some bits over the wall.  We’re done when we’ve satisfied the requirements and delivered value to the customer.  When reading the diagram, it’s important to understand that we don’t release the feature until we have a consensus that the requirement will be satisfied.  The diagram says that we can’t close the original request until everything is really done and the customer signs off and goes away.

Our new release package can’t be completed until it contains something worth releasing.  The thing that the package contains is made of bits, and the way we make bits is by producing source code,  so the next thing we do is…

3. Request the provision of development resources.  We can’t deliver on our promise to develop the feature now if we don’t have the resources to develop it.  In order to release the feature, we will need a development team and whatever tools they need to create the feature.

But the customer didn’t come looking for any random feature.  She wanted a particular feature, which means that we have to…

4. Produce a design that actually solves the customer’s problem.

In order to design a feature, we have to know what the requirements are.  But we already know that!  Which means something interesting happens now.  Design is the deepest state of our development process.  Even though it seems that we are not done yet, in one sense we are.  We are done decomposing the workflow.  The next steps are not to issue further work requests.  They are to complete the requests that have already been issued and deliver the results.  We are unwinding the call stack.

When the design is complete, we can integrate and verify all of the code that represents that design:

This won’t happen instantaneously, but once enabled, the transition will eventually fire, which will then enable the next transition:

This process repeats for the remaining layers, until we are finally done:

We’ve designed a workflow (though still an abstract one) specifically for pull.  We can check our assertion by asking a few whys:   Why do we design a feature?  So that we can produce code.  Why do we produce code?  So that we can release a feature.  Why do we release a feature?  So that we can satisfy the customer’s demand.  Why do we satisfy the customer’s demand?  So that she will give us her business, now and in the future.

Please don’t be too concerned about the apparent direction of flow in our diagram.  Iteration is possible in this model.  The nodes running down the center of the diagram are in parallel.  The transitions don’t fire until their corresponding processes complete.  Also, it’s just a model.  We don’t throw a design over the wall.  The code isn’t released until everybody is satisfied that the design satisfies the requirement.  But iterative or not, at some moment in time the feature is actually released to production.

Note that this isn’t a pull system yet.  It’s just a workflow.  There’s nothing here to prevent you from starting 100 features and never finishing any of them.  We will have to add something else to implement the mechanics of pull.

Turning a workflow built for pull into a pull workflow

A curious thing happens if you grab our previous workflow diagram by the ears and give it a good stretch:

You transform our nested call-stack view into a sequential instruction-stream view.  Our sequential view highlights both the work stream and the work-in-process tokens that store the return addresses of the work stream.

This sequential projection maps directly to our now-familiar card walls.  Sometimes I draw arcs across the top of the card wall to remind people of the symmetry between design and design verification, or specification and functional verification.  These are a little faint, but you can still make out the dotted lines connecting states:

In order to make our process a pull system, we have to have a way to regulate work in process.  As we described at the beginning of the article, a way that we might do that is to add kanban tokens moving against the direction of flow.  Our new diagram seems to have more than one kind of flow, so where do we put the kanban?  We can try something like our first example and make kanban for “coding-in-process” or “feature-in-process”:

This kind of kanban system is analogous to Microsoft’s Feature Crew process, or the original Scrumban process.  But there also seems to be another type of kanban system, more like a David Anderson-style maintenance process:

Isn’t that interesting?  I knew there were different types of kanban system, but I don’t think I ever quite articulated the difference between the two that are now commonly in practice for software development (there are still others).   If there is more than one, then which is right?  Which should we use?  The first type seems to be looser, more vertical, or product-oriented.  The second type seems tighter, more horizontal, or flow-oriented.  If you are starting a new workflow and you can’t calculate detailed limits yet, then you might start out with one or two limits of the first type.  This is how the Modus Cooperandi card wall is currently set up.  We can have 2 MMFs in process at a time, but no detailed limits on user stories or activities.  Later, we may change that.

You can also have a mix of kanban types.  Here, we can have 3 specifications-in-process at a time, but only 1 of them can be in the specify state:

We can fill out more of the workflow kanban if we decide we want to try to smooth out a lumpy cumulative flow diagram.  Workflow synchronization can help to flatten organizational hierarchy and maximize specialized resources.  But I don’t want to come in with guns blazing and put labels and limits on everything I see.  I want to put just enough control on things to see the constraint appear.  If everybody is carrying a huge pile of inventory, it’s not always immediately obvious who is producing new inventory the fastest.  The combination of kanban styles helps us approach the problem with a lighter touch.