Derivation of the essential development workflow by induction
1. Imagine that you have produced a high-integrity, high-capability design that is suitable for your problem domain. The design is of high quality in every way that matters to you: reliability, usability, security, response time, power consumption, whatever. You have reliable metrics to assess all of your relevant quality attributes. It does not matter how you created this design. It could be a rigid phase/gate process, it could be a million monkeys. It only matters that you are in possession of such a design.
2. Starting with your finished and delivered system, add one feature or improve one performance specification in a way that preserves all of the quality attributes that you could measure in the original system, plus any new attributes that are relevant to your change. Deploy the newly enhanced system and validate. This is your essential development workflow.
3. Repeat step 1, but with one fewer initial feature, or one relaxed performance attribute.
Any sequential development workflow can be pipelined
1. Take all of the steps from your essential development workflow and arrange them in dependency order. Work serially through the steps until a result is produced. If the result is not satisfactory, then repeat the process and apply what you have learned until a satisfactory result is produced. It does not matter if this process is not efficient.
2. Allocate sufficient resources so that two such design increments, following the essential workflow, can execute in parallel without overlapping resources. Create a build/integration process that allows each feature to develop in isolation and integrate through to deployment and validation. Integrations will necessarily be serialized, creating a pipeline. This is your essential development process.
3. Map the value stream of your development process. Add sufficient capacity to each pipeline to realize flow. Add or remove stages to the pipeline to reduce backflows or otherwise reduce cycle time. Substitute new practices or stages as the situation demands and capability allows. Find bottlenecks and relieve them. Share resources between pipelines to improve utilization. Add or remove pipelines as necessary to match demand.
4. Repeat step 3, forever.