design

The difference between software development and software engineering

Software development:

The system performs function A.

Software engineering:

The system performs function A under operating conditions B with operational performance parameters C with tolerances within the probability distribution D and reliability within the probability distribution E and we are legally responsible if it doesn’t.

As you can probably imagine, one of these problems is harder than the other. My interest is in explaining why it is still possible and even desirable to address the second problem with evolutionary design.

Agile methods can explain how to manage evolutionary design for the first problem. They have so little to say about the second problem that they almost appear to deny its existence. Software engineering methods are largely concerned with the second problem, but they usually apply a simple-minded mass production mentality to management. I guess “serious” software engineering researchers haven’t considered management to be a sufficiently interesting problem.

Maybe I’m a dreamer. I want both. In fact, I don’t think Software Engineering can be made to work without evolution. The best that a phase-gate system can hope to offer is to solve yesterday’s problem (i.e. the wrong problem) with great precision. I want the optimum solution to today’s problem, today.

Comments (4)

Print This Post Print This Post

Email This Post Email This Post

Permalink

A good example of autonomation in software engineering

…is the class invariant.

Class invariants and contracts are not about testing. They are about reliability. The purpose of a class invariant is to minimize the state space of the system, both at design time and at run time. A class invariant is an extension to your type system. You will get the most power from them if you think about them and use them in that way. I love dynamic languages, but the people who bitch the most about type systems probably don’t understand how to use them correctly. The compiler is your friend. Static analysis is your friend. Class invariants make types smarter.

Reliable systems fail at the earliest opportunity. Sounds counterintuitive?

The loom stopped instantly if any one of the warp or weft threads broke. Because a device that could distinguish between normal and abnormal conditions was built into the machine, defective products were not produced.

- Taiichi Ohno

Comments (4)

Print This Post Print This Post

Email This Post Email This Post

Permalink

The conditions for lean development

We recently explored some of the possibilities for managing software development explicitly around the principles of value, waste, pull, flow, and perfection. Our hypothesis was:

If we can divide requirements into small and similarly-sized pieces, then one-piece flow and continuous improvement are possible.

Our conclusion is that both analysis and evidence show that it is possible. So the next question should be: under what conditions is it possible? This leads us to question the premise:

Under what conditions can we divide requirements into small and similarly-sized pieces?

In my mind, that is really the fundamental question of iterative product development. In order to address that question, we’re going to have to talk about design. Clarifying our understanding of the nature of design will lead us to discover what we need to know about the process of design.

Comments (1)

Print This Post Print This Post

Email This Post Email This Post

Permalink

Total Design

In the 1970′s and 80′s, Stuart Pugh developed a philosophy of product development, which he called Total Design. This philosophy anticipated many of the values of Agile development and Design for Six Sigma, but in some ways is still more advanced than either of those. Pugh is remembered most often for his set-based development method of Pugh Concept Selection, but there is much more to his philosophy that just that.

Principles of Total Design
  1. The user need/customer requirement/voice of the customer is paramount to the success or failure of the product
  2. All facets of a business need to be involved in (and interact with) the design core in parallel and not sequentially
  3. To satisfy the user need, rigorous systematic working is required throughout the design core using modern methods
  4. A product’s status needs to be assessed accurately before starting any new design
  5. Within systematic working, a cyclical process of synthesis/analysis/synthesis is necessary, brought to a satisfactory conclusion by the appropriate methods
  6. The most up-to-date elements of engineering, based on sound engineering principles, must be used as appropriate
  7. Total design teams must be multi-disciplinary, with sufficient expertise within the team, and sufficient diversity of experience
  8. Consideration must be given to a wide range of alternatives without prior commitment to any particular alternative
  9. The design team must repeatedly scrutinize and test the information and reasoning on which a design is based
  10. People performance is critical to total design performance
  11. Engineering principles are a vital subset of total design; they influence but do not necessarily relate directly to the user need
  12. To minimize the cycle time for completion of the design core (to minimize process losses), systematic working with modern methods and aids is required
  13. Total product quality is only achievable through total design

I know of no popular software development methodology that lives up to these ideals. So, we are trying to define one!

Comments (0)

Print This Post Print This Post

Email This Post Email This Post

Permalink

What is coupling, really?

Any software engineer responsible for more than a few tens of thousands of lines of code ought to be required by law to have the words:

MINIMIZE COUPLING, MAXIMIZE COHESION

…laser etched onto his or her corneas. Software engineering rightly concerns itself with these two properties as the primary considerations of structural design. And it turns out that proper attention to these things is necessary to the effective operation of lean scheduling.

The definitions of cohesion and coupling are interrelated, so that it is a bit difficult to tease one out in isolation, but a simple definition might be:

Cohesion measures the semantic strength of relationships between components within a functional unit

Coupling measures the strength of all relationships between functional units

Cohesion is closely bound to the idea of abstraction. An information system is ultimately composed of physical devices, themselves layers of abstraction, that are assigned logical meanings and are built up into progressively more abstract constructs in order to finally represent human intentions. There are different styles of abstraction, from the functional, to the relational, to the object-oriented, but they’re all after the same goals of modularity, composability, and hierarchy.

Of the two properties, coupling is easier to measure and understand, because it is more mechanical in nature and requires less interpretation. Suppose there are three components A, B, and C. A’s behavior depends upon B in some way (any way), A’s behavior depends upon C in some way, and the behavior of B and C do not appear to depend upon anything else. That’s pretty much it, that’s the coupling of the system according to our definition, and it’s the sort of thing that a machine can figure out by simple measurement.

Cohesion is more difficult to measure directly because it is a measure of the meaning of relationships, and not just their mechanical strength. However, cohesion has a very strong influence on coupling, and systems that feature poor cohesion often feature poor coupling as a consequence. We usually discover poor cohesion when we are looking for the cause of poor coupling, or through casual inspection, because designs with poor cohesion tend to look like gibberish. A machine would have to understand the intention of the designer in order to measure cohesion effectively. I don’t know about you, but I find some comfort in the fact that machines don’t have that capability yet.

So, we can take all of this to heart and build a system that would appear to have low coupling and high cohesion. All of the modules represent clear abstractions with inviolable abstract data types. All variables and tables have clear types with equally clear labels. All functions perform the one and only operation that their identifiers suggest that they do. There is no global state of any kind. Further, modules communicate with one another in such a way that not only avoids dependency cycles, but forms a perfect tree, the very ideal of low coupling.

Can we thus characterize our system as having low coupling and high cohesion? The answer is: it depends.

But…we followed the rules, how can I say that it depends? The answer is: it depends on how the system is used. The system may have a beautiful internal logic, perfect in its self-consistency, but if the semantics of the system are inconsistent with how the system is used in practice, then it may actually exhibit very strong coupling.


Let’s examine why. It turns out that the classic computer science definition of coupling may be insufficient because the definition of cohesion doesn’t necessarily relate the structure of the problem domain to the structure of the solution domain. If there is a direct correspondence between the functional requirements of the system and the design structure of the system, then our original design preserves its status of minimum coupling. If there is no interaction between the functions themselves, then we say that the system is functionally uncoupled.

Still with me? I promise that this is of the utmost relevance to lean project scheduling!

A functional requirement of the system may have dependencies upon multiple design elements. Common dependencies mean that functions have an opportunity to interfere with one another, either at design time or at run time. However, this interference can always be resolved by applying the functions in a certain order. If the system functions can always be resolved by sequencing, then we would say that the system was decoupled. The dependency graph of the functions of the system contains no cycles. Purely uncoupled designs are uncommon, so most workable designs in the world are of the decoupled variety.

By adding only one more design dependency, we have now added a nasty dependency cycle between functions 2 and 3. Because each function shares the same common dependencies on the design structure, there is no way to avoid interference between them. This interference may manifest at design time or run time, but probably both.

What this case means is that a design has been applied to a problem for which it is poorly suited. The design has good internal structure, but because it is poorly adapted to its purpose, it will perform poorly. This design would satisfy the classic software definition of low coupling and high cohesion, but it’s still a badly coupled design.

The importance of functional coupling to lean scheduling is about rework and variation in task duration. A design that contains functional dependencies will necessarily cause retesting and rework as old design decisions must be revisited to accommodate new dependencies. Keeping the functional dependency graph acyclic makes this manageable because the refactoring required to complete any particular new functional requirement can be completed in a time that is a function of the number of shared dependencies. Refactoring of requirements with cyclic dependencies may require more extensive rework and possibly architectural re-engineering.

The lean way to deal with this, of course, is to make satisfaction of global coupling criteria part of the completion of any particular new function, and to allocate a portion of engineering capacity to continuously optimize total system structure. This is essentially the same approach as the agile practices of continuous integration and refactoring, but we can have criteria that are a little bit more precise than “bad smells.”

Some functional coupling is due to factors that are beyond the direct control of the designer. This often involves shared system resources that you can’t change. In these cases, the design may not be amenable to refactoring, and requires the installation of a decoupling device. Decouplers in software systems usually take the form of semaphores, message queues, virtual memory, processor schedulers, and the like. They make a shared resource appear to be dedicated to each client, usually according to some kind of time slicing. It is helpful to see them as a more general example of decouplers, because this perspective makes other design alternatives more apparent.

As it turns out, Extreme Programming addresses the issue of functional coupling, in its typically blurry way, with the idea of Metaphor. The use of metaphor improves the chances that the design will be well suited to its intended function. Further, the practice of regular refactoring ensures that the cohesion of the design remains high, and that the integrity of the metaphor is subject to continuing scrutiny. Feature Driven Development also addresses this issue by construction of a domain model and use of object-oriented design techniques to align the structures of the problem and solution domains. In the right hands, FDD or XP practices are likely to keep a design within the decoupled realm, although in an informal manner. Metaphor is a convenient mechanism for managing this issue, but we don’t have to settle for XP’s hand-waving explanation, because we can have the real thing. There is a neat mathematical description of functional coupling relationships and we will get into more detail about that in a future post.

It is nice that one can simply follow these methods and know that they will produce a good result, but it also nice to know why they work and how to measure the results. Our goal of elevating agile development methods to the level of Software Engineering requires that we do so. A more rigorous definition of coupling also helps us find hidden interactions that we never intended. Because at the end of the day, source code can only describe what we want the system to do, and not what the system actually does.

Comments (5)

Print This Post Print This Post

Email This Post Email This Post

Permalink

Close
E-mail It
Socialized through Gregarious 42