Have you …
- Had an important decision for which you’ve waffled between several viable choices?
- Had a decision that split your team into camps with no consensus and poor buy-in?
- Had a design decision or policy that kept being attacked or reconsidered, months or years down the road?
- Been using Set Based Development — exploring several design alternatives, looking to pick the final choice for this version of the product at the “last responsible moment”?
A great decision making tool for this kind of situation is a Pugh Decision Matrix, with the technique often called Pugh Concept Selection. “Pugh” comes from its originator, Stuart Pugh.
Here’s an example of a spreadsheet, applying our variant of the technique. I was looking at alternatives for buying a cellphone here in the US. Based on what I’ve filled in so far, the Nokia 6682 with T-Mobile is the best choice.
So how does this work? The basic steps of the Pugh Concept Selection Process are
- Brainstorm alternatives, list them across columns of sheet. Make one alternative the “default” — often it’s the “do-nothing” or status quo choice. This choice is rated zero for all criteria.
- Brainstorm criteria and characteristics important to the customer. List them down rows of sheet.
- Begin filling in 1, 0, or -1 ratings in the main area of sheet, based on whether that alternative is better, equivalent, or worse than the status quo for that criteria.
- If some criteria are more important than others, adjust the weights. If some products are much better than others, adjust the rating weights in the main area of the sheet. Don’t go overboard with this.
- Look at what the spreadsheet tells you is the best choice. Do you and the group feel good about that decision? If so, you’re done.
- If not, look again at steps 1-5 — do you have a complete set of criteria, or was something important to the decision missed? Are the weights you’ve assigned close enough?
I’ve found this technique personally useful whenever a simple pro/con sheet didn’t cut it — and taught the technique to a few hundred people at a little company here in Redmond, WA. The response was often positive — “this is a great way to more methodically make a tough decision as a group, and leave behind a record of why we made it.”
Sound interesting? Jump to our Pugh Decision Matrix page to download templatized versions of this spreadsheet to try yourself. Thanks for any feedback you have!
Kanban discussion
Kanban Group
rob | 08-May-07 at 6:28 pm | Permalink
Cool! Inspired by this, I used Pugh to decide how to upgrade one of my machines. It said I should buy a whole new machine, so I like this process.
Seriously, though, I’d be curious about other rules of thumb: Should criteria be entirely qualitative? How do you know when you’ve got the right criteria? (e.g. you’re staring at the result and it doesn’t match your intuition). If you have a quantitative criterion (e.g. time or cost), how do you express values for it?
E.g. one of my criteria was setup time. I had 1.5h, 1.75h, 4.5h (baseline was do nothing, thus 0). If I value these proportionally, I get -1,-1.7,-3.3. That -3 has a huge effect. Does it really make sense that this setup time disadvantage would outweigh three other positive advantages (if I mainly use +/-1)? I can see similar quandaries with cost, or any other criteria you choose where abs(value)>1. I could adjust the weight, but there’s another, qualitative, scale - is a 3x increase in setup time really 3x worse to me?
How best to deal with these?
One additional thing I find useful: don’t do the sums until you’re happy with the matrix. Without this, I’d have skewed my way into a dual quad-core with 4 wheel drive and a flame job. Even if I try to ignore sums, I’ll unconsciously skew results towards favorites.
Bernie | 08-May-07 at 9:05 pm | Permalink
Hi Rob! It’s great that you’re trying personal decisions against this. I’ve found that useful at times, too (but of course, sometimes it’s overkill).
I think the criteria are best when they stay qualitative .. e.g. “quick to set up (~2 hours or less)”. I wouldn’t use setup time as a number/weighting directly on the sheet. You saw it getting you into trouble.
Pugh is most powerful when it is applied as a supporting dose of logical/objective yin, balancing the yang of your instincts. Getting too serious or precise about the numbers will steer you (or a group) astray. They can never be more than approximations — so don’t take them too seriously.
How to find the “missing criteria” when the current best result doesn’t match instinct?
It’s great to be at this point, because Pugh has brought you to the verge of an insight — something important that you haven’t been able to articulate about your instinctual decision — or, perhaps, a realization that you’ve found a new, better choice.
Let us know how it goes!
Bernie
Total Design | Lean Software Engineering | 08-Jan-08 at 12:27 pm | Permalink
[...] than either of those. Pugh is remembered most often for his set-based development method of Pugh Concept Selection, but there is much more to his philosophy that just that. Principles of Total [...]