The February 2008 issue of HBR (yes, I’m behind on my reading) has a fascinating article called “The Experience Trap.” The tagline for the article says “As projects get more complicated, managers stop learning from their experience.”

Yikes!

They ran a computer simulation with very experienced project managers from the IT industry, and nearly all of them failed miserably in complex projects (and who doesn’t have complex projects these days). The reasons for the learning breakdowns were important.

The first they cited was “time lags between causes and effects.” In complex systems, we can’t always make cause-effect links, and even when they can be made, there is often a time lag, so that by the time you figure out the cause and effect, it’s too late to do anything about it. The experienced managers did not take time lags into account when making their decisions—basically making similar decisions to a rookie. The researchers found that they operated on mental models that were overly simplistic, despite their experience.

This is not new stuff, by the way. Senge wrote about similar simulations in the Fifth Discipline (if you’ve read it, you remember the “beer game” don’t you?). But the notion of mental models is critical: we are all operating based on unspoken models of how the world works—many of which are inaccurate!

Another issue was “initial goal bias.” In the simulation, they gave different groups different targets initially. One was focused on staying within budget, while the other was focused on reducing defects. They were told clearly that they would be rated in the end on overall success (balance of cost, quality, and timeliness), and that these were initial estimates only. Then a quarter of the way through the game, the scope of the project was increased (as often happens in real life!). The response:

Neither group readjusted targets in light of the new information. Instead, players in both groups stuck to their original targets, and as a result they failed to achieve an optimal outcome.

Hmmm. Sound familiar? Listen to their conclusion:

These results suggest that if not explicitly required to re-evaluate objectives, managers will continue to pursue the targets set at the outset of a project, even when events render the targets inappropriate. It’s not hard to see where that bias comes from. Very early in their careers, people incorporate into their mental models thee notion that it’s important to meet externally set targets. This bias is often reinforced in managerial life. Revising targets is seen as an admission of failure in many companies, and managers quickly realize that their careers will fare better if they stick to and achieve initial goals—even if that leads to a worse overall outcome.

This is one reason why I am skeptical when people tell me that their new method for strategic planning is new and improved. They may talk about it being a living document and revising things as you go along, but I doubt they are taking into account the power of the mental models. That is one reason why Jeff and I talked so much about the capacity for thinking strategically, and also why we tried to get people to examine their internal practices so deeply when moving to implementation. If you can’t unearth the mental models and change them, the best strategy documents simply won’t make things better.