Eight to Late

Sensemaking and Analytics for Organizations

Beware the false analogy

with one comment

Reasoning by analogy refers to the process of drawing inferences based on similarities between different objects or concepts. For example, the electronic-hydraulic analogy is based on similarities between the flow of water in pipes and the flow of electricity in circuits. Closer home, project teams often use analogies when they estimate tasks based on similar work that they did in the past.   Analogical reasoning is powerful because it enables us to leverage existing knowledge in one area to solve problems in other, possibly unfamiliar, areas.  However, such reasoning can also mislead. This post looks at the problem of false analogies in project estimation.

I’ll begin with a story…

Some years ago, I was in a discussion with a client, talking about costs and timelines for an application that the client needed. The application was a sales bonus calculator for front-line sales staff.  The client needed an app that would calculate bonuses for each salesperson (based on some reasonably involved calculations) and display them via a web front-end.  Nothing fancy really,  just a run-of-the-mill corporate web-database application. The discussion was proceeding quite nicely until a manager from the client’s side felt obliged to make a remark based on a false analogy. I can’t recall the conversation word-for-word, but it went something like this:

“It can’t be that hard, he said. ” You guys have built a similar application before, your promotional literature says so”.  He knew from our brochure that we had built a bonus calculator before; problem was he didn’t know the details.

There was a brief silence until my boss said, “Umm…yes we have done a superficially similar project before, but the details were very different from this one.”

“How different can it be?” retorted the manager, “bonuses are based on sales data. You process the data based on rules and display it. Yes, the rules are different, but the concept’s the same. You should be able to do it in half the time you’ve quoted.”

My boss countered the argument politely, but the manager would not let it go. They went back and forth a number of times until the sponsor stepped in and asked my boss to ensure that the manager’s concerns were addressed. The issue was resolved later, after my boss stepped the manager through the application, showing him just how different it was from the one they had requested.

The technical manager based  his estimate on a superficial similarity between the app we were building for him and one that we had done earlier. Analogies almost always break down when examined in detail. For example, the electronic-hydraulic analogy mentioned in the first paragraph has several limitations. The same is true when comparing two projects or tasks.

An insidious (and dare I say, more common) occurence of such reasoning is when team members themselves draw false analogies.  This happens when they make seemingly harmless (and often tacit) assumptions regarding similarities between tasks  that are actually dissimilar in ways that are important.  See my post on the reference class problem for a discussion of an estimation technique that is prone to incorrect analogical reasoning.

Estimates based on false analogies are a reflection of poorly understood requirements.   This begs the question: why are  requirements misunderstood when most projects involve countless meetings to discuss scope and requirements? In my opinion this happens because talking about requirements doesn’t mean that everyone understands them in the same way. In fact, in most cases different stakeholders walk away from such meetings with their own version of what needs to be done and how to do it.  The first step towards curing the problem of false analogies is to ensure that all stakeholders have a shared understanding of the requirements. This applies  particularly to those who will create the product and those who will use it.  Dialogue mapping, which I’ve discussed at length in several posts on this blog, offers one way to achieve this shared understanding.

Of course, a deep understanding of  the requirements does not by itself   cure the problem of false analogies.  However, it does make estimators aware of what makes a particular project different from all the other ones they’ve done before. This makes it unlikely that they’ll use a false analogy when making their estimates.

Written by K

July 9, 2010 at 5:56 am

One Response

Subscribe to comments with RSS.

  1. […] superficially similar to the one at hand, but actually differs in critical ways.  See my posts on false analogies and the reference class problem for more on this […]

    Like


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: