Streamline Training & Documentation: Learning from "Mistakes"

Saturday, June 03, 2006

Learning from "Mistakes"

Several weeks ago, I posted about ways of examining bits of conventional wisdom to determine their validity. An article in the June issue of the Harvard Business Review offers another valuable technique.

In "The Wisdom of Deliberate Mistakes," Paul Schoemaker and Robert Gunther explain how learning can by accelerated by taking actions that violate accepted practice, or that violate a working hypothesis not yet definitively proved or disproved. Schoemaker and Gunther illustrate this principle with a simple example:

Suppose you are asked to determine the underlying pattern in the sequence 2, 4, 6. To gather data to help you uncover the underlying pattern, you can propose other sets of three numbers to a person who knows the correct answer, and ask if the proposed sets fit the pattern.

The first working hypothesis you come up with for the 2, 4, 6 sequence is likely to be "ascending adjacent even integers."

So the question now is: How to test this hypothesis?

Schoemaker and Gunther find that the favorite approach in their training groups is to suggest sequences that fit the working hypothesis, e.g.,

4, 6, 8
10, 12, 14
120, 122, 124

In fact, all three of these sequences do fit the underlying pattern, and Schoemaker and Gunther report that their training groups tend to conclude with great confidence after getting three Yes's to their proposed sequences that the "ascending adjacent even integers" hypothesis is correct.

Unfortunately, this conclusion is wrong.

In order to arrive relatively quickly at the correct answer, you need to take a different tack. Namely, you need to propose sequences that violate your working hypothesis and see whether, by any chance, any of these "mistakes" turn out to fit the underlying pattern you're seeking. For example, to see whether the pattern actually requires adjacent even integers, you could propose:

4, 6, 11

If you're told, "Yes, 4, 6, 11 fits the pattern," you know it's time for a new hypothesis. Maybe you decide to see whether the pattern is "positive integers either ascending or descending." You could propose:

5, 2, 1

If the answer comes back, "No, 5, 2, 1 does not fit the pattern," you can see whether maybe the key feature is that the integers are ascending, without regard to whether they are positive or negative. To test this possibility, you could propose:

-10, 0, 546

If the answer you get is, "Yes, -10, 0, 546 fits the pattern," you can proceed to check several more sequences, some of which ascend, some of which descend, some of which go up and then down, and some of which go down and then up.

In this particular example, you will be able to conclude in due course that the pattern is indeed simply "any trio of ascending integers."

Schoemaker and Gunther wrap up their example by observing:

Whenver you have few data points [in the example, it's just three], the chances are low that you'll be correct in your first guess about how they fit together. The fastest way to find the pattern is to try many disconfirming tests. How many decisions in your own business are based on limited data? Are you testing only confirming hypotheses, or are you also making deliberate mistakes?

Aside from the minor quibble that I think it's clearer to put "mistakes" in quotation marks, since disconfirming tests are more quasi-mistakes than actual mistakes, I recommend Schoemaker and Gunther's technique as a highly useful way of scrutinizing conventional wisdom.

###

Labels: Decision-making

<< Home