Streamline Training & Documentation: The Testing Effect

Tuesday, December 08, 2009

The Testing Effect

On October 20 Henry Roediger, a psychology professor at Washington University, and Bridgid Finn, a post-doc at the Memory Lab that Roediger heads, published a short article at ScientificAmerican.com reporting on recent research that provides further evidence of the power of the testing effect, defined as "improved performance on a later retention test arising from an earlier test."¹

Roediger and Finn outline the findings from two sets of experiments aimed at determining if testing people on material they have not studied boosts their recall of the material on a subsequent test even though the testees, understandably, get most of the answers on the initial test wrong.

The first set of experiments is described in a paper (pdf) by Nate Kornell, a psychology professor at Williams College, Matthew Hays, a post-doc at USC's Institute for Creative Technologies, and Robert Bjork, a psychology professor at UCLA.². As the abstract explains:

Taking tests enhances learning. But what happens when one cannot answer a test question — does an unsuccessful retrieval attempt impede future learning or enhance it? The authors examined this question using materials that ensured that retrieval attempts would be unsuccessful. In Experiments 1 and 2, participants were asked fictional general-knowledge questions (e.g., “What peace treaty ended the Calumet War?”). In Experiments 3–6, participants were shown a cue word (e.g., whale) and were asked to guess a weak associate (e.g., mammal); the rare trials on which participants guessed the correct response were excluded from the analyses. In the test condition, participants attempted to answer the question before being shown the answer; in the read-only condition, the question and answer were presented together. Unsuccessful retrieval attempts enhanced learning with both types of materials. These results demonstrate that retrieval attempts enhance future learning; they also suggest that taking challenging tests — instead of avoiding errors — may be one key to effective learning.

In sum: the read-only condition emulates someone sitting down and reading a portion of a textbook. How much does the person remember? Not as much as if he/she had been quizzed in advance on the material, even if the answers he/she came up with were largely wrong.

A key point is that the unsuccessful retrieval attempts need to be followed by feedback providing the right answers in order for enhanced learning to occur.

Kornell, Hays, and Bjork (KHB) also found, as have previous researchers, that the benefit of testing increased as the delay between study and the final test of the material increased. The reason seems to be that testing is more effective at preventing forgetting than is a read-only/no-initial-test approach to learning.

KHB offer three conjectures concerning the reason for the testing effect:

Testing enhances deep processing (elaborated thinking about the material), which, in turn, enhances learning.

Retrieval strengthens retrieval routes from the question to the correct answer. And "it could be that exploring incorrect retrieval routes actually weakens, rather than strengthens, those routes."

"... information generated from memory during a retrieval attempt, even if it is incorrect, can serve to cue future recall attempts. In other words, incorrect information can serve as a mediator, connecting the question with the correct answer. ... unsucessful retrieval attempts are likely to produce related information that can serve to mediate recall of the correct answer." [reference omitted]

KHB close their paper with the advice that "educators and learners should introduce challenges into learning situations, including using tests as learning events, even if doing so increases initial error rates."

The second set of experiments is described in a paper (pdf) by Lindsey Richland, an education professor at the University of California, Irvine, Nate Kornell, and Liche Sean Kao, a graduate student in education at Irvine.³ As explained in the abstract:

Testing previously studied information enhances long-term memory, particularly when the information is successfully retrieved from memory. The authors examined the effect of unsuccessful retrieval attempts on learning. Participants in 5 experiments read an essay about vision. In the test condition, they were asked about embedded concepts before reading the passage; in the extended study condition, they were given a longer time to read the passage. To distinguish the effects of testing from attention direction, the authors emphasized the tested concepts in both conditions, using italics or bolded keywords or, in Experiment 5, by presenting the questions but not asking participants to answer them before reading the passage. Posttest performance was better in the test condition than in the extended study condition in all experiments — a pretesting effect — even though only items that were not successfully retrieved on the pretest were analyzed. The testing effect appears to be attributable, in part, to the role unsuccessful tests play in enhancing future learning.

What this research suggests is that if you have a chunk of time available in which students can get themselves "in gear" for a class session on a new topic, it's better to use the time for testing — even if the students give a lot of wrong answers — than to adopt a "study hall" approach of having the students spend the time reading about the topic.⁴

Richland, Kornell, and Kao emphasize the importance of following the pre-testing with instruction that provides answers to the tested questions. They go on to say that

... our data suggest that instruction following testing need not be individualized to learner errors. Rather, instruction that appropriately draws attention to key content may build on the previous cognitive acts performed when attempting to answer a test question. This implies that standardized tests, or other test situations where it is difficult to provide timely item-by-item feedback, could still provide learning benefits for successful and unsuccessful test takers, as long as those test takers are given an opportunity to learn the information on which they were previously tested.

For an overview of research relating to the testing effect, you can see a survey paper published by Henry Roediger and Jeffrey Karpicke in 2006.⁵ In the paper Roediger and Karpicke argue:

The testing effect cannot be explained by additional exposure to the material. This suggests that retrieval processes engaged in during a test are responsible for enhancing learning. More specifically, elaboration of encoding [processing of sensory input into memory], more effortful or deeper encoding, and creation of different routes of access can account for the basic effect. ... The concept of transfer-appropriate processing is also congenial, albeit at a general level, to explaining the testing effect. [As explained earlier in the paper, "The idea behind transfer-appropriate processing is that performance on a test of memory benefits to the extent that the processes required to perform well on the test match encoding operations engaged during prior learning."]

They also note that the testing effect can be exploited with more complex learning than memorization — such as the learning required in college-level courses or at business firms — by adopting an approach of frequent assignments. In other words, "testing" should be interpreted broadly; it's not just multiple choice, true/false, and fill-in-the-blank testing of memory.

In closing, let's return to the Roediger/Finn article. The authors offer this practical advice, reflecting all the insight that has accumulated concerning the testing effect:

By challenging ourselves to retrieve or generate answers we can improve our recall. Keep that in mind next time you turn to Google for an answer, and give yourself a little more time to come up with the answer on your own.

Students might consider taking the questions in the back of the textbook chapter and try to answer them before reading the chapter. (If there are no questions, convert the section headings to questions. If the heading is Pavlovian Conditioning, ask yourself What is Pavlovian conditioning?). Then read the chapter and answer the questions while reading it. When the chapter is finished, go back to the questions and try answering them again. For any you miss, restudy that section of the chapter. Then wait a few days and try to answer the questions again (restudying when you need to). Keep this practice up on all the chapters you read before the exam and you will be have learned the material in a durable manner and be able to retrieve it long after you have left the course.

Of course, these are general-purpose strategies and work for any type of material, not just textbooks. And remember, even if you get the questions wrong as you self-test yourself during study, the process is still useful, indeed much more useful than just studying. [emphasis added]

The key take-away from all of the testing effect research is that testing is useful not only for assessment but also as a tool to promote learning.

__________
¹ Mark McDaniel, Henry Roediger, and Kathleen McDermott, "Generalizing Test-Enhanced Learning from the Laboratory to the Classroom," Psychonomic Bulletin & Review, Vol. 14, No. 2 (2007), pp. 200-206.

² Nate Kornell, Matthew Hays,and Robert Bjork, "Unsuccessful Retrieval Attempts Enhance Subsequent Learning," Journal of Experimental Psychology: Learning, Memory and Cognition, Vol. 35, No. 4 (2009), pp. 989-998.

³ Lindsey Richland, Nate Kornell, and Liche Kao, "The Pre-testing Effect: Do Unsuccessful Retrieval Attempts Enhance Larning?" Journal of Experimental Psychology: Applied, Vol. 15, No. 3 (2009), pp. 243-257.

⁴ It should be noted that Richland, Korness, and Kao did not find that their pretesting slowed the rate of forgetting.

⁵ Henry Roediger and Jeffrey Karpicke, "The Power of Testing Memory: Basic Research and Implications for Educational Practice," Perspectives on Psychological Science, Vol. 1, No. 3 (2006), pp. 181-210.

At his Memory Lab, Roediger oversees the Test-Enhanced Learning in the Classroom (TELC) project, which is based on these four principles: "(1) Testing enhances learning (better than repeated studying), and students should be tested frequently; (2) Production tests (short answer or essay tests) produce better retention at delays than do recognition tests (multiple choice, true/false); (3) Multiple tests are better than single tests in enhancing learning; and (4) Immediate feedback improves the effect of testing, especially for facts missed on the tests."

###

Labels: Classroom training, Cognition, Teaching

<< Home