No Correlation Between Level I and Level III Evaluation
There is an excellent article by Sarah Boehle in the August issue of Training magazine. "Are You Too Nice to Train?" blows the whistle on the widespread misuse of smile sheets in evaluating training.It was the title that caught my eye. Ever since I started school, I've accepted the idea that you should take what you can from a knowledgeable teacher and not worry too much about whether he or she has a winning personality. And you certainly shouldn't hold it against the teacher if he or she challenges you to think more clearly and to meet high standards.
Unfortunately, in the world of business training, evaluation is most often confined to Donald Kirkpatrick's Level I, which means that the facilitator's personality, and willingness to flatter participants, play a disproportionate role in determining whether the training is viewed as successful.
Put another way, if a company begins and ends training evaluation with smile sheets, they are depending on feedback suitable for a motivational speaker, whose job it is to get people into an "I will do it" frame of mind. What they should be doing is gathering information on whether the training has advanced people to the point where they "can do it" and "do do it."
"Can do" is measured at Kirkpatrick's Level II (see above graphic). Actually "doing it," which is what really matters, is measured by observing whether learning is being applied (Level III), and whether the training has led to better business results (Level IV).
Boehle reports that research indicates that "there is an exceedingly weak correlation among the various levels of training evaluation." Furthermore, there seems to be a negative correlation between Level I results and Level III results, i.e., high ratings on the smile sheets tend to be associated with poor on-the-job application of learning, and vice versa.1 Apparently some of what is necessary to make learning stick, can lead to participants' giving a facilitator relatively low smile sheet scores.
This finding reinforces the importance of doing a sufficiently meaningful Level III evaluation to know whether needed skills and knowledge are being acquired and used. Meanwhile, smile sheets can serve the very useful purpose of gathering ideas from training participants concerning how the training can be improved.
__________
1 Boehle cites Turning Research into Results A Guide to Selecting the Right Performance Solutions, by Richard E. Clark, a professor at the Rossier School pof Education at the University of Southern California, and Fred Estes, a training designer and an adjunct professor at USC.
Labels: Evaluation of training
posted by Karen Nelson @ 7:27 AM
<< Home