Streamline Training & Documentation: Failing to Evaluate Training

Saturday, September 05, 2009

Failing to Evaluate Training

Anybody in the training field is familiar with the issue of deciding how best to evaluate a particular training program. Most training specialists take it for granted that evaluation is necessary, and then go on to investigate the question of what level of evaluation is feasible and appropriate in a given case.

What the typical training specialist doesn't do is entertain the idea that the evaluation step can be skipped entirely. But skipping evaluation seems to be all too common at the Department of Defense. The particular instance that got me going on this subject today is a column by Senior Chief Jim Murphy (USN, Retired) in the September issue of the Proceedings of the US Naval Institute.

According to Murphy,

For at least the past two years, every uniformed, civilian, and contract member of the Department of the Navy has been required to complete training on protecting PII [Personally Identifiable Information]. ... The new fiscal year likely signals another round of this so-called training. These "courses" are simple PowerPoint presentations that allow learners to click through the slides until reaching the end, signaled by a course certificate with blanks to fill in one's name and completion date.

Computer security instruction is not much better. DOD [Department of Defense] Information Assurance Awareness training, a separate annual requirement for all personnel, also misses the mark. In this product, information is presented in a lectured e-learning format with scenario-based questions on various topics. The training is problematic because it also allows clicking through the lecture, and worse, clicking through dialogue boxes, which explain incorrect responses to assessment questions. Also there is no retesting of missed topics. In fact, an e-learner can advance through the entire curriculum, miss every question, ignore every explanation, and still successfully complete the training.

Certificate of Completion awarded to personnel who page through the Department of Defense course on "Information Assurance Awareness." No proof of actual learning required.
(US Naval Institute Proceedings)

Not surprisingly, Murphy thinks, at a minimum, the training in question should be evaluated through "formative assessments to certify comprehension" — Kirkpatrick Level 2.

Murphy sums up in his conclusion: "Only through effective training development will the Defense and Navy Departments stop wasting training funds, prevent losses of PII, and secure our systems. It's time we get serious about these issues."

###

Labels: Evaluation of training, Military training

<< Home