Evaluating Training Effectiveness

How do you know when your training program has created a real impact for your people and your business?

twitterfacebooklinkedingoogleplus

Why Measure Training Effectiveness?

Measuring the effectiveness of training programs consumes valuable time and resources. As we know all too well, these things are in short supply in organizations today. Why should we bother?

Many training programs fail to deliver the expected organizational benefits. Having a well-structured measuring system in place can help you determine where the problem lies. On a positive note, being able to demonstrate a real and significant benefit to your organization from the training you provide can help you gain more resources from important decision-makers.

Consider also that the business environment is not standing still. Your competitors, technology, legislation and regulations are constantly changing. What was a successful training program yesterday may not be a cost-effective program tomorrow. Being able to measure results will help you adapt to such changing circumstances.

The Kirkpatrick Model

The most well-known and used model for measuring the effectiveness of training programs was developed by Donald Kirkpatrick in the late 1950s. It has since been adapted and modified by a number of writers, however, the basic structure has well stood the test of time. The basic structure of Kirkpatrick's four-level model is shown here.

Figure 1 - Kirkpatrick Model for Evaluating Effectiveness of Training Programs

  • Level 4 - Results
    What measurable organizational benefits resulted from the training in terms such as productivity, efficiency and sales revenue?
  • Level 3 - Behavior
    To what extent did participants change their behavior back in the workplace as a result of the training?
  • Level 2 - Learning
    To what extent did participants improve knowledge and skills and change attitudes as a result of the training?
  • Level 1 - Reaction
    To what extent did the participants find the training useful, challenging, well-structured, organized, and so on?

An evaluation at each level answers whether a fundamental requirement of the training program was met. It's not that conducting an evaluation at one level is more important that another. All levels of evaluation are important. In fact, the Kirkpatrick model explains the usefulness of performing training evaluations at each level. Each level provides a diagnostic checkpoint for problems at the succeeding level. So, if participants did not learn (Level 2), participant reactions gathered at Level 1 (Reaction) will reveal the barriers to learning. Now moving up to the next level, if participants did not use the skills once back in the workplace (Level 3), perhaps they did not learn the required skills in the first place (Level 2).

The difficulty and cost of conducting an evaluation increases as you move up the levels. So, you will need to consider carefully what levels of evaluation you will conduct for which programs. You may decide to conduct Level 1 evaluations (Reaction) for all programs, Level 2 evaluations (Learning) for "hard-skills" programs only, Level 3 evaluations (Behavior) for strategic programs only and Level 4 evaluations (Results) for programs costing over $50,000. Above all else, before starting an evaluation, be crystal clear about your purpose in conducting the evaluation.

Using the Kirkpatrick Model

How do you conduct a training evaluation? Here is a quick guide on some appropriate information sources for each level.

Level 1 (Reaction)

  • completed participant feedback questionnaire
  • informal comments from participants
  • focus group sessions with participants

Level 2 (Learning)

  • pre- and post-test scores
  • on-the-job assessments
  • supervisor reports

Level 3 (Behavior)

  • completed self-assessment questionnaire
  • on-the-job observation
  • reports from customers, peers and participant's manager

Level 4 (Results)

  • financial reports
  • quality inspections
  • interview with sales manager

When considering what sources of data you will use for your evaluation, think about the cost and time involved in collecting the data. Balance this against the accuracy of the source and the accuracy you actually need. Will existing sources suffice or will you need to collect new information?

Think broadly about where you can get information. Sources include:

  • hardcopy and online quantitative reports
  • production and job records
  • interviews with participants, managers, peers, customers, suppliers and regulators
  • checklists and tests
  • direct observation
  • questionnaires, self-rating and multi-rating
  • Focus Group sessions

Once you have completed your evaluation, distribute it to the people who need to read it. In deciding on your distribution list, refer to your previously stated reasons for conducting the evaluation. And of course, if there were lessons learned from the evaluation on how to make your training more effective, act on them!

Expert View Author: AIMM MAITD

You will be interested in
Training Evaluation Toolkit cover

Our comprehensive toolkit can help you in all stages of your evaluation exercise. From initial planning, data collection and analysis to reporting results, our guide has over 20 customizable tools and templates to make your evaluation task as easy as possible. If you are not sure at which of the four Kirkpatrick Levels to conduct your evaluation, this resource will walk you through the decision process.

Plus, you will learn the pros and cons of the various evaluation methods and how to isolate the impact of non-training factors on performance results. If you need to convert training program benefits to financial results, such as Return on Investment (ROI), our guide contains worksheets for all of the common financial measures. Our toolkit contains everything you need to undertake a credible evaluation exercise in one volume.

Training Evaluation Toolkit

Share This

  • twitter
  • facebook
  • linkedin
  • googleplus
  • gmail
  • delicious
  • reddit
  • digg
  • newsvine
  • posterous
  • friendfeed
  • googlebookmarks
  • yahoobookmarks
  • yahoobuzz
  • orkut
  • stumbleupon
  • diigo
  • mixx
  • technorati
  • netvibes
  • myspace
  • slashdot
  • blogger
  • tumblr
  • email
Short URL:http://bit.ly/zgz0QL

SUBSCRIBE NOW




Privacy
Training Evaluation Toolkit

Need help in measuring and reporting the effectiveness of your training programs?

Don't know where to start?

Training Evaluation Toolkit

Check out our comprehensive guide and toolkit.

Complete with a full set of reusable and customizable forms and calculation worksheets.

Buy Now