Professional Documents
Culture Documents
6
Training Evaluation
Training doesn’t cost; it pays. Training process
1 Kirkpatrick (1967, 1987, Four levels: reaction, learning, job behaviour and results
1994)
2 CIPP (Galvin, 1983) Four levels: context, input, process and product
3 CIRO (Warr, 1970) Context, input, reaction and outcomes
4 Brinkerhoff (1987) Six stages: goal setting, program design, program
implementation, immediate outcomes, intermediate or usage
outcomes and impacts and worth.
5 Systems approach (Bushnell, Four sets of activities: Inputs, Process, Out puts and Outcomes.
1990)
6 Kraiger, Ford, and Salas A classification scheme that specifies three categories of
(1993) learning outcomes (cognitive, skill based, affective) suggested by
the literature and proposes evaluation measures appropriate for
each category of outcomes.
7 Kaufman and Keller (1994) Five levels: Enabling and reaction, acquisition, application,
organizational outputs and societal outcomes.
8 Holton (1996) Identifies five categories of variables and the relationships among
them: secondary i nfluences, motivation elements, environmental
elements, outcomes, ability/enabling elements.
9 Phillips (1996) Five levels: Reaction and Planned Action, Learning, Applied
learning on the job, Business results, Return on investment.
Level I - Reaction
Level II - Learning
Level IV - Results
Cont….
Level - 1
Reaction: At reaction level, evaluation is focused on how the trainees felt, and
their personal reactions to the training or learning experience. For example: did
the trainees like and enjoy the training? Did they consider the training relevant?
Was it a good use of their time? Did they like the venue, the style, timing, etc?
Level of participation, ease and comfort of experience, level of effort required to
make the most of the learning perceived practicability and potential for applying
the learning. This is useful information.
Evaluation at this level will convey to us only the satisfaction level of the trainees
and not what they have learnt. Examples of reaction level: typically ‘happy
sheets’ feedback forms based on subjective personal reaction to the training
experience, verbal reaction which can be noted and analyzed, post-training
surveys or questionnaires, grading by delegates, subsequent verbal or written
Cont….
reports given by delegates to managers back at their jobs.
Level 5
What to look for in
training evaluation?
Level 4
Return on
Investment
Results
Level 3
Behaviour
Level 2
Learning
Level 1
Reaction
Cont….
Indicators of training success
Level - 2
Learning: At the learning level, evaluation is aimed at the measurement of
increase in knowledge or intellectual capability after the training. Evaluation at
this level is based on, whether the trainees learn what is expected of a particular
programme? This is an important criterion, which many people in the
organization would expect an effective training programme to satisfy. Measuring
the learning may involve a quiz or a test.
Typically, assessments or tests before and after the training, interview or
observation can be used before and after although this is time-consuming and
can be inconsistent. Methods of assessment need to be closely related to the
aims of the learning. Reliable, clear scoring and measurements need to be
established, so as to limit the risk of inconsistent assessment. Hard copy,
electronic, online tests or interview style assessments are all possible.
Cont….
Level – 3
Behaviour: Behaviour evaluation is the extent to which the trainees applied the
learning and changed their work place behaviour. This can be seen immediately
or several months after the training, depending upon the situation. Did the
trainees put their learning into effect when they returned on the job? Were the
relevant skills and knowledge used? Was there a noticeable and measurable
change in the activity and performance of the trainees? Was the change in
behaviour and new level of knowledge sustained? Would the trainee be able to
transfer his learning to another person? Is the trainee aware of changes in his
behaviour, knowledge, and skill? This is also a critical measure of training
success. We have all come across many employees who know how to do a job
well, but chose not to do. If learning does not result in positive workplace
behaviour of the trainees then the training efforts would be a waste. Measuring at
this level may involve observing employees’ behaviour at work or the feedback
Cont….
from customers, suppliers, bosses, peers, etc.
Level – 4
Results: At this level, the evaluation focuses on the business or environment
resulting from the improved performance of the trainee — it is the acid test.
Evaluation at this level aims at finding out whether the training initiative has
improved the organization’s performance effectiveness. Is the organization more
efficient, more profitable, and better able to serve its clients or customers as a
result of the training programme? Meeting this norm is considered as the bottom
line. It is also the most challenging level to assess, given that many things
beyond employee performance can affect organizational performance. At this
level, the business data and financial data are analyzed to evaluate the training.
Measures would typically be business or organizational. Key performance
indicators are volumes, values, percentages, timescales, return on investment,
and other quantifiable aspects of organizational performance. For instance, the
number of complaints, staff turnover, attrition, failures, wastage, non-compliance,
quality ratings, achievement of standards, accreditations, growth, retention etc.
Cont….
Level – 5
Organizations expect much more than results from training.
Thus, Jack Phillips (1996) has suggested that evaluation must
go beyond Level IV and focus on real measurement of return
on investment. Robinson (1989), whose writing redirected the
attention of trainers to business impact, exhorts trainers to
become “performance consultants” and de-emphasizes
training as an intervention. Robert Brinkerhoff (1987) uses
Jack Phillips
data gathering and evaluation to make the training function
more customer-focused and practice of continuous
improvement.
Many trainers are of the view that ROI can easily be included
in Kirkpatrick’s original fourth level ‘Results’. The inclusion and
relevance of a fifth level is therefore arguably only relevant if
the assessment of Return On Investment might otherwise be
ignored or forgotten when referring simply to the ‘Results’ Cont….
level.
Table gives a quick idea about how to find outcome of training at the five levels
discussed above.
Techniques for finding training outcomes
Level of Evaluation How to find the Outcome?
Level 1-Reaction of the trainees Enquire from trainees orally, or use a feedback form at the end of the
programme or at the end of each day/ session
Level 2-Learning occurred or not Ask questions to trainees orally, give a written test at the end of the
programme or at the end of each session.
Level 3-Behaviour changed or Observe on the job or seek report from the supervisor, peers, customers,
not or subordinates, who are familiar with the post -training performance of
the trainee.
Level 4- Results produced or not Look for the outcomes like increase in sales/productivity, or improvement
in product quality/customer service or profitability.
Level 5- Return on investment Calculate what was the cost of training and what is the monetary value of
got or not performance outcome resulted on account of the said training.
Cont….
Phillips (2005) suggests in this regard the following dimensions to assess
ROI. The words in italics indicate the name of the training programme.
Absenteeism control/reduction: Absenteeism, customer satisfaction, job
satisfaction
Business coaching: Productivity/output, quality, time savings, efficiency,
costs, employee satisfaction, customer satisfaction
Career development/career management: Turnover, promotions,
recruiting expense, employee satisfaction
Communications: Errors, stress, conflicts, productivity, employee
satisfaction
Compensation plans: Costs, productivity, quality, employee satisfaction
Compliance programmes: Penalties/fines, charges, settlements, losses
Diversity: Turnover, absenteeism, complaints, charges, settlements, losses
Cont….
E-learning: Cost savings, productivity improvement, quality improvement,
cycle times, error reductions, employee satisfaction
Employee benefits plans: Costs, time savings, employee satisfaction
Employee relations programme: Turnover, absenteeism, employee
satisfaction, engagement
Gain sharing plans: Production costs, productivity, turnover
Labour-management cooperation programmes: Work stoppage,
grievances, absenteeism, employee satisfaction
Leadership development: Productivity/output, quality, efficiency, cost/time
savings, employee satisfaction, engagement
Marketing and advertising: Sales, market share, customer loyalty, cost of
sales, wallet share, customer satisfaction
Meeting planning: Sales, productivity/output, quality, time savings,
employee satisfaction, and customer satisfaction Cont….
Cont….
Davidson (1998) also suggests a similar approach to measure ROI as per
Table. The table indicates some of the areas to look when trying to demonstrate
results.
Measuring ROI in HR