Kirkpatricks Levels of Evaluation

Presentation by: Hemant Swarnkar

1

Donald Kirkpatrick
Kirkpatrick developed a model of training evaluation in 1959
Arguably the most widely used approach

Simple, Flexible and Complete

4-level model

The Four Levels

Level evaluation evaluation description type (what and characteristics is measured) 1 Reaction Reaction evaluation is how the delegates felt about the training or learning experience. 2 Learning Learning evaluation is the measurement of the increase in knowledge - before and after.

examples of evaluation tools relevance and practicability and methods

'Happy sheets', feedback forms. Verbal reaction, post-training surveys or questionnaires.

Quick and very easy to obtain. Not expensive to gather or to analyse.

Typically assessments or tests Relatively simple to set up; clear-cut before and after the training. for quantifiable skills. Interview or observation can Less easy for complex learning. also be used.

Behaviour Behaviour evaluation is the extent of applied learning back on the job - implementation.

Observation and interview over time are required to assess change, relevance of change, and sustainability of change.

Measurement of behaviour change typically requires cooperation and skill of line-managers.

Results

Results evaluation is the effect on the business or environment by the trainee.

Measures are already in place via normal management systems and reporting - the challenge is to relate to the trainee.

Individually not difficult; unlike whole organisation. Process must attribute clear accountabilities.

Relationship Between Levels

Level 4 - Results
Was it worth it?

KSA being used on the job?

Level 3 - Behavior

Level 2 - Knowledge
Did they learn anything

Level 1 - Reaction
Was the environment suitable for learning?

Each subsequent level is predicated upon doing evaluation at lower level A Level 3 will be of marginal use, if a Level 2 evaluation is not conducted

Types of Assessments Used at Each Level

Type
Level 4 - Results
Was it worth it?

Form
Correlation of business results with other assessment results Observation of Performance 360 Survey Self-assessment

Summative

KSA being used on the job?

Level 3 - Behavior

Summative

Level 2 - Knowledge
Did they learn anything

Diagnostic Test Summative Reaction Formative

Level 1 - Reaction
Was the environment suitable for learning?

Survey Real-time Polling Quizzing

6

Reaction - What Is It?

How favorably participants react to the training (Customer satisfaction)
Collects reactions to instructor, course, and learning environment Communicates to trainees that their feedback is valued Can provide quantitative information
7

Reaction - What It Looks Like

Questionnaire - Most common collection tool
Content: I enjoyed the content. Methods: The seminar approach helped me learn. Media: The AVs were helpful to me. Trainer style: I liked the instructor. Facilities: The room was useful for my learning. Course materials: The materials provided make my learning better.
SD: 1 2 3 4 5 6 :SA

Reaction - How to Perform

Determine what you want to find out Design a form to collect/quantify reactions Do Immediately Develop acceptable scoring standards Follow-up as appropriate
9

Learning - What Is It?

Knowledge Skills Attitudes

10

Learning - What It Looks Like

Media used to measure learning:
Text: I felt the textbook help me learn. Voice: The audio materials increased my knowledge. Demonstration: I learned well from the demonstration.

Methods used to measure learning:

Interviews Surveys Tests (pre-/post-) Observations Combinations

11

Learning - How to Perform

Use a control group, if feasible Evaluate knowledge, skills, and/or attitudes before and after Get 100% participation or use statistical sample Follow-up as appropriate
12

Behavior - What Is It?

Transfer of knowledge, skills, and/or attitude to the real world
Measure achievement of performance objectives

13

Behavior - What It Looks Like

Observe performer, firsthand Survey key people who observe performer Use checklists, questionnaires, interviews, or combinations
I believe coming in early is helpful. I work better on the new production system.
14

Behavior - How to Perform

Evaluate before and after training Allow ample time before observing Survey key people Consider cost vs. benefits
100% participation or a sampling Repeated evaluations at appropriate intervals Use of a control group
15

Results - What Is It?

Assesses bottom line, final results Definition of results dependent upon the goal of the training program

16

Results - What It Looks Like

Depends upon objectives of training program
Quantify: Bottom line, productivity, improvement

Proof vs. Evidence

Proof is concrete Evidence is soft

Sample Questions:
I believe the new system has improved productivity. My training has allowed me to be more productive. My boss is a better leader after my training.
17

Results - How to Perform

Use a control group Allow time for results to be realized Measure before and after the program Consider cost versus benefits Be satisfied with evidence when proof is not possible

18