Module 6

Production &
Developmental Testing

214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.1

 Outline
Purpose of Production and Developmental Testing
Module Objectives
Design and Production of Training Material
Developmental Testing
Quality Assurance Checklist

214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.2

 Training Development Process
Decision Step 1
Step 2 Step 3 Feedback
to proceed Preliminary
with analysis Study Job Analysis Population Analysis on the analysis

Non-training Training Needs

solutions

Step 6 Step 5 Step 4 Feedback

Production & Design of Modules Design of Curriculum on the Design
Developmental
Testing
Step 7
Draft STP Validation & Revision

Implementation
Validated STP

214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.3

Purpose of Production and Developmental Testing
Produce the training material which has either
been drafted or specified during the Design of
Modules step
Try out the material before the final version is
produced
Have a full set of well produced relevant material
ready for the validation delivery of the course
214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.4
 End-of-Module Objective
Given the results of developmental testing
Conditions carried out on a Mastery Test and using a
Test/Item Analysis form

Establish the validity of Mastery Tests and key

Performance Progress Tests

The validity of each test item is checked and

remedial action recommended in accordance
Standard with criteria given in Chapter 7, Section 6.1 of
the TDG

214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.4

 Intermediate Objectives
Produce Instructional Materials

Try out Instructional Materials

Validate the tests to be implemented during the

course

214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.6

 Production of the Training Material
Should commence as soon as possible after design of
each module has been completed

Step Step
5 6
Ideally when complete the design of the final module, the
production of the training material for earlier modules
should be essentially completed
214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.7
Design and Production of Printed Instructional Material
Consider some useful tips:

Editing and Course

Layout of Text Administrators
Forms
Variety and
Coding of
Highlighting
Printed Material

Size and Format Design of

of the Paper Job Aids
214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.8
 Editing and Layout of Text
To improve readability of a text:
Introduce the main idea as early as possible
Start with the basic case or rule, then move to the
exceptions
Write briefly and succinctly as possible
Try to avoid being too long-winded, comprehensive
Avoid (as far as possible) cross references to other parts of
the training material
Use the KISS principle (Keep It Short and Simple)

214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.9

 Variety and Highlighting
Highlighting devices can be used to direct
attention to key points
Text can be highlighted:
Use different styles of letters
Place key word, headline or phrase in a box
Identify sentences or paragraphs by numbering or symbols
Use colours

214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.10

 Size and Format of Paper
The size and the format of the paper depend on:
The way the material is to be used
The availability of paper sizes and facilities for cutting,
binding and reproduction
ISO A series size (A3 or A4) should be used
to facilitate the international exchange of
printed material

214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.11

 Design of Job Aids
Fundamental points to bear in mind when
preparing a printed Job Aid:
Accessibility
Conciseness
Accuracy
Durability
Modifiability
214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.12
 Coding of Printed Material
Coding of modules: In sequential number from
number 1 onwards.
Coding examples: Item Code Prefix
Hand out H
Progress Test PT
Answer A
Exercise Ex
Mastery Test MT

214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.13

Instructor Guide & Course Administrators Forms
Printed material required for an STP will include:
Module Plan form (TDG Form 6)
Course Description form (TDG Form 9)
Course Content form (TDG Form 10)
Instructors Timetable form (TDG Form 11)
Facilities, Instructors Material and Equipment forms (TDG Form 12)

Read TDG Forms

214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.14
 Design and Production of Projected Visual
Material and Video Training Material
Consider some useful tips:

Colour
Readability Combination

Size of the Size of

image to be Letters,
projected Content Symbols and
Figures

Read TDG Ch. 7 Sec. 7.3 and 7.4

214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.15
Design and Production of E-Training Material
Positive aspects:
Flexibility of implementation
Less time spent by trainees in training institutions
Reduced cost associated with instructors
Improvement in raising entry levels to a course
Easy adaptation to target population
High level of motivation for trainees
Presentation of high-quality training materials
A wide range of training applications

214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.16

Design and Production of E-Training Material
Uses that can be made of e-training:
Practice and reinforcement can be provided in
knowledge and skill areas
Tutorials are designed so that the trainee interacts with
the e-training application
Simulations are designed on the computer to imitate real
equipment and situations
Testing can also be integrated into e-training both for
progress and mastery tests

214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.17

 Developmental Testing
Two main goals:

1. Establish the validity and reliability of mastery

tests and key progress tests

2. Verify the quality of the instructional material

214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.18

 Validity of a Mastery Test
Score %
100
Skilled 90 At least 80%
Group 80 get more than 80%
70
60
50
40
30
Unskilled 20 At least 80%
Group 10 get less than 20%
0

214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.19

 Validity of a Mastery Test
If unskilled performers pass the test, then:
The objectives may be irrelevant
The test may be too simple
The scoring key may be inadequate

Under these conditions, it becomes necessary to:

Check the acquired K/S/As of the target population and
the relevance of the objective
If the objective is relevant, redesign the test
214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.20
 Validity of a Mastery Test
If skilled performers fail the test, then:
The test may be inadequately constructed
The objective may be irrelevant
The test may be unreasonably difficult

Under these conditions, it becomes necessary to:

Check the relevance of the objective against the task
description
If the objective is relevant, revise the test
Read TDG Ch. 7 Sec. 7.6.1
214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.21
 Test/Item Analysis Form (Form 5)

214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.22

Developmental testing of Instructional Material
Importance to try out as much of the
instructional material as possible before the
validation delivery of the course.

Projected material should undergo testing in

normal classroom, laboratory, workshop, etc.

214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.23

 Developmental testing of Instructional Material
Technical review and proofreading:
Instructional material to be provided to trainees for
practical exercises should be checked by experienced
personnel
Objectives:
Improve quality
Improve consistency
Error-free instructional material
Read TDG Ch. 7 Sec. 7.6.2
214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.24
 General consideration for developmental testing
Put the trainees at ease.
Encourage the trainee to ask questions.
Do not provide more guidance (or hints) than those
mentioned in the training material.
Keep a careful record of all the problems encountered.
Keep an eye on trainees reactions.

Read TDG Ch. 7 Sec. 7.6.3 PT 6.1

214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.25
Quality Assurance Checklist for Production &
Developmental Testing
Has a complete Instructor Guide been produced for the
validation delivery?
Has all printed instructional material been checked and
finalized?
Has all projected visual material been produced for the
validation delivery?
Has any video training material obtained from external
sources been reviewed?
214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.26
Quality Assurance Checklist for Production &
Developmental Testing
Has any production of in-house video training material
been reviewed?
Have all e-training application been finalized and
checked for the validation delivery (where applicable)?
Have draft copies of the Course Description, Course
Content, Instructors Timetable and Material and
Equipment forms been produced?
214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.27
Quality Assurance Checklist for Production &
Developmental Testing
Has developmental testing been conducted on all
mastery tests to ensure their validity?
Have all mastery and progress tests been reviewed and
printed for the validation delivery?
Has developmental testing been carried out on printed
instructional material?

214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.28

 Any questions?

Thank you

214/01/ICAO TDC Rev 3.3 TRAINAIR PLUS PPT 6.29