CIDTT Guide Assessment

Cambridge International
Diploma Teachers and Trainers

for
GUIDE 2011
1 University of Cambridge International Examinations 2010
Guide for the Cambridge International Diploma for Teachers and Trainers CONTENTS
CONTENTS
Page
INTRODUCTION Welcome The spirit of the Diploma Thinking about yourself as a teacher What makes a good teacher? Personality and professionalism Introducing you to the Diploma syllabus Approaching assignments Your reflective journal Resources Performance observation Being an observer yourself Reflecting on observed sessions Scanning guidelines Images in Word documents Examiners advice to candidates MODULE 1 : DESIGN Identifying learners needs Specifying the learning objectives Planning content, methods and resources Completing the programme plan Specifying the requirements for each learning session Completing session plans Preparing learning materials Preparing equipment and learning facilities Planning for evaluation Preparing the learning environment MODULE 2 : PRACTICE Presenting information Giving instruction and demonstration Using visual aids
4
4 5 6 7 8 9 11 15 16 17 24 25 28 29 31
38
39 45 49 57 61 73 80 86 90 93
97
98 103 109
Guide for the Cambridge International Diploma for Teachers and Trainers CONTENTS
Supervising learning activities Managing the flow of activities Managing the learning environment Motivating learners Encouraging learners Guiding learners Supporting learners MODULE 3 : ASSESSMENT Preparing formative assessments Using formative assessments Analysing formative assessment data Providing feedback about progress Maintaining records of learners progress Preparing summative assessments Using summative assessments Analysing summative assessment data Providing feedback about achievement Maintaining records of learners achievement MODULE 4 : EVALUATION Evaluating learning Using evaluation to plan improvements Making improvements and planning further evaluation Evaluating own practice Identifying goals for improvement Completing a professional development plan Specifying actions and evaluating outcomes NB to help you: This document is bookmarked to help you navigate easily to the section you want. You can also go to the section you want by clicking on its title on this Contents page To return to this Contents page at any time, simply click on the page number of the page youre on.
114 119 124 134 142 148 152
158
159 167 175 181 188 193 204 212 218 221
224
225 232 236 241 247 259 263
Guide for the Cambridge International Diploma for Teachers and Trainers MODULE 3 : ASSESSMENT
Module 3
ASSESSMENT
PREPARING FORMATIVE ASSESSMENTS
Formative assessment tests the current level of understanding and progress at any point in a learning programme and provides feedback information to teacher and learner, to guide the next phase of learning. It is assessment for learning. Summative assessment is used at the end of the programme formally to assess a learners skill, knowledge and understanding gained as a result of that programme. It is assessment of learning. How am I doing? How is my son/daughter getting on? How are my learners progressing with their learning? These are among the most commonly asked questions in everyday education. They illustrate the important point that assessment is a vital teaching-learning process. Learners see the need for it as much as teachers and, of course, parents. Formative assessment, that is the means for assessing learners progress, is a good way of embedding assessment into learning sessions and programmes. Before you start consider these essential points 1. Ritualised versus Targeted Assessment. It's easy to fall into the habit of assessment for assessments sake. This is where you collect in work and mark it with no special purpose on mind. Assessment becomes ritualised Wednesday is test day, the weekly essay. No-one would argue against frequent formative assessment but it must be properly targeted. 2. In order to target assessment you must be clear about what you want to assess and why. The answers to your what and why questions become your assessment objectives. These need careful consideration and precise identification as we shall see later in this task. 3. Once you have identified your assessment objectives you can choose your methods of assessment.
TYPES OF FORMATIVE ASSESSMENT There is a large range of formative assessment methods available. This includes Question and Answer in the Session This is perhaps the most commonly used method and is almost instinctive for teachers. It gives instant feedback, can be used to develop motivation but is largely ephemeral that is to say that it is momentary and difficult to record. Short Tests and Quizzes These are either from textbooks or devised by the teacher. These are informal, can be fun and marks can be simply recorded. Used with care they can become part of every day teaching and learning. Homework Exercises These vary in purpose, design and complexity. Purpose is the key word here. Learners will make good use of homework if they feel it is useful, for example, preparation of material for a class discussion, seeing how a piece of writing ends, developing a skill, are all appealing tasks. Skills Assessment using Formal Assessment Criteria These may be the foundation for many skills-based courses. This method requires experience in on the hoof assessment and systematic recording. Observation of Performance This is often used in the arts such as music and skill assessment such as team and leadership exercises. It needs expert and experienced assessors. Assignments This term spans a vast range of tasks but an example might be individual research assignments say for a group project. A very useful and increasingly used method, especially in conjunction with homework. May involve library and internet investigations, visits and interviews. Difficult to manage and assess. Projects Increasingly used in modern education as it is felt that developing your own learning material/methods gives you an ownership of your own learning experience. The assessment methods of the various project components need careful design and clear communication to the learners. Written Questions / Exercises with Short, Extended or Multiple-choice Answers Very widely used. Easy to design, mark and assess. Simulations, Business Games Almost guaranteed to produce lively learning sessions! Can teach a number of skills imaginatively and effectively. The better ones contain useful directions to possible methods of assessment. May well be time-consuming.
Conferencing / Reviews / Audit This involves sitting down with learners and reviewing their written work/homework/progress in general. A very useful and beneficial process for teachers and learners. Can be used to introduce care, involvement and motivation into the teacher-learner relationship. Three points to watch when operating it as a method. 1. It can be time consuming as you have to give all learners a review session. (If you do not those who are omitted will feel rejected!) 2. If you do it in class you must ensure that those not involved have something useful to be getting on with. 3. Make notes on learner performance immediately after the review, not during it.
REFLECTION
David Kolb described experiential learning as a cyclical process. A learner can start at any point in the cycle but to be really useful the stages should be followed in the order shown on the diagram.
Lets just rest at Stage 2 for a moment and reflect on the kind of formative assessments you have already used. You could have used oral questioning, oral tests, written tests of skills, knowledge and understanding and many other methods. Ask yourself why you chose these methods. Was it because: you had used them successfully before? it was department/programme policy to use them? you felt comfortable handling them as a technique? you felt they did the job you were asking them to do?
There is nothing wrong at all with any or all of these reasons for selection but when we are selecting a method of assessment we do need to think about 1. reasons 2. criteria We can now go on to stage 3 of the Kolb cycle, the abstract conceptualisation stage. Selecting methods The methods of assessment you choose will be those best suited to the learning objectives and material present in your learning programme and learning sessions. Consider what you need to discover from the learners. Then consider the possible assessment methods available and select the most effective method of assessment for your teaching and learning environment. Think also about the position of assessment in the programme and sessions planned. Remember:

most formative assessments tend to be relatively brief so tests and quizzes need to be brief and to the point (hence the need for careful targeting and design) some formative assessment techniques such as conferencing (Audit) are brief in so far as the teacher talks to the learner about his/her work but may be very time consuming when the ten minutes per learner is multiplied by the thirty learners in the class.
Criteria for selecting formative assessment methods Ask yourself these questions: is the method appropriate to the nature and abilities of your learners? can it be deployed quickly, say during a learning session, or will it require a good deal of preparation and planning? how much time will it take in terms of operation, marking and feedback to the learners? how will you record the findings of the assessment exercise? how formal or informal will it be? will learners be able to develop their motivation as a result of using this method?
As explained above there is a large range of methods available. These include: question and answer in the learning session short tests devised by the teacher homework exercises skill assessment using formal assessment criteria observation of performance standardised tests designed by an external agency projects assignments written questions with short, extended, or multiple choice answers simulations online assessments observations presentations to whole group/institution.
The UK Qualifications and Curriculum Development Authority (QCDA) have a useful set of principles for Assessment for Learning. You can find these at: http://www.qcda.gov.uk/4336.aspx How far do you agree with these statements as principles? How practicable to do you think they are? Variety, formality and frequency Keep teaching and learning fresh and engaging. You can assess the same learning objectives in a variety of ways and your learners will thank you for it. You should think of assessment as a partner to learning in the education process. It should not be a dictator, dominating all that happens. Integrate assessment into session activities so that assessment feels like a natural part of learning. Your learners can then get used to assessment and become more comfortable with assessment. You can also vary the formality of assessment. Frame the assessment task and format to the nature of the material and the learning objectives involved. Frequency of assessment also needs some thought. Pestering learners with a stream of questions and tests is not constructive. On the other hand if you are casual about formative assessment, learners may feel they are not receiving sufficient review and guidance about their progress. They may also come to feel that they have had little preparation for the formalities of important internal or external summative assessments. Clearly you need to strike a balance - the best thing to do is to review and to prioritise your learning objectives in terms of skills, knowledge and understanding. Then you can match the appropriate assessment method etc to each.
Validity, reliability and fairness Fairness is most important. For example in criterion-referenced assessment schemes much depends on the opportunity for the candidate to perform. Each person being assessed in this way must have an equal opportunity to demonstrate their skills. Reliability is difficult to achieve in any assessment scheme. Total reliability implies no variation whatever in the application of marks or grades from candidate to candidate, assessor to assessor. Reliability can be improved, for example by strict use of assessment criteria or mark schemes. About validity, Geoffrey Petty says: ..the validity of an assessment depends on whether it actually measures the knowledge or skills it is designed to assess. For example an objective test cannot measure a candidates practical skill, or his or her ability to develop a coherent argument. To be valid, an assessment must also sample across a large proportion of the topics ... (available for assessment) ... the breadth and depth of learning sampled by the assessment must be correctly weighed in the markingValidity is also compromised if questions are difficult for the candidates to understand, or are culturally biased. It is common for teachers to confuse poor learning with a learners difficulty in understanding ... questions. Any assessment you do should ideally be based on the objectives that you have set at the start of a course of learning. You will have identified the skills and knowledge that learners need to acquire and the more specific, carefully defined and elegant these are the easier it will be designing valuable and useful assessments. Any accurate assessment of learners' learning will have assessment criteria at its core. These criteria are what help to make meaning out of the whole assessment process and should reflect the following points:

any assessment should ultimately be standards-based it should support and help to facilitate further learning any assessment methods adopted should be consistent it should achieve what it sets out to achieve it should be appropriate to the subject of the assessment it should embrace, as holistically as possible, the full experience of learning in that subject or topic the justifications for the assessment should be as transparent as possible; in other words, the criteria help to convey exactly why the assessment is taking place at that particular time and in that particular format (note that this is often implicit, rather than explicit, in assessment criteria).
Practice
A teacher of dance, mime and drama is running a course for adults. There are ten learners in the group which meets twice weekly in two hour sessions for ten weeks per term. The course aims to introduce learners to the techniques involved, develop individual skills and lead to a diploma in essential stage skills. Summative assessment at the end of the course will comprise a series of practical and written tests provided by an external awarding body. Clearly the teacher will be responsible for preparing the learners for these assessments. But how can formative assessment by the teacher best support learning and also prepare learners for the summative tests? Imagine yourself in this teachers position and think of answers to the following questions. Dont worry about your lack of knowledge of the course content! If you wish to substitute a similar skills-based programme (sailing, art, wrestling ..) go ahead. Think about the assessment objectives formative assessment issues
The assessment objectives What might these be in such a programme? Perhaps: suppleness/fitness use of balance use of breathing sense of rhythm ability to express emotion mastery of steps/movements creativity working with others.
These might be the kind of things the teacher is trying to assess
Formative assessment What kinds of assessment techniques might be appropriate to these assessment objectives? What advantages and disadvantages might they have in terms of reliability, validity and fairness? Issues There may be some important issues to deal with in assessing a course like this on a session-by-session basis. For example how might you as a teacher deal with matters such as: Differentiation - supposing two or three learners quickly emerge as talented learners with considerable ability and high levels of aspiration while the remainder seem rather slower to progress? How can you deploy appropriate formative assessment without causing resentment, jealousy or other forms of dissatisfaction? Individual versus group assessment - how do you ensure that individuals are aware of their progress and achievement in each learning session? In skills-based courses such as this the teacher can easily speak to the group about progress but how does he/she get round to assessing and re-assessing and motivating individuals? Recording observations - 'on the spot' observation will be a key method of gaining information. How can the teacher keep records of observations?
USING FORMATIVE ASSESSMENTS
Formative assessment is fundamentally diagnostic, testing the current level of understanding and progress at any point in the programme. There are three principles which should be followed: Short term and informal this will take place either during or immediately after a session Question and Answer is the most obvious method, although short tests and homework are also relevant and widely used an emphasis must be on improvements that can be made rather than a mark or grade to be awarded it is a motivational tool.
Methods of assessment of activities in learning sessions will depend heavily on the types of activities undertaken. It is often useful to consider the best methods of assessment to be used before considering the activities to be undertaken to promote learning. No matter which assessment strategy is adopted, adequate time should be built into learning sessions for assessment to take place. With methods such as observation, skills tests and simulations, quality of information recorded depends heavily on both: your ability to design assessment records which will capture a variety of information in the simplest way possible your ability to record what has been observed quickly and without interfering with observation taking place - no one can write and observe at the same time.
Rapid feedback to be most effective, feedback must be immediate or as rapid as possible in the case of written work this is most effective with question and answer, but can be through comments in the classroom or in tutorials one helpful technique is to ask learners to describe what they have done - in this way understanding can be gauged and feedback and support can be immediate rapid may mean comments following a short test or work done, in which case it will not be immediate but within a short time.
Motivational feedback feedback needs to be positive, emphasising what has been done well negative issues should be highlighted, not condemned feedback should end with future action that can be taken.
To apply these principles means that you should plan the use of the techniques into your Learning Session Plan. REVIEWS Assessment of progress on learning programmes must take into consideration the fact that all learners are individual and will progress and achieve at different rates, some more quickly than you expect and some more slowly. For this reason, structured opportunities for reviewing progress should be made available and information on the availability and purpose of these disseminated to all learners. These reviews could be formal or informal and include individual tutorial sessions and more informal reviews. It is good practice to sit down with learners during a class and look through what they have been doing. It is almost always easy to check whether work is complete and if it is not identify what is missing. Praise and encouragement can be given for work well done and learners who went that 'extra mile' to take their efforts to a new level. New ideas can be fed back to the group as a whole so that they can try them out. A brief look around work and files and an encouraging chat with learners can make them feel valued and cared for. It can also short-circuit small problems which may become bigger ones by the time a more formal review is due. The purpose of these opportunities will be to: create opportunities for learners to discuss and review their progress in an informal setting give constructive feedback to learners on what has been achieved and what is yet to be achieved discuss alternatives where expected progress has not been made or where learners have progressed more quickly than expected identify any problems or areas of concern which are preventing expected progress
discuss additional support, perhaps from others within the organisation or from external agencies plan future progress and achievement and agree appropriate action with learners.
It is essential that results of these reviews are recorded fully and accurately and used to inform future sessions. Records would contain information such as: the purpose of the review where and when it took place what was discussed any issues or concerns any problems identified as a result of the review future actions agreed to ensure progress and achievement.
Note that it is NOT a good idea to make copious notes during the review. A few brief notes of important points are all that is required. Then immediately after the review, the teacher can make a fuller record of what went on. Individual review sessions are time intensive but are extremely useful. The quality of the time given is more important than the amount. A CASE STUDY We're going to look at a case study to see how teachers can manage assessment to ensure fairness and to facilitate performance. Rose is teaching a programme for a group of new recruits to a large international corporation. This programme aims to introduce and develop a series of skills which will be vital for the learners to apply in their various roles within the corporation. The programme has a number of learning objectives such as the ability to work constructively as a team member the ability to contribute usefully and effectively to group discussions development of time-management skills development of problem-solving techniques making a presentation constructing a short written report.
It is a formative assessment because it will provide Rose with information to help her prepare the next stage of the course to meet particular needs.
Rose is using a criterion-referenced assessment scheme It works like this
NB: Rose produced this assessment scheme herself. Usually the boxes for scores contain descriptors which are quite specifically worded and go well beyond simple statements like little evidence. Such assessment schemes are widely used for skills-based exercises and activities. Note that the scoring system is augmented by a brief supporting comment. Clearly such scoresheets minimise the amount of preparation you need - they can be reproduced quickly in advance. They involve rapid recording (circling numbers) and only a brief comment so they are easy to use in an observation assessment context. GOOD PRACTICE It's always good practice to outline in advance the purpose, conditions, method of operation and kinds of outcomes to learners before the assessment takes place. Rose prepares her learners by reassuring them about the nature of her assessment, the criteria being used and what she would be looking out for in the discussion.
If time is available you can conduct a dry run of the assessment. It is very good practice in skills assessment for learners to have more than one opportunity to demonstrate their competence at this particular set of skills. Always write a time and a date on the assessment sheet (as Rose has done top right on James Chengs sheet) You should always take care storing such assessment records. They will provide useful evidence for: conferencing with learners about performance and progress compilation of summative and other reports discussions with internal/external visiting moderators reporting to parents, carers, employers, other teachers.
A LEVEL PLAYING FIELD In Roses case it might be good practice to try to assess each learner within the same discussion context (i.e. the same learning session). But is 20 minutes enough time for that? Some learners may offer three contributions; some only one. Has enough time been allowed for this kind of assessment? Probably not. Rose probably needs to see the group in action again to give everyone a fair chance. Does the composition of the group affect the performance of individuals? Does this matter? Even when enough time has been allowed, the discussion may be disrupted - a visit from an outside individual or even a simple sneezing fit. Note that if other groups are in the room at the same time they must be usefully occupied and in no way intrude upon the assessment. FEEDBACK AFTER ASSESSMENT Even the most confident of learners can become concerned about an aspect of performance in a learning session. This means that your feedback should be immediate where at all possible. Learners should leave the session confident about what they have done and can do and know how they can improve their performance, rather than be worried or not knowing about how they have done.
Clearly time and need are important questions here. You should allow time for a few well-chosen and positive general comments reassurance for those who are aware that they have not done as well as they could.
You can then keep more detailed feedback until you can use purpose-built feedback such as conferencing sessions and written progress reports. We have looked at only one method of formative assessment in action. But many of the points above can be applied to more complex schemes relating to role play and simulations or the implementation of simple question and answer sessions. A final word on what makes a good test? Phil Green reckons that the features and characteristics of a good test are: Questions are based on objectives
When they are used to test transfer of knowledge and understanding, the questions must relate to an identified objective. Do not include questions just because they are good. The learner always receives some feedback
You must always give learners feedback to their responses. If there are particular reasons why they should not know whether their answers were right or wrong, give them a thank you message The learner is in control
Because tests are important to learners, you must take whatever steps are necessary to reduce learners anxieties. One source of anxiety is the level of control they have over the questioning. You can overcome this by giving learners the chance to change their answers and letting them review questions they have selected. Tests should be unambiguous
Tests should make clear how the learner should tackle the problem and under what conditions Tests should be valid
Tests should be confined to the content within the learning session. With these points in mind its worth taking time out to think about the way you use your own formative assessments.
INFORMAL EVIDENCE The most informal of formative assessments can happen naturally as part of the teaching-learning process and may be totally spontaneous. Heres an example. A General Studies teacher was conducting a learning session on developments in art in the twentieth century. She asked the learners to consider a slide which showed a Picasso painting of a sad young clown. Spontaneously, one of the learners began to speak about his own views on the colours and construction of the figure in the painting. The teacher invited the learner to take his time to explain to the group what were clearly deeply held personal views about Picassos work. A lively discussion followed. It was a memorable learning session for all present. The learner in question had made a big contribution and had stimulated other learners thinking. He had shown understanding, courage in his own views, ability to support his arguments and presentational skills. This wasn't a prepared assessment task. But it was certainly an opportunity to record evidence of the learner's progress, for feedback and further development. There should be some record of his contribution he should have credit for its content and style and effectiveness. Perhaps the place for this record should be in the journal which the teacher maintains as a record of his/her teaching activities? An event like this would certainly also figure in the teachers regular reflections and ongoing evaluation of his/her learning sessions. In a primary classroom, the teacher is always undertaking informal assessment. Many teachers keep notebooks with a page for each child. Unexpected happenings and evidence of learning can be recorded as they happen. Such a notebook can help the teacher to ensure that, over the course of time, all children have been observed.
Practice
Using a scheme such as Roses criterion-referenced assessment grid raises a number of assessment issues for you to think about. Supposing there are five group members in the discussion exercise: 1. How many learners can Rose usefully and accurately assess during a 20 minute discussion? 2. Supposing a learner makes very little or no contribution, can/should Rose look at another discussion session to see if the learner takes part then? 3. What about conditions in the specific learning environment in which the discussion exercise takes place? Have you any recommendations for the physical conditions for action and assessment? 4. The issue of transparency - should the learners see a copy of the assessment grid before the discussion? 5. How can the teacher best prepare learners for such an exercise and assessment? 6. How might the teacher arrange individual feedback from the assessment to individual learners? 7. What issues of session and programme design might surface as a result of using such an assessment scheme? 8. Is there scope to record any other aspects of learning which Rose had not planned for or anticipated?
ANALYSING FORMATIVE ASSESSMENTS
Quantitative Data Teachers normally accumulate sequences of scores based on exercises held during the course of the module, term or part of the learning programme. They are part of the pattern of formative assessments. For example there are the short vocabulary tests used by language teachers, tests of arithmetic, spelling and short tests of skills. These tests may well have raw marks or percentages as an outcome. Numerical data will fascinate some teachers but may (unnecessarily) scare others! We need to be able to make some sense and some good use of this valuable store of information. Here are ideas to help you analyse numerical data. Supposing a large group of learners take the same short test, it is usual to get a frequency distribution of the marks. It might look like this distribution: Marks Number of learners (Total 113) 0 1 1 1 2 2 3 6 4 15 5 21 6 30 7 20 8 10 9 5 10 2
We can use these raw marks (each out of ten) to construct a frequency distribution histogram:
Test 1 35 30 25 Learners 20 15 10 5 0 0 1 2 3 4 5 Mark 6 7 8 9 10
The teacher needs then to evaluate the distribution. Given the stage of learner development, the purpose of the test and its level of difficulty is this distribution expected? Or are the learners scores skewed towards higher or lower marks? Three other measures can be useful in analysing performance data: mode mean median
The mode is the mark which occurs most commonly. So in our original table of marks the mode was 6. It occurs at the peak of our histogram. The mean is the arithmetic mean. It is calculated by adding all the marks together and dividing by the total number of learners. For our original scores the mean was 653 divided by 113 = 5.78. The median is the mark of the middle learner. So if you had eleven learners in your group and their test scores (out of 20 marks) were as follows: Marks 2 5 6 8 10 12 13 14 15 17 19
The median mark here would be 12. Notice here we have arranged the marks in order and there are as many scores below the median as there are above it. Means can be very sensitive to extremes in data sets but medians will be less affected by one of two very large (or very small) scores.
Ranking learners by scores can also be a useful exercise. For example: Marks Rank 2 11 5 10 6 9 8 8 10 7 12 6 13 5 14 4 15 3 17 2 19 1
All these measures are useful. Mean, median and modal values will give you a picture of average performance. Histograms show the spread of scores among the candidates. Ranking also shows spreads and the order of performances. QUALITATIVE DATA If you have a sequence of qualitative statements, descriptive paragraphs or notes, e.g. in point form, you cannot apply quick arithmetic measures. So most measures of performance tend to be converted into a quantifiable form (for example the criterion referenced scheme used by Rose in the previous section). However you can analyse qualitative data by commenting on the following: Variations in performance
Did the learner perform well (gain positive remarks) right across the range of assessments or did he or she perform much better in some exercises than in others? Did the learner produce performances which require special praise, attention and congratulation? These may be exceptional or typical in the learners list of achievements. Performance below the level required
Where this happens you can say whether it is typical of the assessed performances or whether it has occurred occasionally. POINT TO WATCH Comments and reports based on analysis of assessment data (quantitative or qualitative) should be objective, truthful and evidenced. It doesn't help the learner, the teacher or any other stakeholder in the learners record of achievement to ignore unsatisfactory performances. One of the most useful styles of analysis comes about when quantitative and qualitative evidence is used together. For example: Her performance has been consistently impressive: on no occasion did her marks in tests fall below 70% (mean marks 46%) and her rank order in tests was never less than 5th (out of 26 in the group).
Practice
Expected vs Observed Take a piece of paper and align it vertically so that the list of learner names is revealed but their marks for the last couple of exercises/tests are concealed. Briefly refresh your memory as to the nature of the last couple of assessments. For the first assessment quickly write down beside each name the mark or percentage that you think he/she ought to have achieved (the most likely outcome) - think clearly but dont spend too long pondering. Leave the next column on your paper blank, then draw in a third column and repeat the exercise for the second assessment. You can now reveal the two columns of actual (observed) achievements. For each learner now compare the expected marks (which you have in two columns on your piece of paper) with the observed (actual) marks which you have on your existing lists. Record the differences e.g. +5, +1, 0, -2 etc. The next step is where things get really interesting. In your differences columns are there any: Wild variations big negative and/or big positive differences? Spot ones zero differences? Performance as predicted? Small perhaps trivial or inconsequential differences which can be explained in part by minor variations in performance (and the crudity of the method).
How would you take this information further? Were the variations in the first column of differences repeated to any extent in the second? Are we actually looking at achievers and underachievers here or were there any other reasons for the differences? These may encompass variations in three aspects of your learners make up. Educational ability to respond to tasks of knowledge, skills and understanding Managerial ability to use assessment instructions, manage time, prioritise effort Personal health, motivation, mood, maturation aspects of performance.
Food for thought Marks and quantitative assessment can be fun. Really, they can! Have a look at the following. It is used by John Lewis in his book The Teaching of Skills and has caused much amusement at many a conference. The story begins with a class of twelve learners who took their 'in-house' examinations and the marks were collated. Totals were calculated and a rank order was produced as follows. The question is to whom should the overall prize for best achievement be given?
Maths Alan Belinda Charles Diana Edward Fiona George Helen Ian Jill Kenneth Lucy Fre Eng Geog Hist Bio Chem Phys
TOTAL
RANK
100 90 61 63 56 80 23 40 85 72 50 10
30 38 36 32 55 45 47 35 40 54 56 60
47 43 40 51 41 49 45 52 60 50 55 59
72 6 45 90 82 64 55 70 40 10 34 20
40 20 41 30 45 65 60 56 28 25 70 35
75 65 55 70 40 45 80 20 51 35 60 30
30 48 62 47 49 38 32 60 55 66 36 70
47 70 80 35 41 20 60 65 30 75 10 58
441 434 420 418 409 406 402 398 389 387 371 342
1 2 3 4 5 6 7 8 9 10 11 12
Well, it looks as though the overall form prize should go to Alan. But is this fair? The top mark for mathematics was 100 whereas the best mark for English and French was 60. Perhaps it might be fairer to scale the marks so that the top mark for each subject was 100 and the bottom mark in each subject was 0? If this strategy was put into operation, the results would look this:
Maths Lucy Kenneth Jill Ian Helen George Fiona Edward Diana Charles Belinda Alan Fre Eng Geog Hist Bio Chem Phys
TOTAL
RANK
0 44 69 83 33 14 78 51 59 57 89 100
100 87 80 33 17 57 50 83 7 20 27 0
95 75 50 100 60 25 45 5 55 0 15 35
13 30 0 38 75 56 68 90 100 44 63 78
30 100 10 16 72 80 90 50 20 42 0 40
17 67 25 52 0 100 42 33 83 58 75 92
100 15 90 63 75 5 20 48 43 80 45 0
69 0 93 29 79 71 14 44 36 100 86 53
424 418 417 414 411 408 407 404 403 401 400 398
1 2 3 4 5 6 7 8 9 10 11 12
You can go on with this. Supposing the top learner in each subject is given 1, the second learner is given 2 and so on, with the bottom learner being given 12. The figures can then be totalled and the learner with the lowest mark gets the prize.
Maths Fiona Kenneth Charles Helen Ian Edward Diana George Belinda Alan Jill Lucy Fre Eng Geog Hist Bio Chem Phys
TOTAL
RANK
4 9 7 10 3 8 6 11 2 1 5 12
6 2 9 10 7 3 11 5 8 12 4 1
7 3 12 4 1 11 5 9 10 8 6 2
5 10 8 4 9 2 1 7 6 3 12 11
2 1 6 4 10 5 9 3 12 7 11 8
8 5 6 12 7 9 3 1 4 2 10 11
9 10 3 4 5 6 8 11 7 12 2 1
11 12 1 4 10 8 9 5 3 7 2 6
52 52 52 52 52 52 52 52 52 52 52 52
1= 1= 1= 1= 1= 1= 1= 1= 1= 1= 1= 1=
This final table makes everyone equal! So we can see that different presentations can produce very different results. Food for thought?
PROVIDING FEEDBACK ABOUT PROGRESS
WRITTEN COMMENTS These need careful handling. You can use them for: indicating good work and giving praise highlighting unsatisfactory work and making connections showing where further explanation is needed pointing out where ideas/material are missing setting targets commenting on whether objectives have been achieved.
You need to be careful with the wording of these comments. Obviously you must avoid any threatening, abrasive and excessively negative language. You can get your point across to the learner without having to make destructive annotations! Instead, you should try to be informative, objective and constructive. Always comment on the work and not on the learner himself/herself. You can also address attitude and behaviour, but not at the same time as responding to the work being assessed.
ORAL FOLLOW-UP Many teachers follow up the return of written work with individual and/or group oral remarks. This is a good idea but you need to take care not to embarrass individuals in any way. You can always see learners individually if you know that difficult issues are going to be involved. Some formal assessments are based on complex learning activities such as simulations, games, extended project work, fieldwork and team exercises. These are often rewarding and exacting experiences for teachers and learners - but their assessment regimes can be equally complex involving several sources and times of assessment. So you need to allocate time for careful interpretation of assessment outcomes. Such follow-up sessions can have functions beyond assessment such as: review and reinforcement of themes and topics which learners have found difficult links with sessions which lie ahead in the learning programme advice on assessment handling technique motivational advice linked to further technical practice, private reading etc at this point the teacher might outline techniques for self-assessment by learners. These are particularly useful for revision programmes.
FEEDBACK TO PARENTS Feedback to parents of younger learners may be communicated in a number of ways, eg written reports delivered by hand or postally telephone calls (usually as part of pastoral systems) oral feedback at meetings with parents, parents and learners together, etc.
In each of these you need to be careful with tone and use of language. As with annotation of assessed work it is good practice to be professional, objective and constructive. You need to be able to support what you say with evidence. It is a good idea to have your files with you. If you mark work to high professional standards you can always invite parents to look at their sons/daughters work on a regular basis. Try to give some perspective on your comments, however. If you see signs of potential - say so. If there are problems, outline a few possible ways forward from them. Answer questions honestly and openly.
FEEDBACK TO OTHER STAKEHOLDERS Your learners may well be employees of a company, perhaps even the company which also employs you. Line managers, such as departmental heads, may need regular formal updates on learners progress in which case the same approach as that suggested above for parents is applicable. It may be that you receive a telephone call from a stakeholder demanding instant feedback. This requires deft handling by the teacher. Knee jerk reactions top of the head comments and other forms of reply are not advisable. They may not present a full picture of the learners range of performances. They will give little evidence and may be anecdotal and inaccurate. If you politely reply I will call you back you will have the opportunity to: research reflect report.
SELF ASSESSMENT BY LEARNERS Learners, like teachers, will improve their performance if they are encouraged to reflect on their own progress and performance. They will need guidance from you for this to happen and you will need to help them with processes like: identifying their own learning needs reviewing the rate and effectiveness of their learning progress updating and developing their own aspirations looking at learning outcomes still to be achieved when and if to consult outside agencies.
Introducing such processes is well and good but it is very frustrating for the learner if it is: introduced as an idea but never followed up outside the regular feedback/reporting system not acted upon.
Experienced and mature learners can often articulate their own feelings about their progress and would appreciate a framework on which to develop them. The teacher can help directly. Such self-assessment, done thoroughly, can be a very fruitful exercise which can also help the teacher plan further learning activities. Its use in coaching and other one-to-one learning activities is widely recognised.
The value of self-assessment by the learner as an integral part of providing support and guidance cannot be underestimated. Often in tutorials or review sessions, the teacher receives from, and provides valuable information to, learners on areas of progress and achievement and areas where further development is necessary to achieve learning outcomes. Problems can arise where the perception of learners about their progress and achievement is different from the perception of the teacher. Self-assessment may be defined as: The process in which the teacher asks the learner for information on his or her current ability to perform a task, or after a period of learning. The teacher does not offer any opinion which may bias this information in any way until after the self-assessment, but is seeking information from the learner on which to base constructive feedback. By finding out the learner's perception of 'where s/he is now' before venturing an opinion, you can pitch feedback on performance or progress at exactly the right level using a technique called 'framing'. Think of a painting you like. It is always enhanced by having the best possible picture frame around it. If the frame is not exactly right for the painting, it can detract from the overall effect. In the learning process, framing is where the teacher has already examined the facts about progress and achievement and has formed a view about the feedback s/he will provide and the action the learner needs to take to ensure progress (if any). Since the whole purpose of constructive feedback is to encourage the learner to take responsibility for his or her own progress willingly, the teacher puts a 'frame' round the content of the feedback based on the learner's self-assessment. Here are some examples of 'framing' in different contexts (the 'frame' is in italics) "Jane, I can see you think you are not doing well on the programme. Let me point out all the areas where I believe you have made excellent progress.." "Manuel, you have told me you think you are progressing well in computer skills despite missing several sessions. I know you want to make your career in this area. Let me tell you about where I believe you have progressed well then we will go on to discuss the areas you need to concentrate on if you want to be a good programmer.." "Ahmed, I know from what you say that you don't believe it is important to study independently. Let me explain the advantages of this and how it could improve your work."
Feedback provided in this way is far more effective and motivating because it is based on the opinions held by the learner. Without knowing these opinions, the teacher's feedback may not be pitched at the right level or be based on what the learner perceives to be the case about his or her progress. Constructive feedback by the teacher This type of feedback gives learners specific information about those aspects of their learning which have been done well and those where further development is necessary. Constructive feedback can be defined as: The process in which the teacher communicates his or her decision on the competence of a learner in performing tasks or in progress and achievement on a programme, usually after the learner's self-assessment of how s/he has performed. Constructive feedback involves praise for those aspects of learning or performance which have been done well, plus information on ways in which the learning or performance can be improved, rather than any criticism. Constructive feedback should be given at different points of the learning programme: immediately after a learning activity - to congratulate or provide information on where further progress or learning is needed at structured points of the learning programme - so that each learner is fully aware of progress and achievement to date, and any areas where further progress or learning is needed at specific points of the programme where problems have arisen and must be resolved promptly.
When teaching a new skill, constructive feedback should concentrate on the one or two main aspects of learning which needs improving. Any more than this might be demotivating in a situation where people are learning new skills.
To be effective, constructive feedback should be done quickly and well, keeping comments brief and to the point. Feedback is very important, especially when learning a new skill so remember the key principles:
The Principles of Feedback Give feedback as soon as possible and keep comments brief - dont let a feedback session become an inquisition Base the feedback on the persons self- assessment (where available) before commenting - how well does s/he think progress is being made? Give comments which are constructive and help to perfect skills Concentrate on one or two key areas - rather than measuring against perfection Be positive and give encouragement to motivate Refer only to progress made not the person - people can change their behaviour or rate of progress, but not themselves Offer alternatives - better ways of doing things - rather than criticism Stress the learning achieved and what will happen next Agree actions necessary to make or maintain progress and keep a record of this in readiness for the next feedback session
Obtaining feedback Remember that each review session with learners is an opportunity to gain valuable information about the aspects of the programme which are helping learners to progress and those where improvements can be made. If you can encourage learners to reflect on the value of the learning programme in meeting their aims and learning needs, you can receive essential information about the effectiveness of the programme itself and your own professional practice.
Practice
Many teachers may not have considered building learner self-assessment into their learning sessions or programmes. Such self-assessment is not only related to making assessment as 'transparent' as possible, but also may well add a different dimension to your professional practice. This is an opportunity for you to experiment with the technique to assess its feasibility and effectiveness in your particular teaching-learning context. There are design considerations, however: Consider learner maturity, in terms of: age/maturity, e.g. younger children will need some step-by-step help experience gained of the learning programme. Self-assessment may be something to introduce once the programme is well under way and the learners have developed confidence in it and their own ability to succeed in its various demands.
You'll certainly need to prepare the kind of initial guidance you'd give to learners (an outline of possible ideas is given in self-assessment by learners above - but the list is not exhaustive - you might wish to include other reflective components). Perhaps the most important (and interesting) section of your experiment is to decide on how learner self-assessment will be communicated and then developed by the teacher. A useful way forward might be to link self-assessment to some form of conferencing system with learners, where their work is reviewed and your assessment is matched to the learners self-assessment. New goals for the individual could then be agreed on the basis of shared input to reflection. We are getting into some very powerful evaluation here! It can also be linked to your record-keeping. There are of course constraints - time comes to mind straight away - but this is for the moment an experiment - a one-off. You can't make an informed assessment of its effectiveness until you've designed it, tried it and reflected upon it.
MAINTAINING RECORDS OF LEARNERS' PROGRESS
THE NEED FOR RECORDS Teachers need to give this area of professional practice more attention than ever before. There are many reasons for the increased interest in record-keeping including: Requirements of national government bodies concerned with educational achievement and standards for evidence of learner performance with accompanying remarks/commentary Inspection of educational establishments, teacher competence and other performance issues Increasing involvement of parents, management and other stakeholders in learner achievement. For example sponsorships and scholarships may require considerable material about individual learners progress as part of the application process The technology for keeping records is changing rapidly. Paper-based systems are being replaced by computer storage which can now be space-saving and very versatile. They enable hard copies and electronic transfer of extensive data to be made, literally, at the press of a button New assessment methods such as school-based assessment in public examinations often require greater accuracy, fairness and sophistication of record storage and maintenance.
Teachers are therefore accountable for ways in which they gather, analyse, store and publicize assessment data.
ISSUES These changes mean that you need to be aware of issues which are developing concerning the maintenance and storage of records of learners progress. Who has access to such records? What procedures are in place to give colleagues and line managers easy access to your records? Do the learners have any rights of access to these records? If so how and when can they exercise such rights of access? Confidentiality is a problem for teachers and learners. This involves taking and making records (say of review and conferencing sessions) as well as storing and using such information Security is also an issue. Its not just a matter of physical safety of filing cabinets and store cupboards, but system and network safety and the need to keep back-up copies and anti-virus protection for electronic storage systems. There may be insurance and even health and safety considerations at issue here, too Updating of records must be regularly (preferably immediately) and thoroughly carried out. Design of well-planned, easy to operate, easy to replicate paper of software systems is well worth time and effort Teachers need to be on the lookout for new developments in information storage and retrieval systems, as they need to keep ever-increasing volumes of data about learner development.
MANAGEMENT OF INFORMATION This has become a crucial issue for educational establishments and training organisations. Some learning programmes involve many different assessments and the number of learners involved may number thousands per year. Data quickly accumulates but the same demands may be placed on its input, storage and retrieval. Teacherassessed and practical assessment schemes may make big demands on space for learner portfolios, files of work and product of practical work. Given the importance and sheer size of such material and data flows, we need to ask questions about human resources. Who manages these systems? Who operates them? Who is responsible for maintaining them? Who advises on legal systems which may change and in turn cause new requirements to operate? Who in the institution is ultimately responsible for such activities?
POINTS TO WATCH Some societies are becoming increasingly litigious. Education and training can be caught up in this. Any form of legal action will ultimately call for evidence and evidence is mainly found in records. The more complete, relevant and up to date your records are the better position, you, your learners and your employers are likely to be in. Issues of management may call for changes to be made in your information systems. This may well be a new requirement in your programme design, or one to which you need to give more importance or attention.
Practice
Remember: performance records are information which is in demand such records form part of a range of professional information important issues surround access to storage and retrieval of such information.
AND NOW AN INSPECTOR CALLS!
A simulation: Imagine that you face an official external inspection within the next 28 days Your head of department or training manager has given you the task of reporting on your 'maintenance of systematic, useful and usable records of learners progress. Ever mindful of your enthusiasm for and skill in handling issues such as this your line manager has proposed the following steps in producing your review. Conveniently these address possible questions which the visiting inspector might pose.
1. Current procedures On a pro-forma like this indicate how information on your learners performance is managed and maintained (you will need to produce a larger version - this version is to show column headings only). Examples are given.
Types of assessment information recorded
observations of drama techniques
Method of recording
notes made on observation of performance
Method of storage
placed in files stored in secure staff office
Access
me H of Dept
Additional notes
Files are colour coded by course: yellow = basic red = standard blue = advanced
2. Management of information Who in your institution is responsible for: Direction of overall input, storage and retrieval policy? Designing and maintaining records of assessment? Supervising security and storage of record material? Updating staff on legal and regulatory developments regarding records e.g. data protection legislation? What, if any, training have you received on issues affecting the nature and maintenance of such information?
3. Recommendations for development What recommendations would you set out for improving your own use and management of assessment information?
PREPARING SUMMATIVE ASSESSMENTS
Summative assessment is used at the end of a programme formally to assess a learners skill and knowledge gained as a result of that programme. The table provided below illustrates assessment methods and gives advantages and disadvantages of each:
Assessment method 1. Observation of performance Advantages The most reliable method of assessing performance of practical activities Can be used to assess application of knowledge and understanding into a real work environment as well as skills performance If carried out using a welldesigned checklist, observation can capture evidence of performing a range of subject or vocational skills plus interpersonal and communication skills to measure actual performance against desired outcomes Most useful when used to individuals rather than groups Observation can be carried out by others (e.g. supervisors) and used as evidence of reaching learning outcomes May require more than one observation to assess consistency of performance Can be combined with oral questioning to confirm knowledge and understanding or any areas not observed 193 University of Cambridge International Examinations 2010 Disadvantages Time consuming, especially if teacher has to travel to carry out observations May not be suitable for assessing in group situation as evidence of individual performance must be recorded simultaneously Not all skills can be observed readily Requires careful planning for situations when teacher can observe the maximum possible range of tasks being carried out Learners may feel under pressure Learners may be nervous when they know they are being observed and may make mistakes If carried out under simulated conditions, may not assess learner's ability to perform in a real work environment (see simulation)
2. Skills test
Can be effectively used to assess ability to carry out a practical task to the required standard after training has taken place, e.g. speed, accuracy, performance to standards Learner completes task against given parameters which are then used to measure successful performance Can be used to assess application of knowledge and understanding into a real work environment as well as skills performance Can be effectively used as tool to measure current level of performance and note areas where skills must be improved during diagnostic assessment Successful achievement may be measured through observing skills test or by examination of products completed during skills test May be carried out through computer-generated programmes where learners work through scenarios and have to successfully complete each one before starting the next
As with observation Can be expensive to set up in terms of provision of resources and equipment
3. Simulation
4. Projects and assignments
Can be effectively used to simulate realistic conditions, e.g. flight simulator If conditions for simulation reflect pressures and constraints of a real work environment, enables learner to perform tasks in a learning environment before undertaking them in the work environment Suitable for assessing performance where assessment of performance in real work environment is not suitable, e.g. where unsuccessful performance could cause danger or damage, e.g. assessing ability to fly an aeroplane Successful achievement may be measured through observing or by examination of products completed during simulated activity Useful for providing opportunities to learners where real work environment is not available, e.g. provision of a model office where learners can practice skills by carrying our real tasks for members of staff May be used successfully using appropriate IT programmes Work-based projects are an effective method of assessing application of skills, knowledge and understanding in a real work environment Can be used to assess learners problem-solving and decisionmaking strategies as well as ability to complete to required standard Can be used effectively with open or distance learning Most effective if used with marking scheme and grading criteria (if relevant) which provides basis for assessment and can be used for fair and objective assessment of results
May be difficult to reflect pressures and constraints of a real work environment Expensive to set up and resources and equipment required for successful simulation may not be available IT skills may be needed and success may depend on how well these are used
May require a level of literacy and/or numeracy which is not a requirement of the task Need to ensure that material submitted has been completed by the learner Requires careful preparation of learners by teacher to ensure ability to complete and full understanding of what is required
5. Oral questioning
Can be combined successfully with observation, skills tests or simulation to assess knowledge and ability to apply understanding in different contexts Useful where learners have literacy problems or disability as alternative to written tests Useful to assess retention of knowledge Multiple-choice tests and those which require learner to adapt learning to provide answers offer opportunities to assess understanding All learners complete same test under same conditions, thus is an objective measure
Requires considerable expertise from teacher in interviewing and questioning techniques Learners are under pressure or may be nervous
6. Written and oral tests
Assesses knowledge and possibly understanding if well designed Considerable expertise is required to ensure tests are fair and at correct level Adult learners may have been adversely affected by past experience in the educational system Does not provide a true reflection of a learner's ability to perform tasks May require literacy or numeracy skills which are not immediately required for performance of a task Rationale of tests depends on ability of learner to recall selected information 'on demand' May not provide the opportunity to review performance and identify future learning or actions unless these are planned into time available Requires 'examination techniques' on the learner's part Otherwise, as for 'tests'
7. Examinations
As for tests Examinations can be used effectively as a final measure to assess achievement of learning outcomes on a learning programme
PREPARING YOUR LEARNERS Formative and summative assessments complement each other, assessing progress and achievement in relation to the learning objectives. This continuity should be clear to your learners as well as to you the teacher, and any other stakeholders in the successful delivery of the learning programme. So the summative assessments themselves should hold no surprises in terms of concepts, topics and skills for those who have to undertake them. Your learners need to be prepared. You should properly introduce summative assessments to your learners. They should be aware of: the purpose of the assessment process the nature of the assessment objectives and criteria for success appropriate methods of response to/involvement in assessment methods any revision or preparation requirements permissible learning aids e.g. calculators, laptop computers regulations about what cannot be used in the assessment room, e.g. programme notes, mobile telephones the exact duration of the assessment sessions : dates, times and venues arrangements for the communication of results and feedback regulations from the external awarding body (if any) or company administering the training programme.
POINTS TO WATCH You should be well prepared yourself for these sessions. You should have to hand necessary exemplar material, schedules and copies of regulations (if these are appropriate/relevant to learners). You will need to be formal when administering summative assessments but in practice assessments and preparatory sessions you can reassure learners should be reassured and reduce nervousness. With young learners, many teachers undertake summative assessment procedures without taking the learners through all of the items above - this is a matter of judgement about how much and what information will reassure. Also younger learners will need to know the procedures they can follow e.g. can they ask questions? can they use dictionaries?
ASSESSING FINAL ACHIEVEMENT Assessment of all agreed learning outcomes may take place in several ways, depending on the types of outcomes required, the assessment strategy selected and the centre in which the learning programme took place. In an educational institution, final assessment may be through: formal examination, at the end of the programme or part programme (module) a mixture of coursework and module tests or examinations module assignments or projects, each contributing towards the final marks obtained a portfolio of work gathered over the length of the course.
In a training organisation or a company where the learning focus is skills development, final assessment may be through: skills tests assessment of performance in the learners work environment e.g. through observation evaluation of information from the learners supervisor or line manager relating to successful and consistent performance to the standards required.
DESIGNING YOUR OWN SUMMATIVE ASSESSMENTS You will find it useful to gain experience of this and the various disciplines involved. Opportunities may arise for such design work at the end of modules for internal school or college examinations or tests. If you are going to design your own summative assessment material, these points will help: 1. Check out the relationship between programme aims and objectives, syllabus and session content and your summative assessment objectives. You and others should be able to identify continuity between them In the case of written assignments such as tests and examinations make sure that you design a full mark scheme to accompany your question paper. This is particular important if more than one person is going to mark learners' work. When setting out your question paper pay particular attention to: The rubric (instructions to candidates - including time allowed) The layout and wording of questions. Questions should be clearly and unambiguously worded Options and sections should be clearly identified Any additional material (such as maps, diagrams etc) should be clearly produced and headed Check that the learners know what has to be handed in at the end of the assessment session
2.
3.
Ensure that the allocation of marks is clearly shown on the paper itself Make sure that sufficient copies of the test/examination paper are available with spare copies in the assessment session. Check validity and reliability.
The following table illustrates possible assessment strategies for the activities listed:
Activity 1. Practical tasks Most suitable assessment methods
Observation combined with oral questioning to confirm knowledge and understanding
2. Activities in small groups
Observation where assessment of the process is a key component, e.g. where learners are developing interpersonal, communication, and teamwork skills Examination of the results of activities Learner's self-assessment may contribute Video-recording of activity and scrutiny of final tapes
3. Simulation
Observation Examination of products created during simulation Video-recording of simulation and scrutiny of final tapes
4. Role-play
Observation Learner's self-assessment may contribute Peer assessment
5. Assignments and projects
Scrutiny of assignment or project using marking scheme as basis for assessment Scrutiny or discussion of written or oral summary of findings
6. Case studies
7. Business games
Scrutiny or discussion of written or oral summary of findings Observation of process Observation combined with oral questioning to confirm knowledge and understanding
8. Skills practice
ADMINISTERING SUMMATIVE ASSESSMENT You must be careful with the environment in which assessment takes place. For example, if learners are asked to prepare for a one hour set of written tests, then the room must be quiet, well-lit, well-ventilated and so on. There should be restrictions on conversation and comment so that everyone has the same chance to think and write in comfort and confidence. This enables equal opportunity for each learner - the level playing field. The term level playing field is commonly used in discussions about assessment. It has come to mean a combination of concepts which include: 1. Fairness/equality of access to assessment
In both formative and summative assessment teachers should ensure that all learners involved in the assessment exercise should have: Equal notice of the time, date, venue of the exercise Clear instructions as to what materials they are to bring along A clear picture of the skills, knowledge and understanding to be assessed Guidance over what to prepare and how Time schedule for marking, results and feedback. 2. Fairness/equality of operation The assessment environment should be managed in such a way that: All learners have the requisite test/examination materials, paper, etc There is silence within the room so that all may concentrate at ease Matters of comfort such as light/shade, temperature, ventilation, etc. have been dealt with in advance Teacher supervision ensures that no form of copying or discussion takes place during the assessment session External interruptions and disturbances are minimised Timings are clearly stated and adhered to. 3. Marking, results and feedback The time schedule for these should be familiar to all learners and should be adhered to There should be as little variation as possible in scheduling if more than one group undertakes the same assessment exercise Clearly the same mark scheme must be operated and the same methods of results communicated and feedback to stakeholders employed if more than one group undertakes the same assessment exercise.
Such practical considerations need even more attention when important observation of performance assessments are taking place (eg for coaching and sports certificates, drama and music qualifications, oral examinations for languages, workplace skills, teamwork demonstrations etc). Here environmental considerations can include the noise levels and activities outside the test location as well as conditions inside the venue. Sports and outward bound assessment need to pay careful attention to: weather forecasts and conditions issues of health and safety transport to and from outdoor venues suitability of candidate's clothing.
MORAL ISSUES As well as fairness in assessment (the level playing field) a number of moral issues may arise in the context of assessment. These include.: Transparency To what extent should our designs, purposes and operation of assessment regimes, justification of choice of assessment and communication of results and feedback be transparent? Why, to what extent and to whom should such disclosure be effected? There is no single simple answer to these questions. There may be school, local or national regulations to guide you on this issue or it may be left to your own professional discretion. If there is choice available on this issue be careful to consider the likely consequences of your options. Performance and Personality It is important to emphasise, especially in feedback to learners, that any criticism you may advance is of a learners performance and not of his/her personality. Nervousness and Sensitivity Learners are understandably nervous about three stages of assessment: Preparation for assessment exercises (revision) The assessment exercise itself Feedback; getting the results and dealing with the feedback and other consequences An understanding teacher will reassure and clarify in the first stage, administer the second stage rigorously and fairly and handle the third stage sympathetically and honestly.
Public v Private Results/Feedback This may be a difficult balance to strike, especially with summative assessment results. Many teachers feel that a factual communication of results by lists on notice boards, websites, postal information, etc followed by individual feedback which stresses ways forward for the learner is an effective combination of methods especially in cases of difficulty and/or disappointment. Even in the communication of formative assessment results care needs to be taken when work is returned to learners in class. Those with problems can be helped by: Realistic and constructive comments on returned scripts A quiet chat after the class
Practice
Revising assessment procedures Check out these questions against your experience with a summative assessment: Validity - did the assessment test all required levels and subjects? Relevance - did the assessment meet the needs of all stakeholders? Did the learners view the assessment as valid and reliable? Was the standard of the assessments appropriate to the programme? Were most learners able to succeed? Was the time allowed for each assessment task reasonable? Were tasks clear, technically correct and appropriate? Were tasks relevant - not going beyond the content of the programme?
POINTS TO WATCH There are number of factors which may impact on the teachers ability to be fair and impartial such as:
learners are unaware of, or do not fully understand, the criteria against which they are being assessed learners are assessed before they are fully prepared and at a level where they are able to demonstrate competence learners are nervous and intimidated by the environment or administrator of the assessment process, and make small mistakes as they perform tasks, although they have the ability to perform competently learners assessment is carried out using methods that are inappropriate to the skills that must be demonstrated although criteria for successful performance have been set, the teacher adds additional standards based on his or her opinion of what the learner must demonstrate or interprets the criteria in too stringent a manner the teacher does not like the learner and is biased OR the learner does not like the teacher and does not expect a fair assessment younger learners may be fearful of expectations, processes or outcomes.
Avoid these pitfalls and you will be
USING SUMMATIVE ASSESSMENTS
SEQUENCE OF USE Here we are managing summative assessment to ensure fairness and to facilitate achievement. This is a sequence of linked processes: 1. Design of assessment tasks and mark schemes 2. Instructions of and preparation of learners 3. Management of assessment sessions 4. Assessment of outcomes - marking/grading 5. Results and feedback to learners 6. Evaluation of the summative assessment process
CASE STUDIES In this section we are going to look at two case studies:
Christina is dealing with a summative assessment exercise for her sequence of drawing classes
Etienne is setting up an end of module examination in AS level physical geography. Case Study 1 Christina has decided to run a 2 hour practical assessment of her learners drawing skills. The assessment objectives follow closely the technical instruction themes carried out in the sequence of learning sessions. These include: ability to use line, methods of shading composition use of tone (light and shade) confidence in handling appropriate materials observational skills.
She has devised an assessment grid which includes her marking scheme. The levels of performance are graded from poor to acceptable to outstanding. There are no quantitative marks as such.
Her assessment system looks like this: (only part of the grid is included)
Skill Requirements Level of achievement 1 2 3 Poor Acceptable Excellent Assessor Signature Date
Ability to locate drawing sensibly and accurately on paper Ability to apply basic principles of composition to still life assembly Ability to use line and shading confidently and appropriately
This scheme is Christinas own design. Case Study 2 Etienne has set up an examination paper for his AS level geography learners. This is how it begins (again, only part of the paper is shown) Question paper AS LEVEL PHYSICAL GEOGRAPHY INFORMATION FOR CANDIDATES The number of marks is given in [ ] at the end of each question or part question All the figures referred to in the questions are contained in the Insert Sketch maps and diagrams should be drawn whenever they serve to illustrate your answer Make sure your name is on each piece of paper you use for your answer and you indicate clearly which question you are answering Answer one question from Section A and one question from Section B TIME ALLOWED 1 HOURS
Section A Hydrology and Fluvial Geomorphology 1. Study diagram 1
(a) What kind of rainfall is being generated on the diagram?
[1]
(b) Give an example of an area of the world where this type of rainfall is frequently seen [1] (c) The diagram shows a cross-section through a drainage basin. What effects might the basin suffer if much of its vegetation is suddenly removed? [11] (d) What kinds of human and physical forces might be responsible for the rapid removal of vegetation from the basin [12] 2. (a) Outline the processes by which rivers erode their channels (b) Explain the effect of erosion on the form of river channels [7] [8]
(c) Using examples discuss the effects human activities may have upon the different flows that occur within a drainage basin system [10]
Mark scheme Here is part of Etiennes mark scheme for question 1 (a) orographic or relief rainfall (b) west coast of USA or South America (except desert areas of Peru and Chile) (c) for 9-11 marks High quality explanation with abundant illustration: wide range of points covered. Mentions and explains incompetence of soil from removal of root system; collapse of channel banks; increased sediment yield; clogging of channels, lakes and reservoirs. Reduced transpiration from surface. Reduced interception therefore more direct pounding of surface; gullying and other accelerated erosion. Destruction of humic layout of soil with reduction in soil fertility. Rapid run-off. e.g. Cumberland or West Virginia in 1930s; slopes of 1990s Madagascar; deforestation in Nepal, Brazil, Indonesia. for 5-8 marks Explanation of some of the above points with some examples. Partial answer. for 1-4 marks Poor quality level of explanation; answer skeletal with one, two or no examples. Only a few explanatory points identified. (d) for 9-12 marks High quality explanation with wide use of a range of illustrations. Mentions and explains: (i) Physical forces of removal: fires (natural) e.g. Rocky Mountain National Park; volcanic eruptions e.g. slopes of Vesuvius, Etna, St Helens; increasing acidity e.g. Sahel (ii) Human forces of removal: creation of ski-runs/slopes e.g. French Alps: Albertville; overgrazing e.g. West Africa, Sudan; fires (accidental and deliberate) e.g. S France, Portugal 2003 for 5-8 marks Explanation of some of the above points with some examples. for 1-4 marks Poor quality level of explanation; skeletal answer only. [1] [1]
POINTS TO WATCH 1. When he was developing his summative assessment paper, Etienne modelled his rubric (instructions to candidates) on past examination papers. His question 2 is in fact taken verbatim from the specimen paper for AS Geography which was published by Cambridge International Examinations. These steps give his learners some experience of summative assessment which is close to the kind of examination experience they will shortly be preparing for. 2. On the other hand Christina designed her entire assessment scheme. It has some weaknesses and may lead to a high degree of subjectivity. However it could be developed into a more effective criterion-referenced scheme of assessment. For example she can improve the descriptors for levels of performance. 3. Many of your teaching colleagues can be of direct help to you - even if they conduct entirely different learning programmes. A second opinion on assessment objectives, layout of questions, mark schemes and wording of assignments is often very useful. A colleague will often identify omissions, highlight ambiguities and suggest amendments from his/her experience. This is good professional practice and its good to give your summative assessment material such a critical read/review before it gets to the learners! 4. If a number of your learners use English as a Second Language (and the test is in English) you need to check that they are disadvantaged by the vocabulary used in the assessment. REVISING ASSESSMENT PROCEDURES In all programmes it is necessary continually to check and perhaps revise the assessment procedures being used. You need to ensure that the validity of assessment (checking against a syllabus or Programme Plan to ensure a representative sample of topics and abilities are being assessed) and the reliability of them (through a mark scheme to ensure that all assessments are consistent for all) continually improve. Changes can and should be made were it is found necessary or desirable. It is useful periodically to ask the following questions: validity: does the assessment assess all required levels and subjects? is the assessment meeting the needs of stakeholders how do the learners view the assessment (valid and reliable?) is the standard of the assessments appropriate to the programme (check against the Programme Plan)? are most learners succeeding? ( are assessments too difficult, too easy, not related to the teaching); if not, the reason will need to be identified
are results what you expected? If not, why not? were assessments achievable in the time allowed? do assessments allow for the assessment of all levels of ability, or were some disadvantaged because assessment tasks (e.g. questions) were too easy/hard and were therefore unable to show their true skills? were tasks clear, technically correct and appropriate? were tasks relevant - not going beyond the content of the programme?
Assessments must be fair, objective and impartial. Internal assessments need to be above reproach with no bias, and the scrutiny by external agencies e.g. awarding bodies or internal personnel help to ensure this. Most educational institutions will have verifiers who check assessors work and external agencies will moderate a centres grades and systems. There are a number of factors which may impact on teachers ability to be fair and impartial. Such factors include: lack of awareness by learners of the criteria for assessment learners are assessed before they are fully prepared learners are intimidated by the environment, teacher of the assessment process inappropriate assessment methods additional criteria added lack of time or resources for assessment.
Practice
Check out the room in which your summative assessment session is to take place and the staff who may be involved in it. 1. Step One Talk the Talk Imagine you had to hold a summative assessment session of your own design (i.e. outside the normal schedule of public examinations, etc) (a) who would you contact to check room availability? (b) who would you contact to arrange invigilation (supervision) if you could not be available yourself? (c) what arrangements would you have to make with the invigilator to: check presence or absence of examinees? Give instructions to examinees? supply script and question papers? return scripts and question papers (if required)?
2. Step Two Walk the Walk Go to the room selected and check it out for yourself for the following: potential causes of exterior disturbance e.g. building work potential causes of interior disturbance e.g. noisy fans count numbers of desks and chairs. If insufficient, where can extra supplies be obtained? Include chair and table for invigilator arrange furniture so that desks are equally spaced construct notices asking for Quiet Please; Examination in Progress any potential difficulties with heating, lighting, ventilation and/or air conditioning.
ANALYSING SUMMATIVE ASSESSMENT DATA
MARKING In written summative assessments it is essential to mark work with accuracy, consistency and thoroughness. This means you need to refer closely and carefully to mark schemes and assessment criteria. If you know you have a great many scripts (i.e. completed assessment written work from learners) to mark, be fair to yourself and your learners by breaking up your marking sessions, taking proper comfort breaks and maintaining your concentration. If you are part of a marking team you may have to submit samples of marked work to the team leader who then checks your marking and compares your marking with those other markers in the team. There may be a meeting then to agree how the marking will be interpreted and details managed. This process is called standardisation of marking. It is an essential part of the formal assessment of public examinations. In a skills-based context, similar advice applies, although here you could well be dealing with a series of exercises involving use of assessment criteria. This can be very taxing and a great deal of concentration is required. Moreover, such assessment may require more than just circling of letters of numbers on a scoresheet. Some of these assessment systems ask for the provision of comments and remarks. This is not easy at the best of times and teachers who work in outdoor environments can describe the difficulties of writing assessments in snow, blistering heat and driving rain!
Not many teachers see marking as being the highlight of their professional activity. Can it be elevated above the level of necessary chore? It can certainly be made more interesting and more worthwhile if it is geared to a variety of assessment designs which are in turn, linked to specific sections of teaching and learning. It therefore shows learners varying abilities to respond to different learning experiences. If we see marking as part of a process or design, enquiry and professional development then it can act as a key source of information on the way in which learners can act as a key source of information on the way in which learners can develop their own skills, understanding and knowledge. Marking is at its fairest and most productive if a mark scheme is consistently developed applied. Make marking work for you. Merely ticking, correcting and scoring does not make the most of a valuable opportunity to feedback directly to the individual learner. Not many such opportunities may exist. So take the opportunity to: Highlight successes (however small) Correct errors or misconceptions Say what a mark means Show the learner ways in which his or her work can be developed Set new goals try this
Marking makes much more sense if it is seen by learners and teachers to perform a series of valuable purposes. So it must be carefully targeted and interwoven into the Programme Plan. It should be a natural follow-up to learning and assessment objectives. These, not some misconceived ritual or habit should drive the frequency of marking.
PROCESSING OF EXAMINATION DATA Let's assume that we have collected summative assessment data from our group of learners who have just completed a written examination. The examination was designed and conducted in-house and there are 25 learners in the group. Have a look at the table which shows the results obtained. Weve substituted learners names with letters of the alphabet.
Candidate A B C D E F G H I J K L M N O P Q R S T U V W X Y Percentage 61 92 57 68 48 32 43 57 37 88 20 47 86 70 57 44 85 45 50 57 45 51 46 56 55 Rank 7 1
8=
6 16 21 22 8= 23 2 25 17 3 5 8= 21 4 19= 15 8= 19= 14 18 12 13
You will see from the table that we've already processed the data by producing a rank order, giving the relative positions achieved by the learners. We can see the range of the data from the highest (1st) of 92% to the lowest (25th) of 20%. We can calculate the mean mark - 55.88% the median - 55% and the modal mark - 57%.
Very often such measures are recorded on and form the basis of summative assessment reports. How much detailed analysis you enter on such reports is determined by: i. ii. iii. iv. school or college or departmental policy your own preference perhaps a combination of i and ii requirements of local or national bodies
You can, given the choice, extend your analysis by reference to percentiles, standard deviation and other such measures. But theres an important question for you to consider here: Who is going to be making use of such data? There are two main groups of users: 1. Learners, parents and other stakeholders These will want to know the marks or percentage achieved i.e. a measure of outright/raw attainment how the learner did relative to other learners in the same class/group/year/cohort what the teacher made of this performance.
In these circumstances raw marks or percentages, plus rank order and mean marks (for set and/or group/year) will give an accurate numerical response to the first two requirements but what did you as teacher make of the performance? Here we need to look at the relationship of the expected to the observed performance there is no reason, having constructed the test paper and the mark scheme, why you should not pencil into a fourth column on the results table the mark or % you would expect each candidate to achieve. In a fifth row you can indicate with + and - signs the differences between the expected and the observed performance .... an interesting exercise!
2. Departmental, school, college users Here there are other considerations such as how did groups perform relative to each other. We can look at comparisons of marks ranges, means, medians, and modes and we could if necessary look at percentiles and standard deviation. We can also look at year-on-year comparisons for the same or similar tests. You can end up handling great masses of data (as examination boards do) and there is no doubt that computerisation can help you enormously especially through adept use of software packages (eg spreadsheets).
SKILLS-BASED DATA Supposing our group of learners has been undertaking a course in communication skills. Such skills might include: making a presentation speaking and listening conducting an interview note taking producing a short report. As we've already seen, many such skills can be assessed using a criterion-referenced assessment scheme. These feature levels of competence or performance plus descriptors of qualities needed from the learner to establish themselves at that level. A simple example of such a scheme in the UK is the Diploma of Achievement 16-18 skills-based course. In this scheme a six mark range is employed but the descriptors are written for 2, 4 and 6 only eg: Making a presentation 2
The presentation is complete
4
As 2, and the presentation is interesting and competent
6
As 4, and the presentation is confident, enjoyable and appropriate to the audience
Here you have three levels of achievement. But when the scheme of assessment was trialled, teachers thought it better in practice to include three other marks - 1 (not quite a 2), 3 (not quite a 4) and 5(not quite a 6). This was a useful suggestion, enabling teachers to make finer judgements, which was in the end adopted into the assessment scheme. Notice that we have written marks in single inverted commas because strictly speaking 2, 4 and 6 are not really marks. They are levels or bands of skills attainment. They could just as easily be called gold, silver or bronze or A, B, C or whatever. You will come across similar criterion-referenced schemes. You can use limited numerical analysis of performance but be very careful of what such measures actually mean. Much depends on how the descriptors are constructed and worded and what the designers expectations of performance were and are.
Practice
Next time you're involved in the processing of summative assessment data, try working as a team. When examination boards undertake this task they work through a number of examination teams, each with its team leaders, chief examiners and so on. Your team could actually manage, process and evaluate the entire summative assessment system for your particular set of responsibilities. Individual team members can be assigned to: draw up possible questions, assignments and assessment tasks produce mark and assessment schemes develop score sheets, marksheets, report forms handle the marking of a component, question or paper be responsible for making calculations based on data produce graphs, charts etc of data produce written analysis of findings from data processing.
The whole team could finalise task designs agree mark schemes standardise marking and assessment procedures help process data evaluate the assessment exercise(s).
Alternatively, if you work in a small institution (eg a small primary school), you can collect example of learners work for each of the levels identified in the institutions curriculum. You can annotate samples with descriptors of ways in which they have met the required standards (the criteria/mark scheme employed).
PROVIDING FEEDBACK ABOUT ACHIEVEMENT
'HIGH STAKES' Communicating results of summative assessments to learners is always a momentous event. You can use all kinds of methods including sending results by post, making them available for collection at institutions, pinning them up on university noticeboards, reading them out in learning sessions and even publishing them in newspapers. The results of examinations, especially public examinations, performance on training courses and skill tests are becoming more and more important. They are pivotal to academic, professional and vocational qualifications. They are not only indicators of success but also milestones in people's lives. They will almost certainly affect prospects for future courses, future employment and future life styles. Of course the most important and immediate 'consumer' of such results of summative assessments is the learner himself/herself but there may well be other users of such information: Within companies results of training programmes held in-house or externally may well affect the direction of a trainees' career within the organisation. Some employment activities cannot and will not accept applicants who do not possess nominated fundamental qualifications. Employers are users of this data. Success in one set of summative assessments may allow entrance into new courses and further qualifications e.g. the transition into higher education depends to a great extent on summative assessment data. The institution itself will wish to record and store internal and external summative assessment data. Even once the learner or trainees has left, others may request information about individuals' performance for reference purposes. In some cases the publication of a schools results will affect parental choice of school for their children.
Given the growing importance attached to performance in summative assessments it is vital for us to consider the way we as professionals support learners when they receive such results. FEEDBACK FOR THE NEXT STAGE Where groups and/or contacts with learners resume after the results of summative assessments have been made known, the teacher should: Debrief learners on the outcomes of the assessment, being sensitive to the emotional needs of the learners Set out ways forward for all the learners, reassuring those who have not done so well. This will include details of new courses to apply for, new sources of information, details of retake opportunities where applicable and other potentially useful ideas. It is important to be positive as well as realistic. Clear up any individual problems arising from marking or other technical issues.
This kind of feedback session needs to be carried out promptly once the summative assessment results have been made known. FINAL FEEDBACK Where the learners leave the institution and have no more learning sessions with the teachers - usually a learner's situation after external examination results. Here the learners may call at the centre to find out their examination results Where this happens (and it happens at most schools or colleges) then staff should be available on-site or at least on-call to deal with enquiries and agree new career or educational strategies as and when these may be required. It may be necessary for learners who are applicants to colleges or universities to telephone these establishments. It is good to set aside a room with telephone facilities for this purpose. Directories, career data and notebooks also come in handy. There may be a health and safety issue. These are stressful events for even the most 'laid back' of learners and there might be some risk of illness or injury if results are disappointing. Senior colleagues may need to be present to deal with the possibility of queries about marks/grades. Stress is also an important feature in the lives of younger learners. Knowing the learners is a key quality for a sensitive teacher to have.
Practice
1. For teachers of older learners: Canvass the opinions of former and current learners who have undergone the stress and worry involved in obtaining external examination results. Some questions for them include: How did they feel about the way their learning centre handled the distribution of results? What was their view of the role played by learning centre staff in acting as counsellors and 'back-up' in the hours after results were made known to learners? Can they suggest any ways in which: results might be better, more thoughtfully, distributed? the facilities open to learners at 'results times' might be improved?
2. For teachers of younger learners: Consider how learners are helped to cope with success and failure. This could be done in the context of winning and losing school matches, quizzes etc.
MAINTAINING RECORDS OF LEARNERS ACHIEVEMENT
FORMAT? We've looked at problems involved in storage, access and security. Now we're going to focus on the format of records. In one way this seems to involve a question with a straight choice between: hard copy or disk storage? Increasingly institution and teachers are opting for the latter, though there are advantages and disadvantages with each format. HARD COPY? Advantages It's already in existence - a known set of methods such as filing and cheap, easy to operate systems such as folders, filing cabinets etc It's paper-based and so copies can be readily made It requires no power source other than human labour to operate it It requires little investment in machinery and training though it can become unwieldy without photocopying facilities.
Disadvantages It's bulky and time-consuming. Special rooms and cupboards may be needed for physical storage of files, registers etc It's paper-based. Paper is both heavy and inflammable It's clumsy in terms of entry of data onto pro-formas etc It's not easy either to use for cross-referencing and or to access and replicate.
DISK STORAGE Advantages Disks, CDs, etc can hold enormous amounts of data on a very small physical space Once saved, data can be quickly and easily retrieved (but also easily erased!) Even cheap printers can produce copies of very acceptable quality very rapidly Spreadsheets and other software applications make entry and processing of information easy The price of PCs is gradually decreasing There is little doubt that technological developments will make this option even more attractive in the future in terms of size, speed, applicability, versatility and price The machinery concerned takes up very little space and can be operated in a wide variety of teaching/learning areas. Laptops, especially, make proper recording of data highly mobile.
Disadvantages Cost: equipment is still more expensive to purchase than hard copy but it is decreasing and reliability is increasing Distribution: if the institution is to purchase these machines then which members of staff, which departments will receive them and where will they be located? Training : some basic training in procedures and keyboard skills may be required Security: of the machines themselves. As they are versatile they do have resale value so are attractive targets for theft of material stored on disks, hard disks, etc. This can be attached by hackers and corrupted by viruses
Individuals: computer-based records work well for the recording of statistical information but may not reflect other areas of a learners progress and achievement.
Practice
Think about the following questions: How long should you maintain records of learners' achievements? What kind of material needs to be stored for a long period? Does your learning centre have any policy framework for dealing with the difficulty of length of storage of information like summative assessment results? Can long-term storage be concentrated around one or two 'master' or 'central' storage facilities?
POINT TO WATCH Is your institution affected by regulations brought about by data protection legislation and, if so, how? Check your understanding if data protection legislation.
Guide for the Cambridge International Diploma for Teachers and Trainers MODULE 4 : EVALUATION
Module 4
EVALUATION
EVALUATING LEARNING
REFLECTION You've planned and taught your programme, and assessed the learners progress and achievement. You've been keeping notes and reflections which you have kept since you started your Diploma experience. Let's remind ourselves about the Kolb experiential learning cycle
Looking at the diagram you can see how we could almost rename it 'the reflective learning cycle'. Reflection is a most important part of our professional practice. But we all know that reflection is often put off until later, squeezed out by other pressing needs. Then by the time we get round to it, we can't quite remember all the points which were important, and we've lost the freshness of the ideas. Or we may just simply forget. Many teachers would benefit immediately from devoting more time to the process of evaluation. Some shy away from it because they are worried about 'criticism'. Be positive! Think of reflection as being an exercise to build strength. This series of activities will build up your awareness and confidence. It will also enhance the variety of your professional practice.
PLANNING FOR EVALUATION Successful evaluation can only be carried out if you have three things in place: 1. A set of criteria against which success can be measured 2. Effective methods of gathering and analysing information 3. Procedures covering the ways in which evaluation is to take place and the part to be played by everyone concerned. You can't measure unless you have something to measure against. The criteria for evaluation may comprise any or all of the following: performance indicators measurable objectives national or professional standards centre standards personal expectations or goals.
Properly worded, detailed performance indicators describe the standards which should be reached by anyone performing a skill, i.e. they indicate the required performance levels. Whether these five types of indicators of success are designed by the teacher as measurable objectives, are part of national or professional standards or are set by the centre, they are essential as a measurement tool. BASIC STEPS IN PROGRAMME EVALUATION You should have given these some thought along the way as you have been teaching your programme. 1. Decide what you want to find out about 2. Decide from whom you can obtain the variety of information needed for successful evaluation. Use notes on self-evaluation, too 3. Decide the methods you will use to collect the required information 4. Design your criteria for success against which you can evaluate the information you will collect. Remember: this is about evaluating the programme, not only about the assessment of learning.
METHODS OF EVALUATION Evaluation can be achieved by using a wide variety of methods: discussion planned oral feedback during the programme written feedback spontaneous feedback.
People who can be involved in evaluations can include: learners colleagues or members of a teaching team, other non-teaching staff working alongside you line managers other stakeholders funding agencies awarding bodies and other external agencies.
POINT TO WATCH It's good to aim for a variety of evaluation sources as each of the above will have their own (probably different) perspective on the learning programme. WHAT SHOULD BE EVALUATED? All aspects of the programme can be evaluated, but the main aspects include: learning methods/learning strategies organisation of learning sessions and the learning programme itself use of resources the content of the programme individual satisfaction of results and addressing needs contribution of activities to other aspects of learner development e.g. language skills.
POINTS TO WATCH 1. It is important that you ensure everyone involved in evaluation has a clear idea of their role and the desired outcomes. So: Keep your contacts with people 'warm' - do a bit of 'networking' Go that extra mile - go to see them - make a special effort Have your questions sharp and ready - don't waste people's time
2. Feedback information needs careful examination and analysis which may be quantitative as well as a qualitative. When designing questions think of the best format for answers in terms of processing those responses 3. Make sure you can list main points arising from your feedback sources. Use appropriate methods to support and illustrate these points
4. If you are going to use methods of quantitative analysis, there is no need to involve yourself in sophisticated statistical techniques - unless you want to. Simple frequency charts, means etc will suffice. Look for what is significant in your data and any anomalies which occur 5. Listen carefully to oral responses to your questions. Note down key issues which are raised. Think of yourself as a journalist looking for quotes. Make sure you make a clear written record of what is said 6. Do not forget to review and use your own reflections from your journal made during the progress of the learning programme. This is vital reflective evidence!
Practice
Now try your hand at designing your own evaluation questionnaire. Questionnaires are by far the most widely used method for collecting information, so here are steps you can take in designing your own questionnaire. 1. Decide what want to find out 2. Decide from whom you can obtain evaluation information 3. Methods of collecting information? Step 1: Decide what you want to find out
You'll want to discover the strengths and weaknesses of your learning programme. It will help to know which parts of the programme you organised and carried out well and where your professional practice could be improved. The following are important questions to ask but don't be afraid to add in your own: did the methods used achieve the outcomes required? were sessions organised effectively? did activities used within sessions achieve the desired outcome? were resources used effectively? was the content of the programme suitable to achieve desired outcomes? were both learning outcomes and individual learning needs satisfied for all learners taking part?
You can add your own specific question(s). Step 2: Decide from whom you can obtain evaluation information
Clearly you'll wish to get feedback information from the learners themselves. There could be a great many of them, so processing the data could be very time consuming. You could go for responses which are quantifiable and easy to process eg The learning sessions stimulated my interest in the subject as a whole (Circle the number to indicate your response)
Strongly agree 1 2 3 Agree 4 5 Disagree 6
As well as from the learners you might wish to obtain information from: teachers, support teachers, and others who have played a part in the learning programme line managers responsible for learners and their progress and achievement, both in your own centre and in companies where learners might operate
learner's employers (if relevant) funding agencies who have paid for all or part of the learning programme (where relevant) Methods of collecting information?
Step 3:
You could use postal or other methods to send your written questionnaire to the evaluator. You would then expect a written, postal response. These tend to be either slow or 'overlooked'. More effective as a method is to take your questionnaire with you and use it as a working basis for interview. You might want to use the questionnaire at events such as parents' evening, open days etc - but don't make the questionnaire too long and cumbersome! You can build in regular times during the programme when you encourage the learners to reflect on their own learning. POINTS TO WATCH 1. You might wish to consider different types of questionnaire tailored to different events/circumstances and different types of respondent. You could keep the main approach of the questioning the same but amend the length and format. 2. This will certainly become necessary if you're dealing with very young learners. But, you can still get a flavour of their views! How did you like the session on friendship? (Colour in the face which says your answer)
3. Think about the spatial design (layout) and quality of production of the questionnaire itself. Don't produce a 'rushed' version. Make sure it is typed or word processed. Write a clear, professionally phrased letter to accompany it if you are using a postal method.
HIGH FIVES AND LOW NOS Look at the information you have been collecting through evaluation during the programme. Filter out from the feedback information the teaching and learning activities which went really well (let's call them the High fives) and those which misfired (let's call these the Low nos). In this activity you will look in turn at 1. the activities which went well. Identify reasons for their success 2. the activities which didn't go so well. Identify reasons for their relative lack of success. High fives Activities which went well need careful thought. How are you going to define what went well? Perhaps they were frequently mentioned in feedback? Perhaps they were easier to manage in the classroom? Perhaps they motivated learners or helped them achieve learning objectives successfully. Take one example of a high five and write the reasons for its success in your journal. Remember, your journal is for you to include as much detail as you think will be useful for future reference, for example, when completing your assignment. 'Low nos' Activities which didn't go so well also need careful thought. In the same way as you analysed the reasons for the success of high fives look in detail at the reasons for lack of success in the low nos. Take one example of a low no and write the reasons for its lack of success in your journal
USING EVALUATION TO PLAN IMPROVEMENTS
ANALYSING INFORMATION FROM EVALUATION SOURCES For quantitative responses, tabulate your data, draw your histograms, calculate your percentages and your means. Use spreadsheets where applicable or possible. For qualitative responses, look for emerging trends which repeat themselves from a range of sources. Note dissenting views and 'anomalous' remarks. Identify and use statements which seem significant for any reason. Don't be afraid to quote directly from your own notes or remarks made in writing. For younger learners, count those smiling faces! Also remember that younger learners remember more about how they learn than what they learn. Young learners will also need to learn the vocabulary needed to express an opinion or to give an example to support what they are thinking.
Generally look particularly for responses which will help you: make changes to learning methods make for more effective organisation make changes to topics and content make more effective use of resources devise more effective methods of satisfying learning outcomes.
This list is just a start. You'll have had subject specialist and other questions which will produce useful feedback. DETERMINING FEASIBILITY AND BENEFITS The first stage of planning for improvements is to review the feedback you've had and physically highlight (or write out afresh) all potential improvements. Don't be afraid to recognise the strengths in your programme.
After you've done this, you can select the improvements which are necessary. These will be necessary for a variety of reasons, listed in order of priority: 1. Identified problems relating to health and safety issues and must therefore take priority to ensure the health and welfare of learners and safety of the learning environment 2. A problem has been identified which is preventing the majority of learners from achieving their identified outcomes 3. The benefits to be obtained from implementing the improvement will outweigh the costs or time implications in implementation 4. The improvement is within the teachers control or level of authority, is easy to implement and will resolve an issue of which the teacher was unaware before evaluation. FORMULATING AN ACTION PLAN An effective action plan will provide the following details. Bear in mind that your recommendations might need approval by other persons. They'll need factual information on which to make decisions about expenditure in terms of costs and time: the improvements need to be made, listed in priority order the benefits to be obtained from each improvement how each improvement will be implemented by whom each improvement will be implemented by when each improvement will be implemented.
Objectives must be: Specific Measurable Everyone knows exactly what has to be done The teacher can evaluate whether or not improvements have actually brought about the intended benefits and have been achieved by the planned deadlines by nominated persons The desired outcomes can be achieved to the required level by the deadline required The outcomes can be achieved within existing resources and practicalities Everyone knows the intended deadline for implementation
Achievable Realistic Time-scaled
Here is a sample from an action plan

Area for improvement 1. Photocopying of materials Benefits to be obtained Some of these are admittedly poor as copies are then photocopied. By improving the quality of master copies, photocopies will in turn be legible and of higher quality. Allowing more time for photocopying by support staff will avoid these being rushed. Photocopies to be checked prior to sessions to ensure legibility and ease of understanding by learners 2. Timing of presentations and Q&A sessions Etc Shorter presentations will allow for improved knowledge retention Selection of vital information for Etc Etc Etc Etc How improvements will be made Prepare set of master copies for each handout Store these in folders under name of session Meet with support staff to discuss how quality of copies can be improved Allow time for checking of copies prior to session Sessions plans to be examined and improvements made to timings Teacher Immediate and ongoing By whom Teacher Support staff By when Immediate
Practice
Fine - you've got ideas for improvements. But how are you going to put ideas into practice? Various factors may have an effect, including

people resources organizational structures.
What factors will help, and how can you get them to help you? Jot down your ideas in your journal. Are there any factors which will cause problems, and how can you avoid these? There may be many reasons why the best laid plans do not work out in practice. You can make a start on your own action plan You can use a pro-forma such as the one used in our 'sample from an action plan'. But, you don't have to take it as the only way to produce an action plan. If you want to redesign, reword the headings, go ahead. If you want to use a different format - give it a try! You might want to include in your plan an area for notes on progress of each 'area of improvement'. That's a good idea. It converts a document into a living document. It might be useful to you to devise a way of mentioning the source prompting the 'area for improvement'. Was it the learners, a line manager or something from your own reflection? It could be good to have this as you'll be looking at this document quite frequently. A point of good practice here - go for 'agreement in advance' over as many points as you can. Talk to those involved in implementation as early as you can, get their agreement and keep talking to them! POINT TO WATCH The very act of listing your areas for improvement and methods of tackling them will etch this plan on your subconscious mind. In order to internalise your goals it's helpful to write them down as part of your commitment to making improvements in your professional practice.
MAKING IMPROVEMENTS AND PLANNING FURTHER EVALUATION

CLOSING THE CIRCLE
We've now got to stage 4 in the Kolb learning cycle. This is where we get to put plans into action. Planned improvements must be put into action effectively however and we must plan for evaluation of the results of the improvements. This means that when we come to the next round of design, practice and assessment, we can include changes we have ourselves identified as a result of on-going evaluation. POINT TO WATCH Implementing change is one of the most difficult tasks you can set yourself. Changes in education and training will affect not only learners but colleagues, line managers and any other stakeholders. This is about dealing with people effectively. Why planned improvements sometimes fail
Let's be realistic. Well planned improvements may fail for a variety of reasons, eg: staff are not committed to the ethos of continuous quality improvement implementation of improvements has not been planned so that it does not interfere with the learning process in any way
no mechanisms have been planned for checking progress and/or for evaluation of successful implementation of improvements time or cost implications cause practical difficulties in implementing improvements staff are constrained by other priorities or have insufficient time to carry out improvements staff are unable to see that benefits will outweigh efforts in implementing changes resources which are needed to implement improvements are not made available.
You'll see that several of the difficulties may be beyond your control. For example a sudden change by your managers may result in an overall reduction in your budget or contact time with the learners.
WAYS FORWARD
It might be that you did not make your case for improvement particularly well
Don't be put off! Here some practical ways of making progress: 1. Introduce the idea of making changes gradually, by stages. 'Warming' colleagues and line managers to your thinking reduces suspicion. It enables others to respond in a considered fashion rather than give a 'knee-jerk' reaction which is usually negative 2. Involve people in your thinking. Make them feel part of your ideas. Then they will feel more receptive 3. Do as much research as you can. Try to find out what the budgetary constraints are and are likely to be. Frame proposals in that light 4. Do not be put off by 'failure'. Failure is part and parcel of professional experience as it was for: Thomas Edison - failed then thousand times before he perfected a lamp which could be worked by electricity J K Rowling - turned down by several literary agencies before Harry Potter became a worldwide success
Guiseppe Verdi - rejected by a music conservatoire in his native Italy, went on to write Aida and other operas. It is said that one million people attended his funeral.
5. Always picture success in your mind. Don't get 'attached' to outcomes - one or more details in your plan might not get realised. It wasn't you who 'failed'! 6. It helps to share experiences. It's a good idea to talk through your ideas with someone else. They might be a colleague - maybe from a different department or discipline. They might be someone from a different institution or company. You'll probably know someone whose professional thinking 'chimes' with yours. You'll be able to 'bounce' ideas off them, sound out ways forward. This is a very powerful method of development and can effectively produce a third thinking force: 1 + 1 = 3! 7. Persistence - do not pester, but be persistent! 8. Early warning - always be up to date. Keep your eyes on notice boards, websites, newspapers, professional journals - any source of information which might convey changes in your institution, its operation and development, in requirements of external bodies e.g. awarding bodies. EVALUATING THE SUCCESS OF PLANNED IMPROVEMENTS Here you are planning for a fresh round of evaluation data to be collected. Targets without evaluation have a nasty habit of disappearing from sight! So keep your eyes firmly on your targets through evaluation. This data may be collected through a variety of methods including: Evaluation of learning sessions on a continuous basis. Oral or written feedback from learners and others involved in the next programme Final evaluation of the next learning programme Own observation to ensure planned benefits have been achieved Observation carried out by others to obtain an objective opinion.
We've almost closed the Kolb learning cycle we're at Stage 4.
We're planning 'active experimentation' before we incorporate it into our next design session our next Module 1.
WORKING WITH OTHERS Our trainers travel worldwide to introduce the Diploma to new centres. The training often involves group exercises. It's noticeable how quickly and effectively teachers respond to group and team challenges. Teachers make fine team members and leaders, yet it's surprising how little they use teamwork techniques in getting their own work done. In reflecting upon your course plan and considering improvements to it for the next teaching-learning cycle, you should think about using teamwork to greater effect. Which of the questions on this list might be more effectively and efficiently tackled by teamwork? designing course plans designing session plans designing assessment schemes managing assessment developing use of audio-visual aids developing use of ICT programme plan evaluation designing improvements to plans implementing improvements to plans trialling new techniques evaluating new techniques attending external courses, exhibitions, organising learner visits, fieldwork designing and monitoring induction.
Teamwork enables you to bounce ideas off each other, discuss the pros and cons of methods, allocate tasks, motivate each other, produce developments quickly, monitor and evaluate improvements effectively. Teaching can often be seen as a lonely business. Lonely in the sense that teachers may see their tasks as being solitary or individual. This may sometimes be true but there are ways of breaking the mould and learning a lot by doing so. We have looked at teamwork as a way forward but there are other ways. 1+1=3 is a technique frequently used by professional people. It is a kind of sharing. You will find it useful and therapeutic to talk to someone else about professional matters. This kind of interaction can be very effective and productive (hence 1+1=3 not 2). The other person may be from a different discipline, department or field of interest. Its good if they are. Think now about who you talk to frequently and frankly. Always choose someone with a positive approach. 'Mutual moan' sessions produce even more negativity avoid them at all costs!!
Practice
Two things for you to try. 1. What improvements are you going to make to evaluation features in the next and revised programme plan? Evaluation itself can be improved so as to be as streamlined and effective as possible. Some thoughts to help you with this: Was your on-going evaluation of the learning sessions and the learning programme strong enough and consistent enough to be of much use to you when final evaluation was considered? If you're not really satisfied, then you need to make some practical changes. Did you really give enough time and opportunity for learner feedback? Even when you are not taking your learning programme as a context for undertaking this Diploma, might there still be a case for inviting external observation of your learning sessions? Do you need to expand your canvas of the opinions of other stakeholders in the learning programme's success. Such involvements can have all kinds of professional benefits.
2. Have a good think about 1 + 1 = 3. Think about sharing your ideas with one other colleague, or with a support group. Such discussions can be professionally and personally beneficial. Sharing successes with a colleague prepares the ground for those times when you'll need to share the failures too.
POINTS TO WATCH Always choose someone with a positive approach to life as well as with an open and positive professional standpoint. Avoid people who'll tell you 'it can't be done' and 'you'll never get that off the ground!' If the Wright Brothers had listened to them at Kitty Hawk they would have made just another boring canoe!
EVALUATING OWN PRACTICE
WHY? There are many reasons for evaluating your work as a teacher. Some of these reasons might be to determine what changes you may need to make in your planning or classroom practice, or to recognise when a particular session or technique has worked effectively. Evaluation doesn't always have to be done to identify change. Sometimes it can establish that change is not necessary. It's not possible to evaluate yourself as a teacher simply through the assessments that you do of your learners' learning. Although there will be a natural correlation between the quality of teaching and the quality of learning, this is not as clear cut as it may sound. Many factors impact upon what learners learn. As well as teaching styles for example, there's also the actual programme or scheme of work that you must deliver that can have its effect too, as can the resources available to you and the time scales in which you must work. In short, evaluation of the work of a teacher is necessarily complex, and not possible to achieve on the basis of one single line of enquiry, such as the educational outcomes achieved by learners. Although evaluation is complex, it is also something that can be achieved on an ongoing basis. If you get into the habit of spending time regularly reflecting on your practice in the classroom and in the profession generally, you will amass a significant amount of evidence on which to base your self-evaluations. BE POSITIVE One of the curious things in life is how little time we spend thinking about ourselves and what we do. We tend to give 'ourselves' remarkably little space. This Diploma asks you to take a planned approach to evaluating your own professional practice, including courses, criteria and goals. This is a positive, beneficial and fascinating set of processes. It involves you as a person as well as you as a professional. It is interesting that in teaching we can often learn as much if not more when things go wrong as when they go well. If a session appears to have been
successful we often dont feel the need to reflect. However, if the learners have been difficult, or do not appear to have learned anything, we need to reflect on reasons and think about the changes which we might need to make. COLLECTING DATA It's good to gather information from a variety of sources. Select those which are likely to provide valid and reliable evidence of your own professional practice. These include: self assessment against set criteria, which may have been developed by either the teacher or the institution work results - learner qualifications etc results of observed performance results of assessment by others against national and/or centre standard results of own evaluations of learning sessions and learning programmes results of appraisals of performance ongoing feedback from others including learners and others in the centre information on future changes relating to areas of work.
The principles for evaluating your own professional practice as a teacher are very similar to those used for evaluating learning programmes. Again, three things must be in place at the planning stage: 1. a set of criteria against which success can be measured 2. effective methods of collecting and analysing information 3. procedures covering the ways in which evaluation takes place and the part to be played by everyone concerned. CRITERIA, METHODS AND ANALYSIS Criteria can include: agreed performance indicators measurable objectives organisational standard.
These can be quantitative and/or qualitative. Methods include self-analysis and analysis with professional colleagues. We 've already mentioned the benefits of '1 + 1 = 3' - they are seen clearly when your colleagues help you to conduct evaluation of your own practice. These benefits include: simple re-assurance - 'talking you through' the steps involved checking that you have thought through all aspects of your evaluation helping you to get issues in perspective and identify experiences.
Perhaps other professionals can be of help, too. For example: line managers and heads of departments colleagues with experience of professional development and experience of inspection members and officials of professional and subject associations.
On the next page, youll find a basic self-evaluation tool you can use. These questions are only intended as a guide and you may well find that there are other more suitable questions covering issues that relate to your specific circumstances, which you can add if necessary. Go through the questions one by one, using them to evaluate yourself as a teacher. Although technically they only need a yes, no or sometimes answer, you may wish to think about any further comment that you would like to add in your journal.
Evaluating my work with my learners Answer each statement by circling either yes, sometimes, or no (coded as Y, S, N). Y Y Y Y Y Y Y Y Y S S S S S S S S S N N N N N N N N N My learners feel happy and safe in my classroom They are able to question me when they dont understand something They have the opportunity to develop their thinking skills through their written and oral work I work hard at developing sound working relationships with my learners I help my learners to feel self-reliant and to show self-respect in their behaviour and dealings with others I nurture effective communication in my classroom There is a culture of trust in my classroom My learners develop self-worth and self-esteem in my classroom I help my learners to develop skills such as creativity, literacy and numeracy.
Evaluating my other work as a teacher Y Y Y Y Y Y Y S S S S S S S N N N N N N N My classroom is set up for effective learning to take place I collaborate with other teachers where appropriate The assessments of learners that I do inform my teaching I work within institution-wide policies I teach with emotional awareness in mind I involve parents in learning where appropriate I encourage a relationship between my learners homes and school
Identifying what I do well in my work Which three aspects of your work as a teacher do you do well? 1. 2. 3.
Identifying what I would like to improve in my work Which three aspects of your work as a teacher would you like to improve? 1. 2. 3.
For each of these aspects, how can you improve, who would help you, and what resources are available to you? How 1. Help Resources
2. 3.
Analysis You can use a variety of methods to analyse the impact of your own level of skills and competence on learners and their learning outcomes. You might want to list the areas which seem to you to be of interest and concern and then expand each area, supporting the findings you list with evidence (either qualitative or quantitative) from the data you've collected from the various sources you've used. Look for the expected and the unexpected, the typical and the atypical. Highlight areas of good practice and be frank about areas for further improvement. Chris Kyriacou lists some useful areas for professional reflection: obtaining a 'measure' of the classroom climate or working atmosphere exploring your use of classroom rules exploring how pupils feel about particular topics monitoring a particular learner's curricular experiences for one week examining tasks in terms of their learning demands investigating question and answer sessions evaluating the techniques you use to assess learners' progress reviewing the motivational qualities of different activities looking at the quality of your relationships with learners examining the time learners spend on different types of activities reviewing the work you set for the more able learners reviewing your use of information technology activities.
These tend to be generic professional issues but there may also be subject specific issues which you also wish to identify from your teaching and training experience. For example teachers involved in skills tuition and training courses 'on location' in manufacturing or business workplaces are often involved in demanding learning environments. Teachers of science subjects may wish to look at aspects of experimental and laboratory work. Those dealing with special needs education may have a number of issues which need specific reflection and analysis.
Practice
This is the time for sorting through the information you have collected and making sense of the important themes which you can identify. It's important to make notes on these themes and collate them into a series of expanded points. One way to think of this is to imagine you had to produce a written report on your own practice for some outside body.
IDENTIFYING GOALS FOR IMPROVEMENT
GOAL SETTING We've reached the point where we can identify and record personal goals for improving professional skills and practice. When we talk about 'personal' goals we are talking about goals specific to you as a teacher. Goal setting as a technique involves a lot of quiet contemplation by the goalsetter and imagination. It is an important stage in the professional development sequence:
Goals: can be characterised by their priority and their feasibility should be SMART, that is to say: Specific, Measurable, Achievable, Realistic, Time-scaled
It does help if you can 'picture' where you would like to be with certain aspects of your professional practice by when. Human beings can be very effective at this. But we are often reluctant to use this technique and are even more reluctant to talk about it! The fact is that all successful people have these 'pictures' in their mind and are determined in achieving their aims. Step 1: Consider yourself Your current level of skills and competence Your individual aspirations connected with your current job and future plans learning aims Your current and anticipated job requirements Your preferred learning styles.
Step 2: Think practically how to achieve your goal programme attendance/attendance at professional courses work experience on other programmes of where delivery methods are different job shadowing/ observing a colleague in a similar teaching role coaching from more experienced teachers mentoring from another teacher open and flexible learning, distance learning, e-learning through internet or intranet self-study through books and other publications.
Step 3: Research ways to achieve your goals publications attendance at conferences information from professional bodies through subscriptions and periodicals internet information on government policy relating to your future needs emerging national and international standards, curriculum development and new areas of work.
Step 4: Create a realistic plan of action Remember that professional practice goals need to be Specific Measurable Achievable Realistic Time-scaled Example As a result of a completed Evaluation Plan for her own performance as a teacher Maria identified these goals for developing her own professional practice: 1. Develop use of IT Skills so that I can operate new computerised system of maintaining learner records 2. Address issues of timing of activities within session plans, identifying specific delivery areas where insufficient time has been problematic 3. Improve time management skills to ensure that all resources are available prior to learning sessions 4. Review handouts and materials provided to learners to ensure content and format are clear and accurate and provide all necessary information. Subsequent compilation of a set of master copies for each learning session 5. Review of time allowed for tutorial sessions with Programme Co-ordinator 6. Closer supervision of learners when completing classroom activities. At first glance these seem to be clearly identified goals, but we asked Maria to sharpen them up a bit by considering two additional considerations for each goal Feasibility Priority
These helped Maria gain a fuller understanding of what exactly might be involved in advancing these potential improvements.
For example: In Improvement 6 'closer supervision of learners when completing classroom activities' Maria knew from feedback from learners that this was an important issue for them, so its priority is high. It is a question of redirecting attention and effort in the learning sessions and should be easily achieved. It is therefore feasible. Thoughts on continuing professional development In her book Continuing Professional Development (RoutledgeFalmer, 2000), Anna Craft emphasises the wide range of methods of professional learning. These include:

action research self-directed study as well as teacher research linked to awards such as masters and doctorates using distance-learning materials receiving and/or giving on-the-job coaching, mentoring or tutoring school-based and off-site courses of various lengths job shadowing and role rotation peer networks membership of a working party or task group school cluster projects involving collaboration, development and sharing of experience and skills teacher placements including those in business but also in other schools personal reflection experimental 'assignments' collaborative learning learning mediated by information technology (for example, through email discussion groups, or self-study using multimedia resources)
THE PERSONAL AND PROFESSIONAL DEVELOPMENT MINDSET The mindset that encourages in you a drive to progress in your career (even if that means maintaining your current position) will undoubtedly enhance your mental and intellectual well-being. In fact, it could be said that a successful approach to professional development is dependent upon your attitude of mind. Regardless of any external factors that may impose limitations on you, the way in which you handle these will ensure you gain as much as you can from what's offered. Factors which indicate the presence of a state of mind that is open to personal and professional development include:

a willingness to seek out learning opportunities a willingness to see positive learning potential in all aspects of life an affinity with the process of reflecting on learning and change an overriding leaning towards curiosity about, rather than resistance to, change.
THE TENSION BETWEEN PERSONAL AND PROFESSIONAL DEVELOPMENT There's no doubt that all forms of personal and professional learning and development can have a positive impact on a school, but this does not negate the inevitable tension that will exist between individual and institutional priorities for development. These will be linked, but they are not interchangeable. Put simply, the ambitions and targets of the individual would, in an ideal world, be perfectly matched to internally and externally perceived development needs that must reflect nationally imposed targets and obligations as well as reflecting the need for the school to be accountable for the education it provides. In order to avoid damaging degrees of tension between the direction in which you, as an individual, want to travel and the direction in which others within your school would like you to develop, a balance needs to be sought. It's no good training to be a pastoral leader because it would really fit nicely with what your principal has in mind for you when what you truly have your heart set on is being a head of department. Responsibility for appropriate development must be accepted both by you as an individual and by the institution in which you work.
WORK-LIFE BALANCE As societies and economies develop and modernise, occupations become more demanding and complex and the balance for individuals between work and life more difficult to manage. Work is occupying an increasing and increasingly dominant part in our lives. Teaching is an excellent example of this. Teachers can point to the growing number of roles expected of them as professionals. As we have seen in this guide, they are becoming counsellors, organisers, designers, mentors, trainers and so on. The needs of learners are becoming more demanding, specific and urgent, as the world around the school becomes more sophisticated. This flower diagram shows a broad classification of activities which make up our lives. Look carefully at the diagram. Each one occupies a space in our life. Each petal represents activities in your life. You can invent new categories and have more petals if you wish. Note that WORKSPACE is only one of our petals. We have drawn the petals as though they occupy equal space in our lives. They never do! But have you got the shape of flower that you would like? Notice the arrows on the diagram. They all emanate from the centre SELFSPACE because it is from our own minds and imaginations that the power to effect changes in the petals originates. Supposing you were to draw each petal proportionate in size to its importance in your life, how big would each be? Sketch your personal flower taking each petal (and the centre) in turn. You could check it out by looking at the actual time you spent on each petal yesterday, last week or last month.
Teaching brings with it many possibilities to expand into other spaces. For example marking, writing and out of school activities can easily invade the spaces around actually being in school. This is understandable most teachers are lively, enthusiastic and care a great deal about their work. The Diploma itself may seem to add to your pressure on time. Our insistence that you consider properly-targeted design and carry out full on-going evaluation adds to the time already spent on other teaching-learning activities or does it? Proper, fully thought-out design can lead to efficiency as well as effectiveness in teaching. A full, detailed teaching course plan can cut out last-minute panics, fraught negotiations over resources, clashes with colleagues and many of the frustrations which can wear you out so easily. Involving others, working as a team or a group can lighten your load by sharing the work equitably. If you plan your time at school effectively you should be able to get some marking done before you leave for home. Try to manage your effort at work more effectively. Perhaps you might take a bit of time for your own selfspace and now that you are in control of your workspace you might look at some of the other petals in the same constructive light.
BRANCHING OUT Teachers can sometimes see themselves as isolated professionals. It's possible to turn up to school, get on with the teaching and return home, and have little contact with other teachers. But let's see things differently. Look at this diagram, which shows a teacher as tree!
In the diagram, the tree is firmly rooted in professional skills and training, subject specific knowledge and grows upward in professional skills and training. Notice the function of the branches. By reaching out to others in the school the tree will

make a contribution to the development of others derive beneficial input from others.
Teamwork is a very strong way in which you can branch out. We're thinking in terms of your own institution and its professional teachers. These trees make up your own 'forest'.
This diagram shows that there are many other forests.
How can you reach out to them? There are plenty of options. The following list will give you a start: join a professional association and attend its meetings self-study through books, publications and on-lone courses observe professional practice in other schools arrange teaching/learning exchanges with other schools.
In your experience, what has worked best for you in terms of interacting with colleagues outside of your own school?
Feeling good and moving on The following table offers a framework of questions that have been designed to help you to focus your thoughts on any development and career planning needs that you may have. The questions in the framework are not exhaustive by any means. Read the questions and think about the responses that you want to record in your journal. Consider these points as you work through the table:

always trust your instincts and intuitive feelings when deciding where you want to go in your career and the path you'd like to take to get there think about the development of new skills and the expansion of existing skills and the relationship between the two gather as much information as possible about the resources available to you consider the short, medium and long-term think about what's urgent to you and what's important to you
Practice
Draft your own development plan. You can identify and refine your own goals for potential improvements in your own professional practice. The themes arising from your analysis of your evaluation should help you immediately. You can now use a pro-forma to state/identify your potential improvements, comment on their feasibility and identify their priority. For example:
Potential improvement Need to keep records up to date Feasibility of making improvement I need to redesign my system so that it becomes easier to operate Priority? Urgent each week I get further and further behind with my record keeping because my existing system is too complex
COMPLETING A PROFESSIONAL DEVELOPMENT PLAN
BE PRACTICAL You can now draw up a development plan to identify and prioritise development needs so that you realise your goals. Many people almost set themselves up to fail in their intentions largely because: they have planned too 'long-term' and been too ambitious and so deadlines appear far away in the future and the impetus of the plan is then lost.
A development plan should concentrate on immediate short-term and mid-term priorities. Long-term priorities can be recognised but only insofar as they provide a working context for more immediate issues. If your institution has an appraisal process in operation, use this opportunity to discuss your perceived needs and bring them to the attention of line managers. COMPLETING A DEVELOPMENT PLAN As we have seen there are essentially four principles in developing your own professional development plan: 1. 2. 3. 4. assessing your own current professional practice identifying areas for potential development prioritising these identifying the most suitable methods of satisfying identified development needs.
SHORT-TERM AND LONG-TERM DEVELOPMENT PLANS The short-term plan is based on the evaluation of your current practice and you use it to develop your professional practice in your current role. The long-term plan is again based on evaluation of current practice but sets out to be your long-term goals to further your career. Both are important in terms of lifelong learning and development. If development is planned only against short-term goals and targets, or to remedy current problems, you may not strive to advance in your career. If development is planned only against long-term goals and targets, your progress may slide as deadlines appear to be far away and not urgent. Development plans should be agreed with everyone else involved in your development, such as your line manager or head of department. An ideal opportunity for raising such issues is at an appraisal. Development plans can be finalised after advice and guidance from these or any other relevant sources. Here is a sample development plan, developed in more detail from the draft plan written earlier.
Area of improvement Information technology training to operate new computerised system of maintaining learner records Actions to be taken Contact Programme Co-ordinator to identify dates and times of IT training programme Register for training programme and carry out all necessary training Target date By the end of this week Criteria for success Suitable dates identified. Permission to attend granted. Training covers all necessary knowledge and skills. Time is allowed for practice to make sure learning is applied to new system before next induction sessions All plans collected and placed in folder by the end of the week. Individual session plans to be reviewed and revised at least 2 weeks prior to each session, with amendments completed in time for photocopy deadline. All amended materials to be supplied to support staff by
Completion by date of implementation of new system
Feedback from learners for selected sessions indicates that support and supervision has been effective and enabled identification of any problems
Collect together all session plans for review at scheduled time Carry out review of all plans and identify those areas where problems have occurred due to insufficient time allowed Rewrite sessions plans to incorporate planned changes
By the end of this week
Deadline in two weeks from start of new recruitment
At least two weeks prior to each session to allow time for
Guide for the Cambridge International Diploma for Teachers and Trainers MODULE 4 : EVALUATION photocopying photocopying deadline. Feedback from learners for selected sessions indicates that support and supervision has been effective and enabled identification of any problems
Closer supervision of learners when completing learning activities
Review of session plans should ensure that time allowed for activities is realistic. Self assessment after each learning session is then necessary to ensure that supervision has been effective
Ongoing during each session Self assessment after each session
POINT TO WATCH You can see that a lot of thought and preparation has gone into the way this plan has been designed and phrased. 'Area of improvement' and 'actions to be taken' have been sharpened up to make them specific. Including the 'criteria for success' column has made them measurable. You can see that the teacher has concentrated on quite detailed short-term objectives which are achievable. The range of measures being considered is also realistic. The 'Target Date' column makes the scheme time-scaled SMART work indeed!
Practice
Over to you! Now's your opportunity to draw up your own professional development plan. It's a good idea to approach this in two clear stages. Stage 1: Short-term development plan
Using the columns and headings in the 'sample development plan' as a working template write in your own: areas of development actions to be taken target dates criteria for success. Long-term development plan
Stage 2:
You can use the same template for this This is more personal and 'timings' may be more difficult Try to use the same phrasing and be specific.
SPECIFYING ACTIONS AND EVALUATING OUTCOMES
ACTIONS AND BENEFITS This is about the ways in which you can put the main points of your plan into action. Then we'll see how these steps can be evaluated. There is no really foolproof way of doing this - much depends on what suits you as an individual. Using the planning example we set out above in the guidance for Completing a Professional Development Plan we can look at what our teacher, Maria, had to say about the actions she intended to take and the benefits she expected to gain. She commented on benefits for her as a teacher:
BENEFITS FOR THE TEACHER Forward planning will help improve time management during the next programme and meeting deadlines New skills learned through the Diploma have already been put into action in the current programme Continuous evaluation and self-assessment help the teacher quickly to identify and act on potential problem areas. The teacher can make small improvements along the way which lead to big improvements in the programme as a whole The teacher can develop time management skills by implementing the development plan to deadlines set The teacher will be able to facilitate learning outcomes more consistently, leading to job satisfaction The teacher will receive a better response from his or her learners by developing better or more appropriate learning materials.
Interestingly, and encouragingly, Maria also looked at benefits for her learners
BENEFITS FOR THE LEARNERS Sessions and activities will be more realistic and help learners gain skills and knowledge through doing rather than listening Learners will be able to learn through reflection on what was achieved and how achieved if more time is allowed for feedback and review of activities Learners will benefit from higher quality of resources and clarity of handouts Learner records of progress and achievement will be computerised and this will lead to greater safety sand security of records Learners will continue to be involved in review and evaluation of the programme and individual learning sessions. This will enable them to give their own views, feel involved and so improve learning on a continuous basis.
MILESTONES AND RECORDS You need to involve yourself in implementing and updating your plan. This includes actually ticking off targets when they have been achieved. It sounds almost nave, but achievement and recording achievement is just as important for professionals like teachers to do for themselves as it is for their learners. Achieving important improvements and planned objectives is a cause of real satisfaction for teachers. You should make a record of what you have achieved and what the benefits of that achievement have been. You can incorporate these records into future design and evaluation, and they can be produced and discussed in appraisal and may be used in inspection or other professional reviews. These records act as milestones in your professional development. There is a saying 'nothing succeeds like success'. All successful individuals and successful organisations are careful to follow up one achievement with another so that a 'culture of success' is developed. Playing a part in constructing such a professional working environment is a great source of professional satisfaction.
CAREER DEVELOPMENT AND LIFE GOALS We have been involved almost completely in explaining goal-setting and development in professional terms. This is fine because this Diploma is about developing professional skills and practice and projecting these developments into the future in a positive and constructive manner. Career development is clearly closely related to professional development but it is part of a much wider range of considerations. These include: family and partner relationships happiness with your current post awareness of fresh opportunities elsewhere financial and accommodation matters leisure and social interests need for further training and qualifications desire for promotion or work in another field desire to travel and/or live and work overseas
There may be many more considerations we have omitted. But notice our choice of words - you can see these as 'considerations' not 'constraints' POINTS TO WATCH You've gained a taste for achievement. Try to picture what you'd like to happen in your life as a whole. You can set yourself 'life goals' and pursue them enthusiastically. If you do set yourself these broader goals: 1. Write them down. People who write their goals down are many more times more successful than those who do not 2. Write them down exactly 3. Pursue them tirelessly. Never give up 4. Don't show them to anyone. Not even friends and relatives. People you know best can be the first to discourage you - without realising. If you can manage your professional work effectively and satisfactorily, you will be in command of your own efforts. You'll be able to manage not only your own work but also the other spheres of activity which make up your life - family, home, friendships, sports and hobbies, spiritual and religious activities The final words in the conclusion to CIE's video about The Reflective Teacher are 'What have you learned for the future?' So, how are you going to teach in the future, and how will your learners learn?
Practice
Our final activity is something for you to think about. In a personal development course a trainer was trying to explain the benefits of 'taking command of your own life'. Perhaps his audience, his learners, were dumbfounded by such a thought; their faces showed little sign of response. With just a little air of desperation the trainer cried out . 'Well what are you then? A piece of driftwood bobbing up and down on the sea or a speedboat cruising smoothly across the ocean en route to Paradise?'
and finally Learners sense a teacher or trainer's feelings If you feel confident, they will feel confident in your ability to pull them through If you enjoy then so will they Enthusiasm is greeted by enthusiasm so in your teaching be confident and enthusiastic, but above all
Enjoy

CIDTT Guide Assessment

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CIDTT Guide Assessment

Uploaded by

Copyright:

Available Formats

Cambridge International

Diploma Teachers and Trainers

1 University of Cambridge International Examinations 2010

114 119 124 134 142 148 152

158 University of Cambridge International Examinations 2010

PREPARING FORMATIVE ASSESSMENTS

159 University of Cambridge International Examinations 2010

160 University of Cambridge International Examinations 2010

161 University of Cambridge International Examinations 2010

162 University of Cambridge International Examinations 2010

164 University of Cambridge International Examinations 2010

These might be the kind of things the teacher is trying to assess

165 University of Cambridge International Examinations 2010

166 University of Cambridge International Examinations 2010

USING FORMATIVE ASSESSMENTS

167 University of Cambridge International Examinations 2010

169 University of Cambridge International Examinations 2010

Rose is using a criterion-referenced assessment scheme It works like this

170 University of Cambridge International Examinations 2010

171 University of Cambridge International Examinations 2010

172 University of Cambridge International Examinations 2010

173 University of Cambridge International Examinations 2010

174 University of Cambridge International Examinations 2010

ANALYSING FORMATIVE ASSESSMENTS

175 University of Cambridge International Examinations 2010

Test 1 35 30 25 Learners 20 15 10 5 0 0 1 2 3 4 5 Mark 6 7 8 9 10

176 University of Cambridge International Examinations 2010

177 University of Cambridge International Examinations 2010

178 University of Cambridge International Examinations 2010

179 University of Cambridge International Examinations 2010

180 University of Cambridge International Examinations 2010

PROVIDING FEEDBACK ABOUT PROGRESS

181 University of Cambridge International Examinations 2010

182 University of Cambridge International Examinations 2010

183 University of Cambridge International Examinations 2010

184 University of Cambridge International Examinations 2010

185 University of Cambridge International Examinations 2010

186 University of Cambridge International Examinations 2010

187 University of Cambridge International Examinations 2010

MAINTAINING RECORDS OF LEARNERS' PROGRESS

188 University of Cambridge International Examinations 2010

189 University of Cambridge International Examinations 2010

190 University of Cambridge International Examinations 2010

AND NOW AN INSPECTOR CALLS!

191 University of Cambridge International Examinations 2010

192 University of Cambridge International Examinations 2010

PREPARING SUMMATIVE ASSESSMENTS

194 University of Cambridge International Examinations 2010

4. Projects and assignments

195 University of Cambridge International Examinations 2010

6. Written and oral tests

196 University of Cambridge International Examinations 2010

197 University of Cambridge International Examinations 2010

Observation combined with oral questioning to confirm knowledge and understanding

2. Activities in small groups

Observation Learner's self-assessment may contribute Peer assessment

5. Assignments and projects

199 University of Cambridge International Examinations 2010

200 University of Cambridge International Examinations 2010

201 University of Cambridge International Examinations 2010

202 University of Cambridge International Examinations 2010

Avoid these pitfalls and you will be

203 University of Cambridge International Examinations 2010