YEREVAN STATE UNIVERSITY

SELF-EVALUATION REPORT
ON STUDENT ASSESSMENT AT YSU

YEREVAN 2012

CONTENTS

INTRODUCTION ............................................................................................................... 3 THE RESULTS OF THE STUDENT SURVEYS.............................................................. 6 THE RESULTS OF THE TEACHING STAFF SURVEYS ............................................ 14 CONCLUSIONS ............................................................................................................... 21 ANNEX 1 .......................................................................................................................... 23 ANNEX 2 .......................................................................................................................... 25

INTRODUCTION

The development of credible and transparent system of student assessment is one of the top priorities in YSU development strategy. That is why special attention is given in the YSU Strategic Development Plan 2010-2014 to promote this area. The YSU current procedures of students assessment is integrated with the common procedure of organization of credit-based education process (Procedure for organization of fulltime credit-based education process: Bachelors and Masters degree study programmes). It was approved by the university Academic Council, 2009 January 29, decision N1 and was amended several times (last time was approved by the Academic Council, 2011 June 2, Decision N6). The procedure formulates the rules of the multifactor system of student assessment. Particularly the system defines the students final grade as a sum of assessment grades for number of specific components which are as follows: Assessment of student participation in the module (taking into account the attendance of the student), Two midterm assessments of implementation of tasks or module subtasks during the semester, Final assessment/examination of the module. The following items are clearly formulated in the procedure: the midterm and final examination modes (written or oral exam/test depending on the nature of the module), midterm and final examination schedules and implementation procedures, a 20-point grading scale, appeal system/procedures, re-examination procedures and schedule, as well as the final attestation procedure and criteria. Although the procedure clearly formulates the assessment modes and examination procedures, it doesnt encourage the application of efficient assessment methods, which would directly measure level of the expected learning outcomes gained by the students. Therefore the organization and implementation of effective assessment procedure as a strategic point is included in YSU Strategic Development Plan 2010-2014 (the strategic goal: Quality Education, objective 4: To review existing forms of students assessment by introducing

learning outcomes-based new assessment modes for general and professional study modules and establishing clear graduation requirements. Based on this objective of the strategic plan, as well as on the necessity of complying the YSU student assessment system with the ESG standards and guidelines, currently the YSU student assessment system is under reconstruction with the aim to establish learning outcomebased credible student assessment system. As a first step in this direction the methodological guidelines for establishing a credible and reliable student assessment system were developed. Specifically, in 2012 A Guideline for Assessment of YSU Student was developed and published which aims to provide the YSU teaching staff with practical and methodological advice on assessment methodology based on standards and criteria. It provides methodological support for the YSU teaching staff in designing and implementing criteria and standards-based assessment of students learning outcomes. The guideline provides principles and techniques on: how to derive assessment criteria (key characteristics) from the learning outcomes, select an appropriate method of assessment that will validly measure the knowledge gained by students, formulate clear assessment standards (levels of quality of performance), and organize all criteria and standards in a marking scheme. Such kind of assessment system makes it possible to combine and comply assessment methods with teaching and learning methods, and assessment criteria with module learning outcomes. The introduction of such assessment methodology will ensure the transparency and credibility of student assessment at YSU. This new assessment methodology was tested at the Educational Research Center of Information Technologies of YSU, where training courses for the teaching staff were organized as well. For the introduction of learning outcomes-based knowledge assessment system the World Bank project on the Development of YSU internal quality assurance system and introduction of self-evaluation process has an objective designed for the Establishment and implementation of transparent and credible student assessment system, where the action plan and the timetable for this specific objective are identified: To train the heads of faculties' QA commissions on new knowledge assessment methodology,

 To develop and formulate for all YSU study courses (study modules) assessment standards and criteria, To make the assessment criteria and standards of all study programmes and modules available for the students including them in Study Guides/Course Catalogues for Bachelor's and Master's Programme Students, Development and enforcement of YSU student assessment updated procedures according to new requirements and standards, Establishment of feedback mechanisms for measuring the effectiveness of students assessment system through annual survey among university students and teaching staff. Despite implementation of the mentioned methodological basis YSU still has not fully introduced an effective student assessment system, since the teaching staff does not yet use learning outcomes-based approach in assessment criteria. This is conditioned not only by the lack of appropriate knowledge and experience among the teaching staff but also by their insufficient motivation and difficulties in implementing the new methodology. Therefore, measures taken in YSU in this area aim at training of the teaching staff and are included in the Structure and Content of Teaching Staff Development Programme as a specific module on Modern methods of teaching, learning and assessment. In order to evaluate efficiency of the student assessment system currently used in YSU the surveys have been conducted among YSU students and teaching staff aimed at finding out strengths and weaknesses of the current assessment system to consider them in the newly developed assessment system. The results of the surveys are provided below.

THE RESULTS OF THE STUDENT SURVEYS

The student survey has been conducted among 259 YSU students from the faculties of Biology (169 students) and Journalism (90 students). It has been carried out according to the prepared questionnaire Student survey on student assessment at YSU (ANNEX 1). The questionnaire contains 8 questions and designed to assess such aspects of student assessment at YSU as assessment criteria, assessment modes and methods, effectiveness of assessment and feedback on assessment provided to students. The evaluation was carried out according to the five-point grading scale, where 1=disagree strongly, 2=disagree somewhat, 3=no strong view, 4=agree somewhat, 5=agree strongly (and 0=not applicable). The template of the questionnaire enabling computer-aided processing of the survey results is attached (ANNEX 1). The results of the conducted survey are presented in the Table 1 and in the following Charts 1-9. According to the survey results, participated in the survey students assessed different aspects of student assessment practices at YSU as follows: 1. The assessment requirements(criteria) are easy to understand: 4.04 2. Teachers follow the assessment criteria in the assessment tasks: 4.03 3. Interim (written) exams allow me to convey what I learned: 3.83 4. Final (oral) exams allow me to convey what I learned: 4.29 5. Multiple-choice tests allow me to convey what I learned: 3.70 6. My performance on exams usually reflects my skill-level completely: 3.61 7. Usually I receive feedback or advice from my teacher after each assessment based on my performance in assessment task: 3.60 8. The appeal system on assessment is efficient: 3.44
6

It should be noted, that mean assessment grades for each category/aspect vary from 3.44 to 4.29. For instance, students are more dissatisfied with the efficiency of assessment appeal system and most satisfied with the final exams, which are conducted in oral mode. The detailed results of the survey are presented in the Charts 1-9.

TABLE 1. THE RESULTS OF THE 259 STUDENTS SURVEY

RESULTS OF THE STUDENT SURVEY ON STUDENT ASSESSMENT AT YSU 1 Assessment criteria are easy to understand 2 Teachers follow the assessment criteria 3 Written exams allow me to convey what I learned 4 Oral exams allow me to convey what I learned 5 Multiple-choice tests allow me to convey what I learned 6 My performance on exams reflects my skill-level 7 I receive feedback from my teacher after each assessment 8 Appeal system on assessment is efficient
not applicable disagree strongly disagree somewhat no strong view agree somewhat agree strongly

Mean grade 4,04 4,03 3,83 4,29 3,70 3,61 3,60 3,44

3 1 8 3 10 9 25 23

8 5 15 6 16 15 14 15

18 24 30 13 41 41 32 38

26 29 27 25 26 28 46 61

109 99 89 66 83 108 83 71

95 100 89 141 82 58 59 50

Chart 1.

Assessment criteria are easy to understand

agree strongly agree somewhat no strong view disagree somewhat disagree strongly not applicable 3 8 18 26

95 109

Chart 2.

Teachers follow the assessment criteria

agree strongly agree somewhat no strong view 29

100
99

disagree somewhat
disagree strongly not applicable 1 5

24

Chart 3.

Written exams allow me to convey what I learned

agree strongly 89

agree somewhat

89

no strong view

27

disagree somewhat

30

disagree strongly

15

not applicable

Chart 4.

Oral exams allow me to convey what I learned

agree strongly 141

agree somewhat

66

no strong view

25

disagree somewhat

13

disagree strongly

not applicable

10

Chart 5.

Multiple-choice tests allow me to convey what I learned

agree strongly

82

agree somewhat

83

no strong view

26

disagree somewhat

41

disagree strongly

16

not applicable

10

Chart 6.

My performance on exams reflects my skill-level

agree strongly agree somewhat 58 108 28 41 15 9

no strong view
disagree somewhat disagree strongly not applicable

11

Chart 7.

I receive feedback from my teacher after each assessment

agree strongly 59

agree somewhat

83

no strong view

46

disagree somewhat

32

disagree strongly

14

not applicable

25

Chart 8.

Appeal system on assessment is efficient

agree strongly 50

agree somewhat

71

no strong view

61

disagree somewhat

38

disagree strongly

15

not applicable

23

12

Chart 9.
Mean grade
Appeal system on assessment is efficient I receive feedback from my teacher after each assessment My performance on exams reflects my skilllevel Multiple-choice tests allow me to convey what I learned Oral exams allow me to convey what I learned Written exams allow me to convey what I learned Teachers follow the assessment criteria Assessment criteria are easy to understand 3,44 3,60 3,61 3,70 4,29 3,83 4,03 4,04

13

THE RESULTS OF THE TEACHING STAFF SURVEYS

The teaching staff survey has been conducted among 50 YSU teachers from the faculties of Biology (38 teachers) and Journalism (12 teachers). It has been carried out according to the prepared questionnaire Teaching staff survey on student assessment at YSU (ANNEX 2). The questionnaire contains 7 questions and designed to assess such aspects of student assessment at YSU as assessment modes and methods, effectiveness of assessment, use of assessment data for improving teaching & learning, and feedback on assessment results provided to students by teaching staff. The evaluation was carried out according to the fivepoint grading scale, where 1=disagree strongly, 2=disagree somewhat, 3=no strong view, 4=agree somewhat, 5=agree strongly (and 0=not applicable). The template of the questionnaire enabling computer-aided processing of the survey results is attached (ANNEX 2). The results of the conducted survey are presented in the Table 2 and in the following Charts 10-17. According to the survey results, participated in the survey teachers assessed different aspects of student assessment practices at YSU as follows: 1. Interim (written) exams allow students to convey what they learned: 3.72 2. Final (oral) exams allow students to convey what they learned: 4.76 3. Multiple-choice tests allow students to convey what they learned: 3.44 4. The assessment methods and criteria used allow to accurately measure of what student learned: 3.78 5. I use student assessment results to improve my own teaching and learning methods: 3.76 6. Usually I provide feedback or advice to students after each assessment based on their performance in assessment task: 4.59
14

7. The appeal system on assessment is efficient: 3.80 It should be noted, that mean assessment grades for each category/aspect vary from 3.44 to 4.76. For instance, teachers are more dissatisfied with the efficiency of multiplechoice tests and most satisfied with the final exams (as in students case), which are conducted in oral mode. The detailed results of the survey are presented in the Charts 10-17.

15

TABLE 2. THE RESULTS OF THE 50 TEACHERS SURVEY

RESULTS OF THE TEACHER SURVEY ON STUDENT ASSESSMENT AT YSU Written exams allow students to convey what they 1 learned 2 Oral exams allow students to convey what they learned Multiple-choice tests allow students to convey what they 3 learned Assessment methods/criteria allow to measure of what 4 students learned I use student assessment results to improve my own 5 teaching methods 6 I provide feedback to students after each assessment 7 Appeal system on assessment is efficient
not applicable disagree strongly disagree somewhat no strong view agree somewhat agree strongly

Mean grade 3,72 4,76 3,44 3,78 3,76 4,59 3,80

0 0 1 0 0 1 2

2 0 2 2 1 0 1

6 0 8 5 7 2 6

4 1 10 7 12 3 10

30 10 23 24 13 8 13

8 39 5 12 17 36 16

16

Chart 10.

Written exams allow students to convey what they learned

agree strongly 8

agree somewhat

30

no strong view

disagree somewhat

disagree strongly

not applicable

Chart 11.
Oral exams allow students to convey what they learned
agree strongly

39

agree somewhat

10

no strong view

disagree somewhat

disagree strongly

not applicable

17

Chart 12.
Multiple-choice tests allow students to convey what they learned
agree strongly

agree somewhat

23

no strong view

10

disagree somewhat

disagree strongly

not applicable

Chart 13.

Assessment methods/criteria allow to measure of what students learned

agree strongly 12

agree somewhat

24

no strong view

disagree somewhat

disagree strongly

not applicable

18

Chart 14.

I use student assessment results to improve my own teaching methods

agree strongly agree somewhat no strong view disagree somewhat 12 13 17

7
1 0

disagree strongly
not applicable

Chart 15.

I provide feedback to students after each assessment

agree strongly 36

agree somewhat

no strong view

disagree somewhat

disagree strongly

not applicable

19

Chart 16.

Appeal system on assessment is efficient

agree strongly
agree somewhat no strong view disagree somewhat disagree strongly not applicable 1 2 6 10 16

13

Chart 17.
Mean grade
Appeal system on assessment is efficient I provide feedback to students after each assessment I use student assessment results to improve my own teaching methods Assessment methods/criteria allow to measure of what students learned Multiple-choice tests allow students to convey what they learned Oral exams allow students to convey what they learned Written exams allow students to convey what they learned 3,72 3,76 3,80

4,59

3,78

3,44

4,76

20

CONCLUSIONS

The conducted surveys allow for making the following conclusions. A) Student survey: 1. Majority of the questioned students evaluate positively the clarity and intelligibility of assessment criteria (requirements) and how the teaching staff follows those criteria during examination process (79% and 77% accordingly). 2. Majority of the questioned students more favour written and oral exams (69% and 80% accordingly) and less countenance to multiple-choice tests (only 64%). 3. Only 22% of the surveyed students find that their performance on exams does not reflect completely their skill-level and 64% of the students consider that there is a match/conformity between exam results and their knowledge. 4. Only 55% of the surveyed students find that they receive feedback or advice from their teachers after each assessment based on their performance in assessment task/exam although 18% of the students is not agree with this statement. As far as these results are above our expectations this issue needs to be explored in more detailed way. 5. The 44% of the questioned students is not satisfied or dispute efficiency of the current appeal system which indicates to necessity of its revision.

B) Teaching staff survey: 1. Majority of the surveyed teachers find written or oral modes of exams more efficient (76% 98% accordingly), and only 46% of the teachers consider multiple-choice tests appropriate for the effective student assessment.
21

2. 72% of the questioned teaching staff finds that the assessment methods and criteria used by them allow to accurately measure of what students learned. 3. Although 60% of the questioned teachers answered that they use student assessment results to improve their own teaching and learning methods/activities, nevertheless these results could not be considered as reliable, therefore the issue needs further exploration. 4. 88% of the surveyed teaching staff answered that they provide feedback or advice to students after each assessment based on their performance in assessment task. As far as these results are above our expectations the issue needs to be further explored. 5. Only 14% of the surveyed teachers consider that the appeal system on student assessment is not efficient.

22

ANNEX 1

STUDENT SURVEY ON STUDENT ASSESSMENT AT YSU

Please indicate your level of agreement on the following statements about student assessment at YSU
Response Options: 0 = not applicable 1 = disagree strongly 2 = disagree somewhat 3 = no strong view 4 = agree somewhat 5 = agree strongly

# Assessment criteria

Survey Question

1. The assessment requirements(criteria) are easy to understand 2. Teachers follow the assessment criteria in the assessment tasks Assessment modes and methods 3. Interim (written) exams allow me to convey what I learned 4. Final (oral) exams allow me to convey what I learned 5. Multiple-choice tests allow me to convey what I learned Effectiveness of assessment 6. My performance on exams usually reflects my skill-level completely Feedback on assessment 7. Usually I receive feedback or advice from my teacher after each assessment based on my performance in assessment task 8. The appeal system on assessment is efficient

23

THE TEMPLATE OF THE QUESTIONNAIRE

24

ANNEX 2

TEACHING STAFF SURVEY ON STUDENT ASSESSMENT AT YSU

Please indicate your level of agreement on the following statements about student assessment at YSU
Response Options: 0 = not applicable 1 = disagree strongly 2 = disagree somewhat 3 = no strong view 4 = agree somewhat 5 = agree strongly

Survey Question

Assessment modes and methods 1. Interim (written) exams allow students to convey what they learned 2. Final (oral) exams allow students to convey what they learned 3. Multiple-choice tests allow students to convey what they learned Effectiveness of assessment 4. The assessment methods and criteria used allow to accurately measure of what student learned Use of assessment data 5. I use student assessment results to improve my own teaching and learning methods Feedback on assessment 6. Usually I provide feedback or advice to students after each assessment based on their performance in assessment task 7. The appeal system on assessment is efficient

25

THE TEMPLATE OF THE QUESTIONNAIRE

26