You are on page 1of 18

SCHOOL-BASED ASSESSMENT AMONG ESL TEACHERS IN MALAYSIAN SECONDARY SCHOOLS Chan Yuen Fook (Ph.

D) Gurnam Kaur Sidhu (Ph. D) Faculty of Education, Universiti Teknolo MARA 40200 Shah Alam, Selangor, Malaysia yuenfookchan@yahoo.com Abstract: School-based assessment (SBA) is a process of monitoring, evaluating and implementing plans to address perceived weaknesses and strengths of the school. Today, policymakers and educators are looking towards SBA as a catalyst for education reform. Since the late 1990s and the beginning of 2000, SBA has slowly made its way into the Malaysian classrooms. Since then, there have been only a few studies that looked into the implementation of SBA in Malaysia. Therefore a study was conducted to investigate the knowledge and best practices of Malaysian ESL (English as a Second Language) teachers conducting the SBA. Such a study is deemed timely and crucial as it could provide a relevant picture for scholars, practitioners and policy makers in relation to testing and assessment. Descriptive research design was employed to examine the level of knowledge and best practices of ESL teachers in SBA. The study indicated that most of the respondents had acquired adequate knowledge in constructing their own tests, but one third of the respondents admitted that they often applied cut and paste method and they did worry about the validity and reliability of the tests constructed. This indicates that there are a number of existing problems in SBA that have to be addressed as soon as possible. Perhaps what is needed is a concerted effort and cooperation from all parties concerned. Keywords: School-based assessment, Knowledge, Best Practices, Secondary School Abstract: Pentaksiran berasaskan sekolah merupakan satu proses pemantauan, penilaian dan pelaksanaan pelan tindakan untuk menangani kelemahan dan kekuatan yang diperhatikan di sesebuah sekolah. Hari ini, pembuat polisi dan para pendidik melihat pentaksiran berasaskan sekolah sebagai pencetus kepada reformasi pendidikan. Sejak akhir tahun 1990an dan permulaan tahun 2000, pentaksiran berasaskan sekolah dengan perlahan-lahan telah meresap masuk ke dalam bilik darjah di Malaysia. Namun demikian, tidak banyak kajian telah dilaksanakan untuk mengkaji keberkesanan pelaksanaan pentaksiran berasaskan sekolah di Malaysia. Oleh itu, kajian ini telah dilaksanakan untuk mengkaji pengetahuan dan amalan terbaik guru Pengajaran Bahasa Inggeris sebagai Bahasa Kedua dalam mengendalikan pentaksiaran berasaskan sekolah. Kajian ini dianggap penting dan memenuhi keperluan semasa sebab ia dapat membekalkan satu gambaran yang relevan kepada sarjana pendidikan, pelaksana dan pembuat polisi tentang aspek pengujian dan pentaksiran. Penyelidikan deskriptif telah digunakan untuk mengenalspati tahap pengetahuan dan amalan terbaik guruguru Pengajaran Bahasa Inggeris sebagai Bahasa Kedua dalam pentaksiran berasaskan sekolah. Kajian ini menunjukkan bahawa kebanyakan responden telah menguasai pengetahuan yang cukup dalam mengendalikan ujian mereka, tetapi satu per tiga daripada responden mengakui bahawa mereka sering mengamalkan kaedah gunting dan tampal dan mereka juga risau tentang kesahan dan kebolehpercayaan ujian yang dibina oleh mereka. 1

Dapatan kajian menunjukkan masih terdapat masalah-masalah dalam pentaksiran berasaskan sekolah yang perlu ditangani dengan segera. Kajian juga menunjukkan bahawa kerjasama erat dan sokongan penuh daripada pelbagai pihak adalah amat dialu-alukan demi menjayakan pelaksanaan pentaksiran berasaskan sekolah. Kata Kunci: Pentaksiran Berasaskan Sekolah, Pengetahuan, Amalan Terbaik, Sekolah Menengah INTRODUCTION For some time now, until the mid nineties testing in Malaysia has been rather centralised and summative in nature. Today, there is sufficient research and literature (Frederiksen, 1984; Cunningham, 1998; Dietel, Herman & Knuth, 1991; Borich & Tombari, 2004: 270) that has shown the shortcomings of one centralised summative examination. Such assessment mechanisms often influence what schools teach and it encourages the use of instructional methods that resemble tests. Besides that, teachers often neglect what external tests exclude. Therefore, there is a global move to decentralise assessments and this has seen the introduction of authentic alternative forms of assessments that are continuous and formative in nature. Since the late nineties and the beginning of 2000, SBA has slowly made its way into the Malaysian classroom. Nevertheless, in tandem with a global shift towards decentralised assessment, similar steps towards decentralising assessment have also been taken by the Malaysian examination bodies. Today, both policymakers and educators are now looking towards School-based assessment (SBA) as a catalyst for education reform. It is seen as a leverage for instructional improvement in order to help teachers find out what students are learning in the classroom and how well they are learning it. Such an opinion was articulated by the former Malaysian Minister of Education, Tan Sri Musa Mohamed on May 7, 2003 when he said that, we need a fresh and new philosophy in our approach to exams . . . we want to make the education system less exam-oriented and (we) are looking at increasing SBA as it would be a better gauge of students' abilities. The former Minister of Education, Tan Sri Musa Mohamed also pointed out that there would be greater reliance on SBA in the future. According to him, such a method of assessment would be in line with current practices in other countries such as the United States, Britain, Germany, Japan, Finland and New Zealand (Musa, 2003; Marlin Abd. Karim, 2002). Thus, the Ministry of Education in Malaysia has looked into ways on how this approach could be expanded to all levels of education. Furthermore, with greater reliance on SBA in the future, some major examinations may be abolished while some would have less bearing on students' overall grades. The Malaysian Examinations Syndicate (MES) holds the view that SBA is any form of assessment that is planned, developed, conducted, examined and reported by teachers in schools involving students, parents and other bodies (Adi Badiozaman, 2007). These kinds of school assessment can be formative throughout the teaching-learning process or summative at the end of it. Such as move would help teachers identify students strengths and limitations. The greater flexibility and reliability offered by SBA has seen the current shift towards decentralisation in school assessment. A similar shift is also seen creeping into the Malaysian schools. Malaysia is currently in the process of revamping its school system to 2

make it less exam-oriented. Consistent with global trends and in keeping with the governments initiatives to improve the quality of human capital development efforts through a holistic assessment system, Malaysian Examinations Syndicate has introduced the National Educational Assessment System (NEAS). The NEAS proposed by the MES promotes a change from a one-off certification system to one that propagates accumulative graduation. In line with that, at an international forum organised by MES in 2007, the former Director of Examinations, Dr. Adi Badiozaman in his concept paper proposed that Primary Year 6 UPSR and the Secondary Three PMR may be abolished and these would be replaced by a broader and more flexible assessment system. This broader concept of assessment requires the assessors to be well-versed in the knowledge of other forms of alternative assessments such as observations, portfolios, peer and self assessments and projects. With the intended implementation of the new concept in the overall school assessment system, it is then the concern of many parties involved in the education system to review current practices and to be well prepared in this regard. The concept of assessment has to be vivid and clear to all, especially the teachers who are to be the implementers. STATEMENT OF THE PROBLEM In line with the changing trends in assessment, SBA or PKBS (Penilaian Kendalian Berasaskan Sekolah) has been introduced into Malaysian schools under the New Integrated Curriculum for Secondary Schools (KBSM). It has adopted coursework for a few subjects in secondary schools such as History, Geography, Living Skills and Islamic Education for the lower secondary classes and Biology, Chemistry and Physics for the upper secondary classes. Beginning 2003 the school-based oral assessment commenced for both Bahasa Malaysia and English Language. Today the school-based Oral English Test is a compulsory component for Secondary Five candidates taking the SPM Examination. Nevertheless, it cannot be denied that SBA is an approach to redesign assessment in schools. It gives all educational stakeholders, namely educators, parents, students and the community-at-large, the power to improve teaching and learning practices. By transferring SBA decisions to schools, educators are now empowered to help students perform better. SBA has a great appeal, as witnessed by the large number of countries all around the globe trying some form of it. Its results, however, have been rather slow as any change requires a paradigm shift for all stakeholders concerned. While some have enthusiastically boarded the ship, there are others who are still sceptical that the current wave of decentralisation could be just another swing of the pendulum. According to Brookhart (2002: 5), studies of what teachers are or should be taught and what they know have generally concluded that teachers knowledge of large-scale testing is limited especially in the important area of communicating assessment results to students, parents and administrators. The findings also indicated that teachers have limited skills at gathering and using classroom assessment information for improving student learning. In fact the findings of many other researchers have also concluded that courses in testing and measurement for teachers should increase emphasis on classroom assessment, and decrease emphasis on large-scale testing (Gullickson, 1993; Marso & Pigge, 1993; Stiggins, 1991; Brookhart, 2002). Thus, we must acknowledge the fact that if teachers possess low levels of knowledge in assessment they may not be able to help improve student learning. These teachers may feel overwhelmed and frustrated and consequently might display undesirable work behaviours 3

towards performing best practices in SBA. If efforts are not made to investigate and to improve the best practices of teachers in SBA, the noble aspirations that the Education Ministry hopes to achieve with the implementation of the SBAs may not be fulfilled. According to Rohizani Yaakub & Norlida Ahmad (2003), improving best practices of teachers in testing and assessment should be an important objective in order to improve the levels of their teachers knowledge in SBA. Unfortunately, until now there have not been many studies conducted to gauge the levels of knowledge and best practices of Malaysian ESL teachers in SBA. The need to measure the levels of knowledge and best practices of ESL teachers is even more pressing considering the importance of English language in todays process of globalisation and the advent of the Information Technologies. Consequently, there is a need to understand further the levels of their knowledge and best practices implemented by ESL teachers in SBA because of the varying perceptions of teachers regarding this pertinent issue. Documenting ESL teachers opinions vis--vis knowledge and best practices in SBA is both timely and significant. Such a study is deemed timely and crucial as it could provide a relevant picture for scholars, practitioners and policy makers in relation to testing and assessment. OBJECTIVES OF THE STUDY The purpose of this study was to examine the level of knowledge and best practices of ESL teachers in SBA as perceived by ESL teachers. The main selected factors were knowledge and best practices in SBA. Specifically, the objectives of the study were to: 1. Describe teachers levels of knowledge in SBA 2. Examine teachers best practices in SBA THE METHOD This is a descriptive study that examined the level of knowledge and best practices of ESL teachers in SBA. Henceforth a descriptive study was chosen to allow a qualitative and quantitative description of the relevant features of the data collected (Cohen, Manion & Morrison, 2003). It involved the use of a questionnaire and structured interviews. The questionnaire consisted of both close and open-ended questions that were used to obtain data both in quantitative and qualitative forms. Quantitative data was analysed by using SPSS package (version 13.0) to identify the mean and standard deviation of each item. The qualitative data was analysed both inductively and deductively to identify the main themes that emerged based on the research questions posed in this study. The target population of this study were secondary school teachers in two states (Negeri Sembilan and Melaka) in Malaysia. The study examined the level of knowledge and best practices of ESL teachers in SBA in the day-time secondary schools. Thus, the study was confined to eight randomly selected day-time secondary schools in the state of Negeri Sembilan and six randomly selected day-time secondary schools in the state of Melaka. Once the schools were identified the multistage cluster sampling method was applied (Cohen, Manion & Morrison, 2003). Each cluster involved the Head of Language Department, Head of English Language Panel and all the English Language teachers in the participating schools.

Respondents Demographic Variables A total of 97 respondents from the 14 schools were involved in this study. Demographic findings (Table 1) indicated that out of the 97 respondents, 83.5% of them were females and 16.5% were males. This is reflective of the fact that a majority of ESL teachers in Malaysian schools are females. As for the ethnic groups, the results indicated that 41.7% of the respondents were Malays, 34.4% Chinese, 21.9% were Indians and the remaining 2.1% were from other ethnic groups. In terms of academic qualifications, the results showed that 80.4% of the respondents had a Bachelors degree, 11.3% had a master degree, 7.2% possessed a Diploma or below, and only 1.0% had a Doctorates degree. Table 1: Gender, Ethnic and Academic Qualification of Respondents Variable Gender Male Female Ethnic Group Malay Chinese Indian Others Academic Qualification Doctorate Degree Master Degree Bachelor Degree Others (Diploma/Certificate) Frequency (n=97) 16 81 40 33 21 2 1 11 78 7 Percent 16.5 83.5 41.7 34.4 21.9 2.1 1.0 11.3 80.4 7.2

The study also looked into the distribution of age, age group, teaching experience and classes or forms the respondents. The demographic analysis displays the frequency distribution, percentages, means and standard deviations of respondents according to the variables mentioned. The results showed that the age of the respondents ranged from 25-56 years with mean of 42.5 and standard deviation (SD) of 7.3. The findings indicated that the majority of the respondents (64.2%) were between 35-49 years. Less than 20% of the respondents were between 50-54 years, approximate to 11% were between 30-34 years and less than 5% were between 25-29 years and 55 and above. In examining teaching experience in the present school, the results revealed that 44.7% of the respondents taught English in the lower forms while another 55.3% taught in the upper forms. FINDINGS Planning the SBA The survey reveals that respondents had sufficient knowledge on many aspects of planning the SBA (Table 2). Respondents did not have problems in writing both the specific and general objectives for their lesson plans. The mean scores for the respondents ability to write the learning objectives are 3.32 and 3.33 respectively, while the standard deviations are 0.57 and 0.59. In relation to the ability to write good objectives, the respondents also 5

indicated that they did not have problems in interpreting the English language syllabus. The mean score is above average which is 3.38, and the standard deviation is 0.62. This is consistent with their ability to write good teaching objectives. The findings also disclosed that the respondents did not have problems in outlining instructional content (mean = 2.99, SD=.66) and objectives for the test (mean = 2.98, SD=.66). Furthermore, most of the ESL teachers in schools were competent in identifying instructional objectives and content for SBA. Apart from that, they should be able to identify the taxonomy of educational objectives. This is to determine the level of difficulty of assessment in accordance to the cognitive, affective and psychomotor domains. It is obvious that the respondents knowledge in the cognitive (mean=2.79, SD=.65), affective (mean=2.73, SD=.68) and psychomotor (mean=2.76, SD=.71) is at the satisfactory level. Thus, the results show that teachers were knowledgeable in developing a table of specifications (mean=2.88, SD=.69). A table of specifications is very important especially at the planning stage because it is like a plan for the preparation of a test paper. Table 2: Knowledge in Planning for SBA Knowledge in Planning the SBA Taxonomy of educational objective: cognitive Taxonomy of educational objective: affective Taxonomy of educational objective: psychomotor English language syllabus Writing general objective for lesson plan Writing specific objective for lesson plan Developing table of specification for test Outlining instructional content for the test Mean 2.79 2.73 2.76 3.38 3.32 3.33 2.88 2.99 Std. Deviation .65 .68 .71 .62 .57 .59 .69 .66 .67

Listing instructional objectives for the test 2.98 Scale: 1 = very limited, 2 = limited, 3 = sufficient, 4 = very sufficient Developing the SBA

Results from questionnaire survey (Table 3) show that the English Language teachers rated themselves as knowledgeable with regards to the construction of MCQ (multiple choice questions) items on several language components, vis--vis; in reading comprehension (mean=3.24, SD=.60), grammar (mean=3.23, SD=.63), language forms and functions (mean=3.20, SD=.65), and language in use (mean=3.17, SD=.62). Besides that, most of the respondents also indicated that they had sufficient knowledge in constructing MCQs based on literary texts (mean=3.00, SD=.67). The responses have high mean scores of 3 and above. This indicates that their knowledge in constructing MCQ items on the listed components is sufficient. On top of that, the respondents also did not have problems in preparing the answer 6

keys for MCQ items (mean=3.15, SD=.69). Again, a high mean score of 3.00 and standard deviation 0.76 reveals that the respondents knowledge in the construction of MCQ items construes with their knowledge in scoring the test papers. The mean scores for their knowledge on the construction of open-ended cloze test (mean=3.14, SD=.64), guided composition (mean=3.21, SD=.64) and free writing (mean=3.14, SD=.66) are all above 3. The respondents also did not have problems in developing analytic scoring rubrics (mean=2.84, SD=.75) and holistic scoring rubrics (mean=2.86, SD=.73) for essay questions. Overall, the findings show that teachers do have sufficient knowledge in scoring essay questions (mean=3.00, SD=.65). Table 3: Knowledge in Developing the SBA Knowledge in Developing and Administering the SBA Mean Constructing MCQ in reading comprehension 3.24 Constructing MCQ in language forms & functions 3.20 Constructing MCQ in language in use 3.17 Constructing MCQ in grammar 3.23 Constructing open-ended questions in cloze text 3.14 Constructing open-ended question in guided composition 3.21 Constructing open-ended question in free writing 3.14 Constructing MCQs based on literary texts 3.00 Constructing questions for listening components 2.91 Constructing questions for speaking components 2.97 Reviewing & assembling classroom test according to format 3.13 Preparing answer keys for MCQs 3.15 Developing analytic scoring rubrics for essay questions 2.84 Developing holistic scoring rubric for essay questions 2.86 Administering the classroom test 3.26 Scale: 1 = very limited, 2 = limited, 3 = sufficient, 4 = very sufficient Std. Deviation .60 .65 .62 .63 .64 .64 .66 .67 .65 .62 .60 .69 .75 .73 .75

Respondents also reported having sufficient knowledge in test preparation with regards to aspects such as the construction of questions for the speaking (mean=2.97, SD=.62) and listening components (mean=2.91, SD=.65). The respondents had sufficient knowledge in reviewing and assembling classroom assessment according to the proper order and format (mean=3.13, SD=.60). At this stage they should be compiling, editing and setting the paper according to the appropriate format before administering the test paper. Apparently, respondents did rate their knowledge in administering classroom test as sufficient (mean=3.26, SD=.75). Hence, it can be concluded that, the respondents understood and knew the principles of constructing both objective (MCQ items) and subjective questions (openended) which was based on the syllabus and lessons they had taught. Scoring, Reporting and Analysing the SBA Even though respondents indicated that they had sufficient knowledge in planning and developing MCQs and essay questions as well as administering the test, findings in the study however reveal that respondents still did not possess sufficient knowledge for the successful implementation of SBA. The results (Table 4) show that they had limited knowledge in analyzing the score and conducting item analysis. The respondents knowledge in analysing 7

and interpreting the scores reflected that they were not familiar in calculating the mean (mean=2.52, SD=.80), standard deviation (mean=2.31, SD=.79), z-score (mean=2.19, SD=2.2) and t-score (mean=2.02, SD=.67). It can be deduced that the respondents most probably did not have practise in interpreting the scores because if they did, they would not rate their level of knowledge in this aspect as limited. This lack of knowledge in interpreting the scores in the form of basic statistics resulted in poor reporting of test scores (mean=2.39, SD=.81). Table 4: Knowledge in Scoring, Reporting and Analyzing the SBA Knowledge in Scoring, Reporting and Analyzing the SBA Mean Scoring the MCQ 3.00 Scoring essay questions 3.00 Calculating mean 2.52 Calculating standard deviation 2.31 Calculating difficulty index 2.20 Calculating discrimination index 2.15 Calculating t-score 2.02 Calculating z-score 2.19 Reporting score 2.39 Forming item bank 2.48 Scale: 1 = very limited, 2 = limited, 3 = sufficient, 4 = very sufficient Std. Deviation .76 .65 .80 .79 .72 .67 .67 2.20 .81 .68

Another obvious indicator of the respondents limited knowledge relates to item analysis data. They rated their knowledge in calculating the indices of difficulty (mean=2.20, SD=.72) and discrimination (mean=2.15, SD=.67) as very limited. Furthermore, the lack of knowledge in data analysis is evident as the respondents did not maintain an item bank as their knowledge is still limited (mean=2.48, SD=.68). In conclusion, it can be said that respondents had limited knowledge in reporting the score and analysing the data of the test. In addition, they also had limited knowledge in setting up an item bank (mean=2.48, SD=.68). Best Practices in SBA

oday SBA is an integral part of the teaching and learning process. Therefore all teachers should possess a good understanding of SBA as a good knowledge of it would lead to the implementation of best practices at different stages in SBA. Such a move would certainly lead to the success of students in the learning process. Thus, the third part of the questionnaire investigated the TESL teachers best practices in a variety of aspects in SBA. Planning Stage The main goal of classroom assessment is to obtain valid, reliable, and useful information concerning student achievement. This requires determining what is to be measured and then defining it precisely so that tasks that require the intended knowledge, skills and understanding while minimizing the influence of irrelevant or ancillary skills can be constructed. A good planning of the test makes expected learning outcomes explicit to students and parents and shows the types of performances that are valued. The likelihood of 8

preparing valid, reliable and useful classroom assessments was greatly affected in this study due to certain identified drawbacks. The results (Table 5) showed that only 69.9% of the respondents prepared a tentative list of instructionally relevant learning outcomes and only 63.3% of the respondents listed beneath each general instructional objective a representative sample of specific learning outcomes that describe the terminal performance students are expected to demonstrate. Even though a majority of the respondents said that they stated instructional objectives in terms of students terminal performance (86.8%) and at the end of instruction (75.6%), and they are able to state general objectives (67.7%) and specific learning outcomes (91.2%) correctly, confusion could be detected in their responses. Many respondents (>80%) admitted that they did state their instructional objectives in terms of teacher performance (84.9%), learning process (90.3%), course content (85.6%) or stating two or more objectives in one objective (89.1%). Table 5: Best Practices in Planning the SBA Yes (%) 69.9 63.3 67.7 91.2 84.9 90.3 85.6 89.1 86.8 75.6 47.9 61.4 48.9

Do you do the following when you are planning the SBA? Prepare a tentative list of instructionally relevant learning outcomes. List beneath each general instructional objective a representative sample of specific learning outcomes that describe the terminal performance students are expected to demonstrate. Begin each general objective with a verb (e.g., knows, applies, interprets). Omit The student should be able to . . . Begin each specific learning outcome with an action verb that specifies observable performance (e.g., identifies, describes). State instructional objectives in terms of: teacher performance. learning process. course content. two or more objectives. students terminal performance. at the end of instruction. Review the list for completeness, appropriateness, soundness and feasibility. Develop a table of specifications to identify both the content areas and the instructional objectives we wish to measure. Use your test and assessment specifications as a guide.

Furthermore, the findings in this study indicated that only 61.4% respondents developed a table of specifications to identify both the content areas and the instructional objectives they wished to measure. However, only less than half of the respondents used their table of specifications as a guide (48.9%) and review the list for completeness, appropriateness, soundness and feasibility (47.9%). One has to understand that to ensure that classroom tests measure a representative sample of instructionally relevant tasks; it is of utmost importance to develop specifications of test items and assessment tasks based on the different cognitive domains. 9

Constructing Stage It can be seen from the result of 81.1% of the respondents did prepare test items well in advance (Table 6). This is a good practice for such a move gives teachers ample time to review or edit the paper to produce a good test paper. This is supported by the evidence that 86.5% of the respondents did edit, revise and recheck the paper for relevance. Findings also indicated that 90.3% of the respondents ensured that the test items and the tasks were appropriate to their students reading level. This was supported by the fact that that 75.3% of the respondents were aware of the elements of validity so much so they agreed that the item constructed would be recognised by experts. Most of the respondents (62.2%) indicated that they are aware of the importance of writing each item that does not provide help in responding to other item. Close to 89.0% of the respondents agreed that the test item and assessment to be performed is clearly defined and it calls forth the performance described in the intended learning outcome. Table 6: Best Practices in the Construction of SBA Yes (%) 38.9 39.3 81.1 89.0 90.3. 62.2 75.3 86.5

Do you do these when you are constructing the items for SBA? Copy all the items from the suitable reference books. Write more items and tasks than needed. Write the items and tasks well in advance of the testing date. Write each test item and assessment task so that the task to be performed is clearly defined and it calls forth the performance described in the intended learning outcome. Write each item or task at an appropriate reading level. Write each item or task so that it does not provide help in responding to other items or tasks. Write each item so that the answer is one that would be agreed on by experts or, in the case of assessment tasks, the responses judged excellent would be agreed on by experts. Whenever a test item or assessment task is revised, recheck its relevance.

On the other hand, more than half of the respondents (60.7%) indicated that they did not prepare more items and tasks than needed due to time constraints and a heavy workload in school. This finding was further validated in the interviews. In the interview sessions, five teachers stated that they had more than 25 periods per week plus other administrative tasks such as being the subject head and co-curricular adviser. However, teachers need to understand the fact that extra items are important especially at the review stage. Should there be any faults with the items selected earlier, then these extra items will be very useful. Furthermore the unused items can always be added to the item bank for future use. Such a practice is not taking place in schools as earlier results already indicated that respondents did not have adequate knowledge to set up item banks. In principle, teachers should construct their own items instead of practising the cut and paste technique. However, slightly more than a third of the respondents (38.9%) admitted that they often took test items from reference books. This fact was confirmed by a majority of the teachers during the interview sessions. Teachers admitted that this was a common practice as there is an abundant supply of good reference books. Teachers cited time constraint and lack of knowledge as the two main reasons for resorting to using items from reference books. 10

Nevertheless a majority of the teachers emphasised that even though they referred to reference books, they always made sure they chose and adapted questions to suit their students proficiency levels. Administering Stage A majority of them (96.5%) observed the basic principles of test administration (Table 7). This is proven in the high percentage for every item responded, the lowest being 86.7% and the highest 98.9%. While administering the test papers, a majority of the respondents having the experience as examination invigilators tried to keep interruptions to a minimum (97.8%) and also highlighted the fact that they did not indulge in unnecessary talk (92.1%). This is to maintain the reliability of test administration. In relation to that, respondents (95.6%) would keep the test reliable by not allowing any form of cheating to occur. By right, teachers should always be alert while carrying out their duty as invigilators. There are too many tricks that students can do in their attempt to cheat in a test. Besides that, the respondents (89.0%) were aware that they should not provide clues or hints to the questions. Respondents also knew their other duties and roles before administering the test papers. They reviewed the test papers for errors (95.5%) and made sure instruction or directions were clear and concise before assembling them for administration (98.9%). Table 7: Best Practices during the Administration of SBA Yes (%) 92.1 97.8 89.0 95.6 95.5 98.9 96.5

Do you do the following when administering SBA? When you are administering the assessment, you abide by these rules and regulations: Do not talk unnecessarily before letting students start working Keep interruptions to a minimum Avoid giving hints to students who ask about individual items Discourage cheating, if necessary Review and assemble the classroom test Ensure the directions are clear and concise Administer the classroom test professionally

The interview sessions also looked into teachers perception as to the quality and administration of SBA in relation to national standardised tests. Approximately 90% of the respondents reported that they often ensured classroom assessments matched the needs of national standardised examinations so that their students will be familiar with the actual examination format. These teachers also stressed that aspects such as content, administration and scoring were strictly adhered to and complied with national high-stake examinations. Scoring Essay Questions for SBA In scoring the essay questions, respondents did agree that they adhered to the basic principles in scoring and grading the scripts (Table 8). A high percentage of between 86.8% to 96.3% responses to the respective items revealed that they were aware of the good procedures in scoring and grading. Firstly, most of the respondents (89.0%) prepared an outline of the expected answer in advance before constructing an answer rubric. Most of the teachers 11

(93.4%) made sure they use the scoring rubric that is most appropriate. They also decided and considered how to handle factors that were irrelevant to the learning outcomes being measured (86.8%). As they marked the papers, they (91.2%) went through one question at a time before marking the other. Majority of the teachers (94.6%) also marked the papers without knowing the name of the students. This is a good practice in order to avoid bias and maintain fairness in scoring. In addition, the respondents (83.9%) also practiced moderation of the scores obtained by having independent ratings before finally grading the test papers. Table 8: Best Practices in Scoring Essay Questions for SBA Do you do the following when scoring essay questions for SBA? Yes (%) While scoring essay questions, you assure the following: Prepare an outline of the expected answer in advance. Use the scoring rubric that is most appropriate. Decide how to handle factors that are irrelevant to the learning outcomes being measured. Evaluate all responses to one question before going on to the next one. When possible, evaluate the answers without looking at the students name. If especially important decisions are to be based on the results, obtain two or more independent ratings. Grading the test Analysing and Interpreting the Test Score for SBA From the findings (Table 9), it can be concluded that most respondents did very basic analysis and interpretation of the score. The most that respondents did was to total up the score. This is usually followed by calculating the mean. Only 51.7% of the respondents did this. This is perhaps true of teachers in schools as many do not need to report the results in terms of mean or standard deviation (33.7% did this). The results also show that only very few teachers attempted to analyse standardised test scores such as t-score (22.5%) and z-score (20.5%) to identify a students actual performance in a particular subject in a class. Table 9: Best Practices in Analysing Test Scores for SBA Do you do the following when analysing the test score for SBA? Scoring the MCQ test by calculating: The total of the score The mean of the score The standard deviation of the score The t-score The z-score The difficulty index of the items The discrimination index of the items 12 Yes (%) 100.0 51.7 33.7 22.5 20.5 33.3 26.3 89.0 93.4 86.8 91.2 94.6 83.9 96.3

Another finding is that only a very small percentage of the respondents analysed the data to determine the difficulty index (33.3%) and discrimination index (26.3%). Then there is only the question as to why the percentage of the responses conducting the index of difficulty and index of discrimination did not tally. As it is, if one did the analysis of data, he would automatically calculate the two indices together. This points to the fact that respondents probably either did not understand what is meant by these two terms or they did not respond honestly to these items. Reporting the Test Score for SBA In Malaysian schools it is a common practice that teachers report mostly to students (92.2%) and parents (83.5%) as stated in Table 10. This is because Form teachers have to prepare report cards as feedback on students performance for parents. Findings further indicate that 60.7% of the respondents agreed that they sent reports to the district and 58.1% sent reports to the state education department. Normally, these reports are only for public examinations like the PMR, SPM and STPM. This finding relates to what the respondents knew about basic statistics and data analysis with regards to interpreting and analysing the scores. Table 10: Best Practices in Reporting Test Scores for SBA Yes (%) 92.2 83.5 60.7 58.1 95.5 28.2

Do you do the following when reporting the test score for SBA? Report the test score to: Students Parents The district education office The state education department Use results for the improvement of learning and instruction Form the item bank

Apart from that, 95.5% of the respondents agreed that they used the results to improve learning and instruction. By analysing the assessment results, a teacher can diagnose the strengths and limitations of their students performance. A good teacher is one that is reflective and improves instruction based on the exam results. As for the item bank, 28.2% of the respondents revealed that they did have an item bank. This percentage is considered small but nevertheless, the setting up of a item bank is an important aspect in testing and assessment and it is something that all schools and teachers should practise. Interview sessions with respondents revealed that they all agreed that SBA results were used in improving the teaching and learning process. A large majority (95%) also admitted that assessment results most certainly gave valuable feedback as to the strengths and limitations of their students. However, 80% admitted to actually using these results for further enhancement of the teaching and learning process. Time constraints and the rush to complete the syllabus were cited as the main reasons that hindered ESL teachers to use SBA results to enhance student learning.

13

DISCUSSION OF FINDINGS Teachers in this study were also aware of the pros and cons of SBA. They were articulate as to the benefits of SBA stressing that it helps to provide them with information and insights needed to improve teaching effectiveness and learning quality. Teachers were also very aware that SBA encouraged classroom assessment to be an ongoing process. As such they felt learning is more meaningful to students as they can obtain immediate feedback on their performance. According to Jonassen (1988) feedback is an important aspect that assists learners in monitoring their understanding. Approximately 80% of the respondents admitted to using test scores for further enhancement of the teaching and learning process. A number cited time constraints, the rush to complete the syllabus, heavy teaching load and administrative duties that hindered them from using SBA results to enhance student learning. On the other hand, the study also indicated that ESL teachers possessed limited knowledge in a number of aspects such as interpreting test scores, conducting item analysis and forming a test bank. Teachers admitted that though they were familiar in calculating the mean, other calculations and terms like standard deviation, z-score and t-score were beyond their means. A number of them informed the researchers that they had heard of these terms during training but since their current job specifications did not require them to look into these aspects; they had forgotten their meanings and functions. A large majority of the ESL teachers were also not familiar with terms such as norm-referenced, criterion-referenced, formative and summative assessments. The mean scores on these items ranged from 10.5% to 23.3%. Terms like validity and reliability which are important aspects in testing and assessment also eluded a large majority. Only 10-14% of the respondents responded correctly to these items. In terms of best practices in SBA, a large majority of the respondents did prepare test items well in advance as this would give them sufficient time to review or edit the paper. The situations of lack of training and lack of positive responses to training in Malaysia are quite similar to what is experienced in advanced countries like the United States. According to Cromey and Hanson (2000), coordinating assessment activities and aligning them with a common "vision" for student achievement is rather demanding work for schools in the United States, particularly when, as research has consistently shown, few teachers and administrators have received formal training in assessment (Stiggins, 1995; Stiggins & Conklin, 1992; Wise, Lukin, & Roos, 1991). The national organisations, the research community, and state and local education agencies have attempted to provide guidance to schools regarding the use of student assessments. Nevertheless, it is not well understood how, or how well, schools are utilizing this support. Therefore schools must implement assessment systems that are not only aligned and integrated with the curricula, instructional practices and professional development strategies, but also contribute to the goal of increasing student achievement based on rigorous content standards. This is complex, demanding work that can last several years. Similarly, managing, synthesising, interpreting and using student assessment data obtained from a multifaceted assessment system can be a daunting task for educators, particularly when (a) the assessment system lacks coherence; (b) school staff do not have training or experience in student assessment; and (c) the time, attention, and energies of teachers and administrators are stretched to personal limits (Cromey & Hanson, 2000). Dwelling into the issues and challenges facing ESL teachers in SBA, a large majority felt the implementation left much to be desired. Even though teachers were aware of the positive 14

effects of SBA they cited time constraints and a heavy teaching load as the main culprits in its effective implementation. Issues of reliability and validity of school-based tests were also raised. Clearly, there are responsibilities associated with the data collection and use of assessment information. Hence, all the teachers should realise that assessment of students will not constitute productive use of time unless it is carefully integrated into a plan involving instructional action. Thus, it is important in a school system to constantly assess the quality of instruction, learning and assessment provided in its schools and to take account of the findings of research in establishing its benchmarks and policies (Hempenstall, 2001). RECOMMENDATIONS The findings of this study imply that the implementation of SBA in Malaysia with regards to the oral English assessment has its share of best practices and gaps that need to be addressed. Given below are some recommendations that could aid in the more effective implementation of the SBA. A large majority (70%) of the ESL teachers in the case study felt they lacked exposure to SBA and felt that any form of training in SBA would be welcomed. One has to understand the fact that any change in policy requires effective training to be provided to all parties concerned. For effective implementation of SBA, training should be on-going in the form of briefings, courses or workshops. Respondents in this study who had gone for training in SBA lamented on the fact that the course and exposure offered at school level was insufficient as the Ministry used the Cascade Model for training. Those who went for training at the national level were exposed to a weeklong training stint whereas at state level training was held for three days and at school level, training ranged from 1-day to 1 hour. A similar sentiment was also voiced by respondents with regards to the recent changes in the school-based oral assessment for PMR and SPM candidates at the Lower Secondary Three and Upper Secondary Five levels respectively. Many felt that providing them with a booklet of the new format did not give a good idea of what was expected. As such interpretations varied from one teacher to another and from one school to another. Hence there is a need for more frequent briefings and workshops for teachers to get hands-on training and exposure to SBA. Effective implementation of any new policy requires effective monitoring and feedback. Interview sessions with respondents indicated that there was little monitoring and supervision with regards to SBA. Monitoring in the school-based oral assessment for ESL teachers is usually conducted by the English language panel head or the Head of the Arts Department. Respondents indicated that many a time both these teachers lacked the knowledge of SBA to help provide effective feedback. Even though the MOEs School Inspectorate Division officers are also responsible for carrying out monitoring, respondents indicated that hardly any of them had provided any feedback into SBA. It was also learnt that regional coordinators (RCs / Pentaksir Kawasan) who are either senior ESL teachers or principals are also appointed by each district to help monitor the implementation of school-based oral assessment. All the schools in this study had not been visited by any RC. Hence, steps must be taken to ensure school top management especially the school principal and the senior assistants be trained to monitor and supervise teachers with regards to SBA. School principals as instructional leaders in the school must play their role to take effective 15

monitoring and supervision measures. More importantly, effective support mechanisms must be provided for teachers who need help. All these findings cited above indicate that perhaps the time has come for the authorities concerned to look into ESL teachers workload. A reduction in workload and a decrease in the number of student enrolment may be steps in the right direction to ensure effective implementation of SBA. Regarding the increase in paper and administrative work such as filling in forms, cards and report books, a large majority of the respondents felt schools should be provided with more office personnel to help reduce their workload. In the light of recent changes in education and a move towards formative SBA, there is a need for a review the implementation of SBA. Effective implementation of SBA requires a set of procedures which applies minimum professional requirements to each level of personnel involved in the process of SBA. Furthermore schools should also move toward the establishment of a network of professional personnel with various levels of responsibility in each school. This network of professional personnel should form a central examining board with trainers to help implementers at the school level. In line with the continuous and formative training at school level, school administrators must ensure regular SBA coordination meetings are held to monitor and evaluate SBA practices. CONCLUSION Good assessments are those that focus on students and their learning. Mitchell (1992) in her book "Testing for Learning" argued for new methods of assessing performance, and asserted that the use of SBA can have a significant impact on teaching style and student achievement. Rather than just the traditional written tests, teachers will need a number of different ways for assessing both the product and process of student learning especially in the teaching of the English language. Good knowledge acquired by the teachers and their best practices will definitely enhance the teaching and learning of the English language subject. Henceforth there is a need for the Ministry of Education to ensure that sufficient in-depth exposure courses are provided to ESL teachers. The current Cascade Training Models is not working well and teachers need hands-on experience in testing and assessment in order to ensure best practices are shared and carried out. Furthermore, ESL teachers are challenged with a new paradigm shift in the current ESL classroom. Teachers have to be innovative and creative to include both the literature component and the Oral English test into their everyday ESL classrooms. In such a situation continuous on-going formative assessment seem to be the call of the day. Only then can one expect to witness an improvement in students performance and teacher instruction.

16

REFERENCES Adi Badiozaman Tuah. (2007). National Educational Assessment System: A proposal towards a more holistic education assessment system in Malaysia. International Forum on Educational Assessmnt System organised by the Malaysian Examinations Syndicate, 8 May 2007, Sunway Resort Hotel, Petaling Jaya. Borich, G.D., & Tombari, M.L. (2004). Educational assessment for the elementary and middle school classroom. New Jersey: Pearson, Merill Prentice Hall. Brookhart, S.M. (2002). What will teachers know about assessment, and how will that improve instruction? In Assessment in educational reform: Both means and end, ed. R.W. Lissitz., & W.D. Schafer, pp. 2-17. Boston: Allyn and Bacon. Cohen, L., Manion, L., & Morrison, K. (2003). Research methods in education. New York: Routledge/Falmer. Cromey, A., & Hanson, M. (2000). An exploratory analysis of school-based student assessment systems. North Central Regional Educational Laboratory. Learning Point Associates. http://www.ael.org/dbdm/resource.cfm?&resource_index=9. Accessed on 7 July 2007. Cunningham, G.K. (1998). Assessment in the classroom: Constructing and interpreteing tests. London: The Falmer Press. Dietel, R.J., Herman, J.L., & Knuth, R. A. (1991). What does research say about assessment? Oak Brook: North central Regional Educational Laboratory http://www.ncrel./org/sdsaraes/stw_esys/4assess.htm. Accessed on 7 July 2007. Frederikesen, N. (1984). The real test bias: Influences of testing on teaching and learning. American Psychology, 39(3), 193-202. Gullickson, A.R. (1993). Matching measurement instruction to classroom-based evaluation: Perceived discrepancies, needs, and challenges. In Teacher training in measurement and assessment skills, ed. S.L. Wise, pp. 1-25, Lincoln, Neb.: Buros Institute of Mental Measurements, University of Nebraskla-Lincoln. Hempenstall, K. (2001). School-based reading assessment: Looking for vital signs. Australian Journal of Learning Disabilities, 6, 26-35. Jonassen, D. (Ed.) (1988). Instructional design for microcomputer courseware. Lawrence Erlbaum: New Jersey. Marlin Abd. Karim. (2002). Pentaksiran berasaskan sekolah: Pentaksiran untuk pembelajaran. Prosiding Persidangan Kebangsaan Penilaian Kemajuan berasaskan Sekolah 2002. Anjuran bersama Unit Penyelidikan Asas, Pusat Pengajian Ilmu Pendidikan, Universiti Sains Malaysia dan Bahagian Perancangan dan Penyelidikan dasar Pendidikan, Kementerian Pendidikan Malaysia.

17

Marso, R.N., & Pigge, F.L. (1993). Teachers testing knowledge, skills, and practices. In Teacher training in measurement and assessment skills, ed. S.L. Wise, pp. 129185, Lincoln, Neb.: Buros Institute of Mental Measurements, University of Nebraskla-Lincoln. Mitchell, R. (1992). Testing for learning: How new approaches to evaluation can improve. American Schools. New York: Macmillan, Inc. Musa, M. (2003). Malaysian education system to be less exam oriented. New Straits Time, 8 May 2003. Rohizani Yaakub & Norlida Ahmad (2003). Teknik alternatif menilai dalam bilik darjah. 2nd International Conference on Measurement and Evaluation in Education (ICMEE 2003). Organized by School of Educational Studies & Centre for Instructional Technology & multimedia, Universiti Sains Malaysia. Stiggins, R. J. (1991). Relevant classroom assessment training for teachers. Educational Measurement: Issues and Practice, 10 (1), 7-12. Stiggins, R. J. (1995). Assessment literacy for the 21st century. Phi Delta Kappan, 77(3), 238-245. Stiggins, R. J., & Conklin, N. F. (1992). Classroom assessment practices. In teachers' hands: Investigating the practices of classroom assessment. State University of New York Press: Albany, NY. Wise, S. L., Lukin, L. E., & Roos, L. L. (1991). Teacher beliefs about training in testing and measurement. Journal of Teacher Education, 37-42.

18

You might also like