Pre-Licensure Examination For Teachers: An Application of Rasch Analysis

138
The International Journal of Educational and Psychological Assessment September 2012, Vol. 11(2)
Pre-Licensure Examination for Teachers: An Application of Rasch Analysis Amalia E. Roldan
Occidental Mindoro State College

Carlo Magno
De La Salle University, Manila

Abstract The purpose of the present study is to construct and validate a pre-licensure examination for the primary and secondary pre-service teachers that include the professional education courses based on the areas covered in the Professional Regulation Commission (PRC) in the Philippines using the Rasch analysis. The professional education courses included were: (1) The teaching profession, (2) social dimension of education, (3) principles of teaching, (4) educational technology, (5) curriculum development, (6) facilitating human learning, (7) child and adolescent development, and (8) assessment of student learning. The proposed test will be useful in the review sessions conducted among graduating and graduate students in preparation for their actual licensure examination. The test was administered to 100 graduating students of Teacher Education department in a province in the Philippines. Rasch measurement was utilized to substantiate the validity of the test. The test obtained moderate person reliability (.57) and high items reliability (.96). All the items fit the Rasch analysis except for one items in the test. The person separation index (1.14, RMSE=.26) can moderately discriminate the person ability while the item separation index (4.97, RMSE=.29) can highly discriminate. The item map showed that person abilities are clustered in the middle of the scale while the items were spread all over the scale. Two extreme items were found on top and at the bottom of the scale. Keywords: Rasch model, licensure examination, professional education courses Introduction As stated in the Philippines Commission on Higher Education (CHED) Memorandum Order (1999), the main mission of teacher education is the preparation of globally competitive teachers who are imbued with ideals, aspirations and values and are adequately equipped with pedagogical knowledge and skills. Quality education needs to be provided in teacher training institution so that preservice teachers will be prepared for their teaching profession. Teacher training institutions are expected to provide strong foundation towards developing preservice teachers knowledge and skills (Libman, 1990; Magno, 2010a). One way to assure that teachers are competent in the profession is gauged through the licensure examination results (Angrist & Guryan, 2008). In preparation for the licensure examination, the present study constructed and validated a pre-licensure
2012 Time Taylor Academic Journals ISSN 2094-0734
139
examination for the elementary and secondary preservice teachers that include the professional education courses based on the prescribed curriculum of the PRC using the Rasch analysis. The Licensure Examination and the Teaching Performance Teacher quality is a key element of student academic success, but little is known about how specific teacher attribute like licensure examination profile influences classroom outcomes. Reviews emphasizes the relevance of the licensure examinations results on teachers academic success (Gitomer, Lathan, & Ziomek, 1999) and teaching performance (Goldhaber & Brewer, 2001; Buddin & Zamarros, 2009). For example, Gitomer, Lathan, and Ziomek (1999) found that teacher academic ability varies widely by the type of licensure sought. Their ability is reflected in the way they teach. Students whose teachers possess a B.A. or M. A. in mathematics outperformed other students in mathematics. Students whose teachers have any kind of certification (standard, emergency, alternative, etc.) outperformed students whose teachers have no certification or are certified in different subjects. The study by Gitomer, Lathan, and Ziomek (1999) shows that having certification in teaching such as a licensure exam brings about success in the achievement of their students. Furthermore, in the primary level (elementary), Buddin and Zamarros (2009) examined whether teacher licensure test scores and other teacher attributes affect elementary student achievement. The results were based on longitudinal student-level data from Los Angeles. California requires three types of teacher licensure tests as part of the teacher certification process: A general knowledge test, a subject area test (single subject for secondary teachers and multiple subject for elementary teachers), and a reading pedagogy test for elementary school teachers. The student achievement analysis is based on a value-added approach that adjusts for both student and teacher fixed effects. The results show large differences in teacher quality across school districts, but measured teacher characteristics explain little of the difference. However, teacher licensure test scores are unrelated to teacher success in the classroom. Similarly, student achievement is unaffected by whether classroom teachers have advanced degrees. Teacher experience is positively related with student achievement, but the linkage is weak and largely reflects poor outcomes for teachers during their first year or two in the classroom. The kind of preparation done for the teacher training institutions can be relevant for their performance in the licensure examination. Houck and Kitche (2010) highlighted in their study about the modal teacher preparation curriculum. Content studies is one of the key aspects of the teacher preparation because stakeholders should support the notion that teachers must know the content they will teach, and they must be aware of how best to teach it (Cruickshank, 1996, p. 11). Their study determined if any amount of quality coursework preparation of preservice agriculture teachers influences their content knowledge. Findings indicated that most preservice teachers are meeting an adequate content knowledge level based on the Praxis II exam scores. They concluded in the study that
140
moderate relationship exists between Praxis II agriculture exam and agriculture content preparation. Unlike Houck and Kitches conclusion of moderate relationship that existed between Praxis II agriculture exam and agriculture content preparation. Arenillo and Arenillo (2009) found in their study that there was a significant relationship between the board performances of graduates with their knowledge obtained in their preservice education. Based on this premise, it can be noted that students with inadequate knowledge would have a less chance of passing the board examination. This limits their opportunity for advancing in their teaching career. The Licensure Examination for Teachers in the Philippines In the Philippines, Republic Act No. 3687, known as Professionalization Act for teachers is implemented to strengthen, regulate and supervise the practice of teaching profession by prescribing a license to teachers certified by the Professional Regulation Commission (PRC). The licensure examination is one of the hurdles that a teacher education graduate take to obtain a license for their career. The PRC has the responsibility to ensure that the examination meets technical, professional, and legal standards, and protect the health, safety and welfare of the public by assessing candidates abilities to practice competently. Once a candidate has passed the Licensing Examination for Teachers (LET), the PRC grants the license, thus assuring the public that the licensee is minimally qualified to practice teaching at the time of initial licensure. The purpose of the licensure examination is to identify persons who possess the minimum knowledge and experience necessary to perform task on the job safely and competently. These licensing examinations are very different from academic or employment examinations. Academic examinations assess how well a person can define and comprehend the terms and concepts. Employment examination can rank order candidates who possess the qualifications for the job. The licensure examinations also assess how the academic programs harmonized their process of developing, maintaining and administering a wholesome academic instruction to ensure students readiness to pass the standard of licensure examinations. The board examination outputs of Higher Education Institutions (HEIs) both private and public are used to evaluate the academic competence of educational programs offered. The examination results serve as an index of performance of the HEIs in general. Institutions which always produce top performers in various programs tend to be identified as the most credible and having the most reliable in offering quality instruction. On the other hand, institutions which continuously have board performance outputs lower than the national passing percentage are compelled to shape up by revisiting and improving their academic inputs. In addition, when the CHED implemented more stringent mechanisms in ensuring quality in HEIs, licensure performance served as key parameters in measuring the capability of an institution. For instance, in the evaluation of Agency Performance Report Rating (APR), board performance outputs is one of the salient
141
indicators in the different key result areas under instruction. Moreover, the implementation of HEIs leveling, normative financing and even in accreditation of courses, the licensure performance outputs is considered one of the important parameters in determining the efficiency and effectiveness of the institution. The present study is undertaken to contribute on the priority concern of the HEIs. The findings of the study can be used in the review sessions conducted among the graduating and graduates of the colleges and universities for their preservice teachers taking the LET. Currently, there are several published examinations that are meant to prepare preservice teachers for the LET but none of them reported the validity, reliability, and functioning of the items based on sound measurement theories. The instrument that was constructed in the study can be used to assess the possible performance of the preservice teachers in taking the actual LET particularly in the professional education courses. Given the instrument, the students can be familiarized with the types of items given in the actual licensure exam since the content of the test is based on the specification provided by the PRC. The latest teacher education curriculum was the source or framework of the items included in the study. The New Teacher Education Curriculum In response to the problem of unqualified and ill-prepared beginning teachers and the rate of passing in the licensure examination for teachers (LET) which ranges from 20 to 30% per year (Pedro, 1996), the Commission on Higher Education (CHED) initiated guidelines for the undergraduate teacher education programs in a Memorandum Order (CMO) 30 Series 2004 otherwise known as the New Teacher Education Curriculum. The aim of the pre-service teacher curriculum is to prepare professional teachers for practice in primary and secondary schools in the Philippines. The memorandum enumerated the 11 competencies expected of graduates of the Bachelor of Elementary Education or BEEd and Bachelor of Secondary Education or BSE. Elementary school teachers are trained to be generalists, that they may be able to teach all the five prescribed learning areas (or subjects) of the basic education curriculum; while secondary school teachers are trained to be specialists in one of the five prescribed learning areas (Fajardo, 2007). The Competency standards are as follows: Graduates of the BEEd (and BSEd) program are teachers who 1. Have the basic and higher level literacy, communication, numeracy, critical thinking, learning skills needed for higher learning; 2. Have a deep and principled understanding of the learning processes and the role of the teacher in facilitating these processes in their students; 3. Have a deep and principled understanding of how educational processes relate to larger historical, social, cultural, and political processes; 4. Have a meaningful and comprehensive knowledge of the subject matter they will teach;
142
5.
Can apply a wide range of teaching process skills (including curriculum development, lesson planning, materials development, educational assessment, and teaching approaches); 6. Have direct experience in the field/classroom (e. g., classroom observations, teaching assistance, practice teaching); 7. Can demonstrate and practice the professional and ethical requirements of the teaching professions; 8. Can facilitate learning of diverse types of learners, in diverse types of learning environments, using a wide range of teaching knowledge and skills; 9. Can reflect on the relationships among the teaching process skills, the learning processing in the students, the nature of the content/subject matter, and the broader social forces encumbering the school and educational processes in order to constantly improve their teaching knowledge, skills, and practices; 10. Can be creative and innovative in thinking of alternative teaching approaches, take informed risks in trying out these innovative approaches and evaluate the effectiveness of such approaches in improving student learning; and 11. Are willing and capable to continue learning in order to better fulfill their mission as teachers. The courses are classified into three components, namely, general education courses, professional education courses and specialization/content courses. General education courses provide basic knowledge in broad fields which provide the foundation on which professional courses are anchored. This is to compensate for knowledge, skills, and attitudes that are not developed and taught during the short pre-collegiate or pre university level (only 10 years). Professional education courses aim to develop the range of knowledge and skills needed in the practice of the teaching profession. There are three categories of courses in this component: (1) theory and concept courses, (2) methods and strategies courses, and (3) field study courses. The last component course is the specialization and content courses which provide basic but essential knowledge in the five prescribed learning areas in elementary school education. The summary of CHED curriculum for Preservice Teacher Education (BSE) is as follows: Part I is the General Education with a total of 63 units; Part II is the Professional Education with 57 units and consists of four areas; (1) Theory/Concepts (Child and Adolescent Development (3units), Facilitating Learning (3 units), Social Dimensions of Education (3 units) and, the Teaching Professions (3 units); (2) Methods/Strategies (Principles of Teaching 1 and 2 (6 units), Assessment of Leaning 1 and 2 (6 units), Educational Technology 1 and 2 (6 units), Curriculum Development (3 units), Developmental Reading 1 and 2 (6 units); (3) Field Study (Field Study 1-6, Practice Teaching (6 units); and (4) Special Topics (3 units); and Part III is the Specialization/Major (63 units) with the total of 183 units.
143
The Rasch Model A variety of national tests are undertaken in the Philippines but none of these test report findings based on the functioning of items. Magno and Gonzales (2011) in their report to UNESCO on the development in Philippine education described that when schools started to privatize testing, more so that information about tests psychometric performance became scarce. One of the movements in the development of national tests in the Philippines is the analysis of test data using Item Response Theory (Magno, 2010b). National examinations such as the licensure examination and the review forms becomes more credible when their actual item reliabilities and test functioning are made known to users. In the present study, the one parameter Rasch model is used to determine the functioning of test for preservice teachers. The Rasch model is am analysis that is referred to as one-parameter logistic model in the item response theory literature. The model estimates the probability of a correct response to a given item function of item difficulty and person ability (Hambleton, Swaminathan, & Rogers, 1991). The primary output of Rasch analysis is a set of item difficulty and person ability values placed along a single interval scale. Items with higher difficulty scores are less likely to be answered correctly, and items with lower scores are more likely to provide correct responses, and those with lower ability are less likely to do so (Magno & Ouano, 2009). Rasch analysis (a) estimates the difficulty of dichotomous items as the natural logarithm of the odds of answering each item correctly (a) log odds, or logit score), (b) typically scales these estimates to mean = 0, then (c) estimates person ability scores on the same scale. In the analysis of dichotomous items, item difficulty and person ability are defined such that when they are equal, there is 50% chance of correct response. As the person ability exceeds item difficulty, the chance of a correct response increases as a logistic ogive function, and as item difficulty exceeds person ability, the chance of success decreases. The formal relationship among response probability, person ability, and item difficulty is given in the mathematical equation by Bond and Fox (2001). A graphic plot of this relationship known as item characteristic curve (ICC), is given for three items of different difficulty levels. One useful feature of the Rasch model is referred to as parameter separation or specific objectivity (Bond & Fox, 2001). The implication of this mathematical property is that, at least in theory, item difficulty values do not depend on the sample used to estimate them, nor do person ability scores depend on the particular items used to estimate them. In practical terms, this means that given well-calibrated sets of items that fit the Rasch model, robust and directly comparable ability estimates may be obtained, from different subsets of items. This, in turn, facilitates both adaptive testing and the equating of scores obtained from different instruments (Magno, 2009). The Present Study The purpose of the present study is to construct and validate a prelicensure examination for the elementary and secondary preservice teachers that
144
include the professional education courses based on the prescribed curriculum of the PRC using the Rasch analysis. It is the intention of the researchers to focus on each particular area covered in the LET of PRC. The researchers decided to construct a test that assesses the students possible performance in the professional education courses. Based on the PRC results, majority of the students got lowest mean scores in the professional education subtest. The following are the subject areas included in the test: (1) The teaching profession, (2) social dimension of education, (3) principles of teaching, (4) educational technology, (5) curriculum development, (6) facilitating human learning, (7) child and adolescent development, and (8) assessment of student learning. The proposed test can be a useful instrument in the review sessions conducted among graduating and graduate students in preparation for their actual Licensure Examination. Method Participants The participants were 100 preservice teachers of one state college in the Philippines. These students are graduating students for the school year 2010 2011 and are expected to take their licensure examination after their graduation. Instrument The instrument which is a pre-licensure examination for preservice teachers is composed of 90 multiple items that was constructed by the researchers. The test includes items that assess the preservice teachers knowledge and skills Table 1 shows the table of specification of the test. Table 1
Table of Specification
Content Areas 1. 2. Teaching Profession Social Dimension of Education 3. Principles of Teaching 4. Educational Technology 5. Curriculum Development 6. Facilitating Learning 7. Child and Adolescent Development 8. Assessment of Student Learning Weights Total Weights 5% 17% 17% 11% 11% 11% 11% 17% 100% 20% 18 Cognitive Domain Knowledge Application Analysis 1-2 3-5 6-7 8-20 21-35 36 -45 52-55 60-65 No. of Items 5 15 15 10 10 10 10 25 21% 19 90
46-50 56-59 66-70
51 71-75
76-90 59% 53
The table of specifications includes 18 items (20%) that measures knowledge, 53 items (59%) application and 19 items (21%) analysis. These three
145
cognitive domains were measured following the objectives and subject areas included in the specifications of the PRC on LET. The items were content validated by a panel of faculty teaching education courses who served as reviewers. The reviewers came from different reputable colleges and universities in the Philippines with good performance in the LET. Procedure The first draft of the test was reviewed by some experts to establish the content validity. The comments and suggestions were integrated in writing the final draft of the test. The final draft of the test was administered to the 100 graduating students of a teacher education department in a state college during their in-house review for the LET. The researchers administered the test among the group and clearly instructed the students on how to answer the test. After the administration of test, the teachers and students were debriefed about the purpose of the study. Data Analysis To describe the distribution of the scores, the mean and standard deviation were reported. The internal consistency of the items was determined using the Cronbachs alpha. Item and person reliability estimates were calculated separately using the Rasch model using the Winsteps software. Rasch model was utilized to determine the item difficulty of the test based on the results of logistic measure indices. The items with positive logits indicate that items are difficult and have the probability to be answered by persons with high ability. On the other hand, the items with negative logits are easy items and have the probability to be answered by persons with low ability. Item in fit and outfit were determined to assess how many of the items fit the analysis as good items and does not fit as items to be revised or deleted. To determine the items that fit in the Rasch analysis, the outfit and infit mean square (MNSQ) indices should be within the acceptable range of 0.70 1.30 (Bond & Fox, 2007). Misfitting items are signs of multidimensionality and model deviance. High values of item MNSQ indicate a lack of construct homogeneity with other items in a scale, whereas low values indicate redundancy with other items (Linacre & Wright 1994). An item map was generated to determine how the items are spread in the entire range of the test. It is also used to show how the test can discriminate persons ability as to high and low abilities. Results Descriptive statistics were reported in Table 1. The mean scores of the eight areas of the review test range from .40 to .84 (closer to a value of 1.0 indicates large proportion of correct answers). Majority of the students obtained correct answers on the items in the area of teaching profession and lowest on educational technology. The rest of the test components got means below the median range except for the principle of teaching and child adolescent development. The whole test obtained mean score of .51 and a standard deviation of .07. The mean score is
146
above the median range (0.65) which indicates that students got more correct responses than the incorrect scores. The internal consistency of the test and its components were established using both the Cronbachs alpha and Rasch analysis. The Cronbachs alpha coefficients of the test and its components range from .00 to .59 which indicates low to moderate reliability. Results of the Rasch analysis show internal consistency of the items and low for person reliabilities. Table 2
Descriptive Statistics of the Eight Areas of the Pre-licensure Examination

M SD Cronbachs alpha Person Reliability Item Reliability
1. The Teaching Profession 2. Social Dimension of Education 3. Principle of Teaching 4. Educational Technology 5. Curriculum Development 6. Facilitating Learning 7. Child and Adolescent Development 8. Assessment of Student Learning The Whole Test
.84 .45 .60 .40 .45 .44 .60 .48 .65
.167 .135 .156 .147 .149 .38 .167 .146 .070
.00 .28 .43 .21 .09 .00 .39 .22 .59
.00 .24 .41 .18 .00 .00 .30 .22 .57
.84 .95 .94 .96 .96 .94 .95 .92 .96
Using the Rasch Model, the item and person separation and reliability were examined. The real person separation is 1.14 with RMSE of .26. This indicates that the test can moderately discriminates among persons. The person separation indicates the number of groups of students that can be separated according to their abilities (high and low ability groups). For the items, the reliability is high (.96). The item separation of 4.97 indicates that the items can discriminate between the high ability and low ability. There is a wide spread of difficulty in the items as the standard deviation of item difficulty estimates 1.48 logits and the separation of 4.97. The item difficulties of the test were determined based on the measure or logistic measures for item difficulty. The item difficulties are reported based on the logits obtained per item (see Table 5). The negative logits indicate that the items are easy and have the probability to be answered by persons with low ability. On the other hand, positive logits indicate that the items are difficult and have the probability to be answered by persons with high ability.
147
Table 5
Difficulty Index, Infit and Outfit of the Test

Test Component Item no. TP 1 2 3 4 5 SDT 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 PT 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 ET 36 37 38
Logistic Measure
1.39 0.88 0.88 -2.45 -0.71 -2.23 -1.93 0.35 -0.04 2.32 1.28 -0.65 4.23 1.09 -1.44 -2.46 -0.22 -0.4 -0.59 0.69 -1.43 3.04 -0.07 0.68 -1.22 0.74 -0.34 -0.34 1.63 -0.34 0.12 -0.94 1.48 -1.22 -1.79 2.42 -1.56 -0.64
SE
0.31 0.33 0.33 1.02 0.48 0.33 0.3 0.26 0.25 0.44 0.31 0.25 1.01 0.3 0.27 0.35 0.25 0.25 0.25 0.27 0.33 0.41 0.25 0.25 0.31 0.25 0.26 0.26 0.28 0.26 0.25 0.29 0.27 0.31 0.37 0.49 0.27 0.25
INFIT MNSQ z
0.94 0.95 1.13 0.98 1 0.96 0.97 0.93 1.01 0.92 1.09 0.92 1.03 1 1.03 1.01 1.15 1.04 0.92 0.96 0.91 1.14 1.03 1.03 0.97 0.83 1.11 0.96 1 0.91 1.1 0.76 1.23 1.11 0.92 0.82 1.08 0.96 -0.64 -0.37 1.09 0.29 0.11 -0.15 -0.13 -0.72 0.1 -0.14 0.53 -1 0.35 0.08 0.31 0.13 1.84 0.49 -1.05 -0.27 -0.37 0.53 0.33 0.37 -0.09 -2.07 1.04 -0.35 0.08 -0.83 1.12 -1.61 1.77 0.64 -0.25 -0.36 0.74 -0.48
OUTFIT MNSQ Z
0.94 0.95 1.14 0.55 0.99 0.84 0.89 0.86 1.05 0.58 1.65 0.88 1.86 0.95 1 1.05 1.21 1.05 0.89 0.97 0.83 1.11 1 0.99 0.91 0.79 1.24 0.93 1.03 0.81 1.12 0.59 1.36 1.17 0.77 0.62 1 0.92 -0.64 -0.35 1.06 -0.11 0.13 -0.45 -0.36 -1.01 0.51 -0.86 2.14 -1.2 0.97 -0.15 0.06 0.26 2.04 0.55 -1.04 -0.09 -0.43 0.37 0.05 -0.02 -0.22 -1.84 1.43 -0.41 0.2 -1.22 0.97 -1.88 1.78 0.64 -0.47 -0.47 0.08 -0.51
148
Cont. Table 5 39 40 41 42 43 44 45 CD 46 47 48 49 50 51 52 53 54 55 FL 56 57 58 59 60 61 62 63 64 65 CAD 66 67 68 69 70 71 72 73 74 75 ASL 76 77 78 79 0.79 1.54 2.42 -1.35 -2.02 -0.05 -1.56 -0.61 -0.93 -1.43 -1.36 -2.68 2.58 1.91 0.83 0.02 1.66 -1.66 -2.18 -0.17 -0.11 0.3 -0.06 0.43 1.56 2.01 -0.11 1.91 -0.18 -2.13 1.91 -0.56 -1.8 -0.72 -0.18 0.23 1.51 -1.2 -0.85 1.22 1.06 0.3 0.37 0.49 0.26 0.29 0.26 0.27 0.25 0.26 0.28 0.28 0.39 0.47 0.37 0.28 0.25 0.34 0.3 0.35 0.24 0.24 0.25 0.24 0.25 0.31 0.36 0.24 0.29 0.27 0.43 0.29 0.28 0.38 0.29 0.27 0.26 0.27 0.27 0.26 0.28 0.27 1.06 1.07 0.94 0.91 0.96 1.13 0.91 1.05 0.81 1 0.97 1.17 1.12 0.97 1.01 0.94 0.99 0.95 0.96 0.98 0.95 1.09 1.05 0.9 0.99 1.15 0.96 1.08 0.9 0.84 1.04 0.87 1.16 0.92 1.11 1.05 0.94 0.97 1 1.07 1.04 0.43 0.37 -0.03 -0.89 -0.26 1.24 -0.79 0.55 -2.01 0.02 -0.19 0.67 0.44 -0.04 0.11 -0.72 0.02 -0.22 -0.09 -0.24 -0.62 1.11 0.69 -1.22 -0.01 0.65 -0.57 0.58 -0.83 -0.45 0.35 -0.98 0.66 -0.53 0.95 0.59 -0.45 -0.14 0.08 0.55 0.37 1.14 1.52 0.65 0.88 0.96 1.42 0.86 1.03 0.73 1.34 0.93 1.4 1.73 1.13 1.02 0.88 0.85 0.88 0.86 0.99 0.97 1.15 1.04 0.84 0.9 1.87 0.94 1.61 0.86 0.7 1.27 0.85 0.93 0.87 1.09 1.13 0.85 1.02 1.05 1.09 0.99 0.6 1.26 -0.41 -0.6 -0.05 2.41 -0.61 0.31 -2.09 1.66 -0.34 0.99 1.22 0.45 0.19 -1.03 -0.34 -0.47 -0.36 -0.08 -0.4 1.43 0.51 -1.43 -0.3 2.12 -0.73 1.9 -0.8 -0.45 0.97 -0.67 -0.01 -0.49 0.54 0.89 -0.63 0.18 0.37 0.49 0.01
149
Cont. Table 5 80 81 82 83 84 85 86 87 88 89 90 -0.01 0.34 0.59 -0.18 -1.86 -0.07 1.14 0.17 1.56 -0.92 0.24 0.25 0.25 0.24 0.31 0.24 0.28 0.24 0.31 0.26 0.99 0.88 1.05 1.06 0.76 0.96 1.04 0.97 1.11 1.06 -0.05 -1.5 0.52 0.8 -1.13 -0.52 0.36 -0.37 0.62 0.53 0.99 0.84 1.03 1.08 0.66 0.96 1.19 1 1.05 1.04 -0.08 -1.6 0.25 0.91 -1.42 -0.39 1.01 -0.01 0.26 0.32
-0.99 0.26 1.04 0.34 1.04 0.3 Note. TF= Teaching profession; SDT= Social dimension of Teaching; PT- Principles of teaching; ET= Educational Technology; FL= Facilitating Learning; CD= Curriculum Development; CDA=Child Adolescent Development;
To determine if the items under each domain has a unidimensional structure, the item fit mean square (MNSQ) was obtained. The results of the fit statistics show that seven items out of 90 items do not fit the Rasch Model following the criteria suggested by Bond and Fox (2007). Items which do not fit the model have outfit and infit mean square (MNSQ) indices outside the acceptable range of 0.70 1.30. Misfitting items are signs of multidimensionality and model deviance. High values of item MNSQ indicate a lack of construct homogeneity with other items in a scale, whereas low values indicate redundancy with other items (Linacre & Wright 1994). Generally, few items did not fit the Rasch model. Only one items do not fit the model in the social dimension in teaching with outfit MNSQ value of 1.65, item 33 of principle of teaching (Outfit MNSQ = 1.36), items 40 and 44 of educational technology with outfit MNSQ of 1.40 and 1.73 respectively. There were two items in curriculum development, the items number 50 and 51 with outfit MNSQ of 1.40 and 1.73 and item number 64 of facilitating learning with outfit MNSQ of 1.87. All misfitting items have high values of outfit MNSQ that indicate of the lack of homogeneity. The itemperson map indicates the spread of the items over the entire range of the test. The numbers on the right indicate items and numbers on left indicate persons. Items and persons placed on top of the scale are more difficult and more competent, respectively. As one goes down the scale, items become easier and individuals become less competent. As shown in the map, majority of the individuals are clustered towards the center of the scale while items are spread all over the scale with two extreme scores, item 13 on the top (very easy item) and item 4 below (difficult item). The person position on the scale indicates homogeneity of their ability. Thus, the test can moderately discriminates the person ability as to high or low abilities. On the other hand, items are normally distributed. The item and person means are matched in the item map.
150
Figure 1. Item-Person Map
151
Discussion It was found in the study that the pre-licensure examination constructed is a test that supports assumptions of the Rasch model. The item reliability of the test is high and can moderately discriminate person ability as to high or low abilities. All the items satisfied the criteria of fit to the Rasch model except for the 7 items. Most of the items have a good fit showing that students with high ability are able to get correct answers to difficulty items, and those with lower ability are most likely not to answer the difficult items. One example of misfitting items is Item number 11 of social dimension in teaching. The item is stated as Teacher Anna teaches to his pupils that pleasure is not the highest good. Teachers teaching is against what philosophy? The options given are a) Existentialism, b) Realism, c) Epicureanism and d) Empiricism. This item does not fit the model due to the large value of outfit MNSQ which is 1.65. The value is outside the acceptable range of 0.7 to 1.30 in the criteria of fitting items suggested by Bond and Fox (2007). This item and the other six misfiting items lack the homogeneity. The items have low level of internal consistency with the rest of the items in the test. It was revealed that most of the items fit the Rasch analysis which indicates unidimensionality of the test or that the test really measures what it intends to measure (the same construct). This was supported by the item map showing the normal distribution of the items as they were spread all over the scale with only two extreme items found on top and at the bottom of the scale. The findings of the research pointed out that the Rasch Analysis is a powerful tool in determining the validity of a teacher-made test. The relevance of the test instrument constructed by this researcher is for the purpose of satisfactorily complying with the CHED Memorandum on licensure examinations for teachers and providing the OMSC with a valid pre-licensure test which could be standardized at the national level. It is highly recommended to administer the same instrument to education students from different schools to create a bigger sample of examinees. Future researches on the said instrument are likewise recommended particularly in establishing the predictive validity of the same. The construction and validation using Rasch analysis on the other two components of the LET, the general and specialization courses are also recommended. Tests with the purpose of practicing students for the licensure exam should have appropriate indicators that they are valid and reliable measures. Educational institutions and review centers when coming up with test items for the LET review should consider showing evidence that the items are performing appropriately.
152
References Angrist, J., & Guryan, J. (2008). Does teacher testing raise teacher quality? Evidence from state certification requirements. Economics of Education Review, 27, 483-503. Arenillo, S. A., & Arenillo, M. T. (2009). Preservice education and performances in teacher licensure examination among graduates of Mindoro State College of Agriculture and Technology. JPAIR Multidisciplinary Journal, 2, 132-143. Fajardo, A. (2007). Curriculum design of preservice teacher education for indigenization of elementary school science college of education. Proceedings of the Redesigning Pedagogy: Culture, Knowledge and Understanding Conference, Singapore, May 2007. Magno, C. (2009). Demonstrating the difference between classical test theory and item response theory using derived test data. The International Journal of Educational and Psychological Assessment, 1, 1-11. Magno, C. (2010a). Looking at Filipino pre-service teachers' value for education through epistemological beliefs about learning and Asian values. The Asia-Pacific Education Researcher, 19(1), 61-78. Magno, C. (2010b). A brief history of educational assessment in the Philippines. Educational Measurement and Evaluation Review, 1, 140-149. Magno, C., & Ouano, J. (2009). Designing written assessments for student learning. Manila: Phoenix Publication. Magno, C., & Gonzales, R. (2011). Measurement and evaluation in the Philippine higher education: Trends and development. In E. A. Valenzuela (Ed.), UNESCO Policy Series: Trends and development in Philippine Education (pp. 47-58). Philippines: UNESCO National Commissions. Bond, T. G., & Fox, C. M. (2001). Applying the Rasch model: Fundamental measurement in the human sciences. Mahwah, NJ: Eribaum. Buddin, R., & Zamarro, G., (2009a. Teacher Qualifications and Student Achievement in Urban Elementary Schools. Journal of Urban Economics, 66, 103-115. CHED (2004). CHED Memorandum Order No. 30 S. 2004 New Teacher Education Curriculum CHED (1999). CHED Memorandum Order No. 11, s. of 1999. Revised Policies and Standards for Teacher Education Gitomer, D. H., Latham, A. S., & Ziomek, R. (1999). The Academic Quality of Prospective Teachers: The Impact of Admissions and Licensure Testing. Princeton, NJ: Educational Testing Service. Goldhaber, D., & Brewer, D. (2001). Evaluating the evidence on teacher certification: A rejoinder. Educational Evaluation and Policy Analysis, 23(1), 79-86. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA:Sage.
153
Libman, Z. (2009). Teacher licensure examination: True progress or an illusion? Studies in Educational Evaluation, 35, 7-15. Linacre, J. M., & Wrigh, B. D. (1994). Reasonable mean square fit values. Rasch Measurement Transactions, 8, 370. Pedro, L. (1996). A quantitative analysis of teacher training institution preservice programs for Mathematics teachers. Unpublished Dissertation: Universty of the Philippines. RA No. 3687. Professionalization Act for Teachers Rasch, G. (1980). Probabilistic models for some intelligence and attainment tests. Chicago: University of Chicago Press. About the Authors Ms. Amalia Roldan is currently a faculty of Occidental Mindoro State College. She is currently finishing her PhD in Educational Psychology major in Quantitative Research in De La Salle University, Manila. Dr. Carlo Magno is a faculty of the College of Education in De La Salle University, Manila. Most of his publication is in line with measurement and assessment studies. He has developed several measures useful in the educational context. Further correspondence can be addressed to him at carlo.magno@dlsu.ph

Pre-Licensure Examination For Teachers: An Application of Rasch Analysis

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Pre-Licensure Examination For Teachers: An Application of Rasch Analysis

Uploaded by

Copyright:

Available Formats

138

Pre-Licensure Examination for Teachers: An Application of Rasch Analysis Amalia E. Roldan

Occidental Mindoro State College

De La Salle University, Manila

2012 Time Taylor Academic Journals ISSN 2094-0734

2012 Time Taylor Academic Journals ISSN 2094-0734

2012 Time Taylor Academic Journals ISSN 2094-0734

46-50 56-59 66-70

Descriptive Statistics of the Eight Areas of the Pre-licensure Examination

.84 .45 .60 .40 .45 .44 .60 .48 .65

.167 .135 .156 .147 .149 .38 .167 .146 .070

.00 .28 .43 .21 .09 .00 .39 .22 .59

.00 .24 .41 .18 .00 .00 .30 .22 .57

.84 .95 .94 .96 .96 .94 .95 .92 .96

2012 Time Taylor Academic Journals ISSN 2094-0734

Difficulty Index, Infit and Outfit of the Test

2012 Time Taylor Academic Journals ISSN 2094-0734

2012 Time Taylor Academic Journals ISSN 2094-0734

2012 Time Taylor Academic Journals ISSN 2094-0734

Figure 1. Item-Person Map

2012 Time Taylor Academic Journals ISSN 2094-0734

2012 Time Taylor Academic Journals ISSN 2094-0734

2012 Time Taylor Academic Journals ISSN 2094-0734

You might also like