Most higher education faculty never took a course in how to write tests. Most tests given in college have weaknesses and are not highly reliable. The following presentation can significant help to improve the quality of the tests.
Most higher education faculty never took a course in how to write tests. Most tests given in college have weaknesses and are not highly reliable. The following presentation can significant help to improve the quality of the tests.
Most higher education faculty never took a course in how to write tests. Most tests given in college have weaknesses and are not highly reliable. The following presentation can significant help to improve the quality of the tests.
of Lucy Jacobs and Clinton Chase from their book Developing and Using Tests Effectively: A Guide for Faculty
The book is a compilation of years of
research on how to develop effective test instruments. Writing Tests Most higher education faculty never took a course in how to write tests. Their experience comes from the many tests they have taken over their many years in school. This personal experience is not a reliable way to learn to develop tests that will determine so many important outcomes for our students.
Even with diligence, time and practice writing
tests, most tests given in college have weaknesses and are not highly reliable.
However, the following presentation can
significant help to improve the quality of the tests Writing Fair and Effective Tests Purpose of Benefits to Tests Students Grades Learn how they Planning content are tested presentations Engage in deep or Facilitate learning surface learning Motivate students based on tests Feedback on learning Test Benefits to Instructors Focus on essential Feedback if course course content objectives were Message to achieved students on what’s Feedback on important effectiveness of Valuable feedback instruction on learning A motivational strategy Problems with College Tests Don’t focus on Too short to what’s important sample content Too little feedback fairly given to students Purpose of test not Questions made clear-so ambiguous or students see it as unclear penal Too few tests- don’t have a broad view of students knowledge How Long Should a Test Be? True-False 30 Matching 30 sec. sec. Multiple choice 1 Short Essay min. 10-15 minutes. Completion 1 Extended essay min. 30 minutes Short answers 2 min. Multiple choice 90sec. higher level thinking Number of Test Items Fifty Minute Tests 40-45 multiple choice 60-80 true-False Fastest students finish in half the time of the slowest Tests should allow almost every student to attempt every item Test Item Difficulty Tests should Difficulty can be: produce a spread A. content it asks of scores Test items should B. cognitive skill be of average level being difficulty measured Items that 50-70% of the students can answer C. Construction of correctly the If they didn’t study item Test Reliability Long enough to Directions should sample content be clear. well Scoring procedure Most student needs to be free of should be able to errors. finish All students should Score range take the same test should be wide Items free of ambiguity and tricks Test Validity Samples the What limits course objectives validity? in proportion to A. Directions not their importance clear Use a test matrix B. Tests skills not to check part of the distribution of course objectives questions by C. Too short content and cognitive level. D. Subjectivity of scoring Multiple Choice Tests Write a stem in the Do not pad the form on an stem-write them incomplete simple and clear. statement or Distracters must question that be plausible could be answered Avoid using humor without look at the four choices Make certain there is only one best Always put a verb answer in the stem Multiple Choice Tests Make distracters Avoid giving fairly grammar clues homogeneous-this careful with will force students specific modifiers-- to be ie. Never, always discriminating Limit use of all of Don’t give the above and irrelevant clues or none of the above tricks-these reduce validity Make answers Multiple Choice Tests Arrange options in Avoid asking trivia logical(alpha ) Ask students to order choose the best Keep correct answer not the response letter correct answer. ( number) balanced Don’t lift stems verbatim from text Arrange answers in vertical columns True-False Tests Advantages Limitations A. Sample many A. Difficult to write more bits of so there not information ambiguous or B. Ask more obvious question in 50 B. Difficult to make minute time clearly true or period helps clearly false reliability statements C. Research C. 50-50 chance affirms True-False D. Educated guess True-False Tests Avoid use of Write questions specific that are clear and determiners ( all, simple to read. never) Make it clear Avoid use of where the answers qualifiers(sometim are to be placed-- es, usually) don’t use + or - Avoid use of indefinite terms (long time ago) True-False Higher Level Thinking Use propositional Setting up a logic--if then problem that approach. requires analysis. Example-If the Example --John gross national bought a car…it product goes up won’t start. Which and inflation is are possible 3.4%…then the reasons? stock market will likely go up. Short Answer or Completion Items Requires recall Questions should rather than have a single word recognition of answer. information. Make certain there Doesn’t test is only one right higher level answer thinking Delete only key Best used to see words what information Make blanks same students have length collected. Don’t lift Matching Tests Stimulus column Put stimuli in on the left alphabetical order numbered-- Use 10-15 items response column Keep entire test on on right lettered. one page. Provide space to Have at least five right answer to the more stimuli than left of stimuli. responses Use only homogeneous material Essay Tests Advantages 4. Can’t simply 1. Can assess recognize an complex learning answer 2. Easy to write 5. Doesn’t allow 3. Emphasize for guessing-- communication except bluffing skills as 6. Can see how fundamental students select, organize and evaluate ideas and apply them Essay Tests Limitations 4. Score 1. Difficult to score influenced by 2. Scores are less readers overall reliable impression of the student 3. Sample only a limited amount of 5. Not favorable to content developing good writing skills Essay Reliability Concerns Less reliable than A second reading objective tests often results in a Environmental different grade. factors influence Expectations of scoring ie. Time of students’ day, mood of performance reader where Physical elements paper is in the of the paper-- stack handwriting Previous paper can Limited number of influence current questions luck of paper the draw Making Essay Tests Better Restrict essays to Keep question assess only focused on a outcomes that single objective require complex All students should higher level respond to the cognitive same questions- functions. don’t give choices Examples Grammar and 1. Compare spelling count only -Contrast if taught as an 2.Present objective in the Making Essay Tests Better Directions need to be very clear and include type of response expected--Example outline, list, discussion List the point value of each question Scoring Essay Tests Conceal students Use a prescribed names reading procedure Skim a few before A. Key procedure starting to grade lays out ideas that to get a sense of should have been responses in the essay Read only one B. Ranking item across all Procedure papers Makes piles based Reshuffle the stack on quality Open-Book Tests Advantages Limitations Used in classes Students tend not where formulas, to prepare well tables or graphs No evidence to are needed to find suggest students solutions do any better Student demonstrates they know how to use references Reflect real life Take Home Tests Advantages Limitations 1. More thorough 1. Question rarely exploration of sample adequately ideas can take course material place 2. Much more 2. No time grading time restrictions 3. Students often 3. Usually better unsure of how written thorough to be. Re-takes of Tests Advantages Limitations Retakes do result A large test bank in better learning is needed Lower test anxiety Grading time Students learn increases from their Students don’t mistakes often prepare well It eliminates the for the first test need for make-up tests Helping Students Prepare for the Tests Review the scope Review make-up of the test policy Use a practice test Provide a review Clarify time limits session Clarify what Make provisions material will be for last minute needed and questions allowed Allow for breaks Review grading during long exams procedure