May 2018

Volume 2, Issue 1

The CALT Group is a professional Editorial Notes

community  of language teachers, Welcome to the third edition of The CALT Report.  In
assessment coordinators, and other this issue, we feature an exploration of reading
educators. It is a community of assessment by Tara Chandler in which she discusses
practice for anyone interested in the many decisions a teacher makes when creating a
effective classroom assessment in Reading test.
ESL/EFL programs. Visit our website Paul Mart reflects on grammar assessment in
at http://groupspaces.com/CALT.  Teacher as Learner after his experience offering
students greater autonomy and creativity in a
classroom grammar test.

Our Member Spotlight focuses on Shelly Stoddard

and her assessment experiences at Virginia Tech's
Language and Culture Institute.

We also draw your attention to two upcoming

regional assessment conferences this fall. Portland
State University will host a conference on October 19
and 20, while Ohio University will host on October 26
and 27. Save the dates and plan to join us for one of
these excellent events!

As always, send your feedback on the newsletter or

Classroom assessment in action with students from Brazil, China,
Iran, Japan, Kuwait, and Saudi Arabia (April 2018, University of
Arizona).
any of the online workshops to
CALT@groupspaces.com. Email us if you would like
to make a contribution to The CALT Report, as well.

1
 Feature Article
Effective Reading Assessment for
 English Language Classes
BY TARA CHANDLER
Tara is a Senior Instructor at the Center for English as a Second
Language (CESL) at the University of Arizona.
INTRODUCTION
Assessing Reading is necessary and is done by
practitioners in many different contexts, yet it has its
share of challenges. In fact, assessing Reading is a difficult
task; since it is a receptive skill and a process more than a
product, one cannot truly observe what the student is
comprehending while reading (Hubley, 2012; Ekbatani,
(2011)). Also, while Reading has been a required activity in
education for decades, the field of assessing Reading is
quite varied and relatively new. A deeper understanding of
Reading contexts and relevant ways to assess them has
only been developing since the 1990s (Grabe and Jiang,
2013). When looking at the educational context as a
whole, one could say that assessing Reading in an L1 is
new, while assessing Reading in an L2 is fledgling. In
addition to these challenges, practitioners must consider
all that teaching the skill of Reading encompasses.
Consequently, both bottom-up skills at the word and
sentence level, as well as top-down processing of larger
texts, are taught to ESL students and should also be
assessed (Coombe, Folse & Hubley, 2007). In the
meantime, all of these ideas should be considered while
still applying the five basic principles of assessment:
reliability, validity, authenticity, practicality and
washback (Abeywickrama & Brown, 2010).
As a point of reference for the rest of the article, here is a
brief explanation of those five basic principles. When
evaluating the reliability of an assessment, one should
2
consider how consistent and dependable it is. For validity,
one must look at whether the assessment truly measures
what it intends to. Because Reading is skills-based and not
content-based, authenticity is also important and focuses
on replicating meaningful and real-world tasks. Also,
Reading can be very time-consuming, so practicality,
whether the assessment works within time constraints
and other administrative factors, is another point of
consideration. Lastly, washback is important  as it can
determine how the assessment positively influences the
teaching and learning in class.
EVALUATING A READING ASSESSMENT
When designing a Reading assessment for the classroom,
there are two areas to pay particular attention to in order
to address all of the aforementioned challenges and
considerations: the tasks and the scoring. The following
section is divided into these two areas.  The purpose of
the following section is to give Reading teachers advice on
how to create an assessment that is purposeful and allows
students to best demonstrate their literacy and
comprehension skills within a practical timeframe. While
this advice is based on current research on the topic, I also
infuse my experience as a Reading teacher of more than 16
years and as a person who has evaluated my colleagues’
Reading assessments in an administrative role at my
university.
TASKS
Commonly, a Reading assessment will contain, but is not
limited to, tasks such as multiple choice, true/false, short
answer, and open-ended or extended-response questions.
Firstly, one of the most important attributes of a good
Reading assessment is that it contains a variety of such
tasks (Alderson, 2000) and tests different Reading
subskills (Hubley, 2012). But there are also advantages
and disadvantages to consider when choosing tasks for
particular Reading comprehension skills. For example,
multiple-choice questions are well-suited for assessing
the difference between main ideas and details (Coombe,
Folse & Hubley, 2007). When considering the five basic
                                                                           continued on p. 3
"Effective Reading Assessment" continued...
principles of assessment, this task is very practical since
students are not producing a lot of language, which can be
very time-consuming, to demonstrate their
comprehension (Alderson, Clapham & Wall, 1995);
although the more choices a student has, the less of a
chance that the student can just guess the correct answer.
Because students have a high chance of guessing the
correct answer, the validity of this task can be quite low.
In my experience, some students also really struggle with
this type of task because they aren’t adept with multiple-
choice questions. Perhaps they come from a culture where
this type of task wasn’t common on tests or because of a
language barrier; they don’t understand the nuances
between answers that are almost correct (and therefore
still wrong) and ones that are best. On the other hand,
some students have been taught by test-taking coaches
how to use the process of elimination to get the correct
score instead of actually demonstrating comprehension of
the text (Alderson, 2000).
The next task to consider is true/false questions. Positively
for teachers, there is great ease of grading as it can be
done quickly and reliably. It should be noted, however,
that it can be difficult to determine whether a student has
written an F or T, so it might help to have those two
choices already written and students must only circle one
of the choices (Coombe, Folse & Hubley, 2007). Also, if the
statements are paraphrased from the text (and not written
verbatim), washback from the test can encourage
vocabulary growth. On the other hand, similar to multiple-
choice questions, the validity is questionable as students
have a 50/50 chance to guess the answer
Learning Center Manager Sheila Mullooly, Volunteer Conver-
sation Partner Program Coordinator Peter Brown dos Reis,
and Learning Center Lab Assistant Abdullah Alothmani at the
IELP Learning Center front desk (Portland State University).
3
correctly (Ekbatani, 2011). A disadvantage for the teacher
who is writing his/her own test is that it can be very
difficult to write questions with such absolutes. To say a
statement from the text is absolutely true or false can take
a lot of forethought.
Another popular task is for students to provide a short
answer. This can be a very authentic task for scanning
charts, tables and advertisements, for instance, and while
students are required to generate answers, they are still
brief and structured, which helps keep reliability high
(Coombe, Folse & Hubley, 2007). Conversely, when
teachers are grading, they might focus too much on other
language factors such as spelling or punctuation, both of
which are not the main point of assessing Reading
comprehension. This can lead to lower validity value to the
assessment. Also, this type of task doesn’t work well for
making inferences and other abstract tasks as it is only
good with specific, detailed information in a text
(Ekbatani, 2011).
  
The last task to evaluate is the extended or open-ended
question task. On a positive note, it contains a higher
degree of authenticity (Ekbatani, 2011) than any other
form of assessment since it most closely resembles the
purpose of Reading in most academic contexts. Also, the
validity of this task is high as students cannot guess the
answer like some of the other tasks previously discussed.
In addition, a higher degree of cognitive ability is required;
however, a possible adverse effect is less reliable scoring
(Bennett & Ward, 1993). Teachers, if there isn’t a clear
                                                                              continued on p. 4
Communication and Culture Program students with instructor
Amber Bliss Calderón at Portland State University's Intensive
English Language Program.
 "Effective Reading Assessment" continued...
focus when grading, can allow too much variability and
perhaps even tighten or loosen their expectations (i.e.,
intra-rater reliability) (Abeywickrama & Brown, 2010) from
the beginning to the end of one set of  student responses.
There are some solutions for maintaining reliability
though. First, it is important to determine an answer key
beforehand and to consider what answers are or are not
acceptable (Ekbatani, 2011). Secondly, similar to writing
assignments, a simple rubric can be used, so that intra-
rater reliability improves. If the rubric is used to grade a
common assessment amongst many practitioners, I also
recommend providing some training on how to use the
rubric and perhaps have teachers score a few examples for
norming purposes.
SCORING
After reviewing the tasks of a Reading assessment, it’s
crucial to evaluate the scoring to determine whether it’s
balanced and appropriate. First of all, the weighting  of
test questions should be clearly marked for the
information of both teachers and students. For students, it
is only fair for them to know how much each question is
worth, so they can use this information when making
decisions while taking the test. for teachers, it can help as
a reminder as to the weight of the assignment when
grading. The determination of the weight should be made
before the students take the test, and the teacher should
consider the difficulty level of the tasks. For example,
Instructor Lisa Graham and Activities Coordinator/Instructor
Ami Hwang lead Film Club: Reading Theater for the Intensive
English Language Program at the University at Albany, SUNY.
4
tasks that require more language production, such as an
extended response, or tasks that require a higher level of
cognitive ability, such as multiple-choice questions
requiring students to make inferences, should be weighed
more heavily than their less difficult counterparts.
Another consideration is to look at how balanced the
scoring is between bottom-up skills and top-down skills
within one assessment. For instance, I often assess
vocabulary words and Reading comprehension in the same
test. It takes more time for students to read a passage and
answer questions, so I usually have fewer Reading
comprehension questions; whereas, vocabulary questions
aren’t as time-consuming, so I have more of them.
Therefore, I want to double-check that the worth of each
of those skills is either even or perhaps more heavily
weighted toward Reading comprehension, since this is the
more important skill, especially in a more academic-
oriented language program.
CONCLUSION
As with designing any assessment, the practitioner has
many decisions to make when creating a Reading test.
While keeping in mind principles such as validity,
reliability, etc., the teacher needs to craft tasks that align
                                                                             continued on p. 5
"Effective Reading Assessment" Regional Assessment
continued...
well with the Reading skill and overall purpose of the Conferences
assessment and determine a scoring system that is fair
and balanced. Of course, a teacher is quite capable of
doing all of this, but I have found it very helpful to
collaborate with colleagues and to get a second or third
opinion. Similar to getting peer reviews before writing for
publication, getting a colleague to give suggestions for
improving an assessment ensures a much more polished
final product. I know I am thankful to have a vocation
where I can work with brilliant and supportive co-workers.

REFERENCES
Abeywickrama, P., & Brown, H. D. (2010). Language
   assessment: Principles and classroom practices. White
   Plains, NY: Pearson Education.
Alderson, J.C. (2000). Assessing reading. New York:
   Cambridge University Press.
Alderson, J.C., Clapham, C., & Wall, D. (1995). Language
   test construction and evaluation. Cambridge, UK:
   Cambridge University Press.
Bennett, R. & Ward, W. (1993). Constructed versus choice
   in cognitive measurement. Mahwah, NJ: Lawrence
   Erlbaum.
Coombe, Folse & Hubley (2007). A practical guide to
   assessing English language learners. Ann Arbor, MI:
   University of Michigan Press.
Ekbatani, G. (2011). Measurement and evaluation in post-
   secondary ESL. New York: Routledge.
Grabe, W. & Jiang, X. (2013, November 6). Assessing
   reading. Volume 1: Abilities, contexts, and learners.
   Retrieved from https://onlinelibrary.wiley.com/doi/
   10.1002/9781118411360.wbcla060.
Hubley, N. (2012). Assessing reading. In The Cambridge
   guide to secondary language assessment (pp. 211-217).
   Cambridge, UK: Cambridge University Press.

A former ESL student celebrates the receipt of her Master's

degree with her husband and faculty/staff at the Ohio Program
of Intensive English (left to right: Becky Challenger, Rena
Peters, and Kim Young).
5
Teacher as Learner BY PAUL MART
Considering the teacher also as a learner in the classroom (and
Western Washington
workplace), in this section we ask a member to describe a memorable
University, WA
assessment experience and the learning that occurred.

Recently, I taught a lower-level grammar class in our IEP three random pictures, and they needed to use

with 14 students from Saudi Arabia, Vietnam, Rwanda, their newly learned grammar skills to comment

and China. In that class, I also mentored a TESOL student- on or query about the pictures for 3–4 minutes.

teacher from West Washington University’s TESOL Student responses were all digitally recorded so

certificate program. I was lucky to have my student- I could review them as needed.

teacher, as I implemented both written and oral tasks on

each of three quizzes throughout the term; my student- The students had many questions during the

teacher could proctor the written exams while I held the first of these quizzes, but once they understood

oral exams individually in another room, or vice versa. my expectations, they tore into the tasks. Who

knew that grammar could be so fun? More than

In this series of exams, I tried to allow my students more once, a student quietly asked me during a

space to be creative when they showed me how they could written quiz which picture they should choose,

use their grammar skills. Also, I tried not to dwell in the only because they had too many ideas of what

lower levels of Bloom’s Taxonomy for the tasks on the to write.

quizzes (Remembering, Understanding). The written I learned that I could steer away from a straight

quizzes had three sections: 1) identifying various grammar ahead, top down, multiple choice, fill-in-the-

forms in sentences or questions, 2) writing short answers blank grammar quiz and offer students a little

(some sentences) and drawing a timeline about a ~300- more room to be creative. The paragraph

word story that I wrote, and 3) writing a 6–8 sentence answers were all very different and fun to read,

paragraph (while using discreet grammar points from and having the discreet grammar points in the

class) about one of three pictures given along with the directions, along with a rubric and the pictures,

quiz. Each student received a clear plastic page protector gave the students a structure upon which to

containing their paper quiz, the rubric for the paragraph, build.

and three random pictures cut from magazines. Every

picture was different, so the students needed to make sure If I could change anything from this experience,

they gave me their page protector back with their finished I might have asked for direct student input as I

quiz, three pictures, and their specific picture clipped to created the writing rubric. Currently, I have an

their quiz. upper intermediate listening/speaking course,

and those students have sent me back to the

The oral quizzes had a rubric, which students had seen in drawing board with at least one impromptu

the previous class and were administered individually in a speech rubric. Their feedback holds much more

separate room. Students once again had a choice from value for me now than before. 

6
 Member Spotlight

Shelly Stoddard
instruction. Many times assessments are created for the
Shelly is an
purposes of assigning a grade to a student. However, I think
Adjunct English
it is more important to see assessments as a way to gather
Instructor at the
student data for analysis. I believe using data to guide
Virginia Tech
instruction is the most important reason to use assessments at

Language and
all. However, at the higher level, when time is short, content

Culture Institute is dense, and differentiation is challenging, instructors use

in Blacksburg, VA assessments for grading rather than instructional design.

Q. Describe your current position and teaching situation. Q. What have you learned about assessment in the past

A. Currently, I am an English instructor at the Language year that you have tried to incorporate into your teaching

and Culture Institute’s (LCI) Intensive English Program. I practices?

teach all areas of English including Listening, Speaking, A. During the last year, my focus has been on instructional

Grammar, Reading, and Writing to a diverse population of strategies and authentic assessment of Listening. Teaching

learners. My students range in level from newcomer to Listening as more than “listen to the audio from the book and

academic user of the language and come from a variety of answer comprehension questions” has been an ongoing

countries around the world including China, South America, challenge for me. As a result, I have chosen to use some of

South Korea, and multiple countries in the Middle East. my professional development time to read articles, research

best practices, and attend a webinar to learn ways to instruct

Q. What assessment responsibilities do you have? and assess in this area.

A. My assessment responsibilities include aligning

summative exams to student learning outcomes, creating Most recently, I have learned that Listening can be assessed

exams for personal use in the classroom, making informal in as little as 5 minutes a day using short, discreet mini

formative assessments, and analyzing test data to create lessons. Creating a short cloze activity that focuses on a

remediation or extension lessons where applicable. grammar point or vocabulary set within the context of

listening has been very effective. It is an easy way to

Q. Does your program have an Assessment Committee? integrate multiple language areas and assess student ability.

A. The LCI does have an assessment committee. The    

committee is made up of the Testing Coordinator and 4 to 5 Q. What is an area of assessment that you want to learn

faculty members. Typically, the committee works on more about?

creating summative exams for the various levels and A. One area that I would like to learn more about is rubric

subjects taught at the LCI. Additionally, the committee creation. I use rubrics, but typically they are just raw score

ensures that these exams align with the student learning ranges with descriptions. I want to learn how to use them as

outcomes, that the tests have a variety of question types, and an instructional tool rather than a way to attach a grade.

that the questions are level appropriate.

Q. When you want to forget about assessment, teaching,

Q. In your context, with your students, what assessment- and work, what do you like to do to relax?  

related issues do you and your colleagues have? A. In my free time, I like to escape by curling up with a good

A. In my opinion, the most difficult issue with assessment is book and warm latte. 

creating authentic assessments that are both teacher- and

student-friendly and using data from assessments to guide  

7
CALT Best Practices
This section presents recommended assessment practices and principles that teachers should
incorporate in the courses they teach to maximize and measure student learning, as well as to
develop their own assessment literacy. (Compiled by Eddy White, PhD) 

THE CALT REPORT Contributors Contributors

MAY 2018 Abdullah Alothmani Ami Hwang
Editor Amber Bliss Calderón Paul Mart
Eddy White Tara Chandler Sheila Mullooly
Production & Design Phoebe Daurio Peter Brown dos Reis
Holly Wehmeyer Lisa Graham Shelly Stoddard