Professional Documents
Culture Documents
Banton
Xiaofei Lu examines one aspect of construct definition that is often included in constructs
underlying speaking and writing assessments. He examines the way in which syntactic complexity is
operationally defined by analyzing three corpus analysis tools that are widely used in research on writing
assessment. Lu begins by carefully unpacking the notion of syntactic complexity and then describes in
detail three prominent analysis tools: the Biber tagger, so-called after its developer, Douglas Biber; Coh-
Metrix, originally developed by Arthur Graesser, Danielle McNamara and colleagues to examine
cohesion in texts (Graesser, McNamara, Louwerse, & Cai, 2004; McNamara & Graesser, 2012); and Lu’s
own Syntactic Complexity Analyzer (Lu, 2010), designed specifically to evaluate syntactic complexity in
second language writing. For language testing researchers who have encountered these tools in their
reading and considered using them in research, this paper provides a welcome analysis of these tools
with implications for their role in operational definitions of syntactic complexity. Furthermore, Lu
provides a useful summary of the linguistic features that have been associated with higher quality scores
on writing in a variety of contexts. Like Römer, Lu argues that findings from corpus analysis might
profitably be used to inform rating scale development. Specifically, Lu points out that many current rating
scales, particularly holistic scales, do not sufficiently distinguish between syntactic variety, on the one
hand, and syntactic sophistication, on the other, both of which contribute to an overall assessment of
syntactic complexity.
References
Biber, D. (1991). Variation across speech and writing. Cambridge: Cambridge University Press.
Biber, D. (2006). University language: A corpus-based study of spoken and written registers (Vol. 23).
John Benjamins.
Biber, D., Conrad, S., Reppen, R., Byrd, P., Helt, M., Clark, V., Cortes, V., Csomay, E., & Urzua, A. (2004).
Representing language use in the university: Analysis of the TOEFL 2000 Spoken and Written Academic
Language Corpus. Test of English as a Foreign Language. ETS Research Report RM-04–03.
Biber, D., & Gray, B. (2013). Discourse characteristics of writing and speaking task types on the TOEFL
iBT® test: A lexico-grammatical analysis. ETS Research Report Series, 2013(1).
Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004). Coh-Metrix: Analysis of text on
cohesion and language. Behavior research methods, instruments, & computers, 36(2), 193–202.
McNamara, D. S., & Graesser, A. C. (2012). Coh-Metrix: An automated tool for theoretical and applied
natural language processing. In P. M. McCarthy & C. Boonthum (Eds.), Applied natural language
processing: Identification, investigation and resolution (pp. 188–205). Hershey, PA: IGI Global.