You are on page 1of 36

The 11th 334 Symposium

Advancing Teaching and Learning: Evidence, Outcomes and E-learning


Lingnan University
4 March 2011

Application of an On-Line
Diagnostic English Language Test

Overview of presentation
Background
The DELTA system
Applications of DELTA

Background

DELTA Context
Opportunity provided by planning for 4-year curriculum
for English language centres to review their curricula and
provision of learning, teaching and assessment
Tertiary educations focus on outcomes-based learning,
teaching and assessment
Increased emphasis on learner autonomy and the
independent learner
Bachman report to UGC on language enhancement

The Bachman Report (1)


While there may be a number of specific, research and
development (R&D) projects involving interinstitutional collaboration among language center staff
at different universities, one that is of particular
importance, in my view, is the collaborative R&D
project involving HKIEd, PolyU, and LU, on the
development of a system for assessing English language
proficiency ... there is a need for much greater
collaboration in this area.
Language Enhancement in Hong Kong Universities: Some Observations
and Recommendations (Annex A, p.4)
Lyle F. Bachman, July 2010

The Bachman Report (2)


2. An overall plan for the development of language assessments could include
the following:
Different tests developed for complementary purposes, some could be CR,
objectives-based, and some could be NR, and independent of any
particular learning objectives.
Tests with some components to be delivered face-to-face and some online.
Integration with a language [learning] management system that could
include E-portfolios for tracking students LE activities and progress in
language proficiency.
A delivery platform that is flexible enough to be used for formative and
summative course-based assessments by all eight institutions. This
flexibility would make it usable for use in general foundational language
courses, language for academic purpose courses, and discipline-specific
language courses, as well as for certification. (Annex A, p.17)

Relevance and Significance of DELTA


The DELTA project is relevant to the partner institutions and
this symposium for several reasons:
It is a genuinely collaborative, inter-university project in which
language centres share knowledge, experience and resources.
It provides additional data about students language proficiency.
Partner universities will be able to use DELTA results for various
purposes.
It will feed into the teaching and learning process with forward
perspective to 2012.
It will create the opportunity to engage in collaborative research.

PolyU assessment outline for 4-year curriculum


Entry

Exit

What is DELTA?
An English language proficiency assessment system
designed specifically for the Hong Kong academic context
which can

diagnose students strengths and weaknesses


track and report on students language gains
help students to plan their language learning
inform curriculum development

Assessment Format

Four components - Listening, Reading, Grammar and


Vocabulary

Academic & general text-based

Multiple-choice items

Items calibrated using Item Response Theory


(Winsteps) analysis

Assessment targeted to each students proficiency level

DELTA Track

DELTA Track and Component Skills Profile

COMPONENT SKILLS PROFILE

Component Diagnostic Report

The DELTA system

The DELTA System


ENROLMENT /
REGISTRATION
student-specific test selection

tracking

TEST DELIVERY
automated scoring & test analysis
REPORTING
report generation & transmission
Learning Management System (Moodle) and/or e-Portfolio

DELTA system (admin)


Customizable set up
Test setup (venue, sessions, enrolment period,
etc.)
Item database (active / inactive items)
Test difficulty level for groups of students
Student report content

Management
Enrolment and attendance
Item database
Results (Export to Winsteps for analysis)

New student (1st attempt of DELTA)


Completes
a generic
test

Receives
email
reminder to
register

Receives an email that


results are available

Sets DELTA
target (with the
advice of a
language
teacher)

2nd year student (2nd attempt of DELTA)


Completes
a targeted
test
(targeted to
proficiency
level)

Receives an email that


results are available

(Re)sets
DELTA target
(with the advice
of a language
teacher)

Receives
email
reminder to
register

Seeks language support


to achieve target

Applications of DELTA
@ CITYU

Relevance to English Language


Teaching and Learning at CityU
Students

-Language support (individualised)


-Tracking information

Providers of
language courses

-Placement (students with no


HKDSE/equivalent)
-Formative information

Parent
departments &
senior
management

-Needs report
-Tracking information
-Overall
-Per college/school

DELTA & English support for CityU


students from 2012
Year

DSE Eng Level


4/+ at entry

DSE Eng Level


3 at entry

DELTA

DELTA

DELTA

English

English

DELTA

English

English

English

Predicted number of test takers


2011

approx 1300 (selected Year 1 students)

2012

approx 5200 (all Year 1 students) +


approx 1300 (from 2011)

2013

approx 2600 (all Year 1 students) +


approx 5200 (from 2012) +
approx 1300 (from 2011)

http://elws2.cityu.edu.hk/Indep
endentLearning/writing2.html

http://elws2.cityu.edu.hk/Indep
endentLearning/writing2.html

Independent
learning
guidance

Language
learning
plan

Meeting
with a
Language
Learning
Advisor

Applications of DELTA
@Lingnan

Application of DELTA at Lingnan


The DELTA system at Lingnan University is informing and
interacting with the teaching-learning process by:
Providing additional information to help in curriculum
planning and decision making
Promoting learner autonomy and guiding students in
independent learning
Providing valuable input to both students and faculty for
academic advising

Curriculum Planning and Decision Making


Planning and developing the 2012 curriculum
All new courses in 2012 will contain significant Independent
Learning (IL) component in which students will plan and
execute their individualised learning pathways with help
from DELTA student reports
CEAL will track language proficiency development of all
students by making the DELTA test mandatory once per year
CEAL currently anticipating proficiency targets and how to
work with parent departments

Promoting Learner Autonomy and Independent Learning


DELTA student reports enable students and CEAL staff to identify:
specific language subskills for IL work
resources that need to be developed

COMPONENT SKILLS PROFILE

Input to provide information and help in Academic


Advising
Lingnan currently using Degree Works as a tool for
advisors and students in academic advising
Students and staff able to discuss English language
proficiency, development and targets
CEAL hopes to work with academic advisors to enable
better understanding and more effective use of DELTA
student reports

Sample student report in DegreeWorks

The DELTA Team


Centre for Language in Education (HKIEd)
Michelle Raquel, Gwendoline Guan

Centre for English and Additional Languages (LU)


Christine Shirley, Jerry Gray

English Language Centre (PolyU)


Alan Urmston, Carrie Tsang, Felicia Fang, Cherry Ng

English Language Centre (CityU)


Jane Lockwood, Eva Lui

External Consultant
Professor Trevor Bond, James Cook University, Australia