Classroom Based Assessment Final Test Project

Developing an Assessment Tool for Year XII of Vocational School

Oktari Aneliya 2215081412 Reg 08 Dik A

English Department State University of Jakarta 2011

1. Introduction
Vocational school is aimed at giving students knowledge and skills to be competent in job field. As stated in Permendiknas no.22 tahun 2008. Pendidikan kejuruan bertujuan untuk meningkatkan kecerdasan, pengetahuan, kepribadian, akhlak mulia, serta keterampilan peserta didik untuk hidup mandiri dan mengikuti pendidikan lebih lanjut sesuai dengan program kejuruannya. Considering to the objective of vocational school, therefore the curriculum and the course program are designed to be agree with the objective of vocational school. All course programs are aimed at promoting students competency in any kind of job field. English in vocational school is considered as English for Specific Purpose in which the focus is on learners need that is being competent in their job field. English then is seen as a communication tool in workplace, therefore the English program is required to help students to communicate in English in workplace context. There are three kind of English proficiency levels in vocational high school: novice, elementary, and intermediate. In novice level (year X), the students learn English in daily life context. In elementary level (year XI), the students learn English that related to the job. In intermediate level (year XII), the students learn English for all vocational competence programs. In this level, the learning materials are more specified to the field of job they want. English language activities that are closely related to the students vocational skill are more developed in year XI and XII. Here, I developed a summative assessment tool that aimed at assessing students knowledge after learning one unit. This assessment tool is designed for year XII students in semester 1 concerned with the topic of taking and making a reservation. At this topic, the students are expected to be able to: 1. 2. 3. 4. Responding to dialogue about reservation (listening) Making reservation (speaking) Identifying the main ideas and supporting ideas of text about reservation (reading) Writing to a reply and completing reservation form (writing)

The assessment tool is developed to assess those four objectives from receptive skills (listening-reading) to productive skills (speaking and writing). In listening and reading, the students are expected to perform their ability in finding specific information about reservation based on dialogue and written text. In speaking and writing, the students are expected to perform their ability in making reservation in written and spoken.

1. Structure of the assessment tool

This assessment tool is developed based on unit 2 which is dealing with clients taken from book entitled Communication Builder: English for Vocational School. For Intermediate Level (Grade XII) of Vocational School (SMK/MAK). The topic of this unit is reservation in which the objectives of this unit are: 1. Listening Responding to dialogues about reservations 2. Speaking Making reservations 3. Reading Identifying the main ideas and supporting ideas of texts about reservations 4. Writing Writing a reply to a reservation letter and completing reservations forms Referring to those objectives, this assessment tool is developed for a summative test at the end of the unit. There are 5 tasks for 4 kind language skills; listening, speaking, reading, and writing. In listening, the students are given dialogues about a hotel reservation then they are asked to answer the questions related to the dialogue. The objective of this task is to assess
Oktari Aneliya (2215081412)

students comprehension in listening by identifying specific information related to dialogues about reservation. In speaking, the assessment tool that is used is interactive speaking test in which the students will do a role play of taking and making a hotel reservation in pairs. One student will be a hotel receptionist and the other will be a guess. The objective of this task is to assess students performance in taking and making a hotel reservation In reading, the assessment tool that is used is interactive reading in which the students are given a text about a hotel reservation confirmation letter then they are required to find answer for specific information related to the text. The students are expected to answer the question with short answer. The objective of this test is to assess students ability in identifying the specific information in a hotel reservation confirmation letter. In writing, there are two tasks that the students will do. One, the students are required to make the hotel reservation confirmation letter and the other is the students are required to fill in the hotel reservation form. These two tasks are tested because in the learning program, the students are given two kinds of written text of making a reservation. The objective of these tasks is to assess students ability in making reservation in written form. Those five assessment tasks are chosen under the consideration that in this unit the students have already learn how to take and make a reservation in spoken or written form. This summative test is conducted in two meetings (4 X 45 minutes) which one meeting lasts approximately 2 X 45. In the first meeting the students will be assessed on their listening, speaking, reading, and writing skill. in speaking task, it may need e lot of time, therefore in the second meeting the students will also be assesses on their speaking skill. Here is the time allocation: Task Time allocation listening comprehension speaking task 10 minutes 10 minutes for a pair of students = pretend that in the class there are 30 students so the time allocation: 10 X 15 = 150 minutes (2 hours 30 minutes) 10 minutes 15 minutes TOTAL 185 minutes = 3 hours 5 minutes

reading comprehension writing task

Oktari Aneliya (2215081412)

1. The assessment tool

Skill Objective Content standard : Listening : students are able to find specific information related to the hotel reservation : students will identify the main components of reservation such as guess name, date of guess arrival, kind of room that the guess want, and the room rate. Process standard : students will answer the short-answer question related to the dialogue.

Task 1: Listening comprehension

You will hear 2 kind of conversation. Listen to the conversations and answer the questions! Part A
In this section, you will hear a conversation between 2 people. Each conversation is followed by 5 questions about it. Each question in this part has three answer choices. You should cross on the best answer to each question. Answer the questions on the basis of what is stated or implied by the speakers On the recording, you will hear:
The script: conversation 1

Hotel Clerk: Hello. Sunnyside Inn. May I help you? Man: Yes, I'd like to reserve a room for two on the 21st of March. Hotel Clerk: Okay. Let me check our computer here for a moment. The 21st of May, right? Man: No. March, not May. Hotel Clerk: Oh, sorry. Let me see here. Hmmm. Man: Are you all booked that night? Hotel Clerk: Well, we have one suite available, complete with a kitchenette and a sauna bath. And the view of the city is great, too. Man: How much is that? Hotel Clerk: It's only $200 dollars, plus a 10% room tax. Man: Oh, that's a little too expensive for me. Do you have a cheaper room available either on the 20th or the 22nd? Hotel Clerk: Well, would you like a smoking or a non-smoking room? Man: Non-smoking, please. Hotel Clerk: Okay, we do have a few rooms available on the 20th; we're full on the 22nd, unless you want a smoking room. Man: Well, how much is the non-smoking room on the 20th? Hotel Clerk: $80 dollars, plus the 10% room tax. Man: Okay, that'll be fine. Hotel Clerk: All right. Could I have your name, please?

Oktari Aneliya (2215081412)

Man: Yes. Bob Maexner. Hotel Clerk: How do you spell your last name, Mr. Maexner? Man: M-A-E-X-N-E-R. Hotel Clerk: Okay, Mr. Maexner, we look forward to seeing you on March 20th. Man: Okay. Goodbye. taken from: www.esl-lab.com

I.

Choose the best answer!

1. The man makes a reservation finally for which day? a. March 20th b. March 21st c. March 22nd 1. What kind of room does the man prefer? a. a non-smoking room b. a smoking room c. either one is okay 1. Why doesn't he want to reserve the suite? a. It doesn't have a nice view. b. It doesn't come with a sauna bath. c. It's too expensive. 1. Including tax, how much is the man's room? a. 80 dollars b. 88 dollars c. 96 dollars 1. How do you spell the man's name? a. Maxner b. Maexner c. Mexner

Part B

In this section, you will hear a conversation between 2 people. Each conversation is followed by 5 questions about it. Each question in this part has three answer choices. You should cross on the best answer to each question. Answer the questions on the basis of what is stated or implied by the speakers On the recording, you will hear

Oktari Aneliya (2215081412)

The script: conversation 2

Receptionist : Good afternoon, San Felice hotel. May I help you? Guess : Yes. I'd like to book a room, please.

Receptionist : Certainly. When for, Madam? Guess : March the 23rd.

Receptionist : How long will you be staying? Guess : Three nights.

Receptionist : What kind of room would you like, madam? Guess : Er double with bath. I'd appreciate it if you could give me a room with a view over the lake. Receptionist : Certainly, Madam. I'll just check what we have available. Yes, we have a room, the 4th floor with a really splendid view. Guess : Fine. How much is the charge per night?

Receptionist : Would you like breakfast? Guess : No, thanks.

Receptionist : It's 84 per night excluding VAT. Guess : That's fine.

Receptionist : who is the booking for, please madam? Guess : Mr. and Mrs. Ryefielt, that's R-Y-E-F-I-E-L-D.

Receptionist : Okay, let me make sure I got that: Mr. and Mrs. Ryefield. Double with bath for March the 23rd, 24th, and 25th. Is that correct? Guess : Yes, it is. Thank you.

receptionist : Let me give your confirmation number. It is 7576385. Ill repeat that 75766385. Thank you for choosing San Felice hotel and have a nice day. Goodbye. Guess : Goodbye taken from: www.audioenglish.net

I.

Choose the correct answer!

1. When will the woman stay in? a. May the 10th b. March the 11th c. March the 23rd

1. What kind of room does the woman want? a. Double with bath b. Double bed c. Double with breakfast

Oktari Aneliya (2215081412)

1. The woman has a special request, what is that?

a. A room with a view over the valley. b. A room with a view over the lake. c. Any kind of view.

1. How much does the room cost per night?

a. 84 per night excluding tax b. 85 per night excluding tax c. 84 per night including tax

1. For whom does the book for? a. Mrs. Ryefield b. Mrs. And Mr. Ryefield c. Mrs. And Mr. Wrightfield

Skill Objective Content standard

: Speaking : students are able to take and make a hotel reservation : given a situation, students will use the language expression in taking and making a hotel reservation as well as they fulfill all the requests given

Oktari Aneliya (2215081412)

Process standard Task 2: Speaking

: students will perform their ability in taking and making a hotel reservation

Working in pair, one of you will be a hotel receptionist and the other will be a guess. You are about to make a reservation in Hilton Waikoloa Village under the name of Mr. Welirangan from 1-7 July 2011. The room you want to have is a single presidential suite with breakfast, a safe in the room, free internet access in the room, air conditioning & DVD Player, Private wireless telephone, and a small kitchen. You also want to a small conference room to be available on 3 July.

Skill Objective Content standard

: Reading : students are able to find specific information based on reservation confirmation letter. : students will identify the main components of reservation confirmation letter such as guess name, date of guess arrival, kind of room that the guess want, and the room rate.
Oktari Aneliya (2215081412)

Process standard

: students will read a letter of reservation confirmation and answer the short-answer quedtions.

Task 3: reading comprehension Read this letter and then answer the questions.
MULTI LOGISTIK Headquarters: Jalan A. P. Pettarani No. 5 Makassar Phone: 62411-425501

June 28th 2011

The Manager Hilton Waikoloa Village 425 Waikoloa Beach Dr. Waikoloa, Hawaii 96738

Reservation Confirmation

Dear Mr. Mappangara, I would like to confirm our booking in this morning for a single presidential suite room under the name of Mr. Welirangan. I would also like to confirm the following amenities are included in the cost of the stay: Breakfast A Safe in the Room Free Internet Access in the room Air Conditioning & DVD Player Private wireless telephone, and A small kitchen

The reservation is for full board from 17 July 2011. I am also confirming the reservation for a small conference room to be available on 3 July, when Mr. Welirangan will be holding a meeting with our international clients. Please find enclosed our bank draft (No. 92109 423 4431) for $10, 000,000.00 as a deposit. The balance will be paid as soon as we receive the confirmation from you. We look forward to hearing from you soon.

Sincerely yours, Melina Latuihamallo Personnel Secretary to Mr. Welirangan Encl. City Bank Draft

1. What is the email about? 2. Who is the writer of the email?

Oktari Aneliya (2215081412)

3. To whom is the email for? 4. Who is Melina Latuihamallo? Who is she work for? 5. Who is Mappangara? 6. What are the reservations for? 7. Where does Mr. Welirangan stay in? When does he stay in?
8. What kind of room does the Mr. Welirangan want?

9. What does Mr. Welirangan want to have besides the room?

10. What kind of payment does Mr. Welirangan use?

Skill Objective Content standard

: Writing : students are able to make reservation confirmation letter : students will make a reservation confirmation letter based on the situation given that should include the identity of the writer, the letter receiver, and the guess, the date and time of the reservation, the room and board preferences that has been reserved, the payment that has been transferred, and contact information.

Process standard

: given a situation, students will make a hotel reservation confirmation letter

Task 4: Writing You are a personnel secretary of PT. Sampoerna Tbk manager. You are about to make a reservation confirmation letter for Mrs. Julia Roberts staying in Sheraton hotel from 4-10 July 2011. The room you are reserved is Suite room with breakfast and lunch, double bed and a mini kitchen. She has transferred for the deposit of Rp. 12,000,000.00 under her bank account (99234.125.9654). Now write an email of the reservation confirmation!

Oktari Aneliya (2215081412)

Skill Objective Content standard

: Writing : students are able to fill a hotel reservation form : students fill in a hotel reservation form that should include the identity the guess, the date and time of the reservation, the room and board preferences, the payment method, contact information, etc.

Process standard

: students will fill in the hotel reservation form.

Task 5: hotel reservation form Fill in this reservation form based on the previous writing task.
Hilton Waikoloa Village (on the Big Island) 425 Waikoloa Beach Dr. Waikoloa, Hawaii 96738
FAX: Reservations (808) 886-2902 Or call: (808) 886-1234 or 1-800-HILTONS Or visit www.hiltonwaikoloavillage.com (enter Group Code "HIC" ) Your conference registration will be verified prior to your arrival. Name_____________________________________________________________________________ Address __________________________________________________________________________ City _____________________________ State ________ Zip _______ Country_______________ Phone (Home) ________/__________________ (Office) _______/_________________________ (Fax)_______________________ e-mail address _______________________________________

Oktari Aneliya (2215081412)

ARRIVAL Date / Time / Airlines /Flight #

DEPARTURE Date / Time / Airlines /Flight #

ACCOMMODATIONS $149/night $169/night $199/night

#nights at hotel _____

Garden/Golf/Mountain View _____ Partial Ocean View _____ Deluxe Ocean View _____

All rates subject to HI state room tax (currently 11.41%). Single or double occupancy Additional person @ $35 Hotel policy limits 3 adults or 2 adults with 2 children per room. Children under 18 are complimentary when accompanied by parent.

Sharing room with ___________________________________________________________

Name of individual (If children, please list names & ages)

Special Requests (not guaranteed): ___One King OR ____2 Double Beds ___Smoking OR ___ Non-smoking PAYMENT METHOD

By Personal Check or By Credit Card -Type: Visa

By Company Check Check # _______

1. Quality of the assessment tool

The quality of the assessment tool is developed based on five principles of language assessment proposed by Brown; practicality,

MasterCard

American Express

Discover

Diner's

Other card __________________

Card Number: ___/___/___/___ ___/___/___/___ ___/___/___/___ ___/___/___/___ Cardholders Name: _________________________________________(Please Print) Signature____________________________________________________ Date_____________________

reliability, validity, authenticity, and washback. Here is the analysis of the assessment tool quality. Based on the first criteria, practicality, this assessment tool is developed reasonably within the set time frame. Listening, reading, and writing test can be administered in 35 minutes whereas the speaking test might need more time because the test are designed based on performance test. It may need 10 minutes for one pair of students. Even though the time allocation of speaking test need a lot of time, the speaking test might be useful in knowing students speaking ability. The scoring system is also feasible in the teacher time frame because the test items in listening and reading are easy to be administered. Even though the reading test requires short answer test, the answer of this test is only has one possible answer. Meanwhile in speaking and writing test might need extra attention in the scoring system because this kind of performance task is less practicality but has high content validity and authenticity. It is said so because this performance tasks enable students to perform their ability in meaningful way. Related to the second criteria, reliability, this assessment tool is require more than 1 rater because the test such as speaking and writing might need more time to be scored. It is also suggested to avoid bias and subjectivity in the scoring. Meanwhile the reading and speaking scoring procedures leave little debate about correctness of the answer. Based on the third criteria, validity, this assessment tool is developed based on the objectives of the unit therefore every task from listening to writing is valid in the content because it match to the objective. For instance, in task 1; listening comprehension, the objective of the listening of this unit is students are able to respond to dialogues about reservation (see the objective on page 2). The task which is developed to achieve this objective is students are asked to listen to a hotel reservation dialogue and find specific information based on the dialogue. The test items which are short answer task represent a communicative stimulus-response task because
Oktari Aneliya (2215081412)

it focuses on the information of the dialogue of the hotel reservation. It also develops students ability to remember certain details from a conversation. In task 2; Speaking test, the objective of the speaking of this unit is students are able to make a reservation (on page 2). The task which is developed to achieve this objective is doing guided role play about making a hotel reservation. This task enables the students present their ability in making reservation creatively. Referring to the face validity, the assessment tool has clear direction for the test taker in understanding what they are going to do. The structure of the test is also organized logically from the easiest to the hardest. Based on the fourth criteria, authenticity, this assessment tool provides natural language that is people normally used in taking and making reservation. To make sure the language as natural as possible, the writer has observed from different kind of sources such as internet and book to find an appropriate language expression, kind of letter of reservation confirmation, reservation form, and reservation dialogue which are used in real life. The tasks also represent real world task. In the listening task, the students are exposed to making a hotel reservation in which they are expected to find specific information especially related to the information about the guess. This represents of real task of a receptionist who is expected to identify the guess information. In speaking task, the students are also exposed with approximately real world task because they are expected to be able to take and make a hotel reservation. This kind of activity is needed for students of vocational school when they are in workplace especially in tourism or hotel business. In reading task, the students are exposed to a text of reservation confirmation letter in which they will face it in real life in workplace as an assistant, secretary, etc. in writing task, the students are expected to make a reservation confirmation letter and fill in reservation form. These tasks also reflect the real world task for students of vocational school. The last is washback. Washback is the effect of test on teaching and learning activity for both the teacher and the students. This assessment tool is expected to have a good reflection on students understanding in this unit.

2. Rubric and scoring of the assessment tool

a. Listening task There are 10 questions in listening comprehension task. Each question score 1 point. Therefore the total score of listening task is 10 point b. Speaking task In order to know students speaking ability, the writer will use a rating scale as proposed by Harris to score the students' result in speaking test. This rating scale measures the students' speaking skills especially in pronunciation, grammar, vocabulary, fluency and comprehension. The rating scales for the speaking test modified from David P. Harris and Walter Bartz can be seen as follow: The rating scale

language aspect Pronunciation

score

characteristics

5 4 3

Speech consists of almost appropriate pronunciation Speech consists of hardly incorrect pronunciation Speech consists of some inappropriate pronunciation
Oktari Aneliya (2215081412)

2 1 grammar 5 4 3

Speech consists of mostly inappropriate pronunciation Speech consists of very poor pronunciation Makes few (if any) noticeable errors of grammar or word order Occasionally makes grammatical and/or word-order errors which do not, however, obscure Makes frequent errors of grammar and word order which occasionally obscure meaning

Grammar and word-order errors make comprehension difficult. Must often rephrase sentences and/or restrict himself to basic pattern. Errors in grammar and word order so severe as to make conversation virtually unintelligible Use of wide range of vocabulary taught previously Sometimes uses inappropriate terms and/or must rephrase ideas because of lexical inadequacies Frequently uses the wrong words; conversation somewhat limited because of inadequate vocabulary Misuse of words and very limited vocabulary make comprehension quite difficult Vocabulary limitations so extreme as to make conversation virtually impossible Speech is quite flowing style, mostly easy to understand Speed of speech seems to be slightly affected by language problems Speed and fluency are rather strongly affected by language problems Usually hesitant; often forced into silence by language limitation Speech is so halting and fragmentary as to make conversation virtually impossible Ideas highly organized, covers all of the elements of the content Ideas well organized, covers almost all of the elements of the content Ideas less organized, some missing parts of the elements of the content Ideas less organized, covers only the main elements of the content Unorganized ideas, a lot of missing parts of the elements.

1 vocabulary 5 4 3 2 1 fluency 5 4 3 2 1 comprehend 5 4 3 2 1

The scale of 0 up to 100 will be used to make the scoring easier. Here, the maximum rate students get is 25. So, the obtained rate is multiplied by 4. c. Reading test There are 10 questions in reading comprehension task. Each question score 1 point. There is only one possible answer for each question. d. Writing task

Oktari Aneliya (2215081412)

There are 2 kind of task in writing that are composing a reservation confirmation letter and fill in reservation form. In the first task, the students letter composition should include: 1. The identity of the writer 2. The identity of the letter receiver 3. The identity of the hotel guess 4. The room and other preferences being reserved 5. The payment that has been transferred 6. Contact information The students also have to consider the clarity of expression of the ideas, the logic of the sequence and connections, the cohesiveness or unity of the letter composition, and the overall effectiveness or impact of the paragraph as a whole (Brown 2004:253). The students will also be assessed on the content, organization, vocabulary, language use, as well as mechanics. COMPONENTS LEVEL OF PERFORMANCES CONTENT 4 3 2 1 ORGANIZATION 4 Related ideas Occasionally unrelated ideas Very often unrelated ideas Irrelevant ideas Effective and well organized occasionally ineffective, 3 weak transition and incomplete organization 2 1 VOCABULARY 4 3 2 lack organization little or no organization effective word choice mostly effective word choice frequently error in word choice mostly ineffective word choice grammatically correct mostly grammatically correct frequently error in grammar very often error in grammar few errors in spelling, 4 punctuation, capitalization, paragraphing 3 occasionally errors in spelling, punctuation,
Oktari Aneliya (2215081412)

DESCRIPTORS

1 LANGUAGE USE 4 3 2 1 MECHANICS

capitalization, paragraphing frequent errors in spelling, 2 punctuation, capitalization, paragraphing dominated by errors in 1 spelling, punctuation, capitalization, paragraphing

If the students meet those requirements in composing the letter, they will be given score of 100. In the second test, the students should fill in the information needed in reserving a hotel room. That it should be:

Filling up the identity of the guess: name, address, contact information Filling up the room reservation information: number of people (include how many of them are adults and children), the type of room, duration of stay, number of nights, date and time of arrival and departure, and expected amenities.

Filling up the payment method.

If the students fill all those information needed, they will be give score 100. From all those tests, the total score would be: Test Percent of total grade speaking test 40% 5 aspects X 5 (the highest score) = 25 X4 = 100. listening test 10 % 10 items @ 1 point each = 10 reading 10% 10 items @ 1 point each = 10 writing 40% task 1: 20% of 100 task 2: 20% of 100 Total Adapted from Brown scoring p, 61 20 20 100 10 10 Calculation Possible total correct 40% X 100 = 40

1. Conclusion
Here, the writer has developed an assessment tool for year XII of vocational school. The assessment is aimed at knowing how far the students grasp the lesson within the unit that is taking and making reservation. This assessment is developed based on four language skills; listening, speaking, reading, and writing. With the objective of the unit Being able to take and make a reservation, the writer focuses the assessment on that aspect. In listening test, the students are given a set of conversation related to the reservation and are given a set of questions about the conversations. In speaking test, the students are required to do role play in making and taking reservation especially a hotel reservation. In reading test, the students are given a text of reservation confirmation letter then they are asked to answer a set of questions related to the text.
Oktari Aneliya (2215081412)

In writing test, the students are given two kinds of test; composing a reservation confirmation letter and filling in a reservation form. All those tests are chosen under the consideration that in this unit the students have already learnt about all those materials in taking and making reservation. This assessment tool is also developed by considering the five principles of language assessment proposed by Brown. The practicality, reliability, validity, authenticity, and the washback of the assessment are counted in developing the assessment. Considering the practicality and the validity of the speaking test, this test might be less in practicality but agree with the validity. However, the writer hoped that this assessment tool could be as source for assessing year XII of vocational school in reservation topic.

References:
Books
Brown, H. D. (2004). Language Assessment Principles and Classrooom Practices. New York: Pearson Education. Haris, D. P. (1969). Testing English as a Second Language. london: Mc.Graw Hill Company. Kurniawan, Eri and Kurniawan, Arif. (2008). Communication Builder: English for Vocational School for Intermediate Level (grade XII) Vocational School (SMK/MAK). Jakarta: Pusat Perbukuan, Departemen Pendidikan Nasional. websites

http://www.hicss.hawaii.edu/hicss_35/hotelreservation.htm http://www.esl-lab.com/audio/mp3/hotel1.mp3 http://www.audioenglish.net/english-learning/english_dialogue_hotel_booking_a_room_2.htm

Oktari Aneliya (2215081412)