Appendix VIII: Quantitative Reasoning: Findings and Actions First Level Mathematics Courses- 16% of the students achieved

the expected performance level of 15 or more correct answers (70%) in the Campus General Logical Mathematical Reasoning Test (CG-LMRT). Approximately 50% of the students achieved a performance level of 12 or more correct answers (55%). These low results could be due to: End of semester administration two weeks prior to finals; Test results did not count towards their course grade; Some students did not received on time the learning objectives in which they were going to be tested; Some students were not aware that they were going to be tested. A closer examination of the results revealed that students performed better on the higher order cognitive test items (Evaluation area) than on the lower order cognitive test items (Computing area). These outcomes suggest that some students could not adequately represent the mathematic information provided in order to arrive to a correct answer or that calculation errors resulted in wrong answers. A detailed scheme of the CG-LMRT results is presented in this Appendix. Findings were discussed with the Department of Mathematics Chairperson and with faculty members in a departmental meeting. It was agreed to implement a series of out of class tutoring sessions to students enrolled in MATE 3105, MATE 3001, MATE 3036, MATE 3041, and MATE 3042 courses in order to strengthen their logical mathematical reasoning skills, which are currently underway. Also, during the second semester of 2011-2012, course modifications and tutors training will be address in order to strengthen student learning.

University of Puerto Rico Rio Piedras Campus Dean of Academic Affairs Office of Evaluation of Student Learning Campus General Logical Mathematical Reasoning Test (CG-LMRT) Results - May 2011 Item 1 Area of competence Computing Specific competency (learning outcome) Basic operations (fractions) Experts Delta 7 Difficulty Index 0.52 Equivalent Delta 13 Discrimination Index 0.44 Comments Medium difficulty, high discrimination Medium difficulty, high discrimination Low difficulty, very low discrimination Medium difficulty, high discrimination Low difficulty, low discrimination Low difficulty, high discrimination Low difficulty, low discrimination Medium difficulty, high discrimination High difficulty, high discrimination

Representation

Apply measurement concepts

0.48

13

0.41

Representation

Probability

0.85

0.19

Computing

Equations

0.49

13

0.51

Representation

Graph interpretation

0.87

0.26

Representation

Graph interpretation

13

0.68

11

0.55

Evaluation

Analyze evidences

0.72

11

0.26

Computing

Proportions Use quantitative competencies to analyze and solve everyday situations

10

0.45

13

0.58

Computing

10

0.35

14

0.47

Item 10

Area of competence Representation

Specific competency (learning outcome) Geometry

Experts Delta 10

Difficulty Index 0.60

Equivalent Delta 12

Discrimination Index 0.46

Comments Medium difficulty, high discrimination Low difficulty, high discrimination Medium difficulty, high discrimination Medium difficulty, high discrimination High difficulty, very low discrimination Low difficulty, high discrimination High difficulty, low discrimination Low difficulty, high discrimination Medium difficulty, high discrimination Low difficulty, high discrimination Medium difficulty, high discrimination

11

Computing

Percentage Use statistics principles to describe situations Percentage Analyze information presented on a graph Graphically represent functions Understand the relation between quantitative information and its different representations Understand the relation between quantitative information and its different representations Understand the relation between quantitative information and its different representations Understand the relation between quantitative information and its different representations Analyze evidence

0.72

11

0.47

12

Computing

12

0.46

13

0.44

13

Computing

0.43

14

0.46

14

Evaluation

12

0.12

18

0.09

15

Representation

13

0.37

14

0.46

16

Evaluation

15

0.11

18

0.20

17

Evaluation

14

0.74

11

0.46

18

Evaluation

14

0.59

12

0.54

19

Evaluation

15

0.79

10

0.40

20

Evaluation

13

0.46

13

0.43

Item 21

Area of competence Computing

Specific competency (learning outcome) Use statistics principles to describe situations Understand the relation between quantitative information and its different representations

Experts Delta 13

Difficulty Index 0.26

Equivalent Delta 15

Discrimination Index 0.34

Comments High difficulty, good discrimination Low difficulty, good discrimination

22

Evaluation

12

0.76

10

0.34

Global test delta: 12.5 Medium difficulty test Global test difficulty index: 0.54 College Boards Delta index 6-8 9 - 11 12 - 14 15 - 19 Item difficulty Percentage of students that answered correctly 95 - 90 85 - 70 60 - 40 30 - 5

Internal consistency index (reliability Cronbachs alpha): 0.719

Difficulty Very low low Medium High

Number of items in the test 0 8 9 5

Index 0.40 0.39 - 0.30 0.29 - 0.20 0.19 Negative

Item discrimination Number of items in Discrimination the test High 15 Good 2 Low - Can be revised 3 Check difficulty, instructions, and Very low - does not 2 alternatives discriminate Does not discriminate and should be 0 eliminated.

Notes: The CG-LMRT was administered from May 16th to the 20th, 2011 to 521 students out of a total of 806 enrolled in 34 sections of MATE, ESGE and EDFU courses that comply with the Campus General Logical-Mathematical Reasoning requirement. A total of 65% of the students enrolled these courses sections participated in this assessment. The item difficulty appreciation by experts (delta) of the College Board was established prior to the administration of the CBA-LMRT. An equivalent delta index was calculated using item difficulty statistics to establish a degree of agreement between experts item difficulty appreciation and test results. The item difficulty index indicates the proportion of students that answered each item correctly. Descriptive Statistics of the Campus General Logical Mathematical Reasoning Test (CG-LMRT) Administration Results May 2011

Distribution of Students Total Score in the Campus General Logical Mathematical Reasoning Test (CG-LMRT) Administration Results May, 2011

Expected outcome: 70% of the students enrolled in MATE, ESGE and EDFU courses that comply with the Campus General LogicalMathematical Reasoning requirement. will obtain a score of 15 points or higher. Findings: As presented on the table, 77.1% of the students achieved a score of 14 or lower. Therefore, only 22.9% of the students obtained a score of 15 points or higher. The goal was not met. Median score: 12

Students Item Alternatives Selection Distribution in the Campus General Logical Mathematical Reasoning Test (CG-LMRT) Administration Results May, 2011

Item 1 Frequency Percent A B Valid C D E Total 5 30 14 192 266 14 521 1.0 5.8 2.7 36.9 51.1 2.7 100.0 Item 1 Frequenc Percent y Incorrect 250 48.0 Valid Correct 266 51.1 Total 516 99.0 Missing System 5 1.0 Total 521 100.0 Valid Percent 48.4 51.6 100.0 Cumulative Percent 48.4 100.0 Valid Percent 1.0 5.8 2.7 36.9 51.1 2.7 100.0 Cumulative Percent 1.0 6.7 9.4 46.3 97.3 100.0

Item 2 Frequency Percent A B Valid C D E Total 15 12 119 242 42 91 521 2.9 2.3 22.8 46.4 8.1 17.5 100.0 Item 2 Frequenc Percent y Incorrect 264 50.7 Valid Percent 52.2 Cumulative Percent 52.2 Valid Percent 2.9 2.3 22.8 46.4 8.1 17.5 100.0 Cumulative Percent 2.9 5.2 28.0 74.5 82.5 100.0

Valid

Correct Total Missing System Total

242 506 15 521

46.4 97.1 2.9 100.0

47.8 100.0

100.0

Item 3 Frequency Percent A B Valid C D Total 4 437 38 32 10 521 .8 83.9 7.3 6.1 1.9 100.0 Item 3 Frequenc Percent y Incorrect 80 15.4 Valid Correct 437 83.9 Total 517 99.2 Missing System 4 .8 Total 521 100.0 Valid Percent 15.5 84.5 100.0 Cumulative Percent 15.5 100.0 Valid Percent .8 83.9 7.3 6.1 1.9 100.0 Cumulative Percent .8 84.6 91.9 98.1 100.0

Item 4 Frequency Percent A B Valid C D E Total 28 50 52 83 67 241 521 5.4 9.6 10.0 15.9 12.9 46.3 100.0 Item 4 Frequenc Percent y Valid Percent Cumulative Percent Valid Percent 5.4 9.6 10.0 15.9 12.9 46.3 100.0 Cumulative Percent 5.4 15.0 25.0 40.9 53.7 100.0

Incorrect Valid Correct Total Missing System Total

252 241 493 28 521

48.4 46.3 94.6 5.4 100.0

51.1 48.9 100.0

51.1 100.0

Item 5 Frequency Percent A B Valid C D E Total 2 9 25 449 26 10 521 .4 1.7 4.8 86.2 5.0 1.9 100.0 Valid Percent .4 1.7 4.8 86.2 5.0 1.9 100.0 Cumulative Percent .4 2.1 6.9 93.1 98.1 100.0

Item 5 Frequenc Percent y Incorrect 70 13.4 Valid Correct 449 86.2 Total 519 99.6 Missing System 2 .4 Total 521 100.0 Valid Percent 13.5 86.5 100.0 Cumulative Percent 13.5 100.0

Item 6 Frequency Percent A B C 11 102 24 17 2.1 19.6 4.6 3.3 Valid Percent 2.1 19.6 4.6 3.3 Cumulative Percent 2.1 21.7 26.3 29.6

Valid

D E Total

22 345 521

4.2 66.2 100.0

4.2 66.2 100.0

33.8 100.0

Item 6 Frequenc Percent y Incorrect 165 31.7 Valid Correct 345 66.2 Total 510 97.9 Missing System 11 2.1 Total 521 100.0 Valid Percent 32.4 67.6 100.0 Cumulative Percent 32.4 100.0

Item 7 Frequency Percent A B Valid C D E Total 7 32 72 14 25 371 521 1.3 6.1 13.8 2.7 4.8 71.2 100.0 Valid Percent 1.3 6.1 13.8 2.7 4.8 71.2 100.0 Cumulative Percent 1.3 7.5 21.3 24.0 28.8 100.0

Item 7 Frequenc Percent y Incorrect 143 27.4 Valid Correct 371 71.2 Total 514 98.7 Missing System 7 1.3 Total 521 100.0 Valid Percent 27.8 72.2 100.0 Cumulative Percent 27.8 100.0

Item 8 Frequency Percent Valid Percent Cumulative Percent

A B Valid C D E Total

11 85 50 48 230 97 521

2.1 16.3 9.6 9.2 44.1 18.6 100.0

2.1 16.3 9.6 9.2 44.1 18.6 100.0

2.1 18.4 28.0 37.2 81.4 100.0

Item 8 Frequenc Percent y Incorrect 280 53.7 Valid Correct 230 44.1 Total 510 97.9 Missing System 11 2.1 Total 521 100.0 Valid Percent 54.9 45.1 100.0 Cumulative Percent 54.9 100.0

Item 9 Frequency Percent A B Valid C D E Total 10 77 149 42 179 64 521 1.9 14.8 28.6 8.1 34.4 12.3 100.0 Valid Percent 1.9 14.8 28.6 8.1 34.4 12.3 100.0 Cumulative Percent 1.9 16.7 45.3 53.4 87.7 100.0

Item 9 Frequenc Percent y Incorrect 332 63.7 Valid Correct 179 34.4 Total 511 98.1 Missing System 10 1.9 Valid Percent 65.0 35.0 100.0 Cumulative Percent 65.0 100.0

Item 9 Frequenc Percent y Incorrect 332 63.7 Valid Correct 179 34.4 Total 511 98.1 Missing System 10 1.9 Total 521 100.0 Valid Percent 65.0 35.0 100.0 Cumulative Percent 65.0 100.0

Item 10 Frequency Percent A B Valid C D E Total 7 52 310 68 66 18 521 1.3 10.0 59.5 13.1 12.7 3.5 100.0 Item 10 Frequenc Percent y Incorrect 204 39.2 Valid Correct 310 59.5 Total 514 98.7 Missing System 7 1.3 Total 521 100.0 Valid Percent 39.7 60.3 100.0 Cumulative Percent 39.7 100.0 Valid Percent 1.3 10.0 59.5 13.1 12.7 3.5 100.0 Cumulative Percent 1.3 11.3 70.8 83.9 96.5 100.0

Item 11 Frequency Percent Valid A B 4 9 31 .8 1.7 6.0 Valid Percent .8 1.7 6.0 Cumulative Percent .8 2.5 8.4

C D E Total

371 100 6 521

71.2 19.2 1.2 100.0

71.2 19.2 1.2 100.0

79.7 98.8 100.0

Item 11 Frequenc Percent y Incorrect 146 28.0 Valid Correct 371 71.2 Total 517 99.2 Missing System 4 .8 Total 521 100.0 Valid Percent 28.2 71.8 100.0 Cumulative Percent 28.2 100.0

Item 12 Frequency Percent A B Valid C D E Total 11 65 76 78 233 58 521 2.1 12.5 14.6 15.0 44.7 11.1 100.0 Item 12 Frequenc Percent y Incorrect 277 53.2 Valid Correct 233 44.7 Total 510 97.9 Missing System 11 2.1 Total 521 100.0 Valid Percent 54.3 45.7 100.0 Cumulative Percent 54.3 100.0 Valid Percent 2.1 12.5 14.6 15.0 44.7 11.1 100.0 Cumulative Percent 2.1 14.6 29.2 44.1 88.9 100.0

Item 13 Frequency Percent A B Valid C D E Total 8 77 219 42 157 18 521 1.5 14.8 42.0 8.1 30.1 3.5 100.0 Item 13 Frequenc Percent y Incorrect 294 56.4 Valid Correct 219 42.0 Total 513 98.5 Missing System 8 1.5 Total 521 100.0 Valid Percent 57.3 42.7 100.0 Cumulative Percent 57.3 100.0 Valid Percent 1.5 14.8 42.0 8.1 30.1 3.5 100.0 Cumulative Percent 1.5 16.3 58.3 66.4 96.5 100.0

Item 14 Frequency Percent A B Valid C D E Total 10 12 156 59 31 253 521 1.9 2.3 29.9 11.3 6.0 48.6 100.0 Valid Percent 1.9 2.3 29.9 11.3 6.0 48.6 100.0 Cumulative Percent 1.9 4.2 34.2 45.5 51.4 100.0

Item 14 Frequenc Percent y Incorrect 452 86.8 Valid Correct 59 11.3 Total 511 98.1 Missing System 10 1.9 Total 521 100.0

Valid Percent 88.5 11.5 100.0

Cumulative Percent 88.5 100.0

Item 15 Frequency Percent A B Valid C D E Total 26 69 184 98 90 54 521 5.0 13.2 35.3 18.8 17.3 10.4 100.0 Item 15 Frequenc Percent y Incorrect 311 59.7 Valid Correct 184 35.3 Total 495 95.0 Missing System 26 5.0 Total 521 100.0 Valid Percent 62.8 37.2 100.0 Cumulative Percent 62.8 100.0 Valid Percent 5.0 13.2 35.3 18.8 17.3 10.4 100.0 Cumulative Percent 5.0 18.2 53.6 72.4 89.6 100.0

Item 16 Frequency Percent A B Valid C D E Total 24 218 100 24 54 101 521 4.6 41.8 19.2 4.6 10.4 19.4 100.0 Item 16 Frequenc Percent y Incorrect 443 85.0 Valid Percent 89.1 Cumulative Percent 89.1 Valid Percent 4.6 41.8 19.2 4.6 10.4 19.4 100.0 Cumulative Percent 4.6 46.4 65.6 70.2 80.6 100.0

Valid

Correct Total Missing System Total

54 497 24 521

10.4 95.4 4.6 100.0

10.9 100.0

100.0

Item 17 Frequency Percent A B Valid C D E Total 12 5 66 37 378 23 521 2.3 1.0 12.7 7.1 72.6 4.4 100.0 Item 17 Frequenc Percent y Incorrect 131 25.1 Valid Correct 378 72.6 Total 509 97.7 Missing System 12 2.3 Total 521 100.0 Valid Percent 25.7 74.3 100.0 Cumulative Percent 25.7 100.0 Valid Percent 2.3 1.0 12.7 7.1 72.6 4.4 100.0 Cumulative Percent 2.3 3.3 15.9 23.0 95.6 100.0

Item 18 Frequency Percent A B C D E 14 183 297 11 6 10 2.7 35.1 57.0 2.1 1.2 1.9 Valid Percent 2.7 35.1 57.0 2.1 1.2 1.9 Cumulative Percent 2.7 37.8 94.8 96.9 98.1 100.0

Valid

Item 18 Frequency Percent A B Valid C D E Total 14 183 297 11 6 10 521 2.7 35.1 57.0 2.1 1.2 1.9 100.0 Item 18 Frequenc Percent y Incorrect 210 40.3 Valid Correct 297 57.0 Total 507 97.3 Missing System 14 2.7 Total 521 100.0 Valid Percent 41.4 58.6 100.0 Cumulative Percent 41.4 100.0 Valid Percent 2.7 35.1 57.0 2.1 1.2 1.9 100.0 Cumulative Percent 2.7 37.8 94.8 96.9 98.1 100.0

Item 19 Frequency Percent A B Valid C D E Total 14 20 8 22 55 402 521 2.7 3.8 1.5 4.2 10.6 77.2 100.0 Item 19 Frequenc Percent y Incorrect 105 20.2 Correct 402 77.2 Total 507 97.3 Valid Percent 20.7 79.3 100.0 Cumulative Percent 20.7 100.0 Valid Percent 2.7 3.8 1.5 4.2 10.6 77.2 100.0 Cumulative Percent 2.7 6.5 8.1 12.3 22.8 100.0

Valid

Missing System Total

14 521

2.7 100.0

Item 20 Frequency Percent A B Valid C D E Total 30 48 224 51 55 113 521 5.8 9.2 43.0 9.8 10.6 21.7 100.0 Item 20 Frequenc Percent y Incorrect 267 51.2 Valid Correct 224 43.0 Total 491 94.2 Missing System 30 5.8 Total 521 100.0 Valid Percent 54.4 45.6 100.0 Cumulative Percent 54.4 100.0 Valid Percent 5.8 9.2 43.0 9.8 10.6 21.7 100.0 Cumulative Percent 5.8 15.0 58.0 67.8 78.3 100.0

Item 21 Frequency Percent * A B Valid C D E Total 28 1 138 113 63 130 48 521 5.4 .2 26.5 21.7 12.1 25.0 9.2 100.0 Valid Percent 5.4 .2 26.5 21.7 12.1 25.0 9.2 100.0 Cumulative Percent 5.4 5.6 32.1 53.7 65.8 90.8 100.0

Item 21 Frequenc y Incorrec t Valid Correct Total Missing System Total 362 130 492 29 521 Percent 69.5 25.0 94.4 5.6 100.0 Valid Percent 73.6 26.4 100.0 Cumulative Percent 73.6 100.0

Item 22 Frequency Percent A B Valid C D E Total 23 11 62 16 31 378 521 4.4 2.1 11.9 3.1 6.0 72.6 100.0 Item 22 Frequenc Percent y Incorrect 120 23.0 Valid Correct 378 72.6 Total 498 95.6 Missing System 23 4.4 Total 521 100.0 Valid Percent 24.1 75.9 100.0 Cumulative Percent 24.1 100.0 Valid Percent 4.4 2.1 11.9 3.1 6.0 72.6 100.0 Cumulative Percent 4.4 6.5 18.4 21.5 27.4 100.0

Index of Internal Consistency (reliability Cronbachs alpha) of the Campus General Logical Mathematical Reasoning Test (CG-LMRT) May, 2011

Case Processing Summary

N Cases Valid 415 a Excluded 106 Total 521 a. Listwise deletion based on all variables in the procedure.

% 79.7 20.3 100.0

Reliability Statistics Cronbach's Alpha Based on Cronbach's Standardized N of Alpha Items Items .719 .713 22