for Assessment of Ouality i n Education

I RST INTERNATIONAL C O 1\11 PA RAT I VE STUDY

O f language, mathematics,and associated factors for students i n the third and fourth grade of primal'y KhQOl.
SECOND REPORT
AUTHORS OFTHE REPORT: J U A N CASASSUS. S A N D R A CUSATO. J U A N ENRIQUE FROEMEL. J U A N CARLOS PALAFOX. SPECIALISTS RESPONSIBLE FORTHE ANALYSIS:

DOUGLASWILLMS AND A N N EM A R I SOMMERS. E DEPARTMENTOF EDUCATION OFTHE UNIVERSITYOF NEW BRUNSWICK,CANADA.

CARLOS PARDO.INSTITUTOCOLOMBIANO DE FOMENTO DE LA EDUCACION SUPERIOR, ICFES.

UNESCO
Report prepared by the Latin American Laboratory f o r Assessment o f Quality i n Education.
f the F i r s t Report: Coordinator o Juan Casassus,Regional Specialist i n Educational Planning and Management

UNESCO-SANTIAGO
Permanent Consulting Staff: (responsiblef o r developing this report) Sandra Cusato Juan Enrique Froemel Juan Carlos Palafox Analysts: Carlos Pardo Anne Marie Sommers Douglas Willms Contributors: Mara Ins Alvarez,National Assessment Coordinator (Chile);Asmara Anderson,General Special Projects Coordinator (Venezuela); Judith Barahona.General Director f o r Assessment o f the Q u a l i t yo f Education (Honduras); Susana Barrera,Coordinator o f the System f o r the Measurementand Assessment o f Quality i n Education (Bolivia); Hctor Fernndez,Specialist i n School Organization (Colombia);Mara Ins f Basic Education Gmez de S Pestana,Assessment Director o (Brazil); Marta iafuente, General Directoro f Educational Development (Paraguay); Leonte Ramrez,Director o f National Testing (Dominican Republic); Alejandra Schulmeyer.Consultant f o r the NationalInstitute o f Education Research (Brazil);Hctor Valds,Education System Researcher,Central I n s t i t u t eo f Pedagogical Sciences (Cuba);Lucrecia Tulic,National Director o f Assessment (Argentina); Julio Valeiron, Technical Director o f National Testing (DominicanRepublic ),Victor M.Velzquez,General Director o f Assessment (Mxico).

Published by the Latin American Laboratory f o r Assessment o f Quality i n Education,with the financial support o f the Government o f Spain.

UNESCO-SANTIAGO
Regional Office o f Education for Latin America and the Caribbean Printed by Andros Ltda. Lay-out:Claudia O'Ryan Original:Spanish English Translation:William Gallagher

The opinionscontained i n this Study are not necessarily those o f UNESCO,nor do they i n any way effect the responsibilities ofthe organization.Geographic place-namesused i nt h i s publication,and the presentation o f data herein do not express any opinion o f UNESCO regarding the judicial status o f cities,territories,o r zones, o f that o ft h e i r authorities,nor i n regard t o the delimitation o f borders.

QUNESCO 2000
Santiago,Chile,June,2002.

UNESCO

PRE FACE INTRODUCTION IMPLICATIONS

OFT H E F I N D I N G S FOREDUCATIONAL

POLICIES

THE STUDY SUBJECT MATTERS TESTED MAJOR FINDINGS RESULTS OF THE ANALYSIS OF ASSOCIATED FACTORS 3 SOCIO-CULTURAL STATUS(SCS) 2 O U T C O M E S FOR FACTORS SUBJECT TO MODIFICATION BY EDUCATION POLICIES 2.1 ATTHECENTRAL ADMINISTRATION L E V E L O F T H E SYSTEM 2.2 SCHOOLPRINCIPALS A N D SCHOOL M A N A G E M E N T 2.3 WITHIN-CLASSROOM TEACHER RELATED FACTORS STUDENT~TEACHER RATIO INITIAL A N D IN-SERVICETEACH E R TRAIN ING TEACH E R EXPERIENCE TEACHER ATTITUDES, OPINIONS, A N D WORKING CONDITIONS PERCEIVED TEACHERS CAUSAL ATTRIBUTIONS CLASSROOM STRATEGIES 2.4 THED O M A I NOF STUDENTS A N D T H E I R FAMILY CONTEXTS 3 ACHIEVEMENT BY STRATAA N D ADJUSTMENTVARIABLES
C ~ M P L E M E N T A ANALYSIS RY OF STUDENT ACHIEVEMENT

2
.I*

1 2

ANALYSIS BY TOPIC ANALYSIS BY ACHIEVEMENT LEVELS 2.1 PERFORMANCE LEVELS IN LANGUAGE 2 2 PERFROMANCE LEVELS IN MATHEMATICS 3 REGIONALOUTCOMES BY DEMOGRAPHIC STRATA T REGIONALOUTCOMES IN PUBLIC A N D PRIVATE SCHOOLS RESEARCH

i i

I F S
2% Jh
iia

{i
I -

CONCLUSIONS A N D IMPLICATIONS FOR FUTURE

. * . *

ACKNOWLEDGEMENTS APPENDIX 1. RELATIONBETWEEN SCHOOL O U T C O M E S A N D GENDER, GRADE,A N D SOCIO-CULTURAL STATUS APPENDIX 2.DESCRIPTION OF THE VARIABLES

ir 4

fi

I n 1998,UNESCO'sLatin American Laboratory f o r the Assessment o f Quality i n Education published the F i r s t Comparative Study o f Language,Mathematics,and Associated Factors f o r Students i n the Third and Fourth Grade o f Primary School.That Study presented f o r the f i r s t time a comparative glance o f educationalachievement i n countries which have many cultural elements i n common.This i s an importantsubject,since during the 199Os, education became the major public policy issue i n Latin America. This Second Report points toward other aspects o f education i n Latin America.How may we characterizestudent achievement i n the region? How can the quality o f education be improved? How can it be made more pertinent t o the needs o f the population? How can education improve the future prospects o f millions o f students? These are some o f the questions addressed i nt h i s report through the study o f factors associated w i t h academic achievement. First,the Study sounds an alert. For it shows that average achievement o f students i n the region,i n terms o f what they should learn i n Language and Mathematics,i s low.But besides serving as an alert,this report seeks t o provide information on how t o improve the current state o f education.In so doing,it points i n a number o f directions.Much can be done.Nevertheless,the major challenge i s how t o learn t o manage sets o f factors that operate i n synergy w i t h one another. The Study also highlights UNESCO'simportant role as a forum f o r the exchange o f ideas,and as a facilitatori n the common tasks faced by nations. To carry out this Study,thirteen countries and hundreds o f researchers joined forces t o produce more and better information. The results o ft h i s effort demonstrate how research can produce more solid and well-basedinterpretations o f what can be done t o improve the quality o f schools i n the region.It i s our hope that it w i l l become a useful t o o li n encouraging dialogue and i n developing new educational policies.

Ana Luiza Machado

Director of the Regional Office of Education for Latin American and the Caribbean

I n order t o provide useful information f o r the formulation and execution o f educational policy o f countries i n the region, i n 1997 the Latin American Laboratory f o r the Assessment o f Quality i n Education - LLECE - carried out the F i r s t International ComparativeStudy i n Language,Mathematics,and Associated Factors i n the Third and Fourth Grades o f Primary Education. The Laboratory,coordinated by the UNESCO Regional Office o f Education f o r Latin America and the Caribbean,brings together i n a network various national systems f o r the measurement and assessment o f the quality o f education.

A total o f thirteen countries participated i n the Study. The target population was a l l boys and g i r l s enrolled i n the third
and fourth grades o f primary schools i n Argentina,Brazil,Chile,Colombia,Costa Rica,Cuba,Honduras,Mexico, r guardians,teachers,school principals, Paraguay,Peru,the Dominican Republic,and Venezuela,as well as their parents o and the schools themselves. I n 1996, the t o t a l population enrolled i n both grades within these countries was 19,490, 590 students. The sample taken from this universe was approximately 55,000- a number considered t o be s t a t i s t i c a l l y appropriate. The f i r s t results published a t the end o f 1998 provided some indications o f a clear situation o f low outcomes i n terms o f achievement,and made manifest the importance o f giving p r i o r i t yt o a systematic policy aimed a t raising the academic achievement o f students i n the region. These and other conclusions o f the Study made it possible t o take the pulse o f the education situation o f more than sixty-fourmillion children who make up the universe o f the primary school population i n Latin America. The most significantfinding was the dispersion of results between countries;s o much s o that they were divided into three groups: the f i r s t ,consisting o f one country that obtained scores f a r superior t o the others;the other two groups had closer results,w i t h generally low achievement levels. Moreover,besides comparisonsbetween countries,the Study,from its inception was intended t o carry out a more exhaustive analysis o f outcomes i n order t o identify factors that could explain them. Together w i t h the tests i n Language and Mathematics,a questionnaire was administered t o students,parents and guardians,teachers,and school principals i n order t o collect more information regarding factors associated w i t h achievement levels. A l lo ft h i s information comprises a valuable data base that allows both countries and the region as a whole t o investigate the effect o f a large number o f school achievement variables. This document contains the essential aspects o f the Technical Report,which w i l l be soon be made available t o the general public. In that report,outcomes are detailed and interpreted through analyses o f achievement levels i n Language and Mathematics and the findings on the different factors that explain achievement. This Second Report focuses on discussion o f implicationsf o r and recommendationsregarding educational policies.

A l lo ft h i s background material substantiates the importance o f the report,since it w i l l allow decision-makers t o have
access t o information that offers support and guidance f o r processes directed a t focussing and making rational use of resources, and t o move toward strategiesthat foster greater quality and equity i n education. While it i s true that the outcomes o f the Study confirmmany o f the major concepts that have guided decision-makingi n recent years,it i s also true that they modify others. This i s very important f o r correcting certain viewpoints that have been current i n education policy. The in-depth analysis o f outcomes i n Language and Mathematics can contribute t o the design,o r re-design, o f curricular programs,t o instrumentation and focus o f resource allocation strategies,and t o establishing quality standards from the perspective o f concrete results o f a representative sample o f students within the region. The Study of associated factors, f o ri t s part,makes possible the development o f a suggested Latin American model f o r effective schools. The l a t t e ri s highly significantbecause it shows that crucial variables exist that compensate f o r the possible negative effect o f adverse social-economic and socio-culturalconditions,and that despite coming from unfavorable contexts, studentscan achieve good results. Such conditions point toward the culture,attitudes,practices,and inter-relationships between teachers,students,administrators, and other actors within the school community. A good part o f the variance i n the outcomes - the measure o fv a r i a b i l i t y-i s explained by factors linked to the school. This opens up Significant opportunitiest o apply low-costeducational policies that can modify the present situation and substantially improve student achievement. Social differences i n Latin America are widening,while a t the same time equity has become an ongoing objective of social policy. This Study allows us t o conclude that i n Latin America,contrary t o what has been argued by some specialists, schools do make a difference and can compensate f o r the effects o ft h i s lack o f socialjustice. Furthermore,it i s important t o note that increased budgets alone do not necessarily imply achieving better education. The Study demonstratesthat although budgetary resources have an impact,they are not sufficient i n and by themselves. Current Latin American experience shows that effective schools do not require enormous investmentsi n resources; they require,rather,the efficient and cumulative inter-relationo f a number o f variables that are examined i n detail i n this Study. Finally,it should be noted that although the most modern and universally-recognized methodologies and procedures have been used i n design,procedures,and data analysis,the Study possesses the limitation inherent t o quantitative research o f this kind -the application o f instrumentsbased on pencil and paper exams,using multiple choice questions w i t h only one v a l i d response;exams that were self-appliedand administered t o diverse populations distributed over a wide geographic area.

I n accordance w i t h the purpose o ft h i s report,i n terms o f being more an explanatory than a comparativet o o l of the results o f the F i r s t InternationalStudy,the present section - which deals w i t h repercussions o f the findings f o r educational policies - presents an i n i t i a l explanation of outcomes i n the region.

A number o f implicationsf o r educational policies arise

from a general analysis o f the Study. These may be summarized as follows:

It i s essential that a much greater effort be made i n order t o improve learning i n Language and Mathematics. Special emphasis should be placed on raising learning levels i n Language,since low achievement i nt h i s area affects other kinds o f learning and for the futurep o s s i b i l i t i e so f students within the educational system. If things remain as they are,countrieswithin the region w i l l be poorly prepared t o meet the challenges presented by the new information and knowledge society within a context o f globalization.
The Study has made it possible t o affirm what other research has detected as well: factors outside the school influencewhat happens within the classroom. However, f factors associated w i t h schools explains more the set o than two-thirdso f the variation i n outcomes between schools. This demonstratesthat,although factors outside the school have an impact on achievement,effective teaching leads t o successful learning processes. The lack o f equity may arise from the environment within which schoolsoperate. Differencesi n achievementbetween schools are l e s s than that observed i n the Socio-Cultural Index (SCS), w i t hi t s four indicators o f family context. Actually,what schools i n the region do i s compensate f o r inequalities. They thus have the potential t o alleviate the effects o f social inequalities.

The Study makes it clear that a good education system, o r a good school,need not be especially expensive. Among those schools studied,there are some that operate under unfavorable conditions.Yet compared t o others o f a higher socio-culturallevel,they achieve remarkable results. The way that different actors inter-relatewithin a school i s a subject that deserves greater study,since a good part o f student outcomes depend on the quality o f such links. Interventionshould be directed a t those factorsthat generate a micro-climate favorable t o learning i n the classroom ( i n which students do not fight,do not interfere w i t h each another,and among whom a climate o f friendshipexists) i n order t o improve the quality o f education.ITSHOULDB E
N O T E D THAT EMOTIONAL FACTORS, OF W H I C H THIS FACTOR IS A PART, ARE MORE C O M P L E X T H A N THOSE RELATED TO INVESTMENT IN IMPACT O N ACHIEVEMENT IS MATERIAL INPUTS. THEREFORE,THEIR MORE UNCERTAIN,AND M A Y REQUIRE MORE TIME FOR INSTRUMENTATION.

The differences observed between the outcomes o f demographic and administrative strata obey factors associated with familiesand school processes,and not the strata per se. This means that those children whose parents have similar educationl e v e l s and who attend schools sharing similarprocessesalso have similaroutcomes. The difference does not l i ei n whether a school i s rural o r urban;nor i n whether it i s public o r private. It lies,rather,i n other factors, some o f which may be modified through appropriate policies. In improving the quality o f education,more attention should be paid t o what happens i n education processes and t o the characteristics that differentiate schools within each strata,rather than the characteristicso f the strata themselves.

The following sections are intended t o support the aforementioned repercussions,both i n terms o f the constituent characteristicso f the Study,as well as the results o f the analysis o f Associated Factors.A country analysis by Language and Mathematics topics has been added that i s a complementt o the regional overview,but which looks this time within each country.Finally,w e present an analysis o f both Language and Mathematics by strata and achievement levels a t the regional level.

It has been noted that the Study provides value judgements i n order t o suggesta model o f effectiueschools,understood as those which achieve what i s expected o f them;that is, that their students learn. Based on the resultso f the Study, it appears that the profile of such an ideal school one i n whichl:
The library has instructionalmaterial and books o f sufficient quantity and quality. The i n i t i a l training o f teachers has taken place a f t e r their secondary schooling;the teachers f e e l that their pay i s adequate,and they teach exclusively i n this school. Teachers believe that the positive results o r failure of their students depend largely on the students themselves. There i s a formal student achievement assessment practice i n place. Heterogeneity i s encouraged. Studentsare not grouped according t o a single standard. Attemptsare made t o provide a classroom atmosphere that favors mutual respect and harmony between students. Parents are involved i n the activitieso f the school community.
1

THEE L E M E N T S PRESENTED I NT H I SP R O F I LA ER E LISTED I NT H E O R D E RI NW H I C H THEY A R E ANALYZED IN CHAPTER 111: RESULTSOF THE ANALYSIS OF ASSOCIATED FACTORS,AND NOT IN
TERMS OFTHEIR RELATIVE IMPORTANCE.

Between June and November,1997,third and fourth grade primary school students i n 13 countries were tested i n Language (54,589 students)and Mathematics (54,417 students). Questionnaires designed t o obtain information on learning conditions were administered t o 48,688 students,41,088 parents and guardians,3,675 teachers, and 1,387 school principals i n 1,509 schools. The size o f the samples were similar i n each country,with about 1 0 0schools selected i n each,and w i t h 20 students a t each grade level. The sampling process was adjusted t o current internationalstandards. Most o f the countries complied with the general guidelineso f the corresponding manual,w i t h only two o f them introducing changes into the agreed-uponprocedure2, but always within acceptable standards f o r this kind o f study. The sample was demographically and administratively stratified. The f i r s t criterion made it possible t o distinguish schools located i n large c i t i e s (morethan 1 million inhabitants), urban areas ( c i t i e sw i t h more than 2,500 and less than one million inhabitants)and rural areas (locales o f 2,500 inhabitants or l e s s ) . The second criterion distinguished between publicly-managed (federal, state,o r municipal) and private schools i n large c i t i e s and urban areas,without Considering the source o f resources o f the l a t t e r . f the Study was t o provide useful The basic purpose o information f o r the formulationand execution o f educational policies within countries o f the region. Three central issues were constantly present:

H The Study coincides with the interest i n many countries

to: give priority t o the development o f education; move ahead w i t h educational reform processes; effect profound changes i n management,objectives, content,s k i l l s ,and methodologies o f teaching;and improve physical infrastructureand support materials used i n schools,with strong emphasis on the quality o f education offered and the equity o fi t s distribution. The results o f the Study,together w i t h other educational indicators, w i l l make it possible f o r authoritiescharged with the design and implementationof educational policies t o have available a broad and documented view o f the factors that most influencethe quality o f education i n their countries, as well as t o access database information that can be used i n future studies on the impact o f educationalreforms.

I Only a few countries in the region have participated

i n previous comparative measurements o f the quality o f education.When they have,consideration has not been given t o their curricular and cultural roots. I n recent years,several Latin American countries have put i n place national education quality assessment systems,t o which the present Study can supply comparative and reference information on common curricular issues.

ARGENTINA UTILIZEDA PREVIOUSLYSELECTED SAMPLE FOR ITS NATIONAL SURVEY. BRAZIL.

DUE TO ITS GREAT DISPERSION,PRE-SELECTEDTHREE5TATES.AND WITHIN EACH O F T H E M THREE MUNICIPALITIESWITH HIGH,MIDDLE,AND L O W RATES B A S E D O N T H E UNITED

NATIONS H U M A N DEVELOPMENT INDEX. MEXICO DEFINED AS"RURAL"TH0SE

OF U P T O 5,000INHABITANTS.

POPULATIONS

SUBJECT MATTERS TESTED

Study areas

The Study included two basic subjects - Language and Mathematics - as indicators o f the quality o f education i n each country.With Language,students construct and develop knowledge and learning,giving meaning t o their experiences and sense t o the knowledge o f others,as well as building the basis for development o f the a b i l i t yt o reason and opening the doors o f access t o knowledgeand c r i t i c a l thinking.Mathematics aids i n organizing the bases f o r logical reasoning, develops the capacity t o solve problems, and lends rigor t o the analysis o f data. After analyzing and identifying curricular elements that define the educational process o f studentswithin the region, f i v e topics i n Language and another f i v ei n Mathematics were chosen,t o be used as a basis f o r constructing the instrumentsand f o r subsequentinterpretation o f the results. These were:
LANGUAGE: Identifymgtypes o f texts;distinguishingbetween

STUDENT FAMILY CONTEXT

TEACHER EDUCATIONAL ENVIRONMENT

COMMITMENTS, HABIT5,AND SKILLS

i I

CURRICULUM AND PEDAGOGICAL MANAGEMENT

INSTITUTIONAL MANAG EMENT

POLICY AND INSTITUTIONAL MANAGEMENT

the transmitter and the recipient o f a text;identifying the message i n a text;recognizing specific information within a text,and identifying vocabulary related t o the meaning o f a text.

P R I NC I PAL

i 1 I
1

PUBLIC OFFICIALS

W i t h due regard f o r the complexity o f educational processes

and their close links t o economic and social development, a basic model was defined that integrates four different factors that influence learning:

MATHEMATICS: Numbering,operations w i t h natural numbers;common fractions; geometry and s k i l l s (thel a t t e r understood as the a b i l i t yt o read graphs,recognize trends, have a notion o f probabilities,and discern relationsamong given data).
Studentachievementi n the various topics, and was analyzed i t h their i n each country,and was also contrasted w achievement i n the t e s t as a whole.This made it possible t o produce a performance profile f o r each country that expresses the relative mastery that their students have o f the different topics. The r e s u l t so f both t e s t s are expressed on an ad hoc (Rasch Model)Scale,w i t h a Mean Score o f 250 points and a Standard Deviation o f 50 points.

1) STUDENTSA N D T H E I R FAMILY CONTEXTS (characteristics, commitment, habits,and s k i l l so f students and o f their immediate families); 2) TEACHERS A N D THE EDUCATIONAL DOMAIN (CUrriCUlUm and teaching management variablesu t i l i z e d by teachers i n the classroom);

3) PRINCIPALS A N D T H E SCHOOLMICROCOSM (variablesupon

which i n s t i t u t i o n a l management processeso f the school are supported);

4) PUBLIC AUTHORITIES A N D THE MACROCOSM(aspects

related t o public management o f institutions a t the national l e v e l ) .

MAJOR FINDINGS

The most significantfindings o f the Study may be summarized as follows: Cuban students achieved the highest scores i n Language and Mathematics,and take less time t o complete a grade (AdvancementRate).This i s the case i na l lo f their schools. Differences i n achievement for t h i s country,i n terms o f gender and socio-culturallevels,are also reduced.The relationship between high scores i n the two subjects and advancement rates also occurs i n other countries.

School-relatedfactors- input and processes - are responsiblef o r nearly two-thirdso f the variation i n student achievement outcomes.This finding warrants the need f o r the study and development o f policies aimed a t changing the present situation and improving achievement. There are differences i n outcomes between schools, according t o the strata t o which they belong.But these disappear and,i n some cases,reverse their direction,if an analysis i s carried out controlling f o r family history variables and,even more so,when t o such control we add education process variables.Thus,the importance o f the family i n studentachievement becomes evident.Even more evident are school-relatedfactors which,i n some cases, even neutralize o r compensate f o r the negative effects o f other variables. Rural schools i n Colombia had higher than expected f the urban schools i n outcomes that were above those o that country.This indicates that,even i n unfavorable contexts,the application o f appropriate and consistent measures (EscuelaNueva)can significantly improve student outcomes. Urban schools i n Chile show better outcomes than those i n large metropolitan areas.This may be due t o the effect o f the regionalization model put into practice i n that country.

With the aforementioned exception,achievement levels

i n Language i n the region are quite low. Most students have only a fragmentary understanding o f the texts that they read.They recognize the words o f a text,but are not able to determine why they say what they do,o rf o r what purpose they are said. THIS MAY INDICATE THAT I NTHE R E G I O N ,
STUDENTS ARE T A U G H T TO DECODE; THAT IS,TO TRANSLATE WRITTEN W O R D S INTO ORAL LANGAUGE, BUT W I T H O U T U N D E R S T A N D I N G THE M E A N I N G OF THE TEXT. NOR D O T H E Y INTERPRET THAT W H I C H THEY READ.THEY LEARN TO READ ALOUD, OR TO ENUNCIATETEXTS,BUT THEY D O N O T LEARN T H R O U G H READING.

Outcomes f o r Mathematics,with the exception noted above for the case o f Cuba,were generally even lower and more unequal.STUDENTS DO NOT ASSIMILATE K N O W L E D G E ,N O R
D O T H E Y DEVELOPSKILLS INTHE SUBJECT.THEY RECOGNIZE S I G N S A N D STRUCTURES,BUT HAVE LITTLE ABILITY TO RESOLVE SIMPLE MATHEMATICAL PROBLEMS E N C O U N T E R E D IN DAILY LIFE.

A last finding points toward the importance o f schools

themselvesand t o what happens within them.One o f the most important findings o f the Study i s that student perception o f a favorable classroom environment has more influenceon learning than the combined effect o fa l l the other factors.

Gender differences follow a pattern similar t o that noted above for the case o f Cuba,being significant only i n some cases.Girls show better achievement i n Language and s l i g h t l y poorer achievement i n Mathematics. Although there are similarities between countries,there are complex differences as well. The way that different variables have an impact on school performance i n each county deserves detailed study,inasmuch as their behavior i s not consistenti na l l countries. The fact i s that,as we s h a l l see i nd e t a i l for the selected variables below,f o r some countries such relations are positive,and for others they are negative.

The Study included the analysisthose factors that have an influence one way o r another upon student learning i n Language and Mathematics.The application o f questionnaires t o students, parents and guardians,teachers, principals,as well as the collection o f information on the schools themselves provided a wealth o f data that w i l l make it possible f o r each country,and f o r the region, t o work i n the future toward intensifying research on the effects o f these different pedagogical,economic,social, and cultural factors- t o examine the interrelationsand/or synergy that can take place between them. o be presented i nt h i s chapter have been The variables t chosen according t o two categories.First,w e w i l ll o o ka t those that had a s t a t i s t i c a l l y significanteffect on student scores;second,we w i l l examine those that,i n spite o f not demonstrating a significanteffect i n the Study,have conceptual relevance that merits our attention. I n addition,i n order t oc l a r i f y understanding o f the incidence 3 f the variables considered i nt h i s Study,independent o f stlident Socio-CulturalStatus (SCS), the s t a t i s t i c a l analysis presents the difference o f scores on each variable f o ra hypothetical group o f students that corresponds t o the M e m value o f SCS i n the region.

1. SOCIO-CULTURAL STATUS (SCS;)

The -9 variations i n achievement i n Language and

Mathematics i n regard t o the variables included i n the analytic model o f the Study w i l l be described i n the following sectims o ft h i s chapter,and may be found i n detail i n the table included i n A P P E N D I1Xo ft h i s document. Analytic f each o f the variables considered are included descriptionso i n A P P E N D I2 X .

I n order t o study the incidence o f variables related t o the family contexts o f students,a scale was created called Socio-Cultural Status. This index consists o f four variables: 1) educational level o f parents;2 )number o f hours that parents are a t home during working days;3 ) reading resourcesavailable i n the household,and 4 ) structure o f the core family (whetherit i so ri s not composed o f two parents,irregardless o f marital status). t s analysisusing collected information The SCS index and i shows that t h i s variable changeswithin each school and between schools f o r each country. A f i r s t take on achievement gradients i n Language and Mathematics shows that an increase i n the Mean value o f the schooling o f parents (estimatedt o be 9.3 years i n the Study)results i n an increase i n the achievement o f their children. This i s even more evident as the heterogeneity i n levels o f schooling o f the countries increases.

2.OUTCOMES FOR FACTORS SUBJECT TO

MODIFICATION BY EDUCATION POLICIES
2 . 1

At the CentralAdministrationLevel of the System

The Study shows that within the region i n general,those studentsw i t h exposure t o pre-schooleducation show s l i g h t l y better subsequent academic achievementthan those studentswho have not had such experience.This i s most evident i n the case o f Language achievement. This fact notwithstanding, it i s important t o remember that such behavior was not s t a t i s t i c a l l y significantacross the entire region,that there i s no uniformity i n these outcomes when the analysis i s done by country,and that the data f o rt h i s variable were very often invalid.For this reason, it i s advisable that in-countryanalyses should be intensified i n the future.

Comparison o f the outcomes o f the Study w i t h similar ones i n developed countries showsthat learningachievement i n the thirteen countries (and, by extension,i n Latin America) i s -w i t h the exception o f one country - quite low.This finding reveals the disheartening situation o f our countries within the international context;one which demands the formulation o f educationalpolicies that assure classroom learning and which raise the academic achievement o f future generations. These new educational policies,as we shall see below,do not necessarily involve more o rl e s s spending per student; nor do they imply questioning the need for investing i n education.It i s not so much a question o f the volume o f resources,but rather o f the effectiveness o f measures many o f low cost - that can guarantee quality within the learning and teaching process. Independent o f the outcomes o ft h i s Study,it should be noted within t h i s domain that i n the different educational systems analyzed,reforms are being carried out - highly productive innovative experiences and traditional practices that produce better outcomes.It i sf o rt h i s reason that the exchange o f knowledgebetween countries i s a key element f o r progress i n the region.

2 . 2

School principals and school management

Student academic achievement i s multivariate. What happens within schools, w i t h their multiple variables,i s very important f o r learning. Knowing their influence on school achievement allows us t o act upon them and when appropriate - modify them through educational policy decisions.
FIGURE 2
SCORE DIFFERENCES AS A FUNCTION

O F VARIABLES RELATED T O SCHOOL RESOURCES

LANGUAGE MATHEMATICS

1 0 . 4

10
8

6 4
2

O
INFRASTRUCTURE TEACHING MATERIALS

SIZE OF LIBRARY (> 1,000BOOKS)

FIGURES IN BOLDTYPE ARE STATISTICALLY SIGNIFICANT ATTHE . 0 5 % LEVEL

The Study assessed school resources i n terms o f the availability o f basic teaching materials (black boards, calculators,maps,and others),and found that a t the regional level,an increase o f an item i n the number o f such materiais i s associated w i t h an almost two point increase i n the Language test.Thus,an increase o ff i v e items would imply an approximate increase o f ten points on the same test.The magnitude o f the relationship i n o achievement i n Mathematics i s lower. regard t

o the availability o f books i n This variable i s also related t school libraries.W e note that schools w i t h libraries that have a tl e a s t one thousand books are more associated with those that have higher achievement scores than those schools that have small libraries.It seems reasonable that children who have the support o f materials and access t o libraries learn more than those children who do not.

2.3

Within-classroomteacher related factors

STUDENT/TEACHER RATIO

Beyond intuitive approaches i n regard t o the subject,the outcomes o f the present Study agree w i t h the general trend revealed i n other research,which point out that i n terms o f achievement,a larger number o f studentsper classroom has a negative,but quite small effect.
INITIAL A N D IN-SERVICETEACHER TRAINING

W e see from the Study that there i s a positive relationship between post-secondarytraining o f teachers and student

achievement.O n the other hand,if we observe the variable in-service teacher training(understoodas that which takes place after i n i t i a i training has been completed),i t s influence on achievement,even when positive,i s not s t a t i s t i c a l l y significant. For each additional year o f postsecondary teacher training,students increase their scores by 2.44 points i n Language and 2.06 points i n Mathematics. This means that those studentswhose teachers possess four years o f post-secondarytraining achieve between 4 and 5 points more than those students whose teachers f similar training. have had only two years o

FIGURE 3
VARIATIONS IN SCORES AS A FUNCTION OF CLASSROOM-ASSOCIATEDVARIABLES A N D TEACHER CHARACTERISTICS

7,s
2
1,5

MATHEMATICS

0s
0

-0.5
-1

-0,49

-0.53

TEACH E RI STUDENT RATIO

TEACHER EXPERIENCE

TEACHER EDUCATION

IN-SERVICETEACHER TRAINING

FIGURES IN BOLDTYPE ARE STATISTICALLY SIGNIFICANT ATTHE

.05% LEVEL

TEACHER EXPERIENCE
For its part,the Study did not detect the influence o f teacher experience on student performance,being s l i g h t l y positive f o r both Language and Mathematics.I nt h i s sense, it would seem necessary to revise the weight assigned to the factor o f experience i n the design o f teacher promotion systems,i n view o f the fact that years o f service have a strong influenceon teacher salary increases and promotions.

As was stated,results o f the research show that the education o f teachers i s an important issue t o consider. Therefore,one could require that a l l teachers have a tl e a s t af u l l secondaryeducation,i n view o f the increase i n student achievement related t o one o r more years o f post-secondary teacher education.

TEACHER ATTITUDES, OPINIONS,A N D W O R K I N G CONDITIONS

When teachers perceive that their pay i s adequate f o r the work they perform,their performance r e s u l t si n an increase o f between eight and ten points i n their students' achievement. The contrary i s the case when teachers perform additional work i n order t o enhance their incomes. Those students whose teachers are i n such a situation obtained a Mean Score o f ten points less i n achievement compared t o those who have a full-timeteacher.

FIGURE 4
VARIATIONS IN SCORES AS A FUNCTION OF VARIABLES ASSOCIATED WITH TEACHER ATTITUDES A N D OPINIONS LANGUAGE MATHEMATICS

-5

-10

-15

L
, ADEQUATE SALARY
LITrLE A U T O N O M Y

~ T E A C H EW RH O P E R F O R M OTHER WORK

FIGURES I NBOLDTYPEA R E STATISTICALLY SIGNIFICAN ATTHE T .05% LEVEL

The fact o f teachers not being able t o carry out their professional a c t i v i t i e sw i t h autonomy i n order t o determine the best strategieswith which t o deal with every-day classroom situationsnegatively (althoughnot significantly) affects student achievement. Autonomy,i n this case,i s taken as the Mean value o f four dichotomous variables: whether the teacher feels free t o cary out his o r her

functions;whether he o r she participates i n work-related decisions;whether the teacher has a role i n the selection o f texts and i n the selection o f audio-visualmaterials t o be used.Moreover,if lack o f autonomy i s combined w i t h the fact that teachers are not satisfied w i t h their salaries, and that they hold other jobs,this combination i s related t o a 27-pointdecrease i n student achievement.

PERCEIVED TEACHER CAUSAL ATTRIBUTIONS

Based upon their experiences,teachers offer reasons t o j u s t i f y success o r failure i n achievements o f their students. I n the Study,seven o f these reasons are considered:family support,s k i l l so f the studentsthemselves,self-esteem, the school environment,school resources,teaching methods, and teacher expectations.
FIGURE 5
SCORE VARIATIONS AS A FUNCTION OF VARIABLES ASSOCIATED T O CAUSES OF OUTCOMES AS PERCEIVED BY TEACHER

30

LANGUAGE

MATHEMATICS

-*O -30

t
FAMILY SUPPORT STUDENT SELF-ESTEEM

I
-22,3
SCHOOL RESOURCES TEACHING M E T H O D S TEACHER EXPECTATIONS

USTUDENT SKILLS SCHOOL CLIMATE

FIGURES I N BOLDTYPEA R E STATISTICALLY SIGNIFICAN A T T H E .05% LEVEL

Teachers also attribute great importance t o student s k i l l s (intelligence, aptitude,creativity,disposition). Both f o r Language and f o r Mathematics,the outcomeso f the Study show that student achievement scores are nearly 21 points higher when teachers feel that these outcomes are due t o the a b i l i t i e so f their students. Nevertheless,it i s important t o stress that similarly,teachers attribute the outcomes o f their students t o their own teaching methods. When the l a t t e r occurs,student achievement i s also higher. This i n i t s e l fc a l l sf o r a change i n mentality and f o r professional practices i n which teachers assume a greater share o f responsibility f o r the effect o f their activity.

Students who achieved better academic performance have as teachers individualswho do not attribute poor student performance t o family conditions.O n the contrary, studentswhose teachers believe that success o r failure i s not their responsibility, but rather that o f their students families have t e s t scores 10 t o 22 points below those o f other students.

CLASSROOMSTRATEGIES
The Study found that the existence o fa f o r m a l assessment system o f studentwork has positive effects on achievement. Those who attend schoolsi n which teacherssystematically control learning obtain outcomes between 4.5 and 5.5 points above those who do not receive ongoing assessment. This c a l l sf o r strategiesthat foster an assessment culture, either i ni n i t i a l teacher training courses o r as a part o f inservice training - communicating t o teachers the evidence found i nt h i s and i n other similar studies.

Nevertheless,o fa l l the variables studied,that which has the largest impacton achievementi s that i n which,according t o student perceptions,there i s a favorable learning environment i n the classroom. Students i n classroomsi n which they l i v e together i n harmony,do not quarrel among themselves,and i n which they establish good friendships, attain between 92and 115 points more than students i n classroomswhere such a climate does not exist. This i s linked t o other a b i l i t i e so r attitudes required by teaching that can condition the transmission o f knowledge and the development o f student s k i l l s .
FIGURE 6

SCORE VARIATIONS AS A FUNCTION OF VARIABLES ASSOCIATED W I T H CLASSROOM STRATEGIES LANGUAGE MATHEMATICS

115.03

120 100

92.07

80 60 40
20

MULTI-GRADE GROUPS SYSTEMATIC ASSESSMENT STUDENTS GROUPED BY AB I LITY ICLASSROOM CLIMATE

O
-20
FIGURES I NBOLDTYPEA R E STATISTICALLY SIGNIFICAN AT T T H E .05% LEVEL

Although i n a limited way,the Study also treated the effect ofthe practice ofgrouping studentsaccordingt o a particdar educational o r Socio-culturalvariable. Such variables can be levels ofprevious achievement, gender,orcultural or ethnic characteristics. Here,however,only the f i r s to ft h i s l i s t was considered. In those schools i n which students are grouped according t o their achievement,studentshave nearly 11 points less than i n those schools i n which such grouping does not occur. A logical suggestion i st o take advantage o f the diversity that exists i n each classroom, given the fact that the greater the heterogeneity,the better the achievement.

THE COMBINED EFFECTOFFORMAL ASSESSMENT A N D NON-GROUPING O F STUDENTS ACCORDING TO SOME EDUCATIONAL OR SOCIO-CULTURAL F A POSITIVE VARIABLE,O N THE ONE HAND, A N D THE ACHIEVEMENT O CLASSROOM CLIMATE O N THE OTHER,RESULTS IN INCREASES O F 108 A N D 131 POINTS MORE IN LANGUAGE A N D MATHEMATICS, RESPECTIVELY. THIS, S THE MOST IMPORTANTVARIATION REVEALED BY THE STUDY. TOGETHER,I

2.4.

The domain of students and theirfamily context

The influence o f the a c t i v i t i e so f parents on the achievement o f their children i s unquestionable.The Study shows,f o r example,that if parents read t o their children,their achievementincreasesby between 3 and 6 points,especially i n Language.The a v a i l a b i l i t ya t home o f ten o r more books that students may consult i s a factor associated w i t h increases i n Language scores (4.55) and s l i g h t l y more i n Mathematics (5.23). The combined effect o f reading frequently t o oneschildren and having books available a t home i s another powerful way t o improve achievement i n school.
VARIATIONS IN SCORES AS A FUNCTION OF VARIABLES ASSOCIATED WITH THE D O M A I N STUDENTS

FIGURE 7

25
20

LANGUAGE

MATHEMATICS

15

1 0

5 O

EDUCATION OF PARENTS GRADUATED F R O M lZTHGRADE AT H O M E 3-4HOURS

PARENTS

I I N V O L V E D PARENTS

BOOKS
P A R E N T S O R G U A R D I A N S PARENTS H E L P =INVOLVED PARENTS READ OFTEN WITH HOMEWORK (SCHOOLLEVEL)

FIGURES IN BOLDTYPE ARE STATISTICALLY SIGNIFICANT ATTHE

.05% LEVEL

Parents helping their children w i t h their homework apparently has a negative effect on achievement. This i s open t o varying interpretations, from supposing that greater help from parents i s a consequenceo f poor student achievement,t o the contrary that l o w achievement i sa result o f the lack o f pedagogical s k i l lo f parents,which s that can i n turn confuse the student. A different case i i t h school i n which parents are actively associated w n such cases,there i s a noteworthy increase activities. i i n student achievement.

3. A C H I E V E M E N T BY STRATA A N D T H E ADJUSTMENT VARIABLES One of the f i r s t findings o f the Study was that o f the differencesi n Mean scores by strata. If, however,these are considered f o r a population group with schoolinglevels o f parents o r guardiansand other family background factors equal t o the regional Mean Score,the strata differences tend t o diminish. If, moreover,one looks a t a group o f students with school processes equal t o the regional Mean Score,the strata differences i n scores not only disappear, but tend t o be reversed.
FIGURE 8

The above,i s depicted i n the following graph.It shows differences i n average scores between strata,taking the Rural Stratum as the reference ( w i t ha value o f O)and including the variations f o r the other strata,using three different models of analysis.The f i r s to f those models incorporates the original observed differences w i t h no correction whatsoever;the second includes the same variations but this time for a hypothetical population o f students w i t h equivalent family context variables and;the third refers t o those variations f o r that same hypothetical population,but t h i s time including an additional condition, being that i t s members also share the condition o f having been exposed t o the same kind o f school processes.

__
SECTOR (RURAL=O)

DIFFERENCES IN M E A N SCORES BY STRATA, W H E R E THE RURAL STRATA 0, W I T H O U T ADJUSTMENT, ADJUSTING FOR FAMILY B A C K G R O U N D A N D ADJUSTING FOR SCHOOL PROCESSES LANGUAGE MATHEMATICS

15

30

25 2 0 15
10

30
20
1 0 0 -10

5 0 -5
-10

~~~~MEGA-CITIE ' .PUBLIC

-20
MEGA-CITIE PRIVATE URBAN PUBLIC MODEL I UNADJUSTED M O D E L II ADJUSTED FOR FAMILY BACKG R O U N D MODEL 1 1 1 ADJUSTED FOR SCHOOL PROCESSES

-30
MODEL I UNADJUSTED M O D E L II ADJUSTED FOR FAMILY BACKGROUND MODEL 1 1 1 ADJUSTED FOR SCHOOL PROCESSES

",I",:

FIGURES I NBOLDTYPEA R E STATISTICALLY SIGNIFICAN AT T T H E .05% LEVEL

This means that low achievement levels detected i n rural schoolsare not due t o the f a c t that they are rural,but rather t o the educationalprocesses prevailing within them.Similarly, achievementdifferencesbetween public schools and private schools tend t o dissipate if we adjust simultaneously f o r context and process variables included i n the model.This l a t t e r fact means that,beyond being different i n the way they are managed,they are also different i n the sociocultural level o f parents and i n the way i n which the educational process i s carried out i n the classroom.
23

This chapter presents two different analytic perspectives o f student achievement i n Language and Mathematics. The f i r s t corresponds t o an analysis o f outcomes within each country f o r the different topics o r themes included i n the tests.The second i s an analysisby achievement l e v e l f o r the region as a whole,i n order t o identify the Mean student achievement f o r each o f the different strata considered.

1. ANALYSIS BY TOPIC
Student performance within each country i n each o f the f i v e topics selected i n Language and the same number i n Mathematics was assessed i n relation t o the performance o f these same studentsf o r the t e s t as a whole. This allows us t o construct an achievement profile f o r each country that summarizesthe relative mastery o f studentsexpressed i n values as:

1) 2) 3) 4 ) 5 )

very high; high; fair; low;and very low.

The topics included i n Language were: identifying kinds o f text;distinguishing the transmitter and recipient o fa text;identifyingthe message o f a text;recognizing specific informationi n a text,and identifying vocabulary related to the meaning o f a text. I n Mathematics,topics included were:numbering;operationsw i t h natural numbers; common fractions; Geometry,and s k i l l s .

It i s importantt o note that i n this analysis, which i s o content analysis,relations are established equivalent t between achievements i n different topics within each country,and that i nt h i s sense,it i s not possible t o generate comparisonsbetween them. Thus,if achievement on a topic f o r a country i s considered high, t h i s means only that it i s so i n regard t o the achievement o f students o f the same country on other topics. Nothing can be said, however,regarding whether such achievement can o r cannot be considered highi n regard t o that o f students i n other countries.

For example,the low outcome f o r the language topic (distinguishbetween the transmitter and the recipient of a text)does not mean the same i n Cuba ( w i t hthe highest Mean Score)as i n Paraguay ( w i t ha Mean Score o f 250).This i s to say that it i s evident that i n Cuba,i n absolute terms,student achievementon t h i s topic i s superior t o the achievementi n absolute terms o f studentsi n Paraguay. But i n both countries,the outcome shows that,o fa l l topics assessed on the Language test,this i s particularly the one t o which attentionshould be given i n the educational process. The outcome f o r Mexico on the same topic i s Very Low, while f o r Venezuela it i s High. But it i s possible that Mexico has higher student achievement i n absolute terms,given that i t s Mean Score i s higher. Nevertheless,Mexico should pay more attention it i t s educationalprocess t ot h i s topic than t o any other.I n other words,the real meaning o f the outcomes are found when a country analyzes them f o ri t s e l f .

T
COUNTRIES ARGENTINA BOLIVIA BRAZIL CHILE COLOMBIA CUBA HONDURAS MEXICO PARAGUAY AVERAGES 277 244
IDENTIFY TYPES OF TEXTS

O
DISTINGUISH TRANSMITTER &RECIPIENT OFATEXT

P
IDENTIFY MESSAGE OFATEXT

I
RECOGNIZE SPECIFIC INFORMATION OFATEXT

IDENTIFY VOCABULARY RELATEDTO THE MEANING OFATEXT

269 272
253

342
230 250 250

DOMINICAN REP. 233 VENEZUELA REGION 242

2 6 1
it.iLi..
HIGH FAIR

0
VERY HIGH

0
VERY LOW

LOW

COUNTRIES ARGENTINA BOLIVIA BRAZIL CHILE COLOMBIA CU BA HON D U RAS MEXICO PARAGUAY

AVERAGES 265 251 263

NUMBERS

OPERATIONS WITH NATURAL NUMBERS

COMMON FRACTIONS

GEOMETRY

SKILLS

254 250
357

230
255 246

DOMINICAN REP. 234 VENEZUELA REGION

233

257
0
HIGH FAIR LOW VERY LOW

0
VERY HIGH

The outcomes permit us t o concludethat treatment o f the f i v e Language and f i v e Mathematics topics does not obey standards that one may consider common f o r the region as a whole,nor f o r most country groups.Although there i s obvious heterogeneity i n the outcomes,i n LANGUAGE there are certain identifiable commonalties.O n the one hand,i n most countries,students tend t o exhibit relatively low o r very low achievement f o r the topics Identify Types of Text and Distinguish the Transmitter and Recipient, thus revealing problems w i t h these skills.O n the other,i n most countries,the a b i l i t yt o Recognize Specific Information of a Text appear as a high-achievement topic. I n MATHEMATICS, the only topic that shows a degree o f homogeneity i s Operations with Natural Numbers, although no country shows i t s performances t o be located a t any o f the two significantcategories,high and low. The greatest discrepancies among countries are seen i n C o m m o n Fractions and Geometry. This may be explained by the differences i n emphasis given t o these topics i n the f i r s t years o f schooling i n each participating country.

2.ANALYSIS BY P E R F O R M A N C E LEVELS

Level 111. READING USING INFERENCE.On this level,students fiil empty

The analysis o f outcomes i n Language and Mathematics concentrated on s k i l l s attained by studentsthrough the study o f performance levels. I n the area o f LANGUAGE, the areas explored centered on s k i l l s developed t o understand through written language. i n MATHEMATICS, the t e s t measured s k i l l s attained t o resolve problems that required studentst o use levels and types o f mathematicalreasoning, and demanding the employment o f mathematical reasoning w i t h growing degrees o f complexity. Similart o the analysiso fs k i l l s ,the analysis by performance levels allows us t o identify the trends o f what a student,o r a group o f students,can o r cannot perform,and shows how the various degrees o f competency that are taught t o children are manifested,giving us a view o f the state o f education i n terms o f both quality and equity.I n order t o carry out t h i s part o f the Study,items from each t e s t were examined i n terms o f their degree o fd i f f i c u l t y and the kind o fs k i l l they require o f students.This allowed us t o define three levels i n each test:
2 . 1

spaces o f the text,explain assumptionsabout structure, l i n k propositions a t the micro and macro-textuallevels, and identify different forms o f relations implicit i n the text. Here,questions require the reader t o relate part o f the text t o a partial theme and t o recognize textual outlines.
2.2

Mathematics performance levels

Level I. RECOGNITION A N D EMPLOYMENT O F BASIC MATHEMATICAL o complete FACTS A N D RELATIONS. Students are able t

customary exercises which require them t o superficially recognize mathematical structures.This level requires working i n elementary mathematical languageand a b i l i t i e s liked t o reading and writing numbers, recognizing geometric figures,identifying simple patterns and carrying out elementary operations.
Level II. RECOGNITION A N D USE OF SIMPLE MATHEMATICAL STRUCTURES.

Language performance levels

Level I. SIMPLE LITERAL READING. This is the most basic and simple

Here w e place those students who are able t o recognize simple mathematical structures.They can carry out both routine classroom exercises as well as simple problem situationsthat require them t o carry out the four basic operations.
Level III. RECOGNITION A N D USE O F COMPLEX MATHEMATICAL o STRUCTURES.On this level are those who are able t

level o f reading,and requires the recognition o f explicit local-level structures:t o identify actors o f a story,the key parts o f the argument,and explicitly stated relations.
Level II. PARAPHRASING OF LITERAL READING. Here we have a greater

degree o f reading complexity that requires a translation o f words regulated by the l i t e r a l meaning of the text. There are questions that ask that the text be described i n other words,without a in-depthinterpretation being necessary.

recognizecomplex mathematical structures. They can carry out common,as well as solve more elaborate procedures, and are able t o solve more complex problems that require knowledge o f the structure o f the decimal system and handling positional values i n order t o establish equivalencies.

As a criterion f o r carrying out the analysis,a base-linewas established that representsthe percentage o f students who should have attained each Performancelevel i n order t o consider that the Level i n question was adequately reached.
f the base-linewas not a s t a t i s t i c a ldecision; Determinationo rather,it i s the result of expert judgement o f the following elements:a complete analysis o f the state o f the a r ti n the teaching o f Language and Mathematics,empirical information,(useo f Item Response Theory (IRT] models, a performance index f o rdifferent l e v e ldata,and consistency analysis),the current state o f Latin American education, as well as aspects related t o the tests themselves. Accordingly,it was decided t o establish base-lineso f 90%, 75%, and 50% f o r levels i, 11, and III, respectively. This means that if s k i l l sthat the t e s t assesses are developing adequately,90% o f studentsw i l l attain Level i, 75% Level II, and 50% Level III. Thus,if a country achieves these outcomes,one may conclude that the performance o f its students i s satisfactory.

29

3. REGIONAL O U T C O M E S
BY D E M O G R A P H I C STRATA

LANGUAGE. Regional Mean scores met satisfactorylevels on the three Performance Levels only i n mega- city schools.It i sf e l t that i n such schools,a satisfactory percentage o f children read,recognize meanings,and understand and interpret information.

I n urban schools,achievementi n reading cornprehension s k i l l s can be considered satisfactory only on the f i r s t level, while a t the two other levels that demand mastery o f more complex skills the situation i s 5% t o 10% below the percentage considered satisfactory. Rural schools exhibit deficiencieson the three levels, especially on the l a s t two, where the difference i n regard t o the value considered satisfactory i s 26% and 19%, respectively.

LANGUAGE REGIONAL LEVEL

FIGURE 11

1 00

100
9 0
8 3

s o
80
7 1 )

7 0

6 0
50

6 0

..........*..../--/.u

54'16

4 4 . 6 3
50
40 30
20 1 0

4 0
3 0

20
1 0
MEGACITY

URBAN

RURAL

MEGACITY

URBAN

RURAL

MEGACITY

URBAN

RURAL

LEVEL I. SIMPLE LITERAL READING

M M M M M M MINIMUM EXPECTED PERCENTAGE

LEVEL II. PARAPHRASING OF LITERAL READING.

LEVEL 111. READING USING INFERENCE.

OVERALL,ONE NOTESTHAT A HIGH NUMBER OF STUDENTS READ WITH ONLY FRAGMENTARY A N D LOCALIZED COMPREHENSION OFTEXTS. THEY IDENTIFY PARTS OFTHE INFORMATION CONTAINED IN TEXTS,BUT LACK READING COMPREHENSION BECAUSE THEY HAVE DIFFICULTY IN ESTABLISHING WHYTHETEXTSAYS WHATITDOES. RATHERTHAN UNDERSTANDING THE MEANING A N D INTERPRETING TEXTS,THEYLEARN MORETO PRONOUNCETHE WORDS ALOUD,OR DECODIFY.THE CHILDREN DECODE, BUT DO NOT K N O W H O W TO LEARN THROUGH READING.

MATHEMATICS. Insufficiencieshere are greater,since i n no s t r a t aare minimum expected l e v e l s attained on Performance Levels II and III, while on Levei I satisfactory levels are achieved only i n schools located i n mega-citiesand urban areas.For Level II, the deficiency varies between 21% f o r the mega-city s t r a t aand 35% f o rthe r u r a l strata.Deficiencies f o r Level III reach 32% f o r the mega-c i t y strata and 38% f o r the rural strata.

MATHEMATICS

- REGIONAL LEVEL

FIGURE 12

- 100

. . . . . . . . . . . . . . . . . . . .

80

70

. . . . . . . . . . . . . . . . . . . .
,

60
50

MEGACITY

URBAN

RURAL

MEGACITY

URBAN

RURAL

MEGACITY

URBAN

RURAL

LEVEL I

RECOGNITION ANDEMPLOYMENTOFBASIC MATHEMATICAL FACTS AND RELATIONS

111111 MINIMUM EXPECTED PERCENTAGE

I F V. E L II . . . . . RECOGNITION AND USE OF SIMPLE MATHEMATICAL STRUCTURES.

__

LEVEL III. RECOGNITION AND USE O F COMPLEX MATHEMATICAL STRUCTURES.

OUTCOMES INDICATE THAT STUDENTS CAN RECOGNIZE NUMBERS, MATHEMATICAL SIGNS,A N D NUMERICAL A N D ORDER RELATIONS AT A N ELEMENTARY LEVEL,BUT THE MAJORITY D O NOT POSSESS SATISFACTORY MASTERY IN ORDERTO SOLVE MATHEMATICAL PROBLEMS,WHETHER SIMPLE OR COMPLEX.

4.REGIONAL O U T C O M E S FOR
PUBLIC A N D PRIVATE SCHOOL

Comparing schools by type o f administration again reveals differences, w i t h a slight advantage f o r the private sector. Both i n Language and i n Mathematics performance i s r e l a t i v e l y similar f o r both s t r a t a on Level i, with achievement above the l e v e i considered t o be satisfactory. I n Language, f o r PerformanceLevels II and II, studentsattending private schools achieve performance rates that are approximately 1 0 % above those f o r public schools. I n Mathematics,the difference i n favor o f private school students i s near 8 %f o r Level II items (recognitionand use o f simple mathematical structures).Nevertheless,both public and private schools are well below the minimum expected percentages.The same i s true f o r the case o f Performance Level III (recognitionand use o f complex mathematical structures),where student achievement i n both types o f schoolsi s 35% below the minimum expected percentage.

LANGUAGE REGIONAL LEVEL

FIGURE 13

LOO
95

90
85 80 75 70 65 60

55
50

PUBLIC

PRIVATE

PUBLIC

PRIVATE

45 40
PUBLIC PRIVATE

LEVEL I. SIMPLE LITERAL READING.

M M M M I B MINIMUM EXPECTED PERCENTAGE

LEVEL II. PARAPHRASING OF LITERAL READING.

LEVEL 111. READING USING INFERENCE.

32

_ I

'

100

- 90
___~.

- 80

5614 48,20

. . . . I . . . . . . . . . . . . . -

,~
PUBLIC

PRIVATE

LEVEL I. RECOGNITIONAND EMPLOYMENTOF BASIC MATHEMATICAL FACTSAND RELATIONS.

LEVEL Il. RECOGNITION AND USE O F SIMPLE MATHEMATICALSTRUCTURES.

LEVEL 111. RECOGNITION AND USE O F COMPLEX MATHEMATICAL STRUCTURES.

70 60

--

SO

- 40 - 30
20

1 4 . 9 4

15,Ol

10

PUBLIC

PRIVATE

111111 MINIMUM EXPECTED PERCENTAGE

I n short,the analysis o f learning outcomes i n both Language and Mathematics reveals a clearly deficient situation. Except for Cuba,most third and fourth grade students who participated i n the Study have not developed s k i l l s according t o expectations,remaining on a basic level o f recognition o f signs and structures o f Language and Mathematics.This shows,i n varying degrees,a deficient development of communicationand problem-solvingskills.

I n other words,students learn t o read,but have d i f f i c u l t i e s understanding the meaning o f what they read and interpreting texts.They learn numbers,numerical relations, signsand structures, but are unable t o solve simple problems. Nor can they extrapolate the applicationo f mathematics t o everyday situations.

This chapter has two basic objectives.The f i r s ti st o offer af i n a l synthesiso f the conclusions o f the Study,t h i s time o the magnitude o f presented hierarchically,according t the relative individualeffect that each associated factor proved t o have upon achievement i n Language and Mathematics i n the third and fourth grades,and w i t ha s t a t i s t i c a l significance level o fa t least 0.05%.

It should be emphasized here that the analytic s t a t i s t i c a l instrumentemployed was that o f HierarchicalLinear Models (HLM) which i s based,i n turn,upon general regression models,but incorporates the condition o f carrying out analysis on more than one level o f aggregation a t once. i n the present case,the aggregation levels were basically two:students and schools.The outcomes presented below correspond t o the l a t t e r .Finally,it i s important t o note that the analysis presented here,as an expression o f multiple regrescion models,simultaneously incorporates a l l the variables o f the regression equation simultaneously. Thus,we repeat that t h i s analysis presents score differences i n each variable f o r a hypothetical group o f students that corresponds t o the Mean Socio-CulturalStatus i n the region.
The second objective o f this chapter i st o treat questions that,due t oi t s design,scope,o r non-availability o f resources, the present Study was unable t o answer,and which thus remain themes f o r future research.

1. CONCLUSIONS
The magnitude o f the relation between Associated Factors and Achievement i n Language and Mathematics i s operationally defined i nt h i s chapter as the number o f units i n the standardized scale adopted for achievement (Mean=250points;Standard Deviation=50 points) o f increase o r reduction due to the effect o f the presence o f a specific variable (associatedfactor),correspondingt o the increment o f one unit on the scale o f that variable.For greater c l a r i t y ,operationaldefinitions ( f o r mo f measurement) o f most o f the variables mentioned i n this chapter are presented i n Appendix 2 - Analysis o f Variables.

Next i s the variable,INVOLVEMENTOF PARENTS OR GUARDIANS (SCHOOL LEVEL).Analysis reveals that f o r each additional point i n the scale i n which parents say that they are involved i n the school i n which their children study, student achievement increases by 21.11points i n Language and by 14.98 points i n Mathematics.

II Next i n importance i n terms o fi t se f f e c t on achievement

i S FAMILY S U P P O R T AS A CAUSE ATTRIBUTED FOR STUDENT O U T C O M E S , AS P E R C E I V E D BY TEACHERS. In t h i s case,the

It should be mentioned beforehand that the t o t a l percentages o f variance explained by the model,w i t h i n schools,are discreet,and reach 13.5% for Language and 15.6% f o r
Mathematics.In contrast,t o t a l percentages o f variance explained between schools are notably higher,reaching 68.3%i n Language and 61.3%i n Mathematics.

outcome i s the opposite direction from those seen t o date for the r e s to f the variables.It i s interestingt o note that each point i n which teacherssee studentoutcomes as attributable t o family support correspondsto a decrease o f 10.09 points i n Language and 22.30 i n Mathematics. The e f f e c to f GRADE LEVEL as an Associated Factor shows that being i n the fourth grade i s related to 18.80more more points i n points i n Language, and 15.30 Mathematics than being i n the third grade.

CLASSROOMCLIMATEi s the single variable that demonstrates the greatest positive effect upon both i n Language and i n Mathematics. achievement, I n fact,a value o f 1f o rt h i s variable,meaning a student appraisal o f the climate as "satisfactory", corresponds t o a 92.07 point increase i n Language and 115.03 points f o r Mathematics.It i s interesting t o note that, i nt h i s case,the effect corresponds to almost exactly two (2) standard deviations o f increasei n achievement.

W W e next have the variable TEACHER HASA N ADDITIONAL

exercise another professional a c t i v i t y besides that o f teaching i n the school i n which they were interviewed,student achievementi s 11.20points l e s si n Language and 9.7 1 points less i n Mathematics than for those students whose teachers work exclusively i n the school i n which they were interviewed. The next i n this l i s t i n go f effects o f variables on achievement i s that called STUDENTSG R O U P EBY D ABILITY. This points t o the practice o f segregating students i n school according t o some variable - i nt h i s case their own a b i l i t i e s .Analysis shows that,as the variable moves toward greater homogeneity o f groups o f students i n the classroom, f o r each 0.5 point i nt h i s direction there are reductions o f 10.35points i n Language achievement and 11.64 points i n Mathematics f o r students under these conditions.
JOB. I nt h i s case,f o r teachers who

W The variable that has the second greatest effect,also

positive,i s STUDENTABILITY AS A CAUSAL ATTRIBUTION FOR SCORE OUTCOMES, AS PERCEIVED BY TEACHERS. Analysis shows that each additional score point i n which teachers attribute student achievement t o Student Ability corresponds to a 21.01point increase i n Language achievement and a 21.59point increase i n Mathematics achievement.

I W e next have the variable,SIZEOF LIBRARY. The results

show that studentsenrolled i n schools that have more than 1,000 books i n their libraries earn 10.40 more points i n Language and 9.90 more points i n Mathematics than those attending schools that have less than this number o f books i n their libraries. In the case o f the variable ADEQUATE SALARY,which f teachers regarding whether reflects the judgement o their salariesare sufficient.Where teachers judge their salariest o be sufficient,student achievement i s 7.63 points higher i n Language and 9.59 points higher i n Mathematics than for studentswhose teachers do not s o indicate when asked about their salaries. The variable,GENDER, also has an impact.Being a g i r l i s related to obtaining 6.04 more points more than boys i n Language,and 1.79 points less than boys i n Mathematics.

The next variable i n terms o f influence on achievement i s SOCIO-ECONOMIC LEVELOFTHE SCHOOL. This i s an estimated variable o f the Social-economicLevel o f the School as perceived by the schools Principal.A one-unit increase i nt h i s variable i s related t o increases o f 5.64 points i n student Language achievement and o f 5.88 points i n Mathematics.

R Similarly,the variable,IO OR MORE BOOKS has an impact

on achievement.In t h i s case,we consider households that have 1 0o r more books.Achievement o f children who l i v ei n households which have this number o f books i s 4.54 points higher i n Language and 5.20 points higher i n Mathematics than f o r those from households not having t h i s number o f books.

I Another variable that shows a significant effect i s

GRADUATED F R O MG R A D E 12,which identifiesthe education l e v e lo f parents o f a particular student as being a t grade 12 or higher.The effect o f parents o f students having completed a t least 12 grades o f school i s related t oa studentachievement o f 3.87 points higher i n Language and 3.63 points higher i n Mathematics than those students whose parents do not have this level o f schooling.

R Another achievement-related variable i s PARENTSO R

G U A R D I A NREAD S FREQUENTLY. This refers t o cases i n which parents o r guardians read t o children every day. In the case o f parents who do so,their children have 5.88 more points i n Language,and 4.31 more points i n Mathematics than the children o f those parents o r guardians who do not.

I The variable denominated TEACHER OVER-BURDENED i s

o how the teacher responds when scaled according t r she has o r does not have arduous asked if he o f teacherswho do not report working hours.Students o arduoushours achieve 5.78 more points i n Mathematics than students o f teachers who do s o report.There i s no Significant relationship i n the case o f Language outcomes f o r any of the alternatives.

Next i s the impact o f the variable entitled PARENTSO R GUARDIANS SOMETIMES READ, defined as a c h i l d being read t o more than once per month.Forthose studentswhose parents sometimes read to them,achievement i s 3.14 points higher i n Language and 2.96 points higher i n Mathematics than those students whose parents do not do so.

I Another variable that shows a relationship w i t h

achievement i s TEACHER TRAINING. This variable i s defined as the number o f years o f post-secondarytraining o f teachers.Each year o f training that teachers have corresponds t o an increase i n scores o f their students o f 2.44 points i n Language and 2.06points i n Mathematics.

W A variable that exhibits a peculiar relationship w i t h

achievement i s that entitled PARENTSO R GUARDIANS HELP WITH HOMEWORK. For those students whose parents help them with their homework there i s a reduction o f 2.2 1 points i n Language and 2.73points i n Mathematics,as compared t o those studentswho do not receive help w i t h homework from their parents.

H The next variable,listed i n order o f relative impact,i s

EDUCATION OFPARENTSOR GUARDIANS, defined as the Mean value o f the number o f years o f schooling o f parents and guardians.Each year o f increase i nt h i s variable correspondst o an increaseo f 0.97 points i n Language achievement,and 0.81points i n Mathematics achievement.

II Another variable i s INVOLVEMENT

(CLASSROOM LEVEL).Here we

RATIO shows an OFPARENTSOR GUARDIANS W The variable entitled STUDENT-TEACHER INVERSErelationship with achievement.The data show see that each unitary that an increase i n the number o f studentsper teacher increase i n the involvement o f parents a tt h i s level correspondst o a decrease of0.49 points i n Language correspondst o an additional 1.82 points i n Language achievement and o f 0.53 points i n Mathematics achievement and 2.22 points i n Mathematics achievement. achievement for their children.

I The next variable i s ATHOME~ T O ~ H O U R which S, receives

a maximum score if the parent o r guardian stays a t home f o r3o r 4hours during working days.A positive unit f o r this variable corresponds t o an increase o f 2.00 points i n Language achievement, w i t h no corresponding significantincrease f o r Mathematics scores.

2.IMPLICATIONS FOR

FUTURE RESEARCH

As i s the case i n studies o ft h i s nature,i t s findings can

form the basis f o r new investigations, whether o f the same themes o r on other themes based on new information furnished by the Study.Although the Laboratory i s currently committed t o the development o f a Qualitative Study o f Schools w i t h Outstanding Outcomes,i n seven countries o f the region,thanks t o the support o f the Ford Foundation, i n order t op a r t i a l l y fill the need f o r answers t o such questions,a large number o f these w i l l require research t o be undertaken by other entities.It i sf o rt h i s reason that the present chapter has sought t o identify questions that seem most relevantand which are presented as suggestions.

I Next i s the variable,INSTRUCTIONAL

MATERIALS. Outcomes show that for each additional element o f equipment, f 1.96 there i s an increase i n Language achievement o points for those students enrolled i n schools that offer these conditions.There i s not significantrelationship i n the case o f Mathematics.

The next variable i s that termed BI-PARENTAL. This describes whether the family has one o r two parental figuresi n the household.The presence o f two parents i s related t o an increase o f 1.89 points i n student Language achievement.There i s no significant relationship i n the case o f Mathematics.

Af i r s t aspect arises from findingsi n regard t o variations

i n achievementi n Languageand Mathematicsi nd i f f e r e n t strata,particularly demographic strata (mega-cities, urban,and rural areas).These seem t o suggest that achievement i s not necessarily linked directly t o strata characteristicsper se,as had been suggested i n the literature,but rather that achievementi s related t o other variables,o r through others such as social-economic Characteristicso f students,school pedagogy,classroom practices,o r the interactions between these variables.

It would be interesting t o explore the relation o f other strata,especially those o f a demographic nature,and the characteristicsand practices cited above.

E A theme o f key importance i st o look more deeply into

the relationship between achievement and the chronologicalage o f students.This was touched upon i n the Study by the variable Rate o f Progress. Moreover,t h i si s a subject that reveals a great heterogeneity o f behavior between different countries and which has implicationsas well f o r education policy.

significantly related t o student achievement scores.To this we should note that in-serviceteacher training i s an education improvement t o o l that i s widely used i n o the Region,and one that i s certainly susceptiblet education policy decisions.

E Two lines o f questioning naturally emerge f r o m results

on the indicator entitled Classroom Climate.The f i r s t concerns confirming the percentageso f explained variance,which,given their magnitudes,certainly warrant confirmation,both i n terms o f the contribution o f the indicatoras a whole,as well as those o f individual variables.The second l i n eo f questioning concerns exploration o f the relationships between School r combinations o f Achievement and other variables,o variables, that operationalizethe constructSchool Climate.It seems important that such variables be chosen both from those that stem from the perceptions o f studentsas well as from the perceptions o f teachers o r principals.

E A n additional important aspect t o investigate i s that

related t o the quality o f library materials.The present f materiais i n school Study looked only a t the quantity o libraries,and only i n a dichotomous sense (more o r less than 1,000 books). It would be interesting t o examine the effect o f the quality o f such materials on achievement,since the independent variable,i nt h i s case,i s sensitive t o education policy decisions i n the Region and i s ,moreover,one o f the few input variables that the Study identified as significant.

E It i s important t ol o o k closely a t examples o f outstanding

achievement.This i s the principal subject o f an investigation already underway by the Laboratory.A t least three areas seem t o be o f particular interest.The f i r s to f these i s Cuba,which,given the magnitude o f studentperformance i ni t s case,may representa model f o r the rest o f the Region.The second i s that o f rural schools i n Colombia,which i n some cases showed better performances than urban schoolsi n that country. These may serve as an important example o f putting learning-focused educational policies into practice. The third area i s that o f Urban Strata schools i n Chile, which i n some cases demonstrated better achievement than schoolsi n mega- cities,and which may be an example o f the effects o f administrativepolicies on education variables.

E Another theme worth exploring i s that o f the d i f f e r e n t i a l

effect o fi n i t i a l and in-service teacher training on the product variables.According t o the findings o f the Study,there i s a significant,positive relationshipbetween student achievement and post-secondarytraining o f their teachers.This i s not the case,however,f o r inservice teacher training.It would be interesting t o look simultaneously a t the relation between b o t h independent variables and achievement i n order t oc l a r i f y doubts i n regard t o individual effects,as well as t o possible effects that t h e i r interactionsmay have on student achievement.

E Furthermore,there i s a need t o study the contribution

o f in-serviceteacher training toward explaining student achievement from the perspective o f the quality of differenttypes o f such training.This topic i s extremely relevant,given the fact that the only thing the present Study concludeshere i s that the usual in-servicetraining currently u t i l i z e di n the different countries i s not

I The Study provides some evidence that school and

classroom-relatedvariables seem t o have a greater effect on Achievementthan those linked t o the macro l e v e lo f the system. it also appearsthat process variables explain a higher percentage o ft e s t score variance than input variables.Both focused and general research are necessary i n order t o confirm,reject,o r complement the findings o ft h i s Study. The theme o f the relation between financialinvestment i n education i n the presence o f other process variables would seem t o be an area f o r future research.The f those present Study,although treating the effect o variables that are closely related t o what normally constitutesa large part o f spending on education,does not include the theme o f economicresources i n a direct and deliberate manner.The subject o f financing i s undoubtedly central i n education policy decision-making.

I Although variables related t o the pre-schooleducation

o f studentsdid not reveal a significant relation t o achievement i nt h i s Study,the effects o f pre-school experience,according t o relevant literaturei n the f i e l d , appear t o be considerable.For t h i s reason,the contribution o f these variables t o school achievement needs t o be studied,particularly considering the implicationsthey have f o r education policy.

Governments,institutions,and individuals have made this Second Report possible. Among the f i r s t are the Ministries and Departments o f Education o f the countriesthat comprise the Latin American Laboratory f o r Assessment o f Quality i n Education (Argentina, Bolivia,Brazil,Chile,Colombia,Costa Rica,Cuba, Dominican Republic,Ecuador,E l Salvador,Guatemala,Honduras,Mexico,Panama,Paraguay,and Venezuela)and i n particular those who participated i n the Study and which have assumed the most important part o fi t s financing. The generosity o f Brazil and Chile merits special mention f o r having become underwriters of the Laboratory. Among internationalorganizations, the Laboratory has benefited from the financial and technical support o f UNESCO, the Inter-AmericanDevelopment Bank,the Ford Foundation,the Andrs Bello Agreement Secretariat,Government o f Spain and Fundacin Andes. In the l a s t part o f the Study,we recognize the decided support given by the Senior Technical Committee o f the Laboratory i t s e l f ,composed o f some o f the above institutions, as well as the OECD, IEA, ETC.and the World Bank. The Measurement and Assessment Systems o f the countries that participated i n the Study generously placed their experience,resources,intelligence,and good-will a t the disposal o f the Laboratory,and were the key element o f the research. National Coordinating Groups were l e d by Hilda Lanza and Lucrecia Tulic (Argentina); Mara Ins Gmez de S Pestana (Brazil); Susana Barrera (Bolivia);Mara Ins lvarez,Josefina Olivares,and Ivn Ortiz (Chile);Hctor Fernndez (Colombia); Hctor Valds (Cuba);Leonte Ramrez and Julio Valeiron (DominicanRepublic),Judith Barahona and Cristin Rodriguez (Honduras);Marta Lafuente and Juana Delms (Paraguay);Victor M.Velzquez (Mexico); and Yamila Nadales,Asmara Anderson,and Nelly Chacn (Venezuela). This Second Report o f the InternationalStudy received the support o f a number o f officials,specialists and consultants. Among these were Nigel Brookes,Lesbia Cnovas,Rolando Castaeda,Rubn Cervini,Gustavo Cuadra,Mara Helena Guimaraes D e Castro,Claudio D e Moura Castro,Mara del Carmen Daz,Viola Espinola,Martha Grijalva,Sandy Gutkowski,Yetil Lunge de Baessa,Ricardo Hevia,Noel McGinn,Hctor Muoz,Scott Murray,Sergio Prenafeta, Pedro Ravela,Wilma Santa Mara,Mara Alejandra Schulmayer,and Larry Wolff.

Very special thanks t o Doug Willms,f o r the development o f the Hierarchical Linear Analysis;Richard Wolfe,f o r his support i n the areas o f sampling and s t a t i s t i c a l analysis;Carlos Pardo,f o r his participation i n the analysis by Topics and by Achievement Levels;Martha Castillo and Gloria Inostroza i n Language,and Claudia Salazar and Irene Villarroel i n Mathematics. The Study was carried out by the Latin American Laboratory f o r Assessment o f Quality i n Education. It was directed by the Regional Coordinating Group o f the Laboratory and by i t s National Coordinators. The Regional Coordinating Group,w i t h headquarters a t OREALCLJNESCO i n Santiago,Chile,was composed of Juan Casassus,Coordinator o f the Study,Sandra Cusato,Juan Enrique Froemel,Maite Gonzlez,and Juan Carlos Palafox. Data were processed in-countryand by the Regional Coordinating Group. The analysis benefited from the support o f the Department o f Statisticso f the University o f Ontario,Canada,the School o f Education o f the University o f New Brunswick,and the Colombian Institute f o r Support o f Higher Education (ICFES). The authors express their gratitude t o those who made valuable contributionsto the Report,while making it clear that responsibility f o r what i s expressed therein i s entirely their own.

LANGUAGE
, ADJUSTED MEAN SCORES

MATHEMATICS 261 -1.79 15.30 0 . 8 1 3 . 6 3 1 . 8 9 1.36 5 . 2 0

262 6.04 18.80 0.97 3.87 0.75 2 . 0 0 4.54

GENDER (M-F)
GRADE

LEVEL (GRADE 4-GRADE3)

SOCIO-CULTURAL STATUS Education Level o f Parents o r Guardians r o m Grade 12 Graduated f Two Parents A t H o m e 3t o 4 Hours Ten o r More Books PREVIOUS EDUCATIONAL EXPERIENCE Attended Pre-School Pre-SchoolNot Available Non-ValidData Parents o r Guardians Read Often Parents o r Guardians Read Occasionally Parents o r Guardians Help With Homework IN-SCHOOL RESOURCES Teacher/Student Ratio TSR Squared Infrastructure Instructional Materials Size o f School Library (>1,000 Books) Teacher Experience Teacher Education In-serviceTeacher Training

1.71 -2.58 -8.20 5 . 8 8 3 . 1 4

-2.12

0.75 -0.79 -7.29 4 . 3 1 2 . 9 6 -2.73

-0.53 0.00 0.56 0.22 9.90 0.21 2 . 0 6 0.55

-0.49 0 . 0 1 0.66 1 . 9 6 10.40 0.02 2 . 4 4 0.22

NOTE: FIGURES IN SOLDTYPE ARE STATISTICALLY SIGNIFICANT ATTHE .O5LEVEL

LENCUACE
SCHOOL CULTURE Teachers Hold Other Jobs Teacher Attitudes Adequate Salary Leadership o f Principal Working Conditions Job Satisfaction Teacher Not Over-burdened Autonomy o f Teacher Autonomy o f Principal

MATHEMATICS

-11.20 7.63 5.61 -7.71 -0.90 -4.31 -9.30 -3.13

-9.71 9.59 4.07 0.03 -3.57 -5.78 -9.32 -4.80 -22.30 21.59 -5.84 -1.40 5.54 3.99 0.39

TEACHER-ATTRIBUTED CAUSES OF STUDENT ACHIEVEMENT Family Support -10.09 Student Ability 21.10 Student Self-Esteem -0.31 School Climate -8.96 School Resources -0.93 Teaching Methods 3.85 Teacher Expectations 1.61
CLASSROOMPRACTICES Multi-gradeGroups Systematic Assessment Students Grouped by Ability Parents and Guardians Involved (classroomlevel) Parents and Guardians Involved (schoollevel) Classroom Climate

-4.46 5.58 -10.35 1.82 21.11 92.07

5.64 13.6 68.3

-5.49 4.59 -11.64 2.22 14.98 115.03

5.88 15.6 61.3

SOCIO-ECONOMIC LEVEL OF SCHOOL

PERCENTAGE OF VARIANCE

EXPLAINED

In-school Between Schools

NOTE: FIGURES IN BOLDTYPE ARE STATISTICALLY SIGNIFICANTATTHE .O5LEVEL

W e describe i n detail below the constructs o r indices and the variables used i n the analysis o f Associated Factors.

GENDER Coded O f o r boys and 1 f o r girls,w i t h a Mean Score o f 0.5

GRADE Coded -0.5 f o r Grade 3 and 0.5 f o r Grade 4

EDUCATION OF PARENTSCreated from a question that asked for the education level of the guardian and spouse,rather than the years o f education. A continuousvariable was created f o r each parent,and then the variable was created from the Mean Scores o f the two.The Regional Mean Score o ft h i s variable i s 9.3 years o f education.
COMPLETE SECONDARYCoded 1 if the education ofthe parents o f a given child was 12 years o r more (graduatedfrom secondary school)and O if it was not. The Regional Mean Score i s 0.2277.

Two PARENTSCoded 1 if the child has two parental figures a t home,independently o f whether they are married o r not,and O f o r any other case. The Regional Mean Score i s 0.7789.
3 OR 4 HOURS I NT H E HOUSEHOLD Coded 1 if the responding parent o r guardian spends 3t o 4hours i n the household during work days o f the week (notconsidering hours o f sleep)o r only time between shifts,and O if the case i s otherwise. The Regional

Mean Score i s 0.2258.

1 0ORMORE BOOKS coded 1 if there are 1 0o r more books i n the childs household s 0.4200. and O if there are not. The Regional Mean Score i

SCS (SOCIO-CULTURAL STATUS) A compound measure o f the socio-cultural level o f children,coded from ParentsEducation, Two Parents, 3 or 4 Hours in the Household, and 10 or More Books using factor analysis.It was then standardized i n order t o obtain a Mean Score o f O and a standard deviation o f 1. MEGA-CITY, PUBLIC; MEGA-CITY, PRIVATE; URBAN, PUBLIC; A N D URBAN, PRIVATECoded 1 if the school f e l l within the relevant classification, and O if it did not,w i t h rural schools being the reference category.Regional mean scores were then calculated f o r these variables. They are,respectively, 0.1252, 0.0878, 0.3195, and 0.1189.

STUDENT ATTENDED PRE-SCHOOLCoded 1 if the student was i n some form o f pre-school program before the f i r s t grade,and O if t h i s was not the case. The Regional Mean Score i s 0.7412.
There was,however,much missing i nt h i s variable. Therefore,following the procedure recommended by Cohen and Cohen (19821, missing data f o r Student Attended Preschool were placed a t the variablesMean Score (0.7460) and a dummy variable was constructed which identified respondentswith missing data f o rt h i s variable (missing = 1 and O f o r the contrary). Using t h i s technique,the analysis produced estimates o f the effect o f pre-schoolf o r those who did not have v a l i d data,and estimates o f the differences i n achievement scores between those who had data and those who d i d not.
P A R E N T / G U A R D I AREADS N FREQUENTLY Coded 1 if the guardian read t oh i so r her children almost every day when they were young and O if the contrary was the case. The Regional Mean Score i s 0.3641. P A R E N T / ~ U A R D IREADS A N SOMETIMES Coded 1 if the guardian read t oh i so r her children more than once per month but not every day,and O if the contrary was the case.The Regional Mean Score i s 0.2813. STUDENT/TEACHER RATIO Constructed by dividing a schools enrollment by the number o f teachers i n the school.The Regional Mean Score i s 27.3196.

INFRASTRUCTURE Constructed by counting the number o f infrastructureresourcesfound i n the school.The Regional Mean Score i s 3.5860.
20 TO 999 BOOKS IN THE LIBRARY A N D M O R E THAN 1,000BOOKS Coded 1 if the school library had the relevant number o f books i n the corresponding range,and O if it did not,with the reference category fewer than 20 books. Regional averages are 0.2725 and 0.3375.

TEACHER EXPERIENCE Expressed as the number of years.The Regional Mean Score i s 13.1571. INITIALTEACHER TRAINING Expressed i n years.The Regional Mean Score i s 3.4587.
IN-SERVICE TEACHER TRAINING The number o f training courses taken within the l a s t three years.The Regional Mean Score i s 4.799

TEACHER HASA N ADDITIONAL JOB Coded 1 if the teacher has a job other than teaching i n the school,and O if the contrary i s the case.The Regional Mean Score i s 0.2034. ADEQUATE SALARY Coded 1 if the teacher i s satisfied with h i so r her salary,and O if not.The Regional Mean Score i s 0.2052.

LEADERSHIP OF THE SCHOOLPRINCIPAL Constructed from the Mean values o ff i v e dichotomous (yes=1,no=O) variables, indicating whether principais (a)motivate the work o f teachers,(b)value the work o f teachers,(c)create a flexible educational environment,(d)if teachers have confidence i n the principals expertise,and (e)if teachers feel involved i n the school.The Regional Mean Score i s 0.8119.
WORKING CONDITIONS Constructed using the Mean values o f three dichotomous (yes=l, no=O)variables, indicating if the school provides a safe working environment,if there i s a clear definition o f the roles of teachers and principals,and if the teacher feels isolated from his o r her colleagues.The Regional Mean Score i s 0.6486.
JOB SATISFACTION Constructed using the Means scores o f four dichotomous (yes=l, no=O)variables indicating if the teachersi n i t i a t i v e s are supported by colleagues,if the teacher i s respected by his o r her students,if colleagues consult the teacher regarding teaching materials,and if the teacher enjoys teaching.The Regional Mean Score i s 0.8616.

TEACHER OVER-BURDENED Coded 1 if the teacher does not have an arduous schedule, r she does.The Regional Mean Score was 0.3660. and O if he o AUTONOMY Constructed from the Mean values o f four dichotomous (yes=l, no=O) variables,denoting if the teacher feels free t o carry out his o r her functions,if the teacher participates i n work decisions,i n the selection o f texts,and i n the selection o f the audio-visual materials that he o r she uses.The Regional Mean Score i s 2.4053. AUTONOMYOF PRINCIPAL Constructed from the Mean values o f nine other variables. The f i r s t indicates if the school has complete (=3), partial (=2), o r no (=i)autonomy. The others indicateif the principal has complete (=3),p a r t i a l (=2),o r no (=i)autonomy i n various administrative tasks he o r she carries out,such as budgeting,discipline,and text selection.The compound construct has a Regional Mean Score o f 2.4053. AITRIBUTION OFCAUSE OF RESULTS Constructed using two questions. One asked teachers t o which o ff i v e factors they attributed the academic problems o f their worst students. The other asked about the success o f their best students.A variable was f i r s t created f o r each one o f the response alternatives o f the questions,counting the number o f times that each response was given by the teacher.Factor analysis was then used t o determine the best way t o group the multiple responses into a few more general variables that indicate family support,student a b i l i t y ,student self-esteem, school climate, school resources,and teaching methods.A Mean Score was calculated f o r each one o f these s i x more general constructs. Regional averages were then calculated.They are 0.4734, 0.3544, 0.3193, 0.2839, 0.1922, and 0.4561, respectively.

47

TEACHER EXPECTATIONS Coded from 1 to 5, denoting the different levels o f education that teachers believe that their studentsw i l l reach. The Regional Mean Score i s 2.8762. MULTI-GRADE CLASSROOMCoded 1 if the instructor teachesw i t h many grades present i n the same classroom,and O if t h i si s not the case.The Regional Mean Score i s

0.1372.
STUDENTSEXAMINED Coded 1 if the teachers assesses students using tests,and O if he o r she does not.The Regional Mean Score i s 0.3767.

STUDENTS GROUPED BY ABILITY Coded 1 if the teacher groups students i n a uniform manner according t o learning ability,gender,age,or ethnicity;0.5if students are not grouped o r if an unspecified criterion i s used;and O if students are grouped uniformly by learning ability,gender,and age. The Regional Mean Score i s 0.2464.

PARENTAL INVOLVEMENT Constructed from the Mean values o f other parental variables: if the parent participates i n school-related a c t i v i t i e s (1 = rarely,2=sometimes,3=always), if the parent knows the teacher o f his or her child (1=no,2= a l i t t l e ,3= very well), and if the parent participates i n parent and guardian meetings (i= never or rarely,2=
almost always,3 = always). Parent involvement was calculated for each school. The Regional Mean Score i s 2.5352.
CLASSROOMCLIMATEConstructed from the Mean values o f three other variables denoting:whether there are studentsi n the class who bother others (1 = no,O = yes), if quarrels occur frequently (1= no,O = yes), and if students i n the class are good friends (1= yes,O = no). This discipline- related construct was then calculated f o r each school.The Regional Mean Score i s 0.5995.
INSTRUCTION TIME DEVOTEDTO LANGUAGE A N D INSTRUCTION TIME DEVOTEDTO MATHEMATICS Constructed by multiplying the number o f sessionso f mathematics o r language taught by the teacher during the school week by the duration (inminutes) o f the class o r session.The Regional Averages o f the variables are 293.2590 and 269.4816, respectively.