Mastering Metrics

Illuminating the Path from Cause to Eect

Joshua Angrist
MIT and NBER

EMPIRICAL STRATEGIES: Spring 2013

We are here to make another world.
- W. Edwards Deming
The Evaluation Challenge

Policy-makers and social scientists ask many causal questions

Labor economists ask what happens to earnings when we:
Subsidize worker training or boost schooling
Change labor market institutions (such as minimum wages and
employment protection)
Criminologists ask what happens to recidivism when we:
Change law enforcement protocols
Change sentence length
Education researchers asks what happens to achievement or schooling
as a consequence of changing
Class size
Incentives and compensation
Data and econometric imagination generate counterfactual worlds
Our econometric toolkit includes experiments, IV, DD, and RD
Often (more often than not), we just run regressions
Naive Comparisons . . . are often misleading
Does going to the hospital make you healthier?
Average health (assigning a 1 to poor health and a 5 to excellent
health) contrasting those who have been an inpatient in the past 12
months and those who have not (tabulated from the 2005 NHIS):
Group Sample Size Mean health status Std. Error
Hospital 7774 3.21 0.014
No Hospital 90049 3.93 0.003
This shows a large (and highly signicant) contrast in favor of the
non-hospitalized.
Taken at face value, this comparion suggests that hospitalization
makes people sicker. So why do so many go?
Maybe, its the food
But those who seek treatment were sicker in the rst place (of
course!)
A naive contrast indeed, but not unsual
Experiments Eliminate Selection Bias

Every day we hear of association between diet/lifestyle and

health/morbidity
Selection bias is pervasive in such comparisons
The genius of two-time Nobel laureate Linus Pauling notwithstanding,
Vitamin C does not protect you from the common cold or cancer
Women who used Hormone Replacement Therapy are indeed healthier,
but now we know: HRT is dangerous
How do we know? Convincing answers come from randomized trials
Pre-treatment dierences contaminated Paulings evidence on Vitamin
C and cancer
When people are randomly selected for treatment, subjects in treated
and non-treated groups are ex ante similar
Random assignment guarantees apples-to-apples comparisons
In social science as in health
A Classic Question . . .The Eects of Class Size
Teachers, policy-makers, parents . . . and economists have asked: do
small classes promote learning?
Angrist and Lavy (1999) used a variety of methods to estimate class
size eects
Naive contrasts suggest that children in larger classes have higher test
scores (Angrist and Lavy OLS , , ,)
Selection bias at work?
The Tennesee STAR randomized trial shows that children randomly
assigned to smaller classes have higher test scores
Krueger 1999 , , ,
Its hard to overstate the impact of this work (or related experiments,
like the Perry preschool project)
Economist Paul Gertler on the impact of the randomized Progresa
(conditional cash transfer) evaluation:
"Progresa is why now thirty countries worldwide have conditional cash
transfer programs."
Random Assignment Carries the Day:
Improving College Succes
You can lead em to water, but can you make em drink?

North American universities struggle to improve student grades,

reduce dropout rates, shorten completion times
Most of this eort comes in the form of largely untested remedial and
support services
The students most in need rarely seek assistance; this makes support
services look good, but do these services really improve outcomes?
Angrist, Lang, and Oreopoulos (2009) tested support services and
nancial incentives for college achievement in a randomized trial
Our STAR (student achievement and retention) project had three
treatments:
1 Peer advising and supplemental instruction ("services")
2 Merit scholarships ($1000-$5000 for good grades; "incentives")
3 Combined services and incentives
Angrist, Oreopoulos, and Tyler (2013) follow up
The Student Achievement and Retention Project

STAR starts with the population of incoming freshman at a large

Canadian public university (UTM)
Data come from the university administrative record-keeping system
and our background survey
Participants gave consent by "signing up"
Random assignment divided students into four groups
250 oered services (the student support program; SSP)
250 oered incentives (the student fellowship program; SFP)
150 oered both (SFSP)
Another 1000 in the control group
Program details
SSP included academic advising, orientations, tutoring, supplemental
instruction, writing workshops
SFP award thresholds were customized, higher bar for better students
(based on HS GPA); a little over 20% got something
STAR Study Population

Mostly commuters, many non-native English speakers, many

South/East Asian immigrants, fairly educated parents
About 85% say theyll nish in 4 years, but historically the 6 year
completion rate at UTM is about %70
We studied incoming freshman in Fall 2005
July 2005: background survey (online and by phone) for all incoming
rst-years (90% respond)
August 2005: categorize all incoming by HS grade quartile (from UT),
drop the top, randomize the rest
Services and award receipt in treated groups was conditional on sign-up
(willingness to be in an active study-arm)
Descriptive Stats
We had no direct contact with controls, except to eld a few queries
Toronto STAR Results

Outcome data
Collected mostly from university systems (GPA, credits, academic
probation, retention)
Outcomes collected at the end of rst semester, rst year, second year
We surveyed a subset later, treatment and control (low response)
Focus groups with a few treated students (they loved the program!)
Take-up (sign-up, participation)
Many students who were oered services were uninterested; SSP
sign-up and service use rates were low
Fellowship treatments boosted sign-up
Women signed up more than men
STAR Results , , , (rst year)
Services alone have no eect
Incentives wear o quickly
Top marks for both (at least for women)
These results are mixed; Our follow-up is downright disappointing!
Do You Believe in Parallel Worlds?
Meat and Potatoes

On the menu of identication strategies, Ill usually take IV, RD, and
DD ... in that order
Dierences-in-dierences is our staple, meat and potatoes while were
waiting for something spicier
Lotteries that creates instruments, perhaps
An interesting discontinuity
Yet, dis-in-dis can be compelling ... sometimes it makes a nice
shepherds pie
At the LSE, I had to learn to eat such things (and much worse!)
Alas, you say, only the Labor tribe can partake
Not so: heres a DD banking story for our times
The story may be familiar to you: I recently found this artifact in the
British Museum
Otherwise, mostly boring stu there
Banking on the South
In the late 1920s, Caldwell and Company ran the American Souths
largest bank chain and owned many other businesses
In November 1930, corruption, mismanagement, and drought
collapsed the Caldwell empire
Within 6 months, 29 Mississippi banks folded as depositors panicked
Does this sort of nancial collapse necessarily precipitate a decline in
economic activity?
Policy makers facing a bank run can let credit ow easy or turn o
the tap:
Easy money allows troubled banks to meet withdrawal demands,
staving o depositor panic, and obviating the need for support in the
rst place
Are all liquidity crises merely crises of condence? Bagehot (1873): If
the banks are bad, they will certainly continue bad and will probably
become worse if the Government sustains and encourages them ... aid
to a present bad bank is the surest mode of preventing the
establishment of a future good bank.
One Mississippi, Two Mississippi (based on a story by
Richardson and Troost)

Depression-era regional Feds had considerable independence

The 6th (Atlanta) District favored lending to troubled banks. Within 4
weeks of Caldwells collapse, the Atlanta Fed had increased bank
lending by about 40% in the 6th District.
The 8th (St. Louis) District followed Real Bills Doctrine (dont ask).
In the same period, bank lending by the St. Louis Fed in the 8th
District fell 10%.
The 6th-8th District border runs smack through the middle of Miss.
Think of the 8th as a largely passive control group and the 6th as the
treatment group, where policy was to increase lending
8 months into the crisis, 132 banks were open in the 8th District, 121
open in the 6th, a decit of 11 banks in the 6th District: The
easy-money treatment eect is negative!
Look again: On July 1, 1930, well before the Caldwell crisis, 165 banks
were open in the 8th District, 135 banks open in the 6th district.
Parallel Worlds
DD uses parallelism to adjust for dierences across districts in the
pre-treatment period
Let Yd ,t denote the number of banks open in District d in year t
The DD eect of loose money in the 6th District is

DD = (Y6,1931 Y6,1930 ) (Y8,1931 Y8,1930 )

= (121 135) (132 165)
= 14 ( 33) = 19.

Instead of comparing the number of banks open in the 6th (treatment)

and 8th District (control) after Caldwell, DD contrasts the change in
the number of banks operating in the two districts
W/in-district dierencing kills time-invariant district eects
Every picture tells a story ...
The DD counterfactual
and a longer view (in 1931, the 8th opened the liquidity tap as well)
Lets get real
The DD Setup
The heart of the DD setup is an additive model for potential
outcomes in the no-treatment state:
yd ,t (0) = d + t (1)
where yd ,t (0) is notation for the potential outcome describing what
happens in district d and period t in the absence of intervention
(dened for all d and t)
Assuming that the causal eect yd ,t (1) yd ,t (0) is constant:
yd ,t (1) = d + t +
and
y6,1931 y6,1930 = (1931 + ) 1930
y8,1931 y8,1930 = 1931 1930
Hence, DD captures the causal eect of interest by subtracting
control changes from treatment changes:
fy6,1931 y6,1930 g fy8,1931 y8,1930 g =
How Trendy

The key DD assumption is parallel outcome trends for treatment and

control units (districts) in the absence of policy dierences
Our model for counterfactual no-treatment outcomes in both districts
allows for:
Time-invariant district eects
Common temporal changes
Unique trendiness within districts is deemed excessive (in fact, for
DD, this is fatal)
Common trends can be applied to transformed data, e.g.,

log yd ,t = d + t

Though common in logs does not imply (indeed, contradicts)

common in levels
DD identication is ckle!
Mis-sis-sippi must be DD heaven
Regression DD: All this and standard errors too

Stack data on districts and years in a sample of size 12 (with 6 years

for each district)
Let TREATd indicate data from the 6th District and POSTt indicate
post-treatment (Caldwell crisis) years
The regression DD estimator for the Mississippi two-step, rDD ,
comes from tting

Yd ,t = + TREATd + POSTt
+rDD (TREATd POSTt ) + ed ,t

With only two periods, estimates of DD and rDD coincide. With

more, rDD is more precise than the simple four-number DD recipe
Regression DD also . . .
gives us SEs (beware clustering and serial correlation)
facilitates specication testing
Checking Specs: EPL in America

American labor law is founded on the doctrine of employment-at-will:

employers can discharge employees as they see t
Employment protection takes the form of either anti-discrimination
legislation (protecting specic groups) or common law court rulings
setting precedents
State courts have adopted three exceptions to the employment-at-will
doctrine:
PP: public policy exceptions (cannot be discharged for performing
public duties, like jury service, or exercising rights, like ling a workers
compensation claim)
GF: good faith exceptions (primarly applied to the timing of a
discharge: e.g. cannot lay o worker just before a bonus or commission
is due)
IC: implied contract exceptions (formal, e.g. company personnel
manuals, or informal rules, e.g. longevity of service, company practice,
may form an implicit contract with the worker)
DD Spec Checks: Granger
Autor (2003) investigates the eect of tighter EPL on employersuse
of temporary help
In this context, Granger causality tests ask whether, conditional on
state and year eects, time-varying state policy dst predicts yist while
future dst does not
Autor estimates equations like
m q
yist = s + t + ds ,t + + ds ,t + + Xist
0
+ ist ,
=0 =1

allowing for m lags ( 1 , 2 , ..., m ) or post-treatment eects and q

leads (+1 , +1 , ..., +q ) or anticipatory eects
Here, weve switched from districts (d) to states (s)
The pattern of lagged eects is of substantive interest as well:
Perhaps causal eects grow or fade
Autors DD reg includes leads and lags, running from 2 years ahead to
4 years behind, plotted in MHE Table 5.2.4 (3+ prior is ref.) [emp
results]
Granger at Work

Fixed Effects, DD, and Panel Data 239

50
Vertical lines mark two
standard errors
40

30
Log Points

20

10

-10

-20
2 Years 1 Year Year of 1 Year 2 Years 3 Years 4 or More
Prior Prior Adoption After After After Years After
Time relative to adoption of implied contract exception

Figure 5.2.4 The estimated impact of implied-contract exceptions

employment-at-will doctrine on the use of temporary workers (from
Autor, 2003). The dependent variable is the log of state temporary
help employment in 19791995. Estimates are from a model that
allows for effects before, during, and after adoption.
DD Spec Checks: State-Specic Trends
A closely related check adds state-specic linear trends:
0
ys ,t = 0s + 1s t + t + dst + Xist + ist , (2)

where 0s is a state-specic intercept as before and 1s is a state-specic

trend coe cient multiplying the linear trend variable, t
This allows parameteric uncommonness (and thats often plenty)
At least 3 periods are needed to estimate a model with state-specic
trends; the more the better
Besley and Burgess (2004) include state trends in a study of the
eect of labor regulation on business productivity in Indian states
The unit of observation in Besley and Burgess (2004) is a state-year
average. MHE Table 5.2.3 reproduces the key results

This anguished cry echoed yet again in the twisting alleys of

Westminster: "State trends kill it!"
 240 Chapter 5

Table 5.2.3
Estimated effects of labor regulation on the performance of firms
in Indian states
(1) (2) (3) (4)
Labor regulation (lagged) .186 .185 .104 .0002
(.064) (.051) (.039) (.020)
Log development .240 .184 .241
expenditure per capita (.128) (.119) (.106)
Log installed electricity .089 .082 .023
capacity per capita (.061) (.054) (.033)
Log state population .720 0.310 1.419
(.96) (1.192) (2.326)
Congress majority .0009 .020
(.01) (.010)
Hard left majority .050 .007
(.017) (.009)
Janata majority .008 .020
(.026) (.033)
Regional majority .006 .026
(.009) (.023)
State-specific trends No No No Yes
Adjusted R2 .93 .93 .94 .95
Notes: Adapted from Besley and Burgess (2004), table IV. The table reports
regression DD estimates of the effects of labor regulation on productivity. The
dependent variable is log manufacturing output per capita. All models include
state and year effects. Robust standard errors clustered at the state level are
Getting a Little Jumpy -
The Regression Discontinuity Idea

Earlier, we looked at evidence on class size from a randomized trial

We dont often get such ambitious social experiments
Angrist and Lavy (1999) used the regression discontinuity method to
estimate the eects of class size in Israel
Sharp RD is when treatment is a deterministic and discontinuous
function of a covariate, xi , called the running variable:

1 if xi x0
di = . (3)
0 if xi < x0

where x0 is a known threshold or cuto value.

Figure 6.1.1 (Angrist and Pischke 2009) illustrates RD where those
with xi 0.5 are treated. In Panel A, the trend relationship
between yi and xi is linear, while in Panel B, its nonlinear.
Fuzzy RD is used when the probability of treatment jumps at x0
Maimonides Rule

AL-99 exploit the fact that Israeli class size is capped at 40; if
enrollment exceeds this threshold, classes are split
This suggests we compare, say, kids in schools with 38-40 fth
graders to kids in schools with 41-43 fth graders, where classes are
much smaller
The idea is that this small dierence in enrollment, which generates a
big change in class size, is a "good experiment"
Let msc denote the predicted class size (in a given grade) assigned to
class c in school s, where enrollment in the grade is denoted es .
Maimonides Rule is
es
msc = (e s 1 )
int [ 40 ] + 1

where int (x ) is the integer part of a real number, x.

Enrollment, es , is the running variable
Maimonides Rule Results

Maimonides Rule has a sawtooth pattern with discontinuities at

integer multiples of 40
We used these discontinuities to construct Fuzzy RD estimates of
class size eects (implemented using two-stage least squares, 2SLS)
Maimonides results . . .
Imply eect sizes on the order of those from Tennessee, without the
trouble and expense of random assignment
Well take it, but RD is not as good as a real randomized trial
RD estimates are typically sensitive to the details of model specication
(compare Maimonides 2SLS estimates across columuns, then do the
same for TN STAR)
RD estimates are considerably less precise, especially those in the
discontinuity window, which rely on fewer assumptions
RD research today focuses on these problems
Lessons Learned
Causal confusion abounds: interested parties weave evocative tales
wrapped in plausible theories
From a few cases, Freud reasoned that mental illness comes from bad
parenting, and that talk therapy is the cure
Lobotomy to treat schizophrenia and depression was motivated by
medical theory of the day; its Portugese inventor won the 1949 Nobel
prize for medicine
Randomized trials have since shown that psychiatric drugs are
typically more eective than psychoanalysis; lobotomy is discredited
by terrible long-term results
As in health research, successful metrics puts social and economic
policy on a solid evidence-based foundation
What are the outcomes of interest? Which control group establishes
credible counterfactuals?

Rigorous evidence comes from clear and compelling answers to these

fundamental questions
Prologue Tables and Figures
-

TABLE II
OLS ESTIMATES FOR 1991

5th Grade

Reading comprehension Math Reading comp

(1) (2) (3) (4) (5) (6) (7) (8)

Mean score 74.3 67.3 72.5

(s.d.) (8.1) (9.9) (8.0)
Regressors
Class size .221 2 .031 2 .025 .322 .076 .019 0.141 2 .05
(.031) (.026) (.031) (.039) (.036) (.044) (.033) (.02
Percent disadvantaged 2 .350 2 .351 2 .340 2 .332 2 .33
(.012) (.013) (.018) (.018) (.01
Enrollment 2 .002 .017
(.006) (.009)
Root MSE 7.54 6.10 6.10 9.36 8.32 8.30 7.94 6.65
R2 .036 .369 .369 .048 .249 .252 .013 .30
N 2,019 2,018 2,04

The unit of observation is the average score in the class. Standard errors are reported in parentheses. Standard errors were cor
class eect of about 5 to 6 percentile points. The eect size is about :2 ; where is the standard deviation
of the
- percentile score in kindergarten. The small-class eect is signicantly dierent from zero, while the

Table 2.2.2: Experimental estimates of the eect of class-size assignment on test scores

Explanatory variable (1) (2) (3) (4)

Small class 4.82 5.37 5.36 5.37
(2.19) (1.26) (1.21) (1.19)
Regular/aide class .12 .29 .53 .31
(2.23) (1.13) (1.09) (1.07)
White/Asian (1 = yes) 8.35 8.44
(1.35) (1.36)
Girl (1 = yes) 4.48 4.39
(.63) (.63)
Free lunch (1 = yes) -13.15 -13.07
(.77) (.77)
White teacher -.57
(2.10)
Teacher experience .26
(.10)
Masters degree -0.51
(1.06)
School xed eects No Yes Yes Yes
R2 .01 .25 .31 .31

Note: Adapted from Krueger (1999), Table 5. The

dependent variable is the Stanford Achievement Test

percentile score. Robust standard errors that allow

-

Instrumental Variables in Action 163

Table 4.4.1
Results from the JTPA experiment: OLS and IV estimates of training impacts
Comparisons by Comparisons by Instrumental Variable
Training Status (OLS) Assignment Status (ITT) Estimates (IV)
Without With Without With Without With
Covariates Covariates Covariates Covariates Covariates Covariates
(1) (2) (3) (4) (5) (6)
A. Men 3,970 3,754 1,117 970 1,825 1,593
(555) (536) (569) (546) (928) (895)
B. Women 2,133 2,215 1,243 1,139 1,942 1,780
(345) (334) (359) (341) (560) (532)
Notes: Authors tabulation of JTPA study data. The table reports OLS, ITT, and IV
estimates of the effect of subsidized training on earnings in the JTPA experiment. Columns 1
and 2 show differences in earnings by training status; columns 3 and 4 show differences by
random-assignment status. Columns 5 and 6 report the result of using random-assignment
status as an instrument for training. The covariates used for columns 2, 4, and 6 are high
school or GED, black, Hispanic, married, worked less than 13 weeks in past year, AFDC
(for women), plus indicators for the JTPA service strategy recommended, age group, and
second follow-up survey. Robust standard errors are shown in parentheses. There are 5,102
men and 6,102 women in the sample.
-

Vol. 1 No. 1

angrist et al: incentives and services for college achievement

Table 1Descriptive Statistics

Contrasts by treatment status

Control SSP v. SFP v. SFSP v. F-stat
mean control control control 1 all5control2 Obs.
112 122 132 142 152 162
Administrative variables
Courses enrolled as 4.745 20.053 0.015 20.158 0.702 1,656
of fall 2005 51.3706 30.0954 30.0954 30.1184 10.5512
No show 0.054 0.002 20.030 0.020 1.852 1,656
30.0164 30.0164 * 30.0194 10.1362
Completed survey 0.898 20.018 20.010 20.051 1.228 1,656
30.0224 30.0224 30.0284 * 10.2982
Student background variables
Female 0.574 20.006 0.029 20.005 0.272 1,571
30.0364 30.0354 30.0454 10.8452
High school GPA 78.657 0.170 0.238 20.018 0.276 1,571
54.2206 30.3084 30.3044 30.3844 10.8432
Age 18.291 20.054 20.033 0.026 0.752 1,571
50.6166 30.0454 30.0444 30.0564 10.5212
Mother tongue is English 0.700 0.017 0.009 0.049 0.495 1,571
30.0334 30.0334 30.0414 10.6862
Survey response variables
Lives at home 0.811 20.040 0.009 20.004 0.685 1,431
30.0304 30.0304 30.0384 10.5612
At first choice school 0.243 0.024 0.060 0.047 1.362 1,430
30.0344 30.0334 * 30.0424 10.2532
Plans to work while 0.777 0.031 20.066 0.037 2.541 1,431
in school 30.0324 30.0314 ** 30.0404 10.0552
Mother a high school 0.868 0.015 20.021 20.045 1.040 1,431
graduate 30.0264 30.0264 30.0334 10.3742
Mother a college graduate 0.358 0.053 20.020 20.052 1.487 1,431
30.0374 30.0364 30.0464 10.2162
Father a high school 0.839 0.025 0.008 20.017 0.416 1,431
graduate 30.0284 30.0274 30.0354 10.7412
Father a college graduate 0.451 0.021 20.001 20.024 0.216 1,431
30.0384 30.0374 30.0484 10.8852
-

10 American Economic Journal: applied economicsMonth 2009

Table 3Program Sign-up and Use of Services

Signed up for STAR Received SSP services Met with/emailed an advisor Attended FSGs
Basic All Basic All Basic All Basic All
controls controls controls controls controls controls controls controls
(1) (2) (3) (4) (5) (6) (7) (8)
Panel A. All
Offered SSP 0.519 0.549 0.238 0.255 0.204 0.217 0.106 0.118
[0.032]*** [0.034]*** [0.028]*** [0.029]*** [0.026]*** [0.028]*** [0.020]*** [0.021]***
Offered SFP 0.863 0.867
[0.022]*** [0.022]***
Offered SSP 0.762 0.792 0.412 0.431 0.383 0.397 0.131 0.139
and SFP [0.036]*** [0.036]*** [0.041]*** [0.044]*** [0.041]*** [0.043]*** [0.029]*** [0.031]***
Observations 1,571 1,431 1,571 1,431 1,571 1,431 1,571 1,431
Panel B. Men
Offered SSP 0.447 0.464 0.194 0.206 0.145 0.149 0.096 0.107
[0.049]*** [0.052]*** [0.039]*** [0.042]*** [0.035]*** [0.038]*** [0.029]*** [0.032]***
Offered SFP 0.792 0.806
[0.040]*** [0.040]***
Offered SSP 0.705 0.708 0.298 0.291 0.282 0.270 0.115 0.112
and SFP [0.058]*** [0.065]*** [0.058]*** [0.063]*** [0.057]*** [0.061]*** [0.042]*** [0.046]**
Observations 665 594 665 594 665 594 665 594

Panel C. Women
Offered SSP 0.571 0.605 0.273 0.287 0.251 0.264 0.113 0.124
[0.043]*** [0.044]*** [0.038]*** [0.040]*** [0.037]*** [0.040]*** [0.027]*** [0.029]***
Offered SFP 0.912 0.908
[0.024]*** [0.026]***
Offered SSP 0.800 0.835 0.506 0.532 0.466 0.489 0.146 0.155
and SFP [0.046]*** [0.043]*** [0.056]*** [0.058]*** [0.056]*** [0.058]*** [0.040]*** [0.042]***
Observations 906 837 906 837 906 837 906 837
-

Vol. 1 No. 113

angrist et al: incentives and services for college achievement

Table 5Treatment Effects on First Year Outcomes in the Sample with Fall Grades

SFP by type Any SFP

All Men Women All Men Women
112 122 132 142 152 162
Panel A. Fall grade
Control mean 64.225 65.935 62.958 64.225 65.935 62.958
111.9022 111.3402 112.1602 111.9022 111.3402 112.1602
SSP 0.349 20.027 0.737 0.344 20.014 0.738
30.9174 31.3344 31.2754 30.9174 31.3324 31.2744
SFP 1.824 0.331 2.602
30.8474 ** 31.2334 31.1764 **
SFSP 2.702 20.573 4.205
31.1244 ** 32.0104 31.3254 ***
SFP 1 any2 2.125 0.016 3.141
30.7314 *** 31.1644 30.9724
Observations 1,255 526 729 1,255 526 729
Panel B. First year GPA
Control mean 1.805 1.908 1.728 1.797 1.885 1.731
10.9022 10.9082 10.8912 10.9042 10.9102 10.8942
SSP 0.073 0.011 0.116 0.071 0.008 0.116
30.0664 30.1074 30.0824 30.0664 30.1074 30.0824
SFP 0.010 20.110 0.086
30.0644 30.1034 30.0844
SFSP 0.210 0.084 0.267
30.0924 ** 30.1624 30.1174 **
SFP 1 any2 0.079 20.042 0.147
30.0564 30.0954 30.0734
Observations 1,255 526 729 1,255 526 729

Notes: The table reports regression estimates of treatment effects on full grades and first-year GPA compu
-

216 THE REVIEW OF ECONOMICS AND STATISTICS

Our basic econometric model is FIGURE 1.STATE LOG EMPLOYMENT-TO-POPULATION RATIOS BEFORE AND
AFTER ADOPTION OF IMPLIED-CONTRACT EXCEPTION: MONTHLY LEADS AND
LAGS FROM 4 YEARS BEFORE TO 8 YEARS AFTER ADOPTION
Y st 1 Treat st 2 Post st
(1)
3TreatstPostst st,

where Treatst is an indicator for the period from 24 months

before to 36 months after adoption of a wrongful-discharge
law in state s, and Postst is an indicator for the period 13
through 36 months after adoption. The coefficient of interest
in this equation, 3, is an estimate of the pre-post change in
the outcome variable in adopting states relative to the
corresponding change in nonadopting states. All estimates
are weighted by the share of national residents aged 1864
in each state-year cell.28
We enrich this basic model in three ways. First, in place
of the common main-effect and pretreatment indicators (
and Treatst), we add main effects for each state and their
interactions with a treatment indicator variable. Second, to
control flexibly for common shocks to national employ-
ment, we include an exhaustive set of time dummies,
corresponding to each year and month of the sample. Fi-
nally, to allow for common regional employment shocks,
we also estimate specifications that include interactions
between calendar-year dummies and indicator variables
denoting the four major Census geographic regions. With
region controls included, the parameter is identified by
contrasting contemporaneous employment or wage out-
comes in adopting versus nonadopting states located in the
same geographic regions.29

IV. Impacts on Employment and Earnings

Before turning to estimates of equation (1), we provide a
visual summary of the employment data in figures 1 through
-

Regression Discontinuity Designs 265

40
30
Class size
20
10
A. Fifth Grade
0
40
30
Class size
20
10

B. Fourth Grade
0

20 40 60 80 100 120 140 160 180 200 220

Enrollment count

Actual class size Maimonides Rule

Figure 6.2.1 The fuzzy-RD rst-stage for regression-discontinuity

estimates of the effect of class size on test scores (from Angrist and
Lavy, 1999.)

makes the RD design fuzzy. Still, there are clear drops in class
-

USING MAIMONIDES RULE 545

- 266 Chapter 6

Table 6.2.1
OLS and fuzzy RD estimates of the effect of class size on
fth-grade math scores
OLS 2SLS
Full Sample Discontinuity Samples
5 3
(1) (2) (3) (4) (5) (6) (7) (8)
Mean score 67.3 67.3 67.0 67.0
(SD) (9.6) (9.6) (10.2) (10.6)
Regressors
Class size .322 .076 .019 .230 .261 .185 .443 .270
(.039) (.036) (.044) (.092) (.113) (.151) (.236) (.281)
Percent .340 .332 .350 .350 .459 .435
disadvantaged (.018) (.018) (.019) (.019) (.049) (.049)
Enrollment .017 .041 .062 .079
(.009) (.012) (.037) (.036)
Enrollment .010
squared/100 (.016)
Segment 1 12.6
(enrollment 3843) (3.80)
Segment 2 2.89
(enrollment 7883) (2.41)
R2 .048 .249 .252
Number of classes 2,018 2,018 471 302
Notes: Adapted from Angrist and Lavy (1999). The table reports estimates of equation
(6.2.6) in the text using class averages. Standard errors, reported in parentheses, are corrected
for within-school correlation.
- 254 Chapter 6

A. LINEAR E[Y0i|Xi]
1.5

1.0

Outcome
0.5

0
0 0.2 0.4 0.6 0.8 1.0
X
B. NONLINEAR E[Y0i|Xi]
1.5

1.0
Outcome

0.5

0 0.2 0.4 0.6 0.8 1.0

X
C. NONLINEARITY MISTAKEN FOR DISCONTINUITY
1.5

1.0
Outcome

0.5

-0.5
0 0.2 0.4 0.6 0.8 1.0
X

Figure 6.1.1 The sharp regression discontinuity design.

DD Tables and Figures
-
-

8th District
160
Number of Banks in Business
140

6th District
120

Treatment effect

6th District counterfactual

100

1929 1930 1931 1932

Year
-

180 160
Number of Banks in Business

8th District
140

6th District
120 100

1929 1930 1931 1932 1933 1934

Year
 180 160

8th District
Number of Banks in Business
140

6th District
100 120

6th District counterfactuals

80

1929 1930 1931 1932 1933 1934

Year

-
 1066 journal of political econom

TABLE 8
Decline in Wholesale Trade

Federal Reserve District

6th Atlanta 8th St. Louis
Wholesale firms:
Number in 1929 783 930
Number in 1933 641 607
D% !18.1 !34.7
Net sales:
$1,000s in 1929 140,776 245,486
$1,000s in 1933 59,513 83,727
D% !57.7 !65.9
Source.Census of American Business, 1929 and 1933.

sheets of all banks in each county in operation on July 1 of each yea

- Note that the dependent variable measures the change in a flow, who
sale transactions summed up over the entire year minus wholesale tran
actions summed over a later year. The explanatory variable measur