Lecture 2:

Experiments and Causality

Stanislao Maldonado (UC Berkeley)

Plan
` Idealized experiments and causal effects.
` Potential problems with experiments.
experiments
` Regression estimators of causal effects using
experimental
p data

2
1. Idealized experiments and causal effects
` Key idea:
Causal effects can be measured by randomly selecting
individuals from a population and the randomly giving some of
the individuals the treatment.

` The effect of random assignment:

(1) Yi = β 0 + β1 X i + ui
Where:
Yi : Outcome
O
X i : Treatment level
ui : All additional determinants of Y
3
` You know:

If X is randomly assigned:

E (ui X i ) = 0
9X is distributed independently of the omitted factor u
9Random assignment
g of X implies
p that the ortoghonality
g y
condition holds

` Causal effect on Y on treatment level X:

β1 = E (Y X = x) − E (Y X = 0)

4
` The Differences Estimator

If X binary:

9 Causal effect can be estimated by the difference in the sample

average
g outcomes between the treatment and control groupsg p

9 Equivalently:
β can be
b estimated ti t b if treatment
ti t d bby OLS estimator t t t iis
randomly assigned

5
2. Potential problems with experiments

` Threats to internal validity:

9 Failure to randomize
Ex: using last name to assign the treatment

9 Failure to follow treatment protocol

People don’t do what they are asked to do
• Partial compliance
• Incorrect measurement of treatment

9 Att iti
Attrition
Subjects dropping out of the study after being randomly assigned to
treatment

6
 •Random attrition
E SSelected
Ex: l t d tto training
t i i program th
thatt gett sick
i k

•Endogenous attrition
Ex: More able individual dropping training program for getting job

9 Experimental effects
Being in an experiment change behavior
•Double – blind experiments: Placebos
• Hard to implement in economics

9 Small sample
Not bias but causal effects are imprecisely
p y estimated

7
` Threats to external validity
9 Non representative
N i sample l
9 Non representative program or policy
9 General equilibrium effects
9 Treatments vs eligibility effects

8
3. Regression estimators of causal effects
using experimental data
` If treatment is randomly received:
9 Differences estimator is unbiased
9 But is this efficient ?

` When experiment have some issues of internal validity

validity, then the
differences estimator is biased

` SSolution:
l
Differences Estimator with additional regressors (DER):

(2) Yi = β 0 + β1 X i + β 2W1i + ... + β r +1Wri + ui

` W is a set of “control variables”

9
` What is the difference between a “treatment” and
“control”
control variable?
9 Conditional mean-zero assumption:

E (ui X i ) = 0
9 Conditional mean independence assumption:

E (ui X i , W1i ,..., Wri ) = γ 0 + γ 1W1i + ... + γ rWri

10
` Conditional mean independence implies:
9 u can be correlated with W
9 Gi
Given W
W, u does
d not depend
d d on X

` When this assumption is true ?

9 When E (ui X i ) = 0
9 X is randomly assigned
9 X is assigned randomly conditional on W.

11
` Taking conditional expectations in both sides of equation (2):

(3) E( Yi X i , W1i ,..., Wri ) = β 0 + β1 X i + β 2W1i + ... + β r +1Wri

+ E (ui X i , W1i ,..., Wri )
= β 0 + β1 X i + β 2W1i + ... + β r +1Wri
+γ 0 + γ 1W1i1i + ... + γ rWri
` Evaluating at X=1 and at X= 0 :
β1 = E (Y X = 1,
1 W1i ,..., Wri ) − E (Y X = 0,
0 W1i ,..., Wri )
` W must reflect non experimental – predetermined outcomes

12
` Reasons for using the DER:
` Efficiency
` Check for randomization
` Adj t for
Adjust f “conditional”
“ diti l” randomization
d i ti

13
` Estimation of causal effects for different groups:
` Addi interaction
Adding i i effects
ff when
h characteristic
h i i iis observable
b bl

` Estimation when there is partial compliance:

` X can be correlated with u, so OLS estimator is no longer
consistent
` Solution: IV
` Assigned treatment serves as instrument for actual treatment

14
` Testing for randomization
` T i ffor random
Testing d receipt
i off treatment
X i = γ 0 + γ 1W1i + ... + γ rWri + vi
F-test for null hypothesis that treatment was received
randomly
` Testing for random assignment
Z i = δ 0 + δ1W1i + ... + δ rWri + vi
F-test for null hypothesis that all the slope coefficients are zero

15
Example: Effect of class size reductions
` Project STAR (Student-Teacher Achievement Ratio)
` 4
4-year study,
d $12 million
illi
` Upon entering the school system, a student was randomly
assigned to one of three groups:
` regular class (22 – 25 students)
` regular class + aide
` small class (13 – 17 students)
` Regular class students re-randomized after first year to regular
or regular+aide
` Y = Stanford Achievement Test scores

16
` Internal validity issues:
` Partial compliance
` Attrition
` Empirical
p estimation:
Yi = β0 + β1SmallClassi + β2RegAidei + ui
SmallClassi = 1 if in a small class
RegAidei = 1 if in regular class with aide

17
18
19
` Replicating results in STATA

20
4. Running regressions without apology
` Without random assignment, a regression may or may
not have causal interpretation
p
` But what’s wrong with not having a causal interpretation
for an OLS coefficient?
` Description
D i ti
` Prediction
` Example:
p schoolingg and earnings
g
` On average, people with more schooling tend to earn more
than people with less schooling
` Education predicts earnings in a narrow statistical sense
` Predictive power is summarized by the Conditional
Expectation Function (CEF)

21