Professional Documents
Culture Documents
.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.
American Marketing Association is collaborating with JSTOR to digitize, preserve and extend access to
Journal of Marketing Research.
http://www.jstor.org
CHRISTIAN
HOMBURG*
Manyapplicationsof causal modelingin marketinginvolveselectionamong several competingcausal models. Theauthorinvestigateswhethercommoncriteriafor
model selectionsuch as cross-validationindicesand informationcriteriaare likely
to lead to discoveryof the correctpopulationmodel. Guidanceon the use of these
selectioncriteriain practiceis providedfor substantivemarketingresearchers.Resultsindicatethat the adequacyof cross-validation
dependscriticallyon the method
used for sample-splitting.The author suggests the applicationof Snee's DUPLEX
algorithmin this context. For situationsin whichthe assumptionof multinormally
distributedvariablesis justified,informationcriteriaare foundto be highlyappropriate for model selection, outperformingcross-validationmethodsin several respects.
and
Cross-Validation
Information
Causal
Criteria
in
Modeling
Recent methodological literaturehas discussed the application of causal modeling in an exploratory framework (see, e.g., Bentler 1986; Cudeck and Browne 1983;
Homburg 1989; Homburg and Dobratz 1991; MacCallum 1986). Interestingly, however, the idea of applying causal analysis in an exploratory way was formulated previously by Jdreskog (1971, 1977), who
suggested carrying out methods of stepwise model modification in the context of causal modeling. These methods, frequently referred to as "specification searches,"
are not the focus of this article; they are discussed in
detail by Saris, de Pijper, and Zegwaart (1979), MacCallum (1986), and Homburgand Dobratz(1991), among
others. Examples of marketingapplicationsof such search
procedures are Sujan's (1986) analysis of salespeople's
motivation and Gaul and Homburg's (1988) investigation of the use of data analysis techniques by German
market research agencies.
A second way of applying causal analysis in an exploratory research context consists of specifying several
alternative models and selecting from that set of models
the one that seems most appropriate for describing the
structuresunderlying the observed data. Cross-validation
methods and information criteria are two well-known approaches in this context. The simulation study reported
here investigated and compared the quality of these selection methods in order to provide guidance for substantive marketing researchers on the use of these approaches in practice.
*Christian Homburg is head of the Marketing and Strategy Department, KSB AG, Frankenthal, Germany.
As the article is based on the author's doctoral dissertation, he thanks
Wolfgang Gaul of Karlsruhe University for many helpful discussions
on model selection. Thanks are due to Adolfo Varillas for computational assistance and to Stefan Suitterlinfor useful suggestions. Special
thanks go to three anonymous JMR reviewers, whose comments improved a previous version of the article.
137
Journal of Marketing Research
Vol. XXVIH (May 1991), 137-44
MAY1991
JOURNAL
OF MARKETING
RESEARCH,
138
After a brief discussion of the application of crossvalidation methods and information criteria for selecting
among several alternative causal models, the simulation
study is described in which the diverse approaches to
model selection were analyzed and compared on the basis of artificial data. Concluding remarks address implications for the use in practice of the methods investigated.
CROSS-VALIDATIONAND INFORMATION
CRITERIA
Consider a situationin which several alternativecausal
models have been specified, one of which is to be selected as most appropriatefor describing the structures
underlying the observed data. The basic idea in crossvalidation is to seek a model that will have the greatest
predictive validity in future samples ratherthan a model
that best reproducesstructuresof one specific sample that
may be inappropriateto futureobservationsfrom the same
population. Traditionally, cross-validation has been a
widely used method of model comparison. The researcher divides the sample into two subsamples, estimates the parameterson one, and validates on the other
(work by Mosier 1951 is an early example). The application of cross-validation procedures to causal models
was suggested by Cudeck and Browne (1983); an example from the marketing literature is the MacKenzie,
Lutz, and Belch (1986) study of attitude toward the ad
as a mediator of advertising effectiveness.
Given two subsamples, A and B, the researcher fits
each of the competing models to the data from subsample A and measures the prediction accuracy of one specific model by the cross-validation index
(1)
FAIB = F(SB,
A),
where S, denotes the sample covariance matrix of subsample B and LA is the covariance matrix reproduced by
the model on the basis of subsample A. In this formula,
F denotes an arbitrarydiscrepancy function (see, e.g.,
Browne 1984) commonly used for parameterestimation
in causal modeling. The choice of the model with the
greatest estimated predictive validity is made by selecting the model that yields the smallest cross-validation
index. Cudeck and Browne (1983) suggested that this
process should be repeated with subsample B as the calibration sample and subsample A as the validation sample (double cross-validation), yielding cross-validation
indices FB/A. Ideally, the same model yields the lowest
values with respect to both indices. If this is not the case,
one should decide on a subset of two or more models
for further consideration.
Literatureon cross-validation provides little guidance
on how the researchershould split the sample (see, e.g.,
Dorans and Drasgow 1980). A common solution is to
halve the sample randomly. To the best of our knowledge, in all applications of cross-validation to causal
modeling in which sample-splitting is carried out, some
random method is used (see, e.g., Balderjahn 1986; Cu-
and
(3)
IN CAUSALMODELING
AND INFORMATION
CRITERIA
CROSS-VALIDATION
139
ANALYSIS
Purpose
140
JOURNAL
OF MARKETING
MAY1991
RESEARCH,
Figure 1
TRUEPOPULATION
MODELS
modelA
X,
82 x2 x2
3 1211
X,
*12
Y'
mode B
021
Y3 E
4
, 41224
x
k22
x,
x.
ye
v.
x
model
B
X,
v.X
eu
x
2 X2 x
E2ys
#112
a,, 83
42
84
x
-*E
43
x6
Es5
Y Y2 31
732
..
12
Y3
r42
723
AND INFORMATION
CRITERIA
IN CAUSALMODELING
CROSS-VALIDATION
141
Table 1
STUDY:FREQUENCIES
FROMSIMULATION
FORSITUATIONS
RESULTS
1/2/3a
Sample
size
Population model A
50
75
100
150
200
300
400
500
750
1000
1/1/8
2/4/4
3/3/4
3/4/3
4/3/3
7/2/1
7/3/0
8/2/0
4/1/5
1/3/6
Total
40/26/34
(both pop-
SIC
4/3/3
3/4/3
5/3/2
6/1/3
7/2/1
6/3/1
7/2/1
9/1/0
3/3/4
2/2/6
4/0/6
5/0/5
6/0/4
7/0/3
8/0/2
7/0/3
8/0/2
9/0/1
10/0/0
10/0/0
4/0/6
6/0/4
7/0/3
7/0/3
9/0/1
8/1/1
9/0/1
9/0/1
10/0/0
10/0/0
0/3/7
0/4/6
0/7/3
2/4/4
5/2/3
3/6/1
6/1/3
6/1/3
2/5/3
1/3/6
52/24/24
74/0/26
79/1/20
25/36/39
CVD
CVR
Total
65
/62
Population model B
AIC
CVD
CVR
/73
96
/50
/54
CVR
CVD
AIC
2/2/6
3/2/5
4/1/5
4/3/3
3/5/2
5/3/2
7/3/0
7/2/1
6/3/1
3/2/5
3/0/7
4/0/6
6/0/4
6/0/4
7/0/3
7/0/3
7/0/3
8/0/2
9/0/1
9/0/1
44/26/30
66/0/34
AIC
140 /0/60
SIC
4/0/6
4/0/6
7/0/3
7/0/3
7/0/3
8/0/2
8/0/2
9/0/1
9/1/0
10/0/0
74/1/25
SIC
153 /2
/45
ulation
32.5/31.0/36.5%
48.0/25.0/27.0%
70.0/0/30.0%
76.5/1.0/22.5%
models)
by DUPLEXalgorithm;AICis Akaike
by randommethod;CVD is cross-validation,sample-splitting
"CVRis cross-validation,sample-splitting
informationcriterion;SIC is Schwarzinformationcriterion.
Example:For a samplesize of 200 objectsand populationmodel A, methodCVD identifiedthe truemodel as the uniqueoptimalmodel in
seven cases; in two cases, more thanone model was identifiedas being optimaland the truepopulationmodel was amongthem;in one case,
CVD suggesteda misspecifiedmodel.
142
MAY1991
JOURNAL
OF MARKETING
RESEARCH,
AND INFORMATION
CRITERIA
IN CAUSALMODELING
CROSS-VALIDATION
to scientific progress unless moderatedby careful thought
and judgment on the part of the researcher. Specifically,
as Fornell (1983) pointed out, substantive theory must
be included in the analysis. Marketingresearchersshould
keep that fact in mind whenever applying one of the approaches discussed here.
REFERENCES
Aaker, David A. and Richard P. Bagozzi (1979), "Unobservable Variables in Structural Equation Models With an Application in Industrial Selling," Journal of Marketing Research, 16 (May), 147-58.
Akaike, Hirotugu (1974), "A New Look at the StatisticalModel
Identification," IEEE Transactions on Automatic Control,
19, 716-23.
(1987), "Factor Analysis and AIC," Psychometrika,
52, 317-32.
Anderson, James C. and David W. Gerbing (1982), "Some
Methods for Respecifying Measurement Models to Obtain
Unidimensional Construct Measurement," Journal of Marketing Research, 19 (November), 453-60.
and (1984), "The Effect of Sampling Erroron
Convergence, Improper Solutions, and Goodness-of-Fit Indices for Maximum Likelihood Confirmatory Factor Analysis," Psychometrika, 49, 155-73.
- and (1988), "StructuralEquation Modeling in
Practice: A Review and Recommended Two-Step Approach," Psychological Bulletin, 103 (May), 411-23.
Bagozzi, Richard P. (1976), "Toward a General Theory for
the Explanation of the Performance of Salespeople," unpublished doctoral dissertation, Northwestern University.
(1980a), Causal Models in Marketing. New York: John
Wiley & Sons, Inc.
-(1980b), "Performance and Satisfaction in an Industrial Salesforce: An Examination of Their Antecedents and
Simultaneity," Journal of Marketing, 44 (Spring), 65-77.
(1982), "Introductionto Special Issue on Causal Modeling," Journal of MarketingResearch, 19 (November), 403.
Balderjahn, Ingo (1986), Das umweltbewuBte Konsumentenverhalten. Berlin: Duncker & Humbolt.
Bearden, William O., Subash Sharma, and Jesse E. Teel (1982),
"Sample Size Effects on Chi Squareand Other Statistics Used
in Evaluating Causal Models," Journal of Marketing Research, 19 (November), 425-30.
Bentler, Peter M. (1985), Theory and Implementation of EQS:
A Structural Equations Program. Los Angeles: BMDP Statistical Software, Inc.
(1986), Lagrange Multiplier and Wald Tests for EQS
and EQS/PC. Los Angeles: BMDP Statistical Software, Inc.
- and Douglas G. Bonett (1980), "Significance Tests
and Goodness-of-Fit in the Analysis of Covariance Structures," Psychological Bulletin, 88, 588-606.
Boomsma, Anne (1985), "Nonconvergence, Improper Solutions, and Starting Values in LISREL Maximum Likelihood
Estimation," Psychometrika, 50, 229-42.
Bozdogan, Hamparsum(1987), "Model Selection and Akaike's
Information Criterion (AIC): The General Theory and Its
Analytical Extensions," Psychometrika, 52, 345-70.
Browne, Michael W. (1984), "AsymptoticallyDistribution-Free
Methods for the Analysis of Covariance Structures,"British
Journal of Mathematical and Statistical Psychology, 37, 6283.
143
144
OF MARKETING
JOURNAL
MAY1991
RESEARCH,
107-20.
MacKenzie,Scott B., RichardJ. Lutz, and GeorgeE. Belch
(1986), "TheRole of AttitudeTowardthe Ad as a Mediator
of AdvertisingEffectiveness:A Test of CompetingExplanations," Journal of Marketing Research, 23 (May), 130-
43.
Mosier, CharlesI. (1951), "Problemsand Design of CrossValidation," Educational and Psychological Measurement,
11, 5-11.
Picard,RichardR. and R. Dennis Cook (1984), "Cross-Validation of Regression Models," Journal of the American
Statistical Association, 79 (September), 575-83.
Rust, RolandT. and DavidC. Schmittlein(1985), "A Bayesian Cross-ValidatedLikelihoodMethodfor ComparingAlternativeSpecificationsof QuantitativeModels,"Marketing
ReprintNo. JMR282101
This
is
publication
available in microform.
_
UniversityMicrofilmsInternational
reproducesthis publicationin microform:microfiche and 16mmor 35mmfilm.Forinformation
aboutthis publicationor any of the more than
13,000 titles we offer, completeand mail the
couponto: UniversityMicrofilmsInternational,
300 N. Zeeb Road,Ann Arbor,MI48106.Call
us toll-freefor an immediateresponse:
800-521-3044.Or call collect in Michigan,
Alaska and Hawaii:313-761-4700.
O Pleasesendinformation
aboutthesetitles:
Name
Company/Institution
University
Mi roailms
International
Address
City
State
Zip
Phone (