Sensitivity of Psychiatric Diagnosis

Based on the Best Estimate Procedure

Therese A. Kosten, Ph.D., and Bruce J. Rounsaville, M.D.

Objective: A “best estimate “ diagnosis is one made by expert clinicians on the basis of
diagnostic information from direct interview conducted by another clinician plus information
f rom medical records and from reports offamily members. The authors address the question
of whether the best estimate procedure can enhance the classification ofpsychiatric diagnoses
of subjects who are interviewed directly. Method: Four hundred seventy-five subjects were
interviewed directly: 201 opiate-addicted probands who sought treatment from a university-
based clinic and 2 74 of their spouses and/or first-degree relatives. Subjects were interviewed
by trained clinical assessors using the Schedule for Affective Disorders and Schizophrenia and
classified according to Research Diagnostic Criteria. Two psychologists independently diag-
nosed the same subjects by applying the best estimate procedure. Lifetime rates of major and
minor depressive disorder, antisocial personality, alcoholism, and drug abuse were calculated.
The rates of diagnoses made on the basis of direct interviews alone were compared with the
rates of diagnoses made according to the best estimate procedure. Results: Higher rates of
diagnoses ofall four disorders were made when the best estimate procedure was applied than
when direct interview alone was used; the best estimate procedure also resulted in a minimal
rate of false positives. Conclusions: The higher rate of diagnoses based on the best estimate
procedure may represent an enhancement in the accuracy of psychiatric diagnoses or an in-
crease in erroneous diagnoses. The authors consider the second possibility less likely.
(Am J Psychiatry 1992; 149:1225-1227)

A ccurate
important
classification
for genetic,
of psychiatric
epidemiologic,
diagnoses
and treat-
is tion from
members.
medical
Diagnostic
records and from
information
reports
from
of family
all these
ment research. Misclassification can lead to erroneous sources are compiled and rated by expert clinicians to
conclusions about the etiology or outcome of psychiat- yield a “best estimate” diagnosis (2). These clinicians
rid disorders. Using structured interviews and specific are not involved in the actual interviews of the subjects,
diagnostic criteria ( 1 ) provides a good basis for accu- including those of their family members. This blind
rate classification of psychiatric diagnoses. However, procedure allows for objective evaluation of diagnoses
direct interviews may not be available (e.g., in a family but can lead to disagreements on diagnoses. When this
study when a relative cannot be contacted because of occurs, the cases are reviewed and a consensus is
death, refusal, on other reasons). Moreover, the data reached. This process increases the cost of the study and
from these interviews may be inaccurate if the subject raises the question of whether the best estimate proce-
withholds or provides false information. dune provides enough valuable diagnostic information
To address these problems, psychiatric diagnoses can to warrant its increased cost for subjects who are di-
be obtained by enhancing interview data with infonma- nectly interviewed. For subjects who cannot be inter-
viewed, the best estimate procedure has been shown to
provide good diagnostic information (2).
Received Sept. 10, 1991; revision received Jan. 15, 1992; accepted The present paper addresses the question of whether
Jan. 31, 1992. From the Substance Abuse Treatment Unit, Depart- the best estimate procedure can enhance the classifica-
ment of Psychiatry, Yale University School of Medicine, New Haven,
tion of psychiatric diagnoses for subjects who are inter-
Conn. Address reprint requests to Dr. Kosten, Substance Abuse Treat-
ment Unit, 27 Sylvan Ave., New Haven, CT 06519. viewed directly. Working under the assumption that the
Supported in part by National Institute on Drug Abuse Center grant best estimate procedure is more accurate than direct in-
P50-04060 and Research Career Award DA-00089 from the National terview data because it is based on multiple sources of
Institute on Drug Abuse to Dr. Rounsaville.
information, we compared psychiatric diagnoses ob-
The authors thank Moira Birmingham-Brewczynski, Ph.D., and Di-
ane Sholomskas, Ph.D., for their assistance in preparing best estimate
tamed from direct interview only with diagnoses based
diagnoses. on the best estimate procedure. We also assessed the
Copyright © 1992 American Psychiatric Association. degree to which the direct interview classified a subject

Am] Psychiatry 149:9, September 1992 1225

DIAGNOSIS BASED ON THE BEST ESTIMATE PROCEDURE

as having a psychiatric diagnosis when the best estimate TABLE 1. False Negative and False Positive Rates for Psychiatric
procedure did not. The batter may represent either false Diagnoses Based on Direct Interview Only Compared With Diagnoses
Based on Best Estimate Procedure
positive direct interview diagnoses or erroneous classi-
fications of best estimate diagnoses. False Ne gative Rates False Pos itive Rates

False False
Number Negatives Number Positives
of of
METHOD
Diagnosis Subjects N % Subjects N %

The 475 subjects in this study were obtained from a Depressive

large family study of opiate addiction (3). They in- disorders 275 57 21 200 3 2
Antisocial
cluded 201 opiate-addicted probands who sought treat-
personality 194 80 41 281 4 1
ment from a university-based clinic during the years Alcoholism 235 56 24 240 0 0
1 983-1 985 and 274 of their spouses and/or first-degree Drugabuse 326 23 7 149 0 0
relatives, who were also interviewed directly. All sub-
jects were at least 1 8 years old and gave informed con-
sent in accordance with the Yale University Human In- and drug abuse, had no false positive rate. The other two
vestigations Committee. diagnoses had minimal false positive rates.
Subjects were interviewed by trained clinical asses- We tested the generalizability of the enhanced accu-
sons using the Schedule for Affective Disorders and racy of the best estimates with subsamples of the data.
Schizophrenia (SADS) (4) and classified according to First, we compared the diagnoses of subjects for whom
Research Diagnostic Criteria (RDC) (5). Family history family history reports were available (N=439) with
data were collected by using the family history version those for whom they were not available (N=36). There
of the SADSIRDC. In the present paper, we report the were no group differences in the false negative rates for
lifetime rates (current or past) of major or minor de- the four diagnoses when the direct interview data were
pressive disorders, antisocial personality, alcoholism, compared with the best estimate data (range=0%-5%).
and drug abuse. Second, we compared those subjects whose medical
Two psychologists independently diagnosed the same records were available (N=128) with those whose re-
subjects by applying the best estimate procedure; they cords were not available (N=347). There were no
reviewed the data from direct interview, family history group differences for the diagnoses of depressive disor-
reports, and medical records. They discussed the diag- dens or drug abuse. However, when the direct inter-
noses on which they disagreed and obtained a consen- view diagnoses were compared with the best estimate
sus on each. The internater reliability was very good; diagnoses, the false negative rate for antisocial person-
mean kappa was 0.89 (nange=0.62-1.00). ality was 39% and the false negative rate for alcohol-
ism was 33% for the 128 subjects with medical record
data. These rates were significantly higher than the
RESULTS rates for the 347 subjects without medical records
available, for whom the false negative rate for anti-
Data obtained from the direct interview were avail- social personality was 1 8% (2=l6.2, df=1 , p<0.Ol) and
able for all 475 subjects diagnosed according to the the false negative rate for alcoholism was 15% (x2=
best estimate procedure. Additional information from 9.6, df=1, p<O.Ol).
medical records was available for 128 (27%) of these Finally, because almost half of the subjects were
subjects, and additional information from family his- treatment-seeking opiate addicts, which may have anti-
tory reports (i.e., interviews with family members about ficially enhanced the agreement between direct inter-
the subject) was available for 439 (92%) of these sub- view and best estimate data, we compared the diagnos-
jects. The mean number of family history reports from tic agreement between the two procedures for the
spouses and/on first degree relatives per subject was 2.2 treatment-seeking probands (N=201 ) and their family
(SD=2.2). members (N=274). We did not examine the group dif-
We calculated the percent of subjects who were given ferences for the diagnosis of drug abuse because the
each of the diagnoses according to the best estimate proband group was defined by this diagnosis. Thene
procedure but who were classified as not having the di- were no differences between the probands and their
agnosis according to the direct interview data. These family members for depressive disorders and alcohol-
false negative rates are given in table 1 . The rate of false ism. However, the false negative rate for antisocial pen-
negatives for all diagnoses ranged from 7% to 41%. sonality was 48% for the probands, compared with 9%
The greatest false negative rate occurred for antisocial for their family members (x2=7O.S. df=1, p<O.OO1).
personality, and the lowest occurred for drug abuse.
We also calculated the percent offalse positive diagno-
ses-subjects who were given the diagnosis by the direct DISCUSSION
interview method but not by the best estimate procedure.
These data are also given in table 1 . Overall, the false This study shows that when best estimate diagnoses,
positive rates were minimal. Two disorders, alcoholism based on multiple information sources, are compared

1226 Am] Psychiatry 149:9, September 1992

 THERESE A. KOSTEN AND BRUCE J. ROUNSAVILLE

with diagnoses made from direct interview only, there denreponting of cases (8-10). We have now shown that
are many false negatives and few false positives. The the direct interview itself may result in underestimates
false negative rate was particularly high for antisocial of diagnostic rates. When family history information,
personality, but the rates for the depressive disorders along with medical record data, are incorporated into
and alcoholism were also high. Among the drug-abus- the diagnosis by using the best estimate procedure, the
ing probands, the higher rate of diagnosis of antisocial number of positive cases is greatly increased. This in-
personality when the best estimate procedure was ap- crease in diagnoses with the best estimate procedure
plied was particularly striking. However, minimal on no may represent an enhancement in the accuracy of psy-
false positives were obtained from the direct interview chiatnic diagnoses if, as we assumed, this procedure is a
data compared with the best estimate data. These find- better standard, or it may reflect an increase in ernone-
ings support the utility of making best estimate diagno- ous diagnoses because more errors may occur with
ses for interviewed subjects. greater available information. Although we consider
We had expected that using the best estimate pnoce- the second possibility less likely, we have no other
dune would result in more diagnoses of disorders for standard by which we can compare these diagnoses.
which the criteria are based on information that sub-
jects may tend to withhold (6). Our finding that the
REFERENCES
smallest increase was in the rate of drug abuse diagno-
ses was undoubtedly due to the fact that family mem- 1. Dohrenwend BP: “The problem ofvalidity
in field studies of psy-
bers were aware that the probands were seeking drug chological disorders” revisited. Med 1990; 20:195-208
Psychol
abuse treatment. The higher rates of diagnoses of abco- 2. Leckman JF, Sholomskas D, Thompson WD, Belanger A, Weiss-
man MM: Best estimate of lifetime psychiatric diagnosis: a meth-
hol abuse and, particularly, of antisocial personality, odological study. Arch Gen Psychiatry 1982; 39:879-883
are in line with the concept that interviewees may with- 3. Rounsaville BJ, Kosten TR, Weissman MM, Prusoff B, Pauls D,
hold information related to these disorders but this in- Anton SF, Merikangas K: Psychiatric disorders in relatives of
formation can be gained by obtaining family history probands with opiate addiction. Arch Gen Psychiatry I 991; 48:
33-42
data and medical record data.
4. Endicott J, Spitzer RL: A diagnostic interview: the Schedule for
One finding that was particularly striking was the Affective Disorders and Schizophrenia. Arch Gen Psychiatry
usefulness of including data from medical records in the 1978; 137:837-844
best estimate diagnoses. The rates of diagnosis of two S. Spitzer RL, Endicott J, Robins E: Research Diagnostic Criteria:
of the four disorders-alcoholism and antisocial per- rationale and reliability. Arch Gen Psychiatry 1978; 36:733-782
6. Andreasen NC, Rice J, Endicott J, Reich T, Coryell W: The fam-
sonality-were significantly higher in subjects for whom ily history method approach to diagnosis: how useful is it? Arch
medical record data were available than in subjects for Gen Psychiatry 1986; 43:421-429
whom these data were not available. Previous research 7. Martin RL, Cloninger CR, Guze SB: The evaluation of diagnostic
also shows that medical record data enhance the rate of concordance in follow-up studies, II: a blind, prospective follow-
up of female criminals. J Psychiatr Res 1979; 15:107-1 25
antisocial personality and alcoholism diagnoses as well
8. Kosten TA, Anton SF, Rounsaville BJ: Ascertaining psychiatric
as drug abuse diagnoses (7). The probable reason that diagnoses with the family history method in a substance abuse
we did not find this effect with the drug abuse diagnosis population. J Psychiatr Res (in press)
was that almost half of the subjects were treatment- 9. Thompson WD, Orvaschel H, Prusoff BA, Kidd KK: An evalu-
seeking opiate addicts, which would make the medical ation of the family history method for ascertaining psychiatric
disorders. Arch Gen Psychiatry I 982; 39:53-58
record data somewhat obsolete. 10. Zimmerman M, Coryell W, Pfohl B, Stangl D: The reliability of
Diagnoses based on family history information, corn- the family history method for psychiatric diagnosis. Arch Gen
pared with those based on direct interview, result in un- Psychiatry 1988; 45:320-322

Am ] Psychiatry 1 49:9, September 1992 1227