Acta psychiatr. scand.

1985:72:239-245
Key words: Observer depression scales; Hamilton Depression
Scale; Bech Rafaelsen Melancholia Scale; Montgomery
Asberg Depression Rating Scale.

Comparative analysis of observer depression scales

W.Maier and M. Philipp
Department of Psychiatry (Head: Prof. Dr. O.Benkert),
University of Mainz, W. Germany

ABSTRACT - The Hamilton Depression Scale (HAMD), Bech Rafaelsen Melancholia

Scale (BRMS) and Montgomery Asberg Depression Rating Scale (MADRS) are analy-
zed according to mean discriminatory power, internal consistency. homogeneity and
transferability. The analysis was done separately in different samples of patients with
depressive syndromes: a) operationally defined depressive syndrome; b) Major Depres-
sive Disorder (RDC); c) Major Depressive Disorder. endogenous type (RDC). BRMS
and MADRS were superior to HAMD in all evaluated aspects. Further, the BRMS was
superior to MADRS according to the criteria of homogeneity and transferability.

Received November 25, 1984; accepted for publication February 9, 1985

The Hamilton Depression Scale (HAMD) (1,2)
is the most often used observer scale for the
assessment of severity of depression. Neverthe-
less, there are several shortcomings associated
with this scale: heterogenous and instable factor
analytical structure (3) ; missing general factor
(so-called severity factor) (4); missing course
validation ( 5 ) ; neglection of self-reported feel-
ings of distress in favor of the assessment of
behavioral symptoms and somatic complaints
( 5 ); insufficient separation between different
scores of global assessment of depression (6);
intermingling between frequency and intensity
of symptoms associated with depression (i.e.
patients with many low scoring symptoms and
patients with few high scoring core symptoms
may have the same sumscore in the HAMD).
In recent years two alternative scales have
been developed which are intended to overcome
the shortcomings of the HAMD. These are the
Bech Rafaelsen Melancholia Scale (BRMS)
(7,8) and the Montgomery Asberg Depression
Rating Scale (MADRS) (5). Both scales differ
from the HAMD in the following respects:
1. Categories of degree are more precisely
described.
2. Items are restricted to representing only
those symptoms considered to be core symp-
toms of depressive syndromes.
3. There are fewer items (11 in BRMS and 10 in
MADRS).
4. Items representing somatic complaints have
been reduced.
It is not clarified which observer depression scale
should be used (9).
As a comparative analysis of all three scales
(HAMD, BRMS, MADRS) has not yet been
made, it has been suggested that the scales
should now be compared in order to demon-
strate their strength and deficiencies (10). A con-
sensus on the criteria for judging the validity of a
severity scale of depression is lacking. Cattell
and Bech have introduced a new criterion for
scale transferability which has significance for
severity scales in psychiatry (11).
We aim to apply a list of criteria of adequacy
which should be met by a severity scale of de-
240 W. MAlER AND M. PHlLlPP

pression. In the process we will concentrate on Interview and rating procedure

the cross-sectional analysis of transferability and
homogeneity and ignore the aspects of course.
Patients and methods
Patients
Ratings were done with 151 consecutively admit-
ted in-patients of the Psychiatric Department of
the University of Mainz. All patients were
required to meet the following conditions:
depressive syndrome defined by criteria A and B
of Minor Depressive Disorder of the Research
Diagnostic Criteria (RDC) (12); age between 20
and 60 years; absence of physical disorders.
organic brain damage and drug or alcohol de-
pendence (Table 1).
Raters
Ratings were done by eight different raters. All
raters were physicians who had been trained in
psychiatry for at least 2 years and were familiar
with the Present State Examination (PSE), the
Hamilton Depression Scale and the Research
Diagnostic Criteria. In a test run before the
study itself was begun, interrater reliability on
the H A M D was shown to be sufficiently high in
a test-retest-setting with a time lag of up to 3 h
between two ratings. Each patient was rated by
one rater only, applying all scales mentioned
below in a fixed order.
All patients were interviewed with the Present
State Examination (13) during the first 3 days
after admission. Immediately after the PSE was
given the same rater judged the patient sepa-
rately on a number of rating scales including the
three scales under study. These ratings were
done by an additional free interview using
always the following order of scales: HAMD,
BRMS, MADRS. Finally, several criteria lists of
operational diagnostic systems were checked
including the criteria A and B of Minor Depres-
sive Disorder of the Research Diagnostic
Criteria. The whole rating procedure lasted for
about 2 to 3 h.
Choice of criteria of adequacy
Because of methodological considerations we
decided to restrict the evaluation to those
criteria of adequacy which can be judged by
analyses within a given scale; we did not use
those criteria which rely on a comparison of
scales with each other. Therefore no statements
are given about the validity of the scales in terms
of content validity and concurrent validity.
The following criteria of adequacy were
choosen:
Discriminating power. In item analysis the dis-
crimination coefficient should differ significantly
from 0 in any item and the mean should not be
low (14). A mean scale value of at least 0.32 is
considered to be adequate (c.f. (3)).

Table 1 Distribution of sum scores. In item analysis the

Patients and RDC-diagnoses
distribution of the sum scores should not differ
All patients n = 151 significantly from normal distribution. This
male n = 43 should be true for any sample under study.
female n = 108
Age mean = 38.4years Internal consistency. The scale should show a
range = 20-60years
satisfactory internal consistency as a measure of
RDC 9 MDD definite n = 99 test reliability (the alpha-coefficient of Cronbach
(9f MDD endogenus type n = 66) (see Lienert (15)) should be higher than 0.75).
(7MDDBP1 n = 16)
(8MDDBP2 n = 9)
Content validity. There should be no correlation
RDC 10 MIN definite n = 25
10 MIN probable n = 2 between the sum score and other variables which
RDC 11 INT n = 2 are not connected with the quality to be assessed
RDC 3 SAD depr. type n = 17 by the scale (age, sex, etc.).
RDC 4 DSlresxhiz. n= 6
 COMPARISON OF OBSERVER DEPRESSION SCALES 241

Homogeneity. The relevance of this criterion has
been stressed by Bech (11,16). The concept of
homogeneity aims at the uniformity of the rela-
tion between the scores of each pair of two dif-
ferent items of the scale operationalized by a
coefficient for each pair of items (Mokken,
referring to the Loevinger coefficient); the coef-
ficients can be summed up to the coefficient of
scale ability of the scale. If this coefficient is be-
tween 0.30 and 0.40 the scale is said to be weak;
if it exceeds 0.40 the scale is medium (17).
Transferability. Transferability means independ-
ence of item interrelations of the sample under
study and of the external criteria of the patient
sample. This concept is operationalized in the
statistical concept of local independence, a
charateristic feature of the Rasch model and of
other latent-class models. The importance of this
criterion has been stressed by Cattell (20) and
Bech (11,16,18). Bech has furthermore em-
phasized the relevance of Rasch model testing
for the assessment of transferability.
Lord & Novik (14, p. 382) vote for a high fac-
tor analytical homogeneity (one-dimensionality
in factor analysis of a scale) as a prerequisite of
scales with adequate transferability (latent-class
models with local independence). However, the

Table 2
Descriptive analysis of observer rating scales for depression

Total group Subgroup I Subgroup I1

Depressive syndrome Major depressive Endogenous type
(n=ljl) disorder (n=99) (n=66)

HAMD (21 items)

Mean 23.4 25. I 26.8
Median 24.0 25.5 27.5
SD 0.69 0.67 0.75
Intern. consistency* 0.76 11.64 0.62
Homogeneity** 0.13 0.09 0.08
Discrim.power (mean) 0.39 0.24 (1.23

HAMD (17 items)

Mean 20.8 22.5 24.0
Median 21.3 22.7 24.7
SD 0.62 0.62 0.72
Intern. consistency* 0.77 11.67 0.66
Homogeneity** 0.15 0.11 0.11
Discrim.power (mean) 0.38 0.28 0.28

BRMS
Mean 17.8 19.1 20.5
Median 17.9 1x.x 20.5
SD 0.56 11.62 0.80
Intern. consistency* 0.81 (1.77 0.80
Homogeneity** 0.30 0.26 0.31
Discrim.power (mean) 0.52 0.47 0.50

MADRS
Mean 26.0 29.2 31.5
Median 27.0 30.0 32.2
SD 0.90 0.91 1.10
Intern. consistency* 0.86 0.XI) il.80
Homogeneity* * 0.31 0.26 0.24
Discrim.power (mean) 0.60 11.51 0.59

* alpha-coefficient of Cronbach (15)

** Loevinger coefficient (17)

16
242 W. MAlER AND M. PHlLlPP

relevance of this criterion has been questioned patients with Major Depressive Disorder (RDC)
(Cattell (20, p.379) and Bech (11)). Therefore and patients with Major Depressive Disorder,
factor analysis is not presented here. endogenous subtype (RDC).

Statistical analyses
The above-mentioned criteria of adequacy
require that several different methods of analysis
must be administered. For analysis the following
methods are necessary: item analysis including
the aspects of discriminating power, test of devi-
ation from normal distribution (Kolmgoroff-
Smirnow test (21)), internal consistency,
homogeneity (Mokken analysis (17)) and Rasch
model fitting (19,22).
Both item analysis and analysis of consistency
were made. After dichotomizing the item scores
according to the mean in the sample (in order to
preserve variance (23) a RASCH analysis was
done and the coefficient of homogeneity
(Loevinger; see Mokken (17)) determined. To
control this particular method of dichotomizing
item scores a further method was applied: all
items were dichotomized between score 1 (the
second score) and score 2 (the third score).
We analysed the data of the whole patient
group as well as the data of two subgroups:
Results
The part-whole-corrected coefficient of discrimi-
nation in mean shows lower values with the
H A M D than with the other two scales (Table 2).
Only the H A M D included items which have dis-
crimination coefficients that do not differ sig-
nificantly (P>O.Ol) from zero. This is found for
items 15 (hypochondriasis), 16 (loss of weight)
and 18 (diurnal variation). Items 3 (suicide), 6
(insomnia, delayed), 13 (somatic, general) and
14 (loss of libido) in at least one of the popula-
tions show relatively low coefficients of discrimi-
nation (0.18) indicating high heterogeneity. In
the analysis of Baumann (3) the discriminating
power of items 4 and 6 of the 17-item H A M D
does not differ significantly from zero (P>O.Ol).
In the MADRS and the BRMS all items show a
significant deviation from zero in all groups.
While the MADRS has a discrimination coeffi-
cient of more than 0.40 in all items the BRMS

Table 3
Correlation of observer rating scale sum scores with other variables
(Spearman's correlation coefficient)

Total group Subgroup I Subgroup I1

Depressive syndrome Major depressive Endogenous type
(11=151) disorder (11 =Y9) (r1=66)

HAMD (21 items)

Age 0.13 I).16 0.05
Diagnosis* - - 0.1 1/0.34**

HAMD (17 items)

Age 0.14 0.12 0.10
Diagnosis* - - l).4O/O.32* '

BRMS
Age 0.06 0.20 0.03
Diagnosis' - - 0.3510.30 **

MADRS
Age 0.13 0.18 0.08
Diagnosis* - - 0.46/0.39* *

* Point biserial correlation coefficient

** Left figure: total group; right figure: subgroup of Major depressive disorder
 COMPARISON OF OBSERVER DEPRESSION SCALES 243

has values of 0.10 in item 9 and 0.20 in item 10.
All other item scores are higher than 0.40. The
lower limit of the discriminating power of a scale
should be 0.32 according to Baumann (3). This is
the case for BRMS and MADRS but not for
HAMD in subgroups I and 11.
None of the sum scores of the three scales
shows a significant deviation from normal dis-
tribution in the Kolmgoroff-Smirnow test (21).
This is true for all groups. (Table 2)
The alpha-coefficient of Cronbach is adequate
for the MADRS and the BRMS in all popula-
tions. For the HAMD it is relatively lower.
Nevertheless, the internal consistency of the
HAMD is satisfactory (Table 2).
No significant correlations were found be-
tween the degree of severity and such external
variables as age or sex (Table 3).
After dichotomizing the item scores at the
empirical mean in each sample all Loevinger
coefficients of homogeneity (17) significantly dif-
fer from zero (Table 2). In HAMD they show a
high heterogeneity (0.08 to 0.13); in the other
two scales they are slightly higher than in
HAMD: here they are clearly more acceptable
for BRMS (0.26 to 0.31) and for MADRS (0.24
to 0.31) according to Mokkens criterion of
>0.30 for a weak homogeneous scale, although
even for these scales there are still problems.
The results are identical after dichotomizing the
item scores according to the second method (see
statistical analyses).
When examining the homogeneity of a scale
by the method of Rasch the tranferability for
different populations is of decisive importance.
Therefore the samples under study have been
divided into two parts in which the parameters of
difficulty are estimated independently (ratio
scale). After this, both parameters were com-
pared by a Chi-square test. For our purpose the
total sample was divided according to the follow-
ing criteria: by chance, by age, by the median of
the sum score of the scale under study, and divi-
sion of the sample according to the diagnosis
(ICD 296 versus ICD 11011-296).The fitting of the
Rasch model will be expressed by a Chi-square
value according to Andersen (22) which shows
the significant deviations from the conditions of
the Rasch model.
After division of the samples by chance there

Table 4
Observer rating scales for depression: adequacy of RASCH model
-
Rationale for partition

Randomized Partition Partition Partition Partition

partition by age by sex by diag- by median
Scale nosis of scale

HAMD a) 21.9 (20df) 63.0 50.7 98.7 62.6

(21 items)
b) 0.35 .(I0 .oo .oo .OO

HAMD a) 20.0 (16df) 48.1 39.3 67.9 39.6

(17 items)
b) 0.22 .00 .oo .oo .00

BRMS a) 12.3 (1Odf) 21.9 24.6 20.9 65.6

b) 0.27 0.02 0.01 0.03 .00

MADRS a) 6.0 (9df) 26.1 25.5 34.1 35.5

b) 0.74 .00 .00 .oo .00

a ) Chi-square values b) empirical P-values

*: after dichotomizing at the mean
 244 W. MAlER AND M. PHlLlPP
are no significant deviations from the model in
any of the scales. In all other divisions the
HAMD is more inhomogeneous in comparison
with the BRMS and the MADRS. Neither of the
latter scales meets the conditions of the model in
all division procedures. However, in divisions
according to diagnosis, age and sex, the BRMS,
in contrast to the MADRS, shows no significant
deviation from the Rasch model. The items on
the BRMS which interfere most with the Rasch
model are Nos. 9 (insomnia) and 10 (suicide).
These results are true for the total group as well
as for the patients with Major Depressive Disor-
der. The significant deviations from the Rasch
model (P=0.05 and P=0.01) are identical when
item scores are dichotomized according to the
second method (see statistical analyses).
Discussion
The 21-item version of the HAMD is not shown
to be superior to the 17-item version. None of
the above-listed criteria is better satisfied by the
21-item version than by the 17-item version.
Even the expected gain in reliability is lacking.
Furthermore, high heterogeneity is indicated by
the low mean discrimination power of the items,
the Loevinger homogeneity coefficient and the
Rasch analysis.
Our results confirm those in other studies, e.g.
Bech et al. (24) in which the HAMD only
showed a low degree of homogeneity (Mokkens
coefficient of scalability, mean discriminating
power, internal consistency) and a low Rasch
model fitting. Both BRMS and MADRS have a
higher degree of homogeneity and a better
Rasch model fitting than the HAMD. The
BRMS only deviates from the Rasch model
regarding the partition criterion (partition of the
sample at the median of the sum score of the
scale). This is mostly because items representing
somatic complaints (particularly item 9) are as
frequent and intensive in patients with low sum
scores as they are in those with high sum scores
on the BRMS. This, however, is not the case for
all other items. The MADRS shows an insuffi-
cient Rasch model fitting in all partitions of the
sample with the exception of random partition.
This demonstrates a low degree of transferabil-
ity, although better than is the case with the
HAMD. Thus, according to these criteria the
BRMS is superior. However, both scales need
increased homogeneity and transferability. Our
results indicate that the homogeneity of the
BRMS could be increased by omitting items 9
(insomnia) and 10 (tiredness and pains). The
same applies to item 4 (insomnia) in the
MADRS.
In another study (25) it has been shown that
by shortening to 6 items the HAMD is better
able to meet the criteria of adequacy used here.
In this case the shortened version of the HAMD
would have even better validity than the BRMS
and the MADRS: the homogeneity of the
HAMD subscales is higher (Rasch analysis and
Loevinger coefficient) than in the more com-
prehensive scales HAMD, MADRS and BRMS;
no criterion used in this study has shown a
superiority of these three scales. Further, inter-
rater reliability of the HAMD subscale of Bech
is similar to the reliability of the BRMS (7). This
is another argument for reducing the BRMS and
MADRS to obtain more homogeneous and shor-
ter scales without loss of validity and reliability.
In conclusion we consider that the BRMS and
the MADRS are superior to the HAMD accord-
ing to all criteria under study. These findings
argue for further comparisons of these three
scales in validation studies. We suggest the
BRMS and MADRS should be used in conjunc-
tion with the HAMD in therapy studies. Accord-
ing to the criteria of transferability and
homogeneity the BRMS seems more adequate
than the MADRS. However, the results are not
conclusive and require further affirmation. In
addition, in order to decide between the BRMS
and MADRS, the two scales should be com-
pared on sensitivity to change.
Acknowledgement
This study was supported in part by Smith Mine Dauelsberg
Gmbh, Miinchen and Troponwerke, Koln.
References
1. Hamilton M. A rating scale for depression. J Neurol
Neurosurg Psychiatr 1960:2356-62.
 COMPARISON OF OBSERVER DEPRESSION SCALES
2 . Hamilton M. Development of a depression scale for prim-
ary depressive illness. Br J SOC Clin Psychology 1967:
6:278-296.
3. Baumann U. Methodische Untersuchungen zur Hamilton-
Depressions-Skala. Arch Psychiatr Nervenkr 1976:
222:359-375.
4. Maier W, Philipp M, Gerken A. Dimensionen der Hamil-
ton-Depressions-Skala. Arch Psychiatr Nervenkr: (in
press).
5. Montgomery P, Asberg M. A new depression scale
designed to be sensitive to change. Br J Psychiatry
1979:134:382-389.
6. Bech P. Gram L F. Dein E. Jacobson 0, Vitger J. Bolwig
T G. Quantitative rating of depressive states. Acta
Psychiatr Scand 1975:51:161-170.
7. Bech P, Bowlby T G. Kramp P. Rafaelsen 0 J. The Bech-
Rafaelsen Melancholia Scale and the Hamilton Depres-
sion Scale. Acta Psychiatr Scand 1979:59:420-430.
8. Rafaelsen 0 J , Bech P. Bowlig T G, Kramp P, Gjerris A.
The Bech Rafaelsen Combined Rating Scale for Mania
and Melancholia. In: Achte K. Aalberg V, Lonquist J ,
eds. Psychopathology of depression. Acta Psychiatr Fen-
nica Suppl. 1980.
9. Checkley S A, Rush A J , Beckmann H, et al. Functional
indices of biological disturbances. In: Angst J, ed. The
origins of depression: current concepts and approaches.
Berlin: Springer, 1984.
10. Snaith R P. Rating scales. Br J Psychiatry 1981: 138512-
514.
11. Bech P. Rating scales for affective disorders: their validity
and consistency. Acta Psychiatr Scand 1981: Suppl. 295.
12. Spitzer R L, Endicott J. Robins, E. Research Diagnostic
Criteria: rationale and reliability. Arch Gen Psychiatry
1978: 35:773-785.
13. Wing J K, Cooper J E , Sartorius N. The measurement and
classification of psychiatric symptoms. (An instruction
manual for the PSE and CATEGO system.) London:
Cambridge University Press, 1974.
245
14. Lord F M, Novik M R. Statistical theories of mental test
scores. Reading: Addison-Weslay , 1968.
15. Lienert G A. Testaufbau und Testanalyse. Weinheim:
Beltz Verlag, 1969.
16. Bech P. The instrumental use of rating scales for deprcs-
sion. Pharmacopsychiatry 1984:17:22-29.
17. Mokken R J. A theory and procedure of scale analysis.
The Hague, 1974.
18. Bech P. Assessment scales for depression: the next 20
years. In: Bech P. Hippius H. eds. Acta Psychiatr Scand
1983: Suppl. 310:117-130.
19. Rasch G . Probabilistic models for some intelligence and
attainment tests. Copenhagen: Danish Institute of Educa-
tional Research, 1960.
20. Cattell R B. Personality and mood by questionaire. San
Francisco: Jossey-Bass Publishers, 1973.
21. Kendell M G, Stuart A. The advanced theory of statistics.
vol 2. London: Griffin, 1976.
22. Andersen E B. A goodness of fit test for the RASCH
model. Psychometrika 1973:38:123-140.
23. Cohen J. The cost of dichotomization. Appl Psychol. Mea-
surement 1983:7.249-255.
24. Bech P. Allerup P. Gram F, et al. The Hamilton Depres-
sion Scale. Evaluation of objectivity, using logistic mod-
els. Acta Psychiatr Scand 1981:63:292-299.
25. Kxier W. Philipp M. Improving the assessment of the sev-
erity of dcprcssive states: reduction of the Hamilton De-
pression Scale. Pharmacopsychiatry 1985:18:114-115.
Address:
Wolfgang Maier
Michnel Philipp
Dept. of Psychiatry
University of Mainz
Langenbeckstrasse 1
6500 Maim
W. Germany