This document is copyrighted by the American Psychological Association or one of its allied publishers.

This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Psychology of Addictive Behaviors

1987, Vol 1. No 3. 14A-153

ISSN 0893-164X, Society of

Psychologists in Addictive Behaviors

Advances in the Assessment of Adolescent

Chemical Dependency:
Development of a Chemical Use Problem
Severity Scale
Ken C. Winters and George A. Henly

Chemical Dependency Adolescent Assessment Project

St. Paul, Minnesota
This paper discusses the development of the Personal Involvement with Chemicals
Scale (PICS), a measure of the extent of psychological involvement in alcohol and
drug use by adolescents. Scale development and initial reliability and validity
evidence are summarized.
Today there is a great deal of concern about psychoactive substance use
by American teenagers. The extent of adolescent substance involvement in
this country has been well established (e.g., Johnston, Bachman, & O'Malley,
1986). In response, the adolescent chemical dependency treatment field is
expanding to treat teenagers with the most serious problems.
Unfortunately, the increasing concern about adolescent chemical involvement comes in the absence of extensive scientific knowledge about the problem. Contributing to this knowledge gap is the lack of valid, standardized
assessment tools for measuring clinical characteristics of adolescent chemical
involvement (Owen & Nyberg, 1983; Winters & Henly, in press). While many
such instruments exist for adult populations (e.g., National Institute on Alcohol Abuse and Alcoholism, 1985; Selzer, 1971; Wanberg & Horn, 1983),
only brief screening tools (e.g., the Adolescent Alcohol Involvement Scale; Mayer
and Filstead, 1979) or high school survey instruments (National Institute on
Drug Abuse, 1979) are available for use with adolescents.
The Chemical Dependency Adolescent Assessment Project was established
in 1982 to develop a comprehensive, clinically oriented assessment battery
to aid in identification, referral, and treatment of problems associated with
chemical use by adolescents. The project consists of a consortium of chemical
dependency organizations and researchers in Minnesota. This paper focuses
on a portion of the project's efforts: the development of a general measure

Major funding for this research is provided by a grant from the Northwest Area
Foundation. Additional funding and administrative support services are provided by
the Saint Paul Foundation, in conjunction with the Amherst H. Wilder Foundation.
Data collection associated with this report was greatly assisted by Minnesota agencies
and treatment facilities in the Adolescent Assessment Project Consortium and by Straight,
Inc. (Boston) and the Saskatchewan Alcohol and Drug Abuse Commission.
Request for reprints should be sent to Ken Winters, Wilder Research Center, 1295
Bandana Boulevard North, Suite 210, St. Paul, MN 55108.
146

Adolescent Chemical Dependency

]47

of adolescent chemical use problem severity, the Personal Involvement with

Chemicals Scale (PICS).'

Method

This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Scale Construction
The rationale for developing the PICS was based on the project's discovery during
the course of research in the measurement of multiple dimensions of adolescent chemical abuse that a strong, general factor was present reflecting psychological involvement
with chemicals. Such a scale, by itself, could serve as a research or screening scale. It
would also serve as a complement to the more specific scales of the full multidimensional inventory.
Based on a review of the literature and on consultation with adolescent service
providers, an initial pool of 203 chemical use problem seventy items was written to
reflect 24 identified content categories. Because research indicates that teenagers often
abuse multiple substances (Johnston et al., 1986), the items were worded for chemical
use in general rather than for specific substances. (Other portions of the inventory
address use of specific drugs.) Items were at approximately a sixth-grade reading level
(Fry, 1977) and were formatted to have two (yes/no), three (never/once or twice/more
than once or twice), or four (never/once or twice/sometimes/often) response options.
A subset of 40 items (all with a 4-option format) was identified that (a) had substantial
(>.5O) loadings on the first principal component underlying responses to the 203-item
pool when administered to a chemical dependency treatment sample (N=398) and (b)
clustered along the diagonal when two principal components were extracted and rotated to varimax solution. The purpose of these analyses was to identify a strongly
unidimensional set of items, so that item reponse theory (IRT) scaling procedures could
be applied. When an item set is essentially unidimensional, one or more models from
a family of logistic IRT models will adequately describe the observed responses to the
items of the set. The basic idea behind these measurement models is that both items
and persons can be positioned above a latent trait continuum (in this case, a psychological involvement with chemicals continuum). Also, item responses can be explained,
in a probabilistic sense, by the relative positions of the person and the item.
There are several important advantages with IRT-based measurement (Hubin, Drasgow, & Parsons, 1983). The most salient of these in chemical dependency assessment
is that IRT models provide a formal basis for identifying individuals whose response
patterns are statistically anomalous; that is, improbable under the assumption of the
model (Levine &. Drasgow, 1982). Those identified cases might reflect invalid responding or truthful reporting of behavior so atypical that the measurement process associated with the scale is importantly degraded.
Several measurement models have been proposed that are suitable for use with
Likert format items (Thissen & Steinberg, 1986). Because of the relatively small sample
of respondents, the rating scale model (Andrich, 1978) was chosen over more complex,
more highly parameterized models (e.g., Masters, 1982; Samejima, 1969). Joint maximum likelihood estimation procedures and item misfit statistics suggested by Wright

The PICS is one of several scales that is contained in a recently developed, multidimensional questionnaire, the Personal Experience Inventory (PFJ). Interested readers
should refer to Henly and Winters (1987, 1988) for more information on the development of the PEI. For information on its publication, contact George Huba, Director
of Research and Development, Western Psychological Services, 12031 Wilshire Boulevard, Los Angeles, CA 90025.

This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

148

Ken C. Winters and George A. Henly

and Masters (1982) were used to arrive at a final set of 29 PICS items that approximate
a statistical fit to the rating scale model.
The Andrich model permits the 29 items to be ranked according to item difficulty.
In determining item difficulty for polychotomous items, the model assigns each item
a position on the theoretical problem severity continuum. Position is based on the
threshold or difficulty associated with the item's response options. Thus, for "easy"
items, the often response option was frequently endorsed by subjects. For "difficult"
items, the often option was infrequently endorsed by subjects. (Conversely, relatively
low endorsement rates of the never option characterize easy items, and relatively high
endorsement rates of the never option characterize difficult items.) Inspection of the
content of items at various points along the continuum suggest five important changes
in the nature of chemical use behavior social/recreational use; mood-related use; adjustment of activities to accommodate use; engaging in deviant behavior to accommodate use; and use related to withdrawal effects.1

Scale Evaluation Measures

Internal consistency reliability. Coefficient alpha was used to estimate the scale's reliability. Separate analyses were conducted using two chemical dependency samples
(development and replication) and a nondinical school sample.
Temporal Stability. The scale's test-retest stability (product moment correlation) was
determined on two samples: a 1-week retest chemical dependency treatment sample
and a 1-month retest chemical dependency waiting list sample.
Criterion Validity. Criterion-related validity was evaluated by testing for scale score
mean differences among groups expected to differ on problem severity. The groups
were defined by (a) type of referral received (no service received/not referred for chemical dependency treatment/referred for chemical dependency treatment) and (b) drug
preference reported by client (alcohol/marijuana/alcohol and marijuana/other drug or
combination).
1. Type of referral. Study adolescents were grouped according to (a) no service
received (N=100); (b) evaluated but not referred for chemical dependency treatment
(N = 108); and (c) evaluated and referred for chemical dependency treatment (N = 360).
The no service or control group consists of students from a random sample of classrooms
across Grades 7-12 in two St. Paul, Minnesota, schools. Clients in the not referred group
were evaluated for chemical dependency treatment because a problem was suspected,
but they did not receive a primary chemical dependency treatment referral. Clients in
the referred group received a recommendation of primary chemical dependency treatment based upon recommendations made by evaluators who were blind to PICS results.
It was anticipated, then, that the three groups should differ in level of problem severity.
2. Drug preference. Group assignment was determined by the response to drug
preference items embedded among the PICS items. Respondents from the clinical
sample were assigned to one of four groups: (a) alcohol (N= 173), (b) marijuana (N= 198),
(c) alcohol and marijuana (N=173), or (d) other drug or combination (N=55). Since
these group designations reflect increasing chemical involvement in relation to illicitness and number of drugs used, it is expected that the four groups will differ from
each other. (School subjects were not included in this analysis.)

While the ranking of PICS items is suggestive of a stage or progression model of

adolescent drug use, it is important to keep in mind that study subjects were instructed
to respond to the items based on lifetime experiences. Thus, the ordering of PICS items
does not provide direct evidence for the actual temporal sequencing of the experiences
reflected by the scale items.

Adolescent Chemical Dependency

149

This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Subjects
The clinical samples in this study can be divided into three separate groups (A, B,
C). Group A consists of consenting adolescents ages 12 to 18 (N = 646) receiving chemical dependency evaluation or treatment at 16 Minnesota sites. Consent from a parent
or guardian was required for participants who were minors. The Minnesota participating programs include both hospital and free-standing units that provide evaluation
or treatment services on either an inpahent or outpatient basis. Twelve of the 16 sites
are located in the Minneapolis-St. Paul metropolitan area; the remaining sites are in
rural locations. Group B is a 1-week retest sample (N = 63) obtained from two residential
chemical dependency facilities, one based in Minnesota and the other based in Massachusetts. Group C is a 1-month retest sample of consecutive adolescent (N = 42)
referrals who received an outpatient chemical dependency assessment at a Regina,
Saskatachewan evaluation center and were put on a chemical dependency treatment
waiting list.
The school sample was collected from a junior and a senior high school in the St.
Paul school district, both of which are known for relative ethnic diversity. Three classrooms from each grade level (7-12) were randomly chosen. Parental consent was required for student participation. The sample of school participants represents
approximately 66% of the eligible sample.
PICS construction was based on a subset of Group A, a development sample (N = 398)
These consenting subjects, who were approached between March and August 1984,
constituted 63.2% of those eligible to participate (N = 630). The remainder of Group A
(N = 248) constituted a replication sample on whom an independent internal consistency
reliability estimate was based. These subjects consented to participate between August
and November 1984, and represent 54.5% of those eligible to participate.1 The Group
A development and replication samples were combined for the criterion validity analysis.
The 1-week retest sample (Group B), collected between January and December 1985,
represents 100% of the eligible subjects. The 1-month retest waiting list sample (Group
Q consists of consecutive admission adolescents seen at the Regina, Saskatchewan
clinic during a 3-month period in 1986. This sample represents 90% of the eligible
sample.
Table 1 provides a summary of demographic information for the various samples.
Participants are predominantely white and between ages 15 and 17.
Procedure
The PICS items were administered in the context of the larger problem severity
questionnaire. For the development and replication samples (Group A), testing occurred as soon as practical after initial contact with client, typically within the 1st or
2nd day of admission. For the retest samples (Groups B and C), initial testing occurred
during the first contact with the facility. Group B was retested after spending 1-week
in a chemical dependency evaluation and a pretreatment orientation program. Group
C was retested after a 1-month waiting period.
Staff members at participating treatment programs administered the questionnaire
and provided referral decisions. Program staff were blind to PICS results during all

1
Comparisons of participants and nonparncipants were computed separately for the
two Group A samples. Each analysis failed to detect statistically significant differences
between the participant and nonparticipant groups on either demographic (sex, age,
race, prior treatment history) or psychological (Minnesota Multiphasic Personality Inventory scale scores) variables.

Ken C. Winters and George A. Henly

150
Table 1

This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

PICS Sample Characteristics

Clinical samples
Group A
Group B
replication
retest
(N=248)
(N = 63)

School sample
Group C
retest
(JV=42)

(N=100)

Variable

Group A
development
(N=398)

Sex
Male
Female

228
151

149
97

43
20

42
0

66
34

Age
12-15
16-19

148
229

107
138

23
40

7
35

60
37

Race
White
Black
Other

345
7
27

233
3
11

59
0
3

34
0
8

74
11
14

Note. Because of missing data, numbers may not add to the totals.

phases of the study. Research staff members and school teachers administered the
questionnaire to the high school students.
Invalid questionnaires were excluded before analyses were conducted. Exclusion
was based on omitted responses and scores on the invalid response scales embedded
in the larger inventory.4 More conservative rules were employed on the development
sample in order to eliminate as many sources of contamination as possible.

Results
Psychometric Tests
A summary of the internal consistency and test-retest reliability data is
provided in Table 2. The estimate of internal consistency reliability for the
PICS is .97 for the two clinical samples and .95 for the school sample. The
obtained test-retest correlation coefficients for the 1-week evaluation and 1month waiting list samples are .72 and .87, respectively.
Criterion Validity
Table 3 provides a summary of the results of the PICS validity tests. For
the service setting analysis, an overall F test of group differences was not
conducted on the PICS because the homogeneity of variance assumption was
not tenable. Instead, pairwise group comparisons (f-tests) were performed.
All three pairwise group comparisons showed statistically significant differences, in the expected direction.
A one-way analysis of variance testing for drug preference group differences on the PICS was significant (p < .001). Post hoc Student-Neuman-Keuls

* The defensiveness scale consists of 31 items adapted from the Marlowt-Crawne Social
Desirability Scale (Crowne & Marlowe, 1960). The 11-item infrequency scale evaluates
faking bad tendencies (see Henly & Winters, 1988).

Adolescent Chemical Dependency

151

This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Table 2
Reliability of the PICS

Test
Coefficient alpha
Test-retest
correlation

Sample
Group B1
Group A
1-week
Group A
retest
development replication
.97

.97

Group O
1-month
retest

School
.95

.72

.87

One-week evaluation/pretreatment orientation period.

2
One-month waiting list period.

Table 3
PICS Scores by the Criterion Group
Criterion

Mean

SD

Pairwise
comparisons

39.9

13.9

3 vs. 2

1.95 (194.5)' <.O5

54.5

19.4

1 vs. 3

5.17 (345.5)1 <.001

73.8

21.0

2 vs. 3

8.50 (466)

t(df)

P(df)

Type of referral

No service (control)
Not referred for
chemical dependency
treatment
Referred for chemical
dependency treatment

<.001

Drug preference

Alcohol
64.6
22.5
Marijuana
82.2
22.3
<.001 47.25
Alcohol and marijuana 88.2
19.6
(3/533)
Other
96.1
17.7
1
Estimated degrees of freedom for the separate variance West.
2
All possible post hoc pairwise tests for drug preference reveal statistically significant

differences.
tests for mean differences, with experimentwise alpha level set at .05, were
then conducted on the drug groups. All post hoc tests were significant in
the expected direction.

Discussion
Generally speaking, the data presented here on the psychometric properties
of the PICS are encouraging. The internal consistency reliability estimates are
quite high, as is the test-retest stability estimate for the 1-month waiting list
sample. The pattern of relationships observed between PICS scores and the
criterion variables is supportive of the scale's construct validity. These initial
data provide support for the view that the PICS is reliable and can differentiate
groups believed to vary in level of problem severity. While the retest analysis
indicates that the PICS is influenced by siruational variables (particularly in
Group B where formal clinical evaluation and pretreatment orientation had
transpired), the waiting list group results indicate that the scale's stability is
quite good in the absense of evaluation or pretreatment program effects.

This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

152

Ken C. Winters and George A. Henly

The findings reported here should be viewed with caution; additional evaluation of the PICS is needed. The project is preparing reports of studies that
investigate the relationship of the PICS with measures of response bias and
with other indices of chemical use problem severity (e.g., the Adolescent Alcohol
Involvement Scale; DSM-lil-R diagnoses). In addition, the project is collecting
normative data on clinical and nondinical samples, conducting additional
temporal stability tests, and initiating predictive validity studies (e.g., forecasting treatment completion and treatment outcome). If results of these
studies are similarity supportive, it is hoped that the PICS will find widespread use in a field that currently lacks standardized, clinically oriented,
assessment tools.
References
Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrike,
43, 561-573.
Crowne, D. P., & Marlowe, D. (1960). A new scale of social desirability independent
of psychopathology. Journal of Consulting Psychology, 24, 349-354.
Fry, E. (1977). Fry's readability graph: Clarifications, validity, and extension to level
17. Journal of Reading, 22, 242-253.
Henly, G. A., & Winters, K. C. (1987). Development of psychosociat scales for the assessment
of adolescents involved with alcohol and drugs. Unpublished manuscript.
Henly, G. A., & Winters, K. C. (1988). Development of problem severity scales for the
assessment of adolescent alcohol and drug abuse. The International journal of the
Addictions, 23. 65-85.
Hulin, C. L., Drasgow, F., & Parsons, C. K. (1983). Item response theory: Application to
psychological measurement. Homewood, IL: Dow Jones-Irwin.
Johnson, L. D., Bachman, J. G., & O'Malley, P. M. (1986). Monitoring the future: Questionnaire responses from the nation's high school seniors, 1985. Ann Arbor: Survey Research
Center, Institute for Social Research.
Levine, M. V., & Drasgow, F. (1982). Appropriateness measurement: Review, critique,
and validating studies. British Journal of Mathematical and Statistical Psychology, 35, 4256.
Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149174.
Mayer, J., & Filstead, W. F. (1979). The Adolescent Alcohol Involvement Scale: An instrument for measuring adolescents' use and misuse of alcohol. Journal of Studies on
Alcohol, 40, 291-300.
National Institute on Alcohol Abuse and Alcoholism (1985). Alcoholism treatment assessment research instruments (DHHS Publication No. ADM 85-600536). Washington,
D.C.: U.S. Government Printing Office.
National Institute on Drug Abuse (1979). Drug abuse instrument handbook (DHEW Publication No. ADM 79-394). Washington, D.C.: U.S. Government Printing Office.
Owen, P. L., & Nyberg, L. R. (1983). Assessing alcohol and drug problems among
adolescents: Current practices. Journal of Drug Education, 13, 249-254.
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded
scores. Psychometrika Monographs, 17.
Selzer, M. L. (1971). The Michigan Alcoholism Screening Test: The quest for a new diagnostic instrument. American Journal of Psychiatry, 127, 1653-1658.
Thissen, D. & Steinberg, L. (1986). A taxonomy of item response models. Psychometrika,
51, 567-577.
Wanberg, K. W., & Horn, J. L. (1983). Assessmemt of alcohol use with multidimensional concepts and measures. American Psychologist, 38, 1055-1069.

Adolescent Chemical Dependency

153

Winters, K. C, &HenIy, G. A. (in press). Assessing adolescents who misuse chemicals:

The Chemical Dependency Adolescent Assessment Project. National Institute on Drug

This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Abuse Monograph Series.

Wright, B. D., & Masters, G. N. (1982). Rating scale analysis. Chicago: MESA Press.