You are on page 1of 20

Scientometrics (2014) 101:1695–1714

DOI 10.1007/s11192-014-1338-z

Post-interdisciplinary frames of reference: exploring


permeability and perceptions of disciplinarity
in the social sciences

Timothy D. Bowman • Andrew Tsou • Chaoqun Ni • Cassidy R. Sugimoto

Received: 11 November 2013 / Published online: 4 June 2014


Ó Akadémiai Kiadó, Budapest, Hungary 2014

Abstract The ProQuest Dissertations and Theses database contains records for approx-
imately 2.3 million dissertations conferred at 1,490 research institutions across 66 coun-
tries. Despite the scope of the Dissertations and Theses database, no study has explicitly
sought to validate the accuracy of the ProQuest SCs. This research examines the degree to
which ProQuest SCs serve as proxies for disciplinarity, the relevance of doctoral work to
doctoral graduates’ current work, and the permeability of disciplines from the perspective
of the mismatch between SCs and disciplinarity. To examine these issues we conducted a
survey of 2009–2010 doctoral graduates, cluster-sampled from Economics, Political Sci-
ence, and Sociology ProQuest SCs. The results from the survey question the utility of
traditional disciplinary labels and suggest that scholars may occupy a post-interdisciplinary
space in which they move freely across disciplinary boundaries and identify with topics
instead of disciplines.

Keywords Disciplinarity  ProQuest  Permeability  Disciplines  Subject


categories

T. D. Bowman (&)  A. Tsou  C. Ni  C. R. Sugimoto


School of Informatics and Computing, Indiana University Bloomington, Bloomington, IN, USA
e-mail: tdbowman@indiana.edu
A. Tsou
e-mail: atsou@indiana.edu
C. Ni
e-mail: chni@indiana.edu
C. R. Sugimoto
e-mail: sugimoto@indiana.edu

123
1696 Scientometrics (2014) 101:1695–1714

Introduction and background

Disciplines are conceptual entities that are institutionalized into academe in the forms of
schools, departments, and centers. As such, these units are sustained by the propagation of
new scholars under the same disciplinary label (Turner 2000). This label serves as a
symbol of students’ knowledge and skills, as well as facilitating the exchange of students
on the academic labor market (Abbott 1999). However, given the emphasis on interdis-
ciplinarity from institutions and funding agencies (e.g., Morillo et al. 2003), it is no wonder
that disciplinary lines are being continually crossed as doctoral students from one disci-
pline are hired on the faculties of new disciplines (Sugimoto et al. 2011). Furthermore,
many of these doctoral students go into non-academic sectors—perhaps only partially
relying on the domain expertise of their doctoral work.
Studies of disciplinary mobility frequently rely on the disciplinary classification systems
embedded in various indexing systems. For example, citation indices such as the Web of
Science (WoS) and Scopus are often used to extract data on disciplines. The disciplinary
variables are operationalized as the Subject Categories (SCs) of these databases—that is,
SCs are proxies for disciplines. Criticisms of the validity of SCs as proxies for disci-
plinarity have been raised across many decades (e.g., Garfield 1978; Ni et al. 2013).
However, SCs for major databases such as PubMed and WoS remain the gold standard in
evaluative bibliometrics, and practices of using these as proxies for micro and macro
evaluations of the system of science persist (e.g., Leydesdorff et al. 2013, 2012; Ley-
desdorff and Rafols 2009).
These issues are intensified by the growth of interdisciplinarity across sectors. Research
pertaining to interdisciplinarity has become more common in the past forty years with ‘‘a
corresponding increase in the discussion of interdisciplinarity across disciplinary, profes-
sional, and general literature’’ (Klein 1990, p. 38). Porter and Rafols (2009) have suggested
that ‘‘science is indeed becoming more interdisciplinary, but in small steps—drawing
mainly from neighboring fields and only modestly increasing the connections to distant
cognitive areas’’ (p. 719). However, other studies suggest the ubiquity of interdisciplinarity
in modern scholarly activities. Even disciplines such as physics can no longer be con-
sidered to be ‘‘a single, isolated discipline’’ (Klein 1993, p. 201); ‘‘modern research
knowledge’’ is no longer ‘‘exchange restricted within narrow silos’’ (Porter and Rafols
2009, p. 740). The degree to which disciplines interact has been described in levels of
‘‘permeability’’ (Baumann 1975, p. 15; Klein 1996, p. 38). One investigation of perme-
ability has analyzed the degree to which doctoral students from one field are hired in
another and begin training new students (Sugimoto et al. 2011). However, few studies have
used dissertation data as a platform for studies of disciplinarity and interdisciplinarity.
One large-scale index providing information on doctoral education is the ProQuest
Dissertations and Theses database. This dataset contains records for approximately 2.3
million dissertations conferred at 1,490 research institutions across 66 countries (Ni and
Sugimoto 2012), a scope that ‘‘cover[s] 40 % of all dissertations from major universities’’
(Andersen and Hammarfelt 2011, p. 374). This represents the largest historical database of
dissertation and theses, covering works as far back as the early nineteenth century (Su-
gimoto 2014). Despite the scope of the Dissertations and Theses database, it has rarely
been used to analyze the growth and connectivity of disciplines (exceptions include:
Andersen and Hammarfelt 2011; Ni and Sugimoto 2013a, b; Ying and Xiao 2012), and no
study has explicitly sought to validate the accuracy of the ProQuest SCs.
To investigate these issues, we conducted a survey of recent doctoral graduates, cluster-
sampled from three ProQuest SCs: Economics, Political Science, and Sociology. The

123
Scientometrics (2014) 101:1695–1714 1697

resultant findings were supplemented with data unobtrusively gathered about individuals
whose dissertations were classified as being from Chemistry, Computer Science, and
Biology. The primary objective was to investigate the degree to which ProQuest SCs could
serve as proxies for disciplinarity. The secondary objective was to analyze the relevance of
doctoral work to doctoral graduates’ current work and the degree to which they use
theories, methods, and literature from their home discipline and other disciplinary areas.
Finally, the work sought to analyze the permeability of disciplines from the perspective of
the mismatch between SCs and disciplinarity. This work serves to examine the use of
ProQuest SCs for disciplinary analyses, as well as to advance our understanding of in-
terdisciplinarity in the careers of doctoral graduates. Implications from this research can be
used to inform practices and policies of higher education and indexing of scholarly
communication.

Methods

Subject categories

When a doctoral student submits a dissertation to ProQuest, they are instructed to select
relevant subject areas. These suggestions are then used by ProQuest dissertation editors,
who assign SCs to the dissertation. As noted by a ProQuest representative, ‘‘we use the
author’s subjects unless our scope for a subject is entirely different from the author’s
interpretation of it (as sometimes occurs). Subjects are assigned based on the dissertation
topic, not the university department, although naturally those are most often in sync’’
(McLean 2012). Each dissertation in the database is associated with at least one SC,
although an increasing proportion is associated with more.
Each SC is identified by a four-digit unique identification number, known as the SCID.
Figure 1 provides a schema of a dissertation assignment to more than one SC. Figure 2
lists an example dissertation with ID = 15 in the ProQuest database, as well as the three
SCs (identified by SCID) to which it is assigned. Each subject category listed in Fig. 2 has
two levels: the more general first level category and the qualifying second level category.
The first level subject category is considered as a proxy for the discipline, while the second
level subject category is considered a proxy for the specialty. Therefore, dissertation 15 is
assigned to two disciplines and three specialties. It is a multi-disciplinary dissertation, and
can be considered simultaneously as a sociology and health science dissertation. The first
item listed (SCID 0344; Sociology) is considered to be the primary category (McLean
2012).

Data

We constructed a sampling frame of all dissertations completed in 2009 or 2010 for each of
three disciplines: Economics, Political Science, and Sociology. The time frame was
selected in order to ensure that recent graduate students had sufficient time to transition
from doctoral work into employment. A dissertation was considered as part of the disci-
pline if it contained the disciplinary label as at least one of the first level SCs (not
necessarily as the primary SC; that is, where the discipline was listed in the first position).
Five hundred dissertations from each SC were randomly selected for inclusion. Given that
most dissertations from this time period are associated with multiple SCs, we resampled to

123
1698 Scientometrics (2014) 101:1695–1714

Fig. 1 Data schema of ProQuest dissertation

Fig. 2 Example dissertation w/multiple subject categories

ensure that dissertations were only assigned to one discipline. The three disciplines were
selected in order to examine an array of social sciences.

Identifying sampling frame

An online web application was constructed using PHP, jQuery, and HTML, allowing a
research assistant (RA) to search, retrieve, and record additional information about each
subject by using the information in ProQuest as a starting point. The RA then attempted to
find current information about the subject by examining the links on the first three pages
returned from a Google search. The Google searches were conducted using the person’s
first name and last name, their dissertation title, or both. We collected information about
each subject’s current URL, current email, current departmental affiliation, and current CV
link; all of the new information was saved in a PostgreSQL database and linked to the
ProQuest data using bridge tables. This allowed us to identify contact information and
obtain additional contextual information regarding career trajectories. Contact e-mail

123
Scientometrics (2014) 101:1695–1714 1699

addresses for each person could not be located. Therefore, the final sample included 348
individuals in political science, 338 individuals in economics, and 283 individuals in
sociology.

Questionnaire

The questionnaire was designed to gather (a) demographic information, (b) career infor-
mation, and (c) information regarding disciplinary identity: that is, the degree to which the
individual self-associated with the discipline in which they were trained (see Appendix for
the full questionnaire—note that not all questions were utilized in the present study). The
questionnaire was pilot-tested with one faculty member and four graduate students and
built using Qualtrics1 survey software. The questionnaire was available electronically, and
the 969 participants were invited to participate between March 29 and April 18, 2013. No
reminders were sent as the use of follow-ups has been shown to be negatively related to
response rates (Anseel et al. 2010). The response data were exported from Qualtrics into
Excel and SPSS for further evaluation. The main analysis of the survey results was to
determine the degree to which the exact disciplinary labels were used by respondents to
describe their area of work. That is, an exact match between disciplinary labels would be
one in which the discipline under study was Economics and the respondent said that they
received their degree in Economics. A close match would be one that demonstrated the
direct application of the subject area (for example, economics of education). Non-matches
were those that were either specialized (e.g., forecasting) or distinct disciplines (e.g.,
History).

Validation exercise

Data from three additional disciplines were selected as a validation metric. Using the same
mechanism as for the three social science disciplines, 100 dissertations for each of three
disciplines (Biology, Chemistry, and Computer Science) were selected at random. These
were selected from the same timeframe as the initial dataset and on the basis of the same
criteria (that is, containing at least one instance of the SC, but not necessarily in the
primary position). We then conducted a series of Google searches to identify the current
departmental affiliations of these dissertations’ authors, as well as to identify the discipline
in which they received their doctorate degree.

Results

This section is organized into three main parts. The first part provides a basic description of
all ProQuest dissertations (in the entire database, not just in the timeframe under analysis)
in the three SCs so as to provide context. The second part provides descriptive data
regarding the respondents. The last part is an analysis of the questionnaire.
1
http://www.qualtrics.com.

123
1700 Scientometrics (2014) 101:1695–1714

ProQuest description

There were a total of 96,434 dissertations (including all of ProQuest dissertations and
theses until 2011) that included Sociology as a SC in ProQuest. Sociology was associated
with 13 s-level SCs (specialities) (Table 1).
Seventy-five percent of dissertations contained another first-level category (discipline).
The ten first-level SCs most commonly occurring with the Sociology SC are listed in
Table 2.
There were 76,336 dissertations with the Economics SC. Economics contains 8 s-level
SCs, present in Table 3.
Only 34 % of dissertations labeled Economics contained another first-level SC. The
most frequently co-occurring SCs are listed in Table 4.
Political science had the lowest number of dissertations in ProQuest at 55,626, as well
as the least number of second-level SCs (4). These are listed in Table 5.
55 % of Political Science dissertations list another discipline. Table 6 shows the most
frequently co-occurring disciplines.
Based on the evaluation of the ProQuest SCs, Sociology is both the most specialized
and the most interdisciplinary of the three selected disciplines; in addition, Economics is
the least interdisciplinary (having the lowest proportion of dissertations with another SC),
and Political Science is the least specialized (as represented by having the fewest number
of second-level SCs).

Description of survey respondents

Our sampling frame of 500 dissertations from each discipline reflects the general patterns
of interdisciplinarity, although the sampled dissertations were more likely to contain
multiple SCs than the numbers reflected here. This is due to the fact that more recent
dissertations are more likely to be associated with multiple SCs. Within the sampling
frame, 68 % of Economics dissertations, 78 % of Political Science dissertations, and 92 %
of Sociology dissertations were associated with another first-level SC.
A 20 % response rate was obtained for the survey, with lower levels for Economics
(n = 45; 13 %) compared to Political Science and Sociology [n = 81, n = 64 (respec-
tively); both 23 %]. This counts only those surveys that were fully completed. The gender,
race, and age of respondents are presented in Table 7, broken down by discipline. Overall,
responders were 51 % male (n = 96) and 49 % female (n = 94); 75 % of the respondents
fell between 31 and 45 years old. As shown, Economics was more male-dominated,
whereas Sociology was more female-dominated. Caucasians were the majority in all
disciplines, and respondents from Economics tended to be the youngest.
Nearly all respondents (98 %) were currently employed, and respondents were from a
diverse set of affiliated institutions. Table 8 shows the respondents’ identification with
various types of organizations. Respondents were highly associated with academic insti-
tutions, particularly when the individuals in question were associated with Sociology.
Counting partially and fully complete response sets, the 48 respondents from Economics
represented 46 different institutions. The majority of institutions (n = 38; 83 %) were in
the USA: ten other countries were also represented (Brazil, Canada, Chile, Germany, Italy,
Lebanon, Mexico, Sweden, Switzerland, and UK). Similarly, 67 affiliations were located
for Sociology doctorates. These were also primarily in the USA (n = 62; 93 %), with only
five other countries represented (Canada, France, Peru, United Arab Emirates, and South
Africa). Political Science doctorates were associated with 81 institutions (with one

123
Scientometrics (2014) 101:1695–1714 1701

Table 1 Specialities of Sociology by number of occurrences


Specialty #Diss 1st appear

Sociology, Individual and Family Studies 23,395 1934


Sociology, General 12,768 1930
Sociology, Social Structure and Development 11,882 1894
Sociology, Criminology and Penology 11,325 1934
Sociology, Public and Social Welfare 9,600 1912
Sociology, Industrial and Labor Relations 7,602 1914
Sociology, Theory and Methods 5,065 1922
Sociology, Demography 4,108 1935
Sociology, Organizational Theory 417 1971
Sociology, Sociolinguistics 166 2008
Sociology, Environmental Justice 67 2008

Table 2 Subject Categories most often occuring with Sociology by number of occurrences
Subject category # Co-occurrences

Psychology 20,685
Education 18,022
Women’s Studies 9,003
Health Sciences 8,149
Political Science 7,992
History 6,946
Business Administration 5,185
Economics 4,838
Anthropology 4,650
Social Work 4,333
Black Studies 4,132

Table 3 Specialties of Economics by number of occurrences


Specialty #Diss 1st appear

Economics, General 29,920 1930


Economics, Finance 18,038 1932
Economics, Theory 11,984 1938
Economics, Agricultural 10,773 1929
Economics, Commerce Business 8,637 1899
Economics, Labor 6,368 1912
Economics, History 3,910 1909
Economics, Environmental 202 2008

individual acting as a freelancer). Ten individuals from Political Science were located
outside of the USA (Bangladesh, Canada, Israel, Mexico, Netherlands, Singapore, Spain,

123
1702 Scientometrics (2014) 101:1695–1714

Table 4 Subject Categories most often occuring with Economics by number of occurrences
Subject category # Co-occurrences

Business Administration 7,233


Political Science 5,104
Sociology 4,838
History 2,628
Education 1,887
Health Sciences 1,798
Agriculture 1,514
Energy 1,444
Environmental Sciences 1,303
Urban and Regional Planning 1,157
Geography 1,041

Table 5 Specialties of Political Science by number of occurrences


Specialty #Diss 1st appear

Political Science, General 31,362 1895


Political Science, Public Administration 15,984 1894
Political Science, International Law and Relations 12,023 1927
Political Science, International Relations 639 1948

Table 6 Subject Categories most often occuring with Political Science by number of occurrences
Subject category # Co-occurrences

History 8,035
Sociology 7,992
Economics 5,104
Education 3,443
Business Administration 2,257
Law 1,958
Philosophy 1,941
Environmental Sciences 1,469
Health Sciences 1,376
Mass Communications 1,370
Women’s Studies 1,271

Switzerland, and UK). While respondents identified with institutions in 19 countries, 91 %


were located in the US, 7 % were in Canada, and only 2 % represented the other 17
countries; there is a clear US bias in the response.
These high degrees of association with academic institutions across all categories
should not be taken to be representative of career trajectories for doctoral students; rather,
they are possibly an artifact of collecting the sample based on identifiable contact

123
Scientometrics (2014) 101:1695–1714 1703

Table 7 Socio-demographic characteristics of respondents, by discipline


Economics (n = 45) (%) Political Science (n = 81) (%) Sociology (n = 64) (%)

Gender
Female 35.6 47.5 60.0
Male 64.4 52.5 40.0
Race
Caucasian 85.4 90.1 79.7
Non-caucasian 14.6 9.9 20.3
Age (years)
26–35 72.7 42.0 30.8
36–45 18.2 35.8 36.9
\45 9.1 22.2 12.3

Table 8 Current institutional affiliation of respondents


Acad. (%) Corp. (%) Govt (%) Non-profit (%) Other (%) No answer (%)

Economics 75.6 6.7 11.1 2.2 2.2 2.2


Political Science 79.0 3.7 8.6 3.7 2.5 2.5
Sociology 86.2 3.1 1.5 6.2 1.5 1.5

information, which was potentially more easily located for those holding academic posts
(due to the public availability of curricula vitae and other relevant material).

Questionnaire results

Figure 3 depicts the degree to which respondents who completed dissertations in certain
SCs identified with the same disciplinary label. The first column represents responses to the
question: ‘‘What was the field of study of your doctorate degree?’’ Questions 2 and 3 asked
respondents to identity the disciplines from which they consumed (2) and to which they
disseminated (3) information. Questions 4 and 5 asked respondents to identify the disci-
plines from which they primarily drew theories (4) and methods (5). Question six asked:
‘‘How would you self-identify your expertise or research area to a future employer?’’
Question seven asked: ‘‘How would you self-identify your expertise or research area to
persons outside the workplace?’’ These used the same coding scheme, where explicit
mention of the disciplinary label was indicated with a darker color and applications
including the disciplinary label (e.g., sociology of x) were indicated in a lighter color.
Our goal was to assess the degree to which the respondents identified with the broad
disciplinary label (i.e., Political Science, Sociology, and Economics), as well as applied
and specialized terms that would be recognizable to an outsider (e.g., sociology of x,
economics of y, politics of z). We did not attempt to differentiate specializations, as our
aim was only to see how much the broad category was used as an identify marker for the
respondents.
For respondents in the Economics set, 50 % (23/46) answered ‘‘Economics’’ to the first
question (i.e., ‘‘What was the field of study of your doctorate degree?’’). These are indi-
cated in dark red in the figure. Ten respondents (shown in lighter red in Fig. 3) identified a

123
1704 Scientometrics (2014) 101:1695–1714

Fig. 3 Association with


disciplinary labels, by each
survey question. Grey indicates
that no response was received.
Cases with non-response for all
seven categories were eliminated

disciplinarity perspective of economics (e.g., agricultural and resource economics, business


economics, nursing economics, and health economics). A few identified specialties of
economics, including macroeconomics and time series econometrics (also identified in the

123
Scientometrics (2014) 101:1695–1714 1705

lighter shade). Other areas included industrial organization, history, urban and regional
planning, finance, operations research, sociology, and marine biology. No color is repre-
sented for these. Grey indicates that the question was not answered.
Similarly, 45 % of those with Political Science SCs on their dissertations said that their
doctoral degree was in ‘‘Political Science.’’ Three provided specialized terms, such as
comparative politics or political theory. Many listed disciplines outside of Political Sci-
ence, such as Anthropology, Education, Geography, French, and Philosophy. Others listed
specialties that are closely related (and possibly considered within) the field of political
science: public administration, public policy, and international relations (these were not
shaded).
Sociology demonstrated the highest permeability. Only 15 % of respondents with
Sociology SCs indicated that they received their degrees in Sociology. There were a few
instances of specific applications of sociology (e.g., sociology of gender and sexuality;
sociology of culture and inequality; sociology of politics); however, most respondents were
outside of the field of sociology (for example, Anthropology, Nursing, or Business). Social
Work was also fairly highly represented in the responses (this was considered a distinct
discipline).
In Economics, the highest proportion of respondents associated with the explicit dis-
ciplinary label was regarding the use of theories, with 60 % of respondents indicating
‘‘Economics’’ as the primary discipline from which they use theories. The majority of
respondents indicated ‘‘Economics’’ for all questions except question 6 and 7, where only
20 % reported using the term ‘‘Economics’’ to describe their expertise to an employer.
Only a third said that they would use this term to describe themselves to people outside
their workplace.
More individuals identified Political Science as the area of their doctorate (44 %) than
those who indicated that they primarily used political science theories in their work
(38 %). Very few used this label to self-identify to employers (11 %) or to those outside
the workplace (17 %).
The strongest association with Sociology was in terms of methods, with 19 % of
respondents indicating that they primarily drew upon sociological methods in their work.
Nearly 18 % indicated that they drew from sociological theories, fewer than those who
indicated that they received a degree in sociology (15 %). A very small percentage self-
identified with Sociology to describe themselves to employers (5 %) or others (4 %). The
majority of respondents whose dissertations were granted in Sociology did not indicate any
apparent association with Sociology.
Given that the SC that appears first is considered the ‘‘primary SC,’’ it is possible that
more reliable results might have been generated if the analysis were restricted to primary
SCs. This was investigated by analyzing the percentage of all 500 dissertations in the
sampling frame that were associated with the degree as a primary SC, the proportion of
dissertations among the respondents with the selected discipline as the primary SC, and the
percent of these that were ‘‘exact matches’’ with the degree in which the respondent
received their degree. By exact match, we refer to those respondents which mentioned the
disciplinary label in their description of the discipline in which they received their degree.
Table 9 provides the percentages of all dissertations (from the sampled 500) that had the
given discipline as the primary SC (i.e., the first-listed first-level SC); the percent of
dissertations among respondents with the given discipline as the primary SC; the percent of
respondents who gave an ‘‘exact match’’ response to question 1 on the survey; the percent
of respondents with the designated primary SC dissertation who gave an ‘‘exact match’’

123
1706 Scientometrics (2014) 101:1695–1714

answer; and the percentage of respondents without the given discipline as a Primary SC
who gave an ‘‘exact match’’ answer.
As shown, our respondent set matched the proportions of dissertations from the entire
set with the discipline as the primary SC. In all cases, the proportion of respondents with
exact matches improved if only those dissertations with the discipline in the primary
position were included. For example, whereas only 50 % of respondents stated that they
received their degree in Economics, this improved to 84 % if only those respondents whose
dissertations had Economics as a Primary SC were considered. However, although this
eliminates some false positives, it does not completely mitigate the false negative situation:
half of the respondents whose dissertations did not list Economics as a Primary SC
identified Economics as their discipline. There was also large variability across the dis-
ciplines: even when looking at only dissertations with Sociology as a Primary SC, only
36 % were exact matches (however, this is limited by the small sample size of this test).

Validation exercise

In order to supplement these findings, we investigated three disciplines (Biology, Chem-


istry, and Computer Science) that lay outside of the social sciences. A series of Google and
ProQuest searches were conducted on the authors of 100 dissertations for each of these
three disciplines in order to determine the degree to which their field of study accorded
with the SC associated with their dissertation. Table 10 presents the percentage of our
sample that had the designated discipline as a Primary SC; the percentage for whom the
Primary SC and the field of study were an ‘‘exact match’’; the percentage of dissertations
that had the designated discipline as a Primary SC that were an ‘‘exact match’’; and the
percentage of people who had a dissertation with another Primary SC but had received a
degree in Biology, Chemistry, or Computer Science (respectively). As with our previous
analysis, there was an improvement in accuracy when only the primary SC was considered,
although the gain was minimal in Biology and Chemistry. There was also a high degree of
potential false negatives in the field of Chemistry.
We further analyzed the instances in which each discipline was listed as a Primary SC
but was not considered an exact match. There were 55 such instances in Biology. Many of
the degrees in this category were from related areas: Biochemistry, Bioinformatics, Bio-
logical Chemistry, Biological Oceanography, Biomedical Sciences, Biophysics, and Bio-
statistics (these represented 16 of the dissertations). There were also other disciplines such
as Chemistry, Computer Science, Psychology, Ecology, Geology, and Zoology. Finally,
there were specializations such as Neuroscience (8 of the sampled dissertations).
There were nineteen instances in which Chemistry was listed first, but it was not an
exact match. These included some aspects of chemistry (e.g., Chemical and Biochemical
Engineering, Chemical and Biomolecular Engineering, Chemical Biology, and Chemical
Engineering). Other disciplines represented among these dissertations were Physics,
Biology, and Information and Library Science.
There were only two instances in which Computer Science was listed as a primary SC
but was not an exact match with the discipline of the degree. In both cases it was a very
similar term: Engineering (Computer Engineering) and Informatics and Computing.
The validation study confirmed the results of the main study by: (1) suggesting
improved accuracy when Primary SCs were used; and (2) identifying large variability in
both false positives and false negatives across disciplinary areas.

123
Scientometrics (2014) 101:1695–1714 1707

Table 9 Relationship between the primary SCs assigned to the sampled dissertations and the disciplinary
labels used by respondents to describe the discipline in which they received their degree (i.e., Question 1 in
Fig. 3)
Primary SC Primary SCs Exact matches % of primary SC % of non-primary
(out of 500) (from (from that were exact SCs that were exact
(%) respondents) respondents) matches matches

Economics 68 69 % (n = 33) 50 % (n = 23) 84 % (n = 26)a 50 % (n = 7)


Political 42 48 % (n = 40) 45 % (n = 37) 64 % (n = 25) 41 % (n = 17)
Science
Sociology 20 23 % (n = 17) 15 % (n = 11) 36 % (n = 5) 10 % (n = 5)
a
Percentage of those who responded to the question for this and all other similar calculations

Table 10 Relationship between the primary SCs assigned to dissertations and the discipline in which
dissertation authors received their degree
Primary Exact % of primary SC that were % of non-primary SC that were
SC (%) matches (%) exact matches exact matches

Biology 90 37 39 % (n = 35) 20 % (n = 2)
Chemistry 82 72 77 % (n = 63) 50 % (n = 9)
Computer 48 70 96 % (n = 46) 46 % (n = 24)
Science

Discussion and conclusion

The results of our survey question the utility of traditional disciplinary labels in a time of
heightened specialization and interdisciplinary. In examining the entire ProQuest dataset,
Sociology was shown to be the most specialized and also the most likely to appear in the
presence of another first-level SC. These SCs were spread across the disciplinary spectrum,
including medical sciences, social sciences, humanities, and professional fields. Economics
was slightly narrower, with little more than one-third of the dissertations in this area
appearing in conjunction with another SC. Political Science was the least specialized,
according to ProQuest’s schema, but it was still fairly interdisciplinary, with more than half
of the dissertations occurring in the presence of another first-level SC.
The goal of the questionnaire was to identify the ways in which the dissertation authors
associated with the SC listed on the dissertation. Many elements were investigated: indi-
viduals’ degrees, their use of theory and methods, the disciplines from which they con-
sumed and disseminated knowledge, and the way they described their expertise to
employers and the public. As shown in Fig. 3, there were varying degrees to which
respondents identified with the disciplinary labels into which they were classed. Sociology
was the most diffuse, with very few people using this term to describe their scholarly
activities or identities. A majority of respondents did not refer to ‘‘sociology’’ in any of
their responses, even though their dissertation was classified under this SC.
One concern could be that Social Sciences are particularly permeable, interdisciplinary,
and more diffuse than STEM disciplines. To test this, we examined a random sample of
dissertations in Biology, Chemistry, and Computer Science. As was shown, there was high
variability in the degree to which the SCs captured the disciplinary labels under which the
disciplines were conferred. Biology was the most diffuse and had the highest potential for

123
1708 Scientometrics (2014) 101:1695–1714

obtaining false positives, even when restricting the analysis to primary SCs. Computer
Science had the most reliable results (although still exhibiting a fairly high level of false
negatives) when the sampling was restricted to primary SCs.
These findings suggest that caution be exercised when using ProQuest SCs as proxies
for disciplinarity. However, the variability across the disciplines suggests that ProQuest
may not be systematically misclassifying dissertations; rather, it may well be that the
traditional disciplinary labels no longer hold. In the qualitative component of this study,
many respondents provided detailed and specialized responses—rather than giving a single
word disciplinary label, they described at length the topic that they studied. It is possible
that topics carry more weight than disciplines in specialized environments.
Specialization may also be characterizing the marketplace, as epitomized by the degree
labels for Biology. Degrees were no longer conferred in ‘‘Biology,’’ but rather in Bioin-
formatics, Biological Oceanography, and Biostatistics. There is a ‘‘balance between spe-
cialization and integration’’ (Klein 1990, p. 38) that must be carefully managed.
Furthermore, it begs the question of what institutions gain by specializing their degrees. Is
there a newcomer’s advantage to being the first school to offer a degree in a specialized
topic? Perhaps traditional disciplines are no longer the Gold Standard, thus reflecting a
marketplace that is now nimble enough to evaluate people on their topical distinctions
rather than on their disciplinary affiliations.
In this case, ‘‘interdisciplinary’’ may not be the appropriate term to characterize the new
marketplace for doctoral students. We may live in a post-interdisciplinary space in which
scholars wish to move more freely across disciplinary boundaries and aggregate topically
more than disciplinarily. Whitley suggested this was the case as early as 1984 when he
wrote that ‘‘the discipline as a set of research activities has outgrown the departmental
basis of employment and careers’’ and that the ‘‘identification of intellectual disciplines
with university undergraduate departments is no longer a social reality in many science’’
(Whitley 1984, p. 18). There remain, however, some disciplines that are less permeable and
present more barriers for entry (Klein 1993), and it seems that disciplines still remain a
dominant organizing agent for institutions of higher education and retain ‘‘enormous
influence over the organization and production of knowledge’’ (Klein 1993, p. 185).
This begs an analysis of the ways in which disciplinary labels may connote something
different between academic markets and professional labor markets. There is a historically
complicated relationship between academic knowledge and professionalism. Abbott (1988)
argued that the ‘‘ability of a profession to sustain its jurisdictions lies partly in the power
and prestige of its academic knowledge’’ (pp. 53–54). However, there are numerous
conflicts over jurisdictional boundaries both among and within disciplines, one that is
complicated by the growing specialization of research (Jacobs 2013). There are also many
professionals who continue to contribute to academic knowledge once they have formally
left academe (Breimer and Leksell 2013). Future research is needed to understand whether
specialization in the labor market is parallel to specialization in the academic research
market.

Limitations

There are, of course, a number of limitations to the present study. As noted above, our
sample is restricted to those with an available e-mail address. It is likely that those with an
e-mail address are more likely to be (a) employed and (b) associated with an academic
institution. Given the ‘‘risks’’ associated with interdisciplinary research (Kniffin and Hanks
2013), it is possible that the sample might skew towards those who are more ‘‘disciplinary’’

123
Scientometrics (2014) 101:1695–1714 1709

in nature. However, current research has shown that individuals conducting interdisci-
plinary dissertation research are more likely to be employed in academe (Millar 2013). Our
results seem to confirm this.
The mode of delivery is also a limitation as e-mail has been shown to have lower
response rates than other modes of survey research (Schaefer and Dillman 1998). Our
response rates were similar to those found in a review of survey research with academic
participants (Dykema et al. 2013), but still lower than averages for other modes of delivery.
We used a number of tested incentives (i.e., personalizing the survey and a monetary
incentive), but it’s possible that our response rate could have been improved with another
incentive structure (e.g., advance notice).
The choice of fields is also a limitation. Our objective was to choose a wide variety of
disciplines, each with their own particularistic identities—in both the main and validation
studies. However, there is always a somewhat arbitrary nature to discipline selection and
we must acknowledge that different results may have been generated given a different
choice of disciplines. Furthermore, we have classified Economics, Sociology, and Political
Science as social sciences and Biology, Computer Science, and Chemistry as non-social
sciences. However, these distinctions may not hold across all subareas of these disciplines.

Future research

Future research should investigate how people can appropriately evaluate these topic-based
scholars, what implications exist for the reorganization of institutions, how these topic
distinctions play into information consumption and disciplinary identity, and how repu-
tation and authority can be conferred given this highly permeable space. Although inter-
disciplinarity is often seen as a positive thing (Rafols and Meyer 2010), there are dangers
and consequences for a lack of disciplinary grounding. Disciplines are not only institu-
tional conventions, but are also bound by shared paradigms: i.e., the types of questions that
can be asked, the ways in which they can be asked, and the answers that can be retrieved
(Kuhn 1970). Members of disciplines establish their own sets of cultural rules and norms,
allowing both members and outsiders to make sense of activities and interactions through
the application of disciplinary-relevant frames (Goffman 1974). Disciplines provide the-
oretical and methodological frameworks for the education and training of doctoral stu-
dents. One might question how students can move freely across disciplinary boundaries—
appropriating theories and methods as necessary—while still maintaining a deeply con-
textualized knowledge of the appropriate use of these theories and methods. Furthermore,
there is the conceptual problem of what happens when disciplinary work is abandoned in
favor of interdisciplinary research. If interdisciplinary research represents ‘‘the juxtapo-
sition of disciplines’’ (Klein 1990, p. 56), who is contributing to the tool chest from which
these researchers are drawing theories and methods? Can interdisciplinary research exist in
the absence of disciplines? Klein (1993) suggested that boundary work involves not only
‘‘crossing’’ and ‘‘deconstructing,’’ but also ‘‘reconstructing boundaries’’ (p. 196). There-
fore, it is possible that interdisciplinary areas are merely pre-disciplinary areas in the
process of reconstructing boundaries. Alternatively, we may be transitioning into a post-
interdisciplinary environment, where identities are constructed in topical, rather than
disciplinary, frames of reference.

Acknowledgments This work was funded by the Science of Science Innovation and Policy (SciSIP)
program of the National Science Foundation (Grant no. 1158670).

123
1710 Scientometrics (2014) 101:1695–1714

Appendix: Questionnaire

123
Scientometrics (2014) 101:1695–1714 1711

123
1712 Scientometrics (2014) 101:1695–1714

123
Scientometrics (2014) 101:1695–1714 1713

References

Abbott, A. (1988). The system of professions: An essay on the division of expert labor. Chicago, IL: The
University of Chicago Press.
Abbott, A. (1999). Department and discipline: Chicago sociology at one hundred. Chicago, IL: The Uni-
versity of Chicago Press.
Andersen, J. P., & Hammarfelt, B. (2011). Price revisited: On the growth of dissertations in eight research
fields. Scientometrics, 88, 371–383.
Anseel, F., Lievens, F., Schollaert, E., & Choragwicka, B. (2010). Response rates in organizational science,
1995–2008: A meta-analytic review and guidelines for survey researchers. Journal of Business and
Psychology, 25(3), 335–349.
Baumann, B. (1975). Imaginative participation: The career of an organizing concept in a multidisciplinary
context. The Hague: Martinus Nijhoff.
Breimer, L. H., & Leksell, J. (2013). Longitudinal and cross-sectional study of registered nurses in Sweden
who undertake a PhD showing that nurses continue to publish in English after their PhD but male
nurses are more productive than female nurses. Scientometrics, 87(2), 337–345.
Dykema, J., Stevenson, J., Klein, L., Kim, Y., & Day, B. (2013). Effects of e-mailed versus mailed
invitations and incentives on response rates, data quality, and costs in a web survey of university
faculty. Social Science Computer Review, 31(3), 359–370.
Garfield, E. (1978). The Gordian Knot of journal coverage: Why we can’t put all the journals you want into
the Current Contents edition you read. Current Contents, 13, 5–7.
Goffman, E. (1974). Frame analysis: An essay on the organization of experience. New York, NY: Harper &
Row.
Jacobs, J. A. (2013). In defense of disciplines: Interdisciplinarity and specialization in the research uni-
versity. Chicago, IL: The University of Chicago Press.
Klein, J. T. (1990). Interdisciplinarity: History, theory, and practice. Detroit: Wayne State University Press.

123
1714 Scientometrics (2014) 101:1695–1714

Klein, J. T. (1993). Blurring, cracking, and crossing: Permeation and the fracturing of discipline. In E.
Messer-Davidow, D. R. Shumway, & D. J. Sylvan (Eds.), Knowledges: Historical and critical studies
in disciplinarity (pp. 185–211). Charlottesville, London: The University Press of Virginia.
Klein, J. T. (1996). Crossing boundaries: Knowledge, disciplinarities, and interdisciplinarities. Char-
lottesville, VA: University of Virginia Press.
Kniffin, K. M., & Hanks, A. S. (2013). Boundary spanning in academia: Antecedents and near-term
consequences of academic entrepreneurialism. Cornell Higher Education Research Institute (CHERI)
Working Paper 158.
Kuhn, T. S. (1970). The structure of scientific revolutions. Chicago: University of Chicago Press.
Leydesdorff, L., Carley, S., & Rafols, I. (2013). Global maps of science based on the new Web-of-Science
categories. Scientometrics, 94(2), 589–593.
Leydesdorff, L., & Rafols, I. (2009). A global map of science based on the ISI Subject Categories. Journal
of the American Society for Information Science and Technology, 60(2), 348–362.
Leydesdorff, L., Rotolo, D., & Rafols, I. (2012). Bibliometric perspectives on medical innovation using the
medical subject headings of PubMed. Journal of the American Society for Information Science and
Technology, 63(11), 2239–2253.
McLean, A. (2012). Personal correspondence to Cassidy Sugimoto via email on Oct 26, 2012.
Millar, M. M. (2013). Interdisciplinary research and the early career: The effect of interdisciplinary dis-
sertation research on career placement and publication productivity of doctoral graduates in the sci-
ences. Research Policy, 42(5), 1152–1164.
Morillo, F., Bordons, M., & Gomez, I. (2003). Interdisciplinarity in science: A tentative typology of
disciplines and research areas. Journal of the American Society for Information Science and Tech-
nology, 54(13), 1237–1249.
Ni, C., & Sugimoto, C. R. (2012). Using doctoral dissertations for a new understanding of disciplinarity and
interdisciplinarity. Poster. In Annual Meeting of the American Society for Information Science and
Technology. Baltimore, MD.
Ni, C., & Sugimoto, C. R. (2013a). Exploring interdisciplinarity in Economics through academic genealogy:
An exploratory study. Poster. In Proceedings of the International Society for Scientometrics and
Informetrics Conference.
Ni, C. & Sugimoto, C. R. (2013b). Academic genealogy as an indicator of interdisciplinarity: A preliminary
examination of sociology doctoral dissertations. In Poster iConference.
Ni, C., Sugimoto, C. R., & Jiang, J. (2013). Venue–author–coupling: A novel measure for identifying
disciplines through social structures. Journal of the American Society for Information Science and
Technology, 64(2), 265–279.
Porter, A. L., & Rafols, I. (2009). Is science becoming more interdisciplinary? Measuring and mapping six
research fields over time. Scientometrics, 81(3), 719–745.
Rafols, I., & Meyer, M. (2010). Diversity and network coherence as indicators of interdisciplinarity: Case
studies in bionanoscience. Scientometrics, 82, 263–287.
Schaefer, D. R., & Dillman, D. A. (1998). Development of a standard e-mail methodology: Results of an
experiment. The Public Opinion Quarterly, 62(3), 378–397.
Sugimoto, C. R. (2014). Academic genealogy. In B. Cronin & C. R. Sugimoto (Eds.), Beyond bibliometrics:
Metrics-based evaluation of research. Boston, MA: MIT Press.
Sugimoto, C. R., Ni, C., Russell, T. G., & Bychowski, B. (2011). Academic genealogy as an indicator of
interdisciplinarity: An examination of dissertation networks in Library and Information Science.
Journal of the American Society for Information Science and Technology, 62(9), 1808–1828. doi:10.
1002/asi.2156.
Turner, S. (2000). What are disciplines? And how is interdisciplinarity different? In P. Weingart & N. Stehr
(Eds.), Practising interdisciplinarity (pp. 46–65). Toronto: University of Toronto Press.
Whitley, R. (1984). The rise and decline of university disciplines in the sciences. In R. Jurkovich & J. H. P.
Paelinck (Eds.), Problems in interdisciplinary studies: Issues in interdisciplinary studies (pp. 10–25).
Aldershot, Hampshire: Gower.
Ying, T. Y., & Xiao, H. G. (2012). Knowledge linkage: A social network analysis of tourism dissertation
subjects. Journal of Hospitality and Tourism Research, 36(4), 450–477.

123

You might also like