388147

91110388147Ingram et al.Assessment The Author(s) 2011 Reprints and permission: http://www. sagepub.com/journalsPermissions.nav

ASM18110.1177/10731

Empirical Correlates and Expanded Interpretation of the MMPI-2-RF Restructured Clinical Scale 3 (Cynicism)
Paul B. Ingram1, Kristy M. Kelso1, and David M. McCord1

Assessment 18(1) 95101 The Author(s) 2011 Reprints and permission: http://www. sagepub.com/journalsPermissions.nav DOI: 10.1177/1073191110388147 http://asm.sagepub.com

Abstract The recent release of the Minnesota Multiphasic Personality Inventory-2Restructured Form (MMPI-2-RF) has received much attention from the clinical psychology community. Particular concerns have focused on Restructured Clinical Scale 3 (RC3; Cynicism). This article briefly reviews the major criticisms and responses regarding the restructuring of Clinical Scale 3. The primary purpose of the article is to provide expanded interpretive perspectives on RC3 by correlating it with a selection of external variables that are conceptually related. Forty undergraduate volunteers completed the MMPI-2-RF as well as measures of Machiavellianism, alienation, faith in people, and locus of control. Results showed strong correlations between RC3 and Machiavellianism (.47) and alienation (.60), with item-level information that may extend existing interpretive statements to include, not just a cynical view of others, but a willingness to intentionally lie, cheat, deceive, and manipulate (i.e., Machiavellianism). Keywords MMPI-2-RF, Restructured Clinical Scale 3, RC scales, cynicism, Machiavellianism

The introduction of the Minnesota Multiphasic Personality Inventory-2Restructured Clinical (RC) Scales in 2003 (Tellegen et al., 2003) and the subsequent release of the MMPI-2-RF in 2008 (Ben-Porath & Tellegen, 2008a) are major events in the history of this important instrument. The RC Scales form the interpretive core of the new restructured test, and they are substantially different from the original clinical scales on which they are based. Although the RC Scales exhibit significant improvements in internal reliability, convergent validity, and, most notably, discriminant validity (see Ben-Porath & Tellegen, 2008b), the disconnection from decades of empirical literature is not a trivial issue. Meyer (2006) edited a special issue of the Journal of Personality Assessment, which provides a fascinating perspective on this ongoing debate. In the following paragraphs, we will very briefly summarize the inherent problems with the clinical scales and how the restructuring effort addressed those problems. We will then briefly review the major criticisms of RC3 (Cynicism), and the major rebuttals to those criticisms, as this has been the most substantially modified of the scales. The current study seeks to expand our understanding of RC3 by examining its associations with measures of other conceptually related constructs. The empirical keying method Hathaway employed in developing the original scales was justifiably considered the single greatest strength of the MMPI, separating it from

the rationally derived and generally ineffective personality tests of that time period (Graham, 1977; Greene, 1980). Over the decades, though, it has become apparent that the empirical keying method itself also produced several serious inherent problems. First, the items that end up on a given scale are those that reflect all differences between the criterion group and the normal group, not just the specific differences that characterize the pathology in focus (e.g., depression). We now know that although patients may be different from each other in specific ways, in other ways they are similar. That is, there is a common factor across pathologies that is represented by the first factor of the MMPI Clinical Scales. Tellegen et al. (2003) have labeled this common factor demoralization and point out that although it is very important clinically, it is not useful to measure it repeatedly as a component of every clinical scale. This one issue results in problematically high intercorrelations among the MMPI-2 clinical scales, with the most glaring example being the correlation of .84 between Clinical Scale 7 (Psychasthenia) and Clinical Scale 8
1

Western Carolina University, Cullowhee, NC, USA

Corresponding Author: David M. McCord, Department of Psychology, Western Carolina University, 301 Killian Building, Cullowhee, NC 28723, USA Email: mccord@wcu.edu

96 (Schizophrenia) in the standardization sample. A second problem with the empirical keying method of scale construction is that in many cases the content of the resulting scale is very heterogenous, including as many as four or five distinct content domains. Scale heterogeneity lowers internal consistency, lowers convergent and discriminant validity, and it makes the scale hard to interpret. Finally, empirical keying identifies items that may have no real association with the pathology of interest but instead reflect idiosyncratic characteristics of the normative group, the criterion group, or both. These items with no logical relationship to the construct being measured have traditionally been labeled subtle items, but most now see these items as simply invalid (Weed, 2006). The steps involved in restructuring the clinical scales are described in detail in the RC Scales monograph (Tellegen et al., 2003) and will be summarized only very briefly here. First, the authors focused on the common factor, labeled it demoralization, and developed a 23-item scale to measure it based on joint factor analysis of Clinical Scales 2 and 7. They then added these items to each clinical scale in turn and factor analyzed the combined set. They hypothesized that a two-factor solution would, in each case, result in, first, a demoralization factor that would not only include the original 23 items but also items in the target scale with a greater association with demoralization than with the other aspects of the target scale, leaving a second factor that would represent the major distinctive core of the target scale, with demoralization removed. In most cases this twofactor solution worked as predicted. For example, in the joint factor analysis of the demoralization items and Clinical Scale 1, the first factor comprised the 23 demoralization items plus 3 more items that were on Clinical Scale 1. The second factor was a clear somatic concerns factor. The twofactor solution for Clinical Scale 2 worked as well, yielding a second factor suggestive of anhedonia, which then became the major distinctive core of RC2. This approach did not work with Clinical Scale 3. The second factor varied across the four development samples and was mixed in terms of content. Thus, a three-factor solution was attempted. This approach worked consistently across the four samples, yielding demoralization as Factor 1, which included 18 of the 60 Clinical Scale 3 items as well, and a very clear somatic concerns factor. Because somatic concerns had been identified as the major distinctive core of RC1, there was no need to measure it again, so the third factor was examined. This factor was formed by a set of items with cynical content and was thus identified as the major distinctive core of RC3. The original Clinical Scale 3, Hysteria, was intended to identify patients with conversion disorder in reaction to stress. The diverse content areas represented by the 60 items of Clinical Scale 3 include general denial of physical health, many specific somatic symptoms, a general denial of psychological or emotional problems, a

Assessment 18(1) denial of discomfort in social situations, and naively optimistic attitudes about other people (Graham, 2006). This last content category, with reversed scoring of items, became the major distinctive core of RC3, labeled Cynicism, a substantially different construct than hysteria. The major change in focus of RC3, compared with its parent scale, has resulted in a variety of criticisms aimed specifically at this scale. We will briefly review three major criticisms and provide references to more detailed presentation. First is the notion of construct drift as a general criticism made by Nichols (2006). This criticism concerns the fact that in the final stages of the construction of the RC Scales, the developers were willing to include items that had not been on the original clinical scale being restructured. The inclusion of new items would introduce elements that had never been a part of the original construct being measured. Although in most cases, this effect would be subtle (and detrimental in Nicholss view), the change in construct for RC3 is substantial, and, as Nichols does acknowledge, intentional. A second, related criticism is that the loss of Clinical Scale 3 (Hysteria) would be seriously detrimental in medical settings, chronic pain assessments, and personal injury cases (Butcher, Hamilton, Rouse, & Cumella, 2006; Butcher & Miller, 2006; Nichols, 2006). Indeed, there is a long history in the MMPI literature of relying on Scale 3 itself, and particularly in the classic Conversion V configuration with high Scale 1 and low Scale 2 score, as a measure of the extent to which psychological symptoms are masquerading as physical symptoms (see, e.g., Graham, 2006). A third criticism of RC3 is that it is entirely redundant with the CYN (Cynicism) Content Scale on the MMPI-2 (Butcher et al., 2006; Nichols, 2006). Specifically, RC3 correlates with CYN around the .90 level, and 80% of the items on RC3 also appear on CYN. Cogent responses to these criticisms have been provided by Tellegen et al. (2006) and Weed (2006). Regarding construct drift, Weed correctly notes that the shifts in conceptual emphasis of the RC Scales, in comparison with their parent scales, are more properly characterized as focusing or sharpening, rather than drifting. Shift in construct definition is an unavoidable outcome of converting a multidimensional measure into a unidimensional measure, an outcome with far more desirable than undesirable effects. In the case of RC3, the construct change was certainly by conscious decision of the developers. The drift criticism, as such, is even less relevant for RC3 than for the other scales. In response to concerns about the loss of the traditional Hysteria scale for use in pain, personal injury, and other medical settings, Tellegen et al. (2006) offer this remarkable conclusion: . . . although there is abundant lore surrounding Scale 3, there are virtually no empirical data to support its purported validity as a measure of somatic symptomatology beyond what is assessed by Scale 1 (p. 163). Tellegen et al. argue that all the multiple constructs assessed

Ingram et al. by the very heterogeneous Clinical Scale 3 are effectively recoverable with the RC Scales. The 60 items of Clinical Scale 3 include a large demoralization element, now captured by RCd. They include a very large somatic concerns component, now measured unidimensionally by RC1. The component of the hysteria syndrome typically described as a navely optimistic view of the motivations of others is adequately reflected by low scores on RC3. The subsequently identified malaise component of Clinical Scale 3 is represented by the MLS (Malaise) Scale on the MMPI-2-RF. Finally, the criticism that RC3 is redundant with the CYN content scale is moot with the introduction of the MMPI2-RF, as CYN was not included in the restructuring. That said, the claim by Butcher et al. (2006) that there is an 80% overlap between RC3 and CYN is not entirely fair. Although it is true that 12 of the 15 RC3 items are also included on CYN, it is also true that only 12 of the 23 CYN items appear on RC3 (52%). The present article starts with the premise that RC3, when introduced in 2003, was indeed a new scale conceptually, with little relationship to Clinical Scale 3. Thus, we knew less about it than any of the other RC Scales. That said, there is now an emerging literature on RC3, including not only internal psychometric properties but associations with external correlates as well. The MMPI-2-RF Technical Manual (Tellegen & Ben-Porath, 2008) reports correlations between RC3 and approximately 30 external variables for samples of mental health and medical outpatients. In both samples, the largest correlations (around .60) were between RC3 and the Machiavellianism-IV Scale (MACH-IV; Christie & Geis, 1970), though moderate correlations are also reported with state and trait anxiety, as well as magical ideation. Independent findings regarding empirical correlates of RC3 have recently been compiled by Tellegen, Ben-Porath, and Sellbom (2009). Briefly, RC3 has significant positive associations with the Angry-Hostility (r = .41) and the Impulsiveness (r = .31) facet scales of the Neuroticism-Extroversion-OpennessPersonality Inventory Revised (NEO-PI-R; Costa & McCrae, 1992), and strong negative associations with the Agreeableness domain scale (r = .43) and especially the Trust facet scale (r = .64; Sellbom, Ben-Porath, & Bagby, 2008). Forbey and BenPorath (2008) examined the RC Scales in comparison with 15 criterion measures in a nonclinical sample; the highest correlation for RC3 was with the MACH-IV Scale (Christie & Geis, 1970), r = .53 for men and r = .56 for women. Trait anger was next for men (r = .36) and magical ideation for women (r = .38). Examining the RC Scales in relation to the Brief Psychiatric Rating Scale (Overall & Gorham, 1988), Handel and Archer (2008) reported statistically significant correlations between RC3 and the Hostility (r = .11) and Suspiciousness (r = .12) scales. In comparison with the Multidimensional Personality Questionnaire (Tellegen, IN PRESS), substantial positive correlations with RC3 are

97 reported for the Negative Emotionality (r = .55) and Alienation (r = .54) scales (Sellbom & Ben-Porath, 2005). Sellbom, Ben-Porath, Baum, Erez, and Gregory (2008) examined convergent and discriminant validity of the RC Scales in predicting a set of historical variables in a sample of partner batterers. RC3 was statistically significantly associated with 2 of the 10 variables: blame externalization (r = .22) and violence disinhibition (r = .19). Finally, in a study of police officer candidates, RC3 scores were positively associated with problematic job behaviors, such as increased citizen complaints, rude behaviors, and externalization of blame (Sellbom, Fischler, & Ben-Porath, 2007). In this article, we compare RC3 scores with those of several other measures of related constructs, including the Machiavellianism Scale (Christie & Geis, 1970), the Alienation Scale (Jessor & Jessor, 1977), Faith in People (Rosenberg, 1957), and locus of control (Nowicki & Duke, 1983). When supported by significant correlations, the focus is on content analysis at the item level, with the goal of expanding our interpretive perspectives on this new scale.

Method Participants
Data were provided by 40 undergraduate students at a midsized state comprehensive university in the rural southeast. There were 31 females and 9 males, ranging in age from 18 to 37 years (mean age = 19.82 years, SD = 3.1). This sample reported the following ethnicities: 92.5% Anglo American, 2.5% Asian American, 2.5% Hispanic American, and 2.5% Native American. Participants volunteered for this study through the human subject pool in partial fulfillment of course requirements. The study was approved by the Institutional Review Board, and appropriate informed consent was obtained from each participant. Participants were to be excluded if the MMPI-2-RF Cannot Say (CNS) score exceeded 15; if VRIN-r, TRIN-r, or L-r exceeded 80; if K-r exceeded 70; or if F-r exceeded 120. However, no participants actually reached these exclusion criteria.

Materials
MMPI-2-RF. The MMPI-2-RF (Ben-Porath & Tellegen, 2008a) is a 338-item personality inventory derived from the original MMPI and MMPI-2. This inventory comprises 50 scales in all, including 9 validity indicators, 3 higher order scales, 9 core scales termed the Restructured Clinical Scales, 25 more narrowly focused special problem scales, and the Personality Psychopathology 5 scales. The MMPI-2-RF exhibits excellent psychometric properties, described in detail in the Technical Manual (Tellegen & Ben-Porath, 2008). Mach IV. The Machiavellianism Scale (Christie & Geis, 1970) was based on the writings of Machiavelli and was

98
Table 1. Descriptive Statistics for Current Sample Compared With Normative and College Student Samples Current Sample n = 40 Scale RCd RC1 RC2 RC3 RC4 RC6 RC7 RC8 RC9 Mean 54.05 57.82 48.98 55.93 51.63 56.07 54.38 56.43 52.93 SD 10.26 10.27 9.82 11.00 9.33 9.52 10.35 12.57 10.80 Mean 49 49 50 51 52 51 48 50 51 MMPI-2-RF Normative Sample Men SD 9 9 10 10 10 10 9 10 11 Women Mean 51 51 50 49 48 49 52 50 49 SD 10 10 10 10 9 10 10 10 9 Mean 54 52 49 57 56 57 53 57 61

Assessment 18(1)

College Student Sample Men SD 11 10 11 10 11 12 12 12 11 Women Mean 56 56 49 56 52 56 56 56 56 SD 11 11 11 10 11 12 12 12 11

Note. MMPI-2-RF = Minnesota Multiphasic Personality Inventory-2Restructured Form; RCd = Demoralization; RC1 = Somatic Complaints; RC2 = Low Positive Emotions; RC3 = Cynicism; RC4 = Antisocial Behavior; RC6 = Ideas of Persecution; RC7 = Dynsfunctional Negative Emotions; RC8 = Aberrant Experiences; RC9 = Hypomanic Activation.

designed to assess the extent to which the test-taker sees other people as gullible or manipulable. Originally 71 items, the most commonly used version in research is the Mach IV, which consists of 20 items. Items are rated on a 7-point Likert-type scale ranging from strongly agree to strongly disagree. Split-half reliabilities on several samples averaged .79. Alienation Scale. The Alienation Scale (Jessor & Jessor, 1977) is a measure of general alienation, including uncertainty about the meaning of ones life as well as a sense of social isolation. The scale has 15 items, with a 5-point Likerttype response format. Two samples used in developing the scale produced the same Cronbachs alpha level of .81 and testretest reliability in one study described by the test authors, spanning a 1-year period, was .49. Validity studies show that this scale correlates in expected directions with measures of self-esteem, locus of control, expectation of affection, and parentfriend compatibility. Faith in People Scale. The Faith in People Scale (Rosenberg, 1957) was developed to assess an individuals degree of confidence in the honesty, trustworthiness, generosity, and goodness of other people. It consists of only five Guttmantype items, two of which are binary forced-choice and three are agreedisagree format. This instrument exhibits high internal consistency (coefficient of reproducibility was .92), and a variety of studies on occupational selection support its validity. ANSIE. The Adult NowickiStrickland InternalExternal Control Scale (ANSIE; Nowicki & Duke, 1983) consists of 40 items that entail yes or no responses. It is meant to measure ones beliefs concerning internal and external locus of control. The ANSIE has reported reliability varying from .83 during a 6-week testretest interval to .56 during a 1-year testretest interval.

Procedures
Participants were tested in small groups ranging from 1 to 5 individuals. They completed all materials using paper and pencil; questionnaires were presented in counterbalanced order. The MMPI-2-RF tests were scored using Q-Local software, and T scores were entered into an SPSS 17.0 datafile. Raw item scores were entered for all other measures.

Results
Table 1 is presented to show how our sample compares with the large MMPI-2-RF normative sample on the RC Scales, as well as with the college sample described in the MMPI2-RF Technical Manual (Tellegen & Ben-Porath, 2008). Generally, this small sample appears to be reasonably representative of the population with regard to the variables of interest. Correlations between each of the RC scales and each of the external measures are shown in Table 2, with the RC3 correlations highlighted. Substantial correlations were found between RC3 and Machiavellianism (r = .47) and between RC3 and Alienation (r = .60). We note that the correlation with Machiavellianism was quite consistent with results reported by Forbey and Ben-Porath (2008), with r values of .53 for men and .56 for women. Correlations between RC3 and the Faith in People Scale, as well as between RC3 and the ANSIE, were modest and not statistically significant in this small sample. Our purpose in this study was not so much to demonstrate convergent validity for RC3, but rather to explore its associations with other variables in order to enhance our understanding of what aspects of personality are being

Ingram et al.
Table 2. Correlations Between External Measures and Restructured Clinical (RC) Scales Machiavellianism Scale RCd RC1 RC2 RC3 RC4 RC6 RC7 RC8 RC9 Correlation .20 .02 .11 .47 .03 .33 .38 .13 .29 p .215 .921 .494 .002 .856 .038 .017 .415 .068 Alienation Correlation .60 .07 .47 .60 .01 .35 .59 .12 .18 p .000 .689 .003 .000 .941 .034 .000 .473 .276 Faith in People Correlation .40 .16 .28 .29 .37 .18 .25 .11 .31 p .012 .326 .085 .065 .017 .254 .118 .511 .050 Locus of Control Correlation .28 .23 .19 .25 .26 .21 .23 .18 .29

99

p .087 .167 .252 .123 .116 .214 .165 .279 .080

Note. RCd = Demoralization; RC1 = Somatic Complaints; RC2 = Low Positive Emotions; RC3 = Cynicism; RC4 = Antisocial Behavior; RC6 = Ideas of Persecution; RC7 = Dysfunctional Negative Emotions; RC8 = Aberrant Experiences; RC9 = Hypomanic Activation.

Table 3. Highest Item Correlates on the Machiavellianism and Alienation Scales Item Q1. Q2. Q5. Q8. Item Q7. Q9. Q11. Q12. Q13. Item Text: Machiavellianism Scale Never tell anyone the real reason you did something unless it is useful to do so The best way to handle people is to tell them what they want to hear It is safest to assume that all people have a vicious streak and it will come out when they are given a chance It is hard to get ahead without cutting corners here and there Item Text: Alienation Scale Correlation With RC3 .51 .54 .39 .37 Correlation With RC3 .51 .50 .49 .42 .51 .46 .43 p .001 .000 .014 .020 p .001 .001 .001 .007 .001 .003 .006

It is hard to know how to act most of the times since you cant tell what others expect Nowadays you cant really count on other people when you have problems or need help I often find it difficult to feel involved in the things I am doing Hardly anyone I know is interested in how I really feel inside I generally feel that I have a lot of interests in common with the other students in the school (reverse scored) Q14. I often feel alone when I am with other people Q15. If I really had my choice Id live my life in a very different way than I do
Note. RC3 = Cynicism.

tapped by the scale. Thus, we took the additional step of analyzing correlations at the item level. We decided that in cases where the overall correlation was significant (i.e., Machiavellianism and Alienation), we would examine specific items that had meaningful correlations with RC3. We initially decided that we would use .40 as the minimum acceptable correlation. However, with the Machiavellianism Scale, despite the high overall correlation, only two items showed correlations greater than .40, and only four items had statistically significant correlations. We chose to include all four of these items for the Machiavellianism Scale, shown in Table 3. For the Alienation Scale, we adhered to the .40 criterion. Table 3 also displays the seven items from the Alienation Scale with substantial correlations.

Discussion
Overall, RC3 appears to correlate meaningfully with a variety of other measures of related constructs, consistent with previous research supporting the convergent validity of this scale. Content of RC3 items as well as external correlates available during the development of the Scale focus on a cynical view of human nature and a generally negative view of the motivations of other people, reflecting the key ideas that other people are looking out for their own interests most of the time and cannot be trusted. Furthermore, high scores reflect hostility, feelings of alienation, and negative interpersonal history. Many of our item-level findings are quite consistent with this interpretive perspective. Most of the significant items on the Alienation Scale fit this

100 category, reflecting distinct feelings of personal and social alienation (e.g., Table 3, Alienation Scale, Items Q11, Q12, Q13, and Q14), as well as disappointment or lack of faith in others (e.g., Table 3, Alienation Scale, Items Q7 and Q9). Further expansion of the RC3 core construct is suggested by the four significant items from the Machiavellianism Scale. Q5It is safest to assume that all people have a vicious streak and it will come out when they are given a chanceis generally consistent with the recommended interpretive concepts, though the idea that everyone has a vicious streak seems somewhat more severe. The other three items add a distinctly new element to RC3 interpretation never tell anyone the real reason you did something unless it is useful to do so (Q1); tell people what they want to hear (Q2); cutting corners is necessary in order to get ahead (Q8). These are, frankly, Machiavellian in nature. Although the basic concept of RC3 has to do with a cynical view of the motivations and trustworthiness of others, these items reflect the notion that other people are gullible and manipulable as well. The developers of the MMPI-2-RF (Ben-Porath & Tellegen, 2008a) report that empirical correlates support the interpretation of low scores on RC3 as well as high scores, with low scorers described as having a positive view of human nature and perhaps being overly trusting or gullible. There is, thus, empirical support from the outset that RC3 reflects a continuous dimension of personality. Current findings suggest a set of more severe and negative set of correlates of RC3, which may be associated with higher scores. One speculation is that high scores may indicate such a negative view of others that deceit and manipulation are justified. That is, if one believes that others are dishonest, untrustworthy, and even vicious when given a chance, then it makes sense to be proactive, to lie, deceive, manipulate, and cheat. Limitations of this study include the sample for two reasons: (a) its small size and (b) it consists entirely of college students. Regarding the sample size, we did restrict our interpretations to substantial correlations, not just statistically significant ones, and the key correlations are large enough that they may be considered robust despite the sample size (Rice, 2007). In addition, the use of college students is not as problematic in this study as it might be with studies of psychopathological constructs, in that the generally normally distributed trait of cynicism should not exhibit significant restriction of range in a normal population. That said, current results are best considered preliminary and in need of replication and extension. A broader and more comprehensive set of external correlates should be considered, including behavioral variables, not just other questionnaires. In particular, our data suggest a possible link between RC3 and some of the core characteristics associated with the classical view of the psychopathic personality. Indeed, from the standpoint of the five-factor model (FFM) of normal

Assessment 18(1) personality, RC3 shows a strong conceptual association with the Agreeableness domain (direction reversed), and a consistent finding among researchers studying psychopathy from the FFM framework is that low A is a key feature of the profiles of the psychopath (see, e.g., Lynam & Widiger, 2001; Widiger, Trull, Clarkin, Sanderson, & Costa, 2002). Consistent with this speculation is the recent report by Sellbom et al. (2008) showing that among the domains and facets of the FFM, RC3 correlated most strongly with the Agreeableness domain (.43) and specifically the Trust facet (.64). In conclusion, our results suggest that RC3 may tap into a personality dimension that goes beyond a general cynicism about people, possibly to a Machiavellianism that justifies the premeditated mistreatment of others. Pending replication and corroboration, these results should certainly be used cautiously, though it does make sense to begin thinking about broader implications of this interesting new scale. Authors Note
Paul B. Ingram is now with Meridian Behavioral Health Services, Sylva, North Carolina.

Declaration of Conflicting Interests

The authors declared no conflicts of interest with respect to the authorship and/or publication of this article.

Funding
The authors received no financial support for the research and/or authorship of this article.

References
Ben-Porath, Y., & Tellegen, A. (2008a). Minnesota Multiphasic Personality InventoryRestructured form: Manual for administration, scoring, and interpretation. Minneapolis: University of Minnesota Press. Ben-Porath, Y., & Tellegen, A. (2008b). Empirical correlates of the MMPI-2 Restructured Clinical (RC) scales in mental health, forensic, and nonclinical settings: An introduction. Journal of Personality Assessment, 90, 119-121. Butcher, J. N., Hamilton, C. K., Rouse, S. V., & Cumella, E. J. (2006). The deconstruction of the Hy scale of MMPI2: Failure of RC3 in measuring somatic symptom expression. Journal of Personality Assessment, 87, 186-192. Butcher, J. N., & Miller, K. (2006). Personality assessment in personal injury litigation. In I. B. Weiner & A. Hess (Eds.), Handbook of forensic psychology (3rd ed.). New York, NY: Wiley. Christie, R., & Geis, F. L. (1970). Studies in Machiavellianism. New York, NY: Academic Press. Costa, P. T., & McCrae, R. R. (1992). NEO-PI-R professional manual. Odessa, FL: Psychological Assessment Resources. Forbey, J. D., & Ben-Porath, Y. S. (2008). Empirical correlates of the MMPI-2 Restructures Clinical (RC) scales in a nonclinical setting. Journal of Personality Assessment, 90, 136-141.

Ingram et al.
Graham, J. R. (1977). The MMPI: A practical guide. New York, NY: Oxford University Press. Graham, J. R. (2006). MMPI-2: Assessing personality and psychopathology (4th ed.). New York, NY: Oxford University Press. Greene, R. L. (1980). An MMPI interpretive manual. New York, NY: Grune & Stratton. Handel, R. W., & Archer, R. P. (2008). An investigation of the psychometric properties of the MMPI-2 Restructured Clinical (RC) scales with mental health inpatients. Journal of Personality Assessment, 90, 239-249. Jessor, R., & Jessor, S. (1977). Problem behavior and psychosocial development. New York, NY: Academic Press. Lynam, D. R., & Widiger, T. A. (2001). Using the five-factor model to represent DSM-IV personality disorder: An expert consensus approach. Journal of Abnormal Psychology, 110, 401-412. Meyer, G. J. (Ed.). (2006). The MMPI-2 Restructured Clinical Scales [Special issue]. Journal of Personality Assessment, 87(2). Nichols, D. S. (2006). The trials of separating the bath water from the baby: A review and critique of the MMPI-2 Restructured Clinical Scales. Journal of Personality Assessment, 87, 121-138. Nowicki, S., Jr., & Duke, M. P. (1983). The Nowicki-Strickland life-span locus of control scales: Construct validation. In H. M. Lefcourt (Ed.), Research with the locus of control construct (Vol. 2, pp. 9-43). New York, NY: Academic Press. Overall, J. E., & Gorham, D. R. (1988). Introduction: The Brief Psychiatric Rating Scale (BPRS): Recent developments in ascertainment and scaling. Psychopharmacology Bulletin, 24, 97-98. Rice, J. A. (2007). Mathematical statistics and data analysis. Pacific Grove, CA: Duxbury Press. Rosenberg, M. (1957). Occupations and values. Glencoe, IL: Free Press. Sellbom, M., & Ben-Porath, Y. S. (2005). Mapping the MMPI-2 Restructured Clinical Scales onto normal personality traits: Evidence of construct validity. Journal of Personality Assessment, 85, 179-187. Sellbom, M., Ben-Porath, Y. S., & Bagby, R. M. (2008). Personality and psychopathology: Mapping the MMPI-2 Restructured

101
Clinical (RC) Scales onto the five-factor model of personality. Journal of Personality Disorders, 22, 291-312. Sellbom, M., Ben-Porath, Y. S., Baum, L. J., Erez, E., & Gregory, C. (2008). Predictive validity of the MMPI-2 Restructured Clinical (RC) Scales in a batterers intervention program. Journal of Personality Assessment, 90, 129-135. Sellbom, M., Fischler, G. L., & Ben-Porath, Y. S. (2007). Identifying MMPI-2 predictors of police officer integrity and misconduct. Criminal Justice and Behavior, 34, 985-1004. Tellegen, A. (IN PRESS). Manual for the Multidimensional Personality Questionnaire. Minneapolis: University of Minnesota Press. Tellegen, A., & Ben-Porath, Y. (2008). Minnesota Multiphasic Personality InventoryRestructured Form: Technical manual. Minneapolis: University of Minnesota Press. Tellegen, A., Ben-Porath, Y. S., McNulty, J. L., Arbisi, P. A., Graham, J. R., & Kaemmer, B. (2003). MMPI-2 Restructured Clinical (RC) scales: Development, validation, and interpretation. Minneapolis: University of Minnesota Press. Tellegen, A., Ben-Porath, Y. S., & Sellbom, M. (2009). Construct validity of the MMPI-2 Restructured Clinical (RC) Scales: Reply to Rouse, Greene, Butcher, Nichols, and Williams. Journal of Personality Assessment, 91, 211-221. Tellegen, A., Ben-Porath, Y. S., Sellbom, M., Arbisi, P. A., McNulty, J. L., & Graham, J. R. (2006). Further evidence on the validity of the MMPI-2 Restructured Clinical (RC) Scales: Addressing questions raised by Rogers, Sewell, Horrison, and Jordan and Nichols. Journal of Personality Assessment, 87, 148-171. Weed, N. C. (2006). Syndromal complexity, paradigm shifts, and the future of validation research: Comments on Nichols and Rogers, Sewell, Harrison, and Jordan. Journal of Personality Assessment, 87, 217-222. Widiger, T. A., Trull, T. J., Clarkin, J. F., Sanderson, C., & Costa, P. T. (2002). A description of the DSM-IV personality disorders with the five-factor model of personality. In P. T. Costa & T. A. Widiger (Eds.), Personality disorders and the fivefactor model of personality (2nd ed., pp. 89-99). Washington, DC: American Psychological Association.