Diagnosis and Classification Research Paper

On Diagnosis and Classification

Folk Taxonomies

Man is by nature a classifying animal. His continued existence depends on his ability to recognize similarities and differences between objects and events in the physical universe and to make known these similarities and differences linguistically. Indeed, the very development of the human mind seems to have been closely related to the perception of discontinuities in nature. (Raven, Berlin, & Breedlove, 1971, p. 1210)

Raven and his colleagues used the phrase folk taxonomy to emphasize their belief that peoples through the ages have developed taxonomies as ways of manipulating knowledge. Although this predisposition is particularly descriptive of specialist subgroups within cultures (e.g., mental health professionals), all cultures have developed taxonomies that—even across diverse cultures—nonetheless tend to take on strikingly similar characteristics.

Taxonomies recognize naturally occurring groups. They are readily identified as natural groups, at least by specialists; accordingly, they are treated as discontinuous from each other. This is certainly the case with psychopathology, in which such illnesses as schizophrenia, mood disorders, substance-related disorders, and others have been viewed as distinct from each other as well as from psychologically healthy states for many decades.

Taxonomies are developed for communication about items of interest to cultural groups that are acquainted with the properties of the items. They are, in effect, a kind of shorthand language that concentrates useful information in the hands of the specialists who require access to that information. The same is true of psychiatric diagnosis, which is the province of the mental health professionals, who daily trade information about their patients back and forth by means of a literature organized around diagnostic classes.

Taxonomies are also organized in a shallow hierarchy; this means that most folk taxonomies focus on generic taxa (categories at a low level of abstraction but not so specific as at the species level). These categories typically comprise a set that is memorizable (typically between 250 and 800) and consist of mutually separate and distinctively named categories with well-understood limits. The more specific or varietal the members of a genus, the more cultural importance that genus tends to have. Thus, the DSM-IV contains around 300 diagnostic categories, and those with the greatest degree of cultural significance—dementia praecox (schizophrenia) in the nineteenth century and (more recently) the mood, anxiety, and substance use disorders—are the most ramified and differentiated.

In other words, the traditions of modern syndromal diagnosis in psychiatry fit well within the limits of folk taxonomies as outlined by Raven and his coworkers. As clinical psychologists, we can take comfort in our adherence to an ancient natural scientific tradition, taxonomy. Said another way, we are not deviating from a putative scientific norm, as some critics of psychiatric diagnosis have alleged (e.g., Albee, 1970; Kanfer & Saslow, 1965; Zigler & Phillips, 1961), when we deploy diagnostic classification systems to categorize our patients.Humansindeedareclassifyinganimals,andwemental health professionals are very human in this regard.

Natural and Prototype Categorization

Rosch (1973) developed the useful concept of nonarbitrary naturalcategories,likenamesofcolors,thatformaroundperceptually salient natural prototypes. Natural categories have eight key attributes, which we illustrate here by using color as the example: (a)They are partitioned—not from discrete clusters but from continua (e.g., wavelength); (b) they cannot be further reduced to simpler attributes (e.g., attributes of color such as saturation and reflectance require a specialist’s skills to describe and understand, and they must be specially learned); (c) some of the examples of these categories are better than others (e.g., true colors vs. off hues); (d) they are not arbitrary; (e) they are easily learned by novices; (f) they attract attention and are easily remembered because they are based on properties that are more salient perceptually than other stimuli in their domains (e.g., sky blue is more salient than aqua, forest green is more salient than lime, snow white is more salient than cream); (g) these salient properties of naturalcategoriesserveasnaturalprototypesfortheorganization of more knowledge; and (h) natural categories have fuzzy (indistinct) boundaries, so they will ultimately encompass both clear-cut and marginal examples. Lilienfeld and Marino (1995) subsequently extended this analysis specifically to psychiatric diagnosis by arguing that psychopathologic entities are Roschian or natural categories because they are partitioned from the continuum of human behavior, they are irreducible to simpler concepts, and some are better examples of firmly bounded categories than others are.

The view that mental disorders represent natural categories complements another influential conceptualization, prototypic categorization, first described by Cantor, Smith, French, and Mezzick (1980). Lamenting the tendency of numerous critics of psychiatric diagnosis to endorse the unhelpful classification standards of what they term the classical categorization model, Cantor and her colleagues proposed instead a prototype categorization model.

The classical categorization model makes the following assumptions about the items to be organized in a diagnostic process that the prototypic categorization model does not: (a) the presence of universally accepted criteria for class membership (e.g., all squares have four sides, and all schizophrenic individuals are autistic); (b) high agreement about class membership among classifiers (e.g., everyone agrees on what is a square, just as everyone agrees on who is schizophrenic); and (c) within-class homogeneity of members (e.g., all squares look alike, and all schizophrenic persons behave the same way). Obviously, this standard is inappropriate to judgments of mental disorder.

Cantor and colleagues’ prototype categorization appears to characterize far better the process of syndromal classification epitomized by DSM-III, DSM-III-R, and DSM-IV. That approach assumes (a) correlated—not necessarily pathognomonic—criteria for class membership, (b) high agreement among classifiers only when classifying cases that demonstrate most of the correlated criteria for class membership, and (c) heterogeneity of class membership because criteria are only correlated, not pathognomonic.

Psychiatric diagnosis and the diagnostic system look reasonably orderly when viewed within the context of [naturalistic classification] systems. Heterogeneity of class membership, borderline cases, and imperfect inter- and intrajudge reliability can all be accepted and studied as fundamental properties of the system, rather than branded as aberrations, errors in measurement or faulty utilization of an otherwise classical scientific system. Revisions in training procedures can be made to better mirror the system as it is actually conceptualized and utilized by practicing clinicians. (Cantor et al., 1980, p. 190)

Utility of Classification and Diagnosis

Birley (1975) suggested that diagnosis should really be viewed as an art. He made this suggestion in the belief that the prime challenge to both the artist and the diagnostician is to grasp a complex and daunting slice of nature and transform it into a condensed, symbolic representation in such a way as to communicate a truth about that slice of nature. However, diagnosis is much more often viewed as a scientific tool. Viewing it this way, Blashfield and Draguns (1976) detailed its diverse scientific purposes as follows: (a) communication, because without a consensual language, practitioners could not communicate; (b) a means for organizing and retrieving information, because an item’s name is a key to its literature and knowledge accrues to the type; (c) a template for describing similarities and differences between individuals; (d) a means of making predictions about course and outcome; and (e) a source of concepts to be used in theory and experimentation.

The view of diagnosis as a scientific tool is best exemplified in the work of members of the neo-Kraepelinian school of American psychiatry. This group of influential thinkers was drawn largely from psychiatry faculty at the Washington University School of Medicine in Saint Louis and the Columbia University College of Physicians and Surgeons in New York. Their diagnostic research during the decades of the 1960s and 1970s laid the groundwork for the revolutionary advances of the DSM-III (APA, 1980).

The neo-Kraepelinians, like their namesake, Emil Kraepelin, endorsed the existence of a boundary between pathological functioning and problems in living. Although this was seen as a permeable boundary (those who exist on one side of it sometimes cross over to the other), it was nonetheless an important distinction to make because the existence of pathology is an important mandate for professional attention. Second, viewing psychiatry as a branch of medicine, the neo-Kraepelinians viewed mental illness, like any other illness, as the purview of medicine. Mental illnesses are real, they are diverse, and—by applying a scientific method of discovery, the neo-Kraepelinians affirmed—they can be affected by studies of their etiology, course, prognosis, morbidity, associated features, family dynamics, and predisposing features.

DSM-I AND DSM-II

Historical Roots

Prior to the philosopher-physician Paracelsus, diagnoses were made on the basis of presumed etiology, as when Hippocrates rooted the illnesses he diagnosed in various imbalances of fluxes and humors (Zilboorg, 1941). This changed with Paracelsus’delineation of syndromal diagnosis, with the syndrome defined as a group ofsignsand symptomsthat co-occur in a common pattern and characterize a particular abnormality or disease state. To our day, syndromal diagnosis has focused on the signs and symptoms of disease entities. Typically, it organizes them hierarchically, by the principles of descriptive similarity or shared symptom pictures. In Paracelsus’system, as well as in each succeeding step toward the modern approach epitomized by the DSM-III (APA, 1980) and DSM-IV (APA, 1994), the etiology of the illness was presumed to be unknown and hence unnecessary for the diagnostic task.

Notable and more complete nomenclatures were subsequently developed, first by Thomas Sydenham (1624–1663) and later by Francois de Sauvages (1706–1767); both developed what were for their time organized and comprehensive hierarchical classification systems. Shortly afterwards Phillippe Pinel, best known for his pioneering efforts to humanize the care of French patients in hospitals for the insane, developedan even more comprehensive classification system (Zilboorg, 1941). The appearance of this nomenclature coincided with the rise of asylums for the insane, for which Pinel was partly responsible. Both Pinel’s system and the new availability of large numbers of patients in asylums paved the way for the marked increase in efforts to categorize psychopathology during the ensuing nineteenth century.

Predictably, superintendents of asylums for mentally ill patients in the nineteenth century were concerned almost entirely with cases of serious and protracted psychopathology— organic mental disorders, severe developmental disabilities, dementia, and what we today call schizophrenia and bipolar disorder (Nathan, 1998; Spitzer, Williams, & Skodol, 1980). Advances in the understanding of these serious disorders accelerated when the German psychiatrist Karl Kahlbaum (1828–1899) discovered the value of knowing premorbid course and risk factors as means of predicting outcome in dementia praecox (today we call the disorder schizophrenia). Additional advances followed Kraepelin’s (1907) research on posthospital course, clinical outcome, and treatment response in cases of major mental illness (Zilboorg, 1941). Kraepelin synthesized these ideas and those of his intellectual forebears in his series called Lectures in Psychiatry, which developed the basic outlines of the first modern taxonomy of mental illnesses, from which many of our current concepts, procedures, and technical terms have developed.

In the twentieth century, psychiatrists and psychologists developed realms of clinical practice discrete from—and very different from—the mental asylums, including schools, the military services, private clinics, and other service outlets. Their experiences, especially during World War II, when psychological casualties took an unexpectedly high toll among combat personnel, required development of a nomenclature that provided substantially greater coverage of the nonpsychotic conditions. Nosologies grew increasingly complex in publications by the National Commission on Mental Hygiene/Committee on Statistics of the American Medico– Psychological Association (1917; Blashfield, 1984), the American Psychiatric Association/New York Academy of Medicine (APA, 1933), and especially the Veterans Administration in the aftermath of World War II. These developments formed the basis for the publication of the first formal nosologies sponsored by the American Psychiatric Association— DSM-I (APA, 1952) and DSM-II (APA, 1968).

Deficiencies

The much-anticipated first edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-I; APA, 1952) was a pioneering comprehensive syndromal classification system. It enabled North American mental health professionals, at least in principle, to employ a common diagnostic language for the first time. At the same time, DSM-I and its like successor, DSM-II (1968), shared serious deficiencies.

The manuals contained relatively little material. The DSM-I contained 130 pages and fewer than 35,000 words; DSM-II was four pages longer but contained about as many words. As a result, they provided only brief, vague descriptions of each syndrome; typically, these descriptions consisted of one or two short paragraphs listing distinguishing signs and symptoms but not detailing them. This information proved insufficient for reliable diagnoses.

An additional problem was that the signs and symptoms linked to each syndrome were not empirically based; instead, they represented the accumulated clinical wisdom of the small number of senior academic psychiatrists who had drafted these two instruments.As a consequence, the diagnostic signs and symptoms were often inconsistent with the clinical experiences of mental health professionals working in public mental hospitals, mental health centers, and the like.

Consequently, clinicians often failed to agree with one another in assigning diagnoses based on DSM-I and DSM-II. They often failed to agree both when they were presented with the same diagnostic information (interclinician agreement; Beck, Ward, Mendelson, Mock, & Erbaugh, 1962; Nathan, Andberg, Behan, & Patch, 1969; Sandifer, Pettus, & Quade, 1964) and when they were asked to reevaluate the same patient after a short time had passed (diagnostic consistency; Zubin, 1967).

Predictably, the low reliability of DSM-I and DSM-II diagnoses affected both their validity and their clinical utility. If clinicians could not agree on a diagnosis, it was unlikely that they would be able to validate the diagnosis against other measures (Black, 1971). For the same reason, they would be unlikely to have confidence in predictions of the future course of diagnosed disorders (Nathan, 1967) or to create the diagnostically homogeneous groups of patients necessary to enable examination of etiological or treatment issues (Nathan & Harris, 1980).

The poor reliability and validity of diagnoses rendered according to DSM-I and DSM-II raised ethical concerns among some practitioners and scholars. Szasz (1960) wrote extensively of the dehumanizing, stigmatizing consequences of psychiatric labeling, ultimately concluding that the modern constructs of psychiatric illness categories were mere myths. Szasz’s ideas were lent empirical substance in 1973 when Rosenhanpublished“OnBeingSaneinInsanePlaces.”Inthis classic study, eight of Rosenhan’s peers, friends, and graduate students self-referred to one of 12 psychiatric hospitals, complaining of hearing voices. Immediately upon admission, they ceased complaining of any abnormal perceptions and manifested no other symptoms. Nonetheless, all eight were diagnosed as psychotic, and their subsequent behavior was construed in conformance to that label. Summarizing his findings, Rosenhan concluded, “The normal are not detectably sane” (p. 252), a damning assertion indeed for advocates of then-current diagnostic systems in psychiatry.

DSM-III and DSM-III-R

The publication of DSM-III (APA, 1980) represented a substantial advance in the reliability, validity, and utility of syndromal diagnosis. DSM-III-R, published in 1987, was a selective revision of DSM-III that retained the advances of the 1980 instrument and incorporated generally modest changes in diagnostic criteria that new clinical research suggested should be a part of the diagnostic system.

Development of DSM-III

In the late 1960s, psychiatrist Robert Spitzer and his colleagues at the New York State Psychiatric Institute undertook research on syndromal diagnosis that ultimately led to the development of several structured diagnostic interviews. These instruments were the first to have the capability of gathering the exhaustive data on clinical signs and symptoms required by an empirically based nomenclature. Chief among these structured interviews were the Mental Status Schedule (Spitzer, Fleiss, Endicott, & Cohen, 1967) and the Psychiatric Status Schedule (Spitzer, Endicott, Fleiss, & Cohen, 1970). Additionally, Spitzer and his colleagues developed two computer programs, called DIAGNO and DIAGNO-II, that were designed to use the clinical information gathered by the Mental Status Schedule to assign reliable clinical diagnoses (Spitzer & Endicott, 1968, 1969).

Diagnostic researchers at Washington University in Saint Louis shared a parallel interest in developing more systematic, empirically buttressed approaches to diagnosis. They published an article in 1972 (Feighner et al., 1972) that set forth explicit diagnostic criteria for the 16 major disorders about which the authors believed enough empirical data had accumulated to ensure reliability and validity. The intent of what came to be called the Feighner criteria was to replace the vague and unreliable material on these disorders in the DSM-I and DSM-II with formally organized, empirically supported diagnostic criteria. The hope was that these criteria could help researchers establish the diagnostically homogeneous experimental groups that diagnostic and treatment researchers were increasingly demanding. The format of the Feighner criteria anticipated—and influenced—the format of the diagnostic criteria subsequently adopted for DSM-III.

In 1975, the Research Diagnostic Criteria (RDC) were proposed. Developed jointly by the New York State Psychiatric Institute and Washington University groups (Spitzer, Endicott, & Robins, 1975), the RDC were designed to permit empirical testing of the presumed greater reliability and validity of the Feighner criteria. In fact, the reliability of the RDC did yield substantially greater diagnostic reliability for diagnoses of the same disorders based on DSM-II (Helzer, Clayton, et al., 1977; Helzer, Robins, et al., 1977). This finding foreshadowed the enhanced reliability of DSM-III diagnoses a few years later.

The DSM-III diagnostic criteria, based in large part on the RDC, constitute the nomenclature’s most significant departure. They are designed to organize each syndrome’s distinguishing signs and symptoms within a consistent format, so that each clinician who is called upon to use them can define each sign and symptom the same way and process the resulting diagnostic information in a similarly consistent manner. Many studies of the DSM-III diagnostic criteria have affirmed the criteria’s success in inducing substantially higher diagnostic reliability, albeit not for every syndrome. However, as Skodol and Spitzer (1987) observed in describing five sources of variation among raters and ratings, not every factor contributing to less-than-perfect diagnostic reliability stems from inadequacies in the diagnostic system.

Subject variance, as the patient’s state changes over time; occasion variance, as the subject is in a different stage of the same condition, or at least reports different information about it; information variance, as different information is obtained from the patient as a result of different examinations; observer variance, as raters differ in their understanding or interpretation of phenomena, such as in rating blunted affect; and criterion variance, as subjects are allocated to different classes because different decision rules are followed. (Skodol & Spitzer, 1987, p. 15)

Around the time of publication of DSM-III, several structured and semistructured diagnostic interviews based on the DSM-III were published in recognition of the impact on diagnostic reliability of information, observer, and criterion variance, all associated with the process by which diagnostic information is gathered. The best known of these instruments is the National Institute of Mental Health (NIMH) Diagnostic Interview Schedule (DIS; Robins, Helzer, Croughan, & Ratcliff, 1981), a structured interview that was taught to nonclinician interviewers for use in the multisite Epidemiologic Catchment Area (ECA) study (Regier et al., 1984). The semistructured Structured Clinical Interview for DSM-III (SCID; Spitzer, 1983; Spitzer & Williams, 1986) was also published around the same time. Both have been widely used, and both appear to contribute to the enhanced diagnostic reliability of DSM-III. These important—and in most ways unprecedented—new instruments and ones like them provided the data-gathering structure both for major new epidemiological efforts (e.g., ECAstudy, Regier et al., 1984; National Comorbidity Survey, Kessler, Sonnega, Bromet, Hughes, & Nelson, 1995) and for a host of clinical and preclinical studies because such instruments ensured the internal validity of research by helping keep samples of human psychopathology well documented diagnostically.

Reliability

The earliest direct tests of the interrater reliability of DSM-III were in the context of the field trials of the instruments that wereconductedinthelate1970s.Datafromtwolargefieldtrials indicated that reliability was adequate but not outstanding, with overall kappa values of .68 and .72 for chance-correct agreement on Axis I disorders. Diagnoses for substance use disorder, schizophrenia, and organic mental disorder were significantly more reliable than were those for the adjustment and anxiety disorders (Spitzer, Forman, & Nee, 1979; Williams, Hyler, & Spitzer, 1982).

The reliability of the disorders of childhood was also examined in the DSM-III field trials; their reliability had been problematic through the years, in part because of the changeability of behavior during this era in a person’s life. Field trial studies of the reliability of criteria for the new DSM-III childhood disorders, markedly expanded from DSM-II to DSM-III, were disappointing—in part because of clinicians’ unfamiliarity with the radically new system (Cantwell, Russell, Mattison, & Will, 1979; Mattison, Cantwell, Russell, & Will, 1979). However, subsequent reliability studies of childhood disorder diagnoses attained from semistructured diagnostic interviews based on the DSM-III criteria and translated into Japanese (Hanada & Takahashi, 1983) and Norwegian (Larsen & Vaglum, 1986) were more promising.

A substantial number of reliability studies of the DSM-III and DSM-III-R diagnostic criteria have been published. In general, these studies point to greater diagnostic stability and greater interrater agreement for these instruments than for their predecessors, DSM-I and DSM-II. Enhanced reliability has been especially notable for the diagnostic categories of schizophrenia, bipolar disorder, major depressive disorder, and substance abuse and dependence.The reliability of the personality disorders, some of the disorders of childhood and adolescence, and some of the anxiety disorders, however, has been less encouraging (e.g., Chapman, Chapman, Kwapil, Eckblad, & Zinser,1994;Fennigetal.,1994;Klein,Ouimette,Kelly,Ferro, & Riso, 1994; Mattanah, Becker, Lexy, Edell, & McGlashan, 1995), in part because of diagnostic stability problems.

Diagnostic Stability

Even though the DSM-III and DSM-III-R diagnostic criteria markedly enhanced diagnostic reliability, diagnostic stability continued to affect the diagnostic process because of naturally occurring changes in clinical course over time. As the research summarized in this section suggests, diagnostic stability depends both on the reliability of the diagnostic instrument and on the variability of clinical course over time.

Fennig et al. (1994) investigated the 6-month stability of DSM-III-Rdiagnosesinalargegroupoffirst-admissionpatients with psychosis—a notably diagnostically unstable group. Affective psychosis and schizophrenic disorders showed substantial diagnostic stability, with 87–89% of patients remaining in the same broad category. Stability for subtypes of these conditions were less stable; only 62–86% of patients remained in the same subcategory. Forty-three percent of these diagnostic changes were attributed to clinical course, and the rest was assumed to reflect the imperfect reliability of the diagnostic process itself.

In a subsequent study of the stability of psychotic diagnoses, Chen, Swann, and Burt (1996) examined changes from schizophrenia diagnoses to those of other disorders and from those of other disorders to schizophrenia in inpatients hospitalized at an urban acute care hospital at least four times over a 7-year period. Only 22% of patients with a schizophrenia diagnosis at the beginning of the study received a different diagnosis during a subsequent hospitalization. Females and patients of Hispanic origin were more likely than others were to experience a diagnostic change from schizophrenia. However, 33% of patients with an initial diagnosis other than schizophrenia were later diagnosed with schizophrenia. Males and African Americans were more likely to change to a diagnosis of schizophrenia. These authors concluded that— contrary to widespread belief—the diagnosis of schizophrenia in current practice is not static.

Coryell et al. (1994) followed up a large group of patients initially diagnosed with major depressive disorder according to the Research Diagnostic Criteria (Spitzer et al., 1975) at 6-month intervals for 5 years and then annually for another 3 years. During this time, most patients had at least two recurrences of the disorder; some had three or four. The kappa statistic was used to quantify the likelihood that patients with psychotic, agitated-retarded, or endogenous subtypes of the disorder in agiven episode would manifest the same subtype in subsequent episodes.The psychotic subtype showed the greatest diagnostic stability across multiple subsequent episodes; for all three subtypes, diagnostic stability was greater for contiguous episodes than for noncontiguous episodes.

Nelson and Rice (1997) tested the 1-year stability of DSM-III lifetime diagnoses of obsessive-compulsive disorder (OCD) in data from the ECA study. The temporal stability of OCD diagnoses over the course of a year turned out to be surprisingly poor: Of subjects in the ECAsample who met criteria for OCD, only 19% reported symptoms a year later that met the OCD criteria. These findings seemed to reflect an excess of false positives for OCD on initial diagnostic examination, raising concerns about the validity of diagnoses of other conditions reported in the ECA study.

Mattanah and his colleagues (1995) investigated the stability of a range of DSM-III-R disorders in a group of adolescent inpatients 2 years after hospitalization. Predictably, diagnostic stability for these subjects was lower than that for the same diagnoses in adults. Internalizing disorders (e.g., the affective disorders) turned out to be more stable but of uncertain reliability because of more new cases at followup, whereas externalizing disorders (e.g., attention-deficit/ hyperactivity disorder; ADHD) were less stable but more reliable because of fewer new cases at follow-up. Surprisingly, personality disorder clusters and substance use disorders were both stable (53%) and reliable.

These studies of diagnostic stability emphasize the extent to which diagnostic reliability depends on both the clarity and the validity of diagnostic criteria, along with the inherent symptom variability of particular disorders over time, as influenced by alterations in environmental and individual circumstances. These findings also suggest that the stability component attributable to DSM-III and DSM-III-R diagnostic criteria caused problems for a number of diagnoses.

Utility and Validity

The developers of DSM-III addressed its predecessors’disappointing clinical validity and utility in several ways (Spitzer et al., 1980). To begin with, the DSM-III and DSM-III-R volumes are much larger than their predecessors—in part to accommodate inclusion of more than three times as many diagnoses and in part to provide detailed information on each syndrome along with its defining diagnostic criteria. (Criticism of this proliferation of diagnoses in DSM-III is reviewed later in this research paper.) The substantial increase in numbers of syndromes made it easier for clinicians to locate and name more precisely the syndromes they observed in their patients. The information about the syndrome included in the volume provided concise summaries of empirical data permitting enhanced understanding of the likely context of the syndrome in the patient’s milieu.

Another of DSM-III’s innovations was its introduction of the multiaxial diagnostic system, which provides for assessment of patients along five dimensions, or axes, rather than only one. The patient’s psychopathology was to be recorded on Axes I and II; medical conditions impacting on the mental disorders were to appear on Axis III; the severity of psychosocial stressors that might affect the patient’s behavior was to be located on Axis IV; and the patient’s highest level of adaptive functioning was to be indicated onAxis V. The range of information available from multiaxial diagnosis was presumed to be more useful for treatment planning and disposition than was the single diagnostic label available from DSM-I and DSM-II.

Despite these substantial changes, it has not proven to be easy to document the enhanced validity and utility of DSM-III. The absence of the kind of definitive, documented etiological mechanisms, with associated laboratory findings, by which the diagnoses of many physical disorders are confirmed—a gold standard for comparative purposes—has made establishing the validity of many DSM-III diagnoses a good deal more difficult (Faraone & Tsuang, 1994). In recognition of the absence of a gold standard for the validation of clinical diagnoses, Robins and Guze (1970) proposed validating diagnoses against “internal criteria, such as consistency of the psychopathological symptoms and signs, and external criteria, like laboratory tests, genetic and family studies, course of illness, and delimitation from other illness” (Skodal & Spitzer, 1987, p. 18).

An editorial in The American Journal of Psychiatry by Andreasen (1995) recalled Robins and Guze’s 1970 proposal for validation alluded to previously, a structure that would include existing validators like clinical description and family studies, but they add a new validator—neurophysiological and neurogenetic laboratory tests. Although she acknowledges that laboratory tests have not yet emerged as prime sources of validation information, Andreasen nonetheless believes that psychiatry’s neuroscience base is key to the continuing evolution of validation. Specifically, she proposed “an additional group of validators . . . to link symptoms and diagnoses to their neural substrates (which) include molecular genetics and molecular biology, neurochemistry, neuroanatomy, neurophysiology, and cognitive neuroscience . . . (linking) psychiatric diagnosis to its underlying abnormalities in DNA” (p. 162). As what follows indicates, the search for neuronal, neurobiological, and genetic-familial validators, along with more traditional clinical and epidemiological ones, characterizes contemporary validation attempts.

Schizophrenic Spectrum Disorders

Subsequent reports on efforts to validate the schizophrenic spectrum disorders have utilized Andreasen’s (1995) “additional group of validators.” Thus, Gur, Mozley, Shtasel, Cannon, and Gallacher (1994) sought to relate whole-brain volume to clinical subtypes of schizophrenia. Magnetic resonance imaging (MRI) measures of cranial, brain, and ventricular and sulcal cerebrospinal fluid volume were examined in schizophrenic men and women and healthy comparison subjects. The MRI measures differentiated males from females, including male patients from female patients, patients from comparison subjects, and subgroups of patients based on symptom profiles. The research revealed two patterns of neuroanatomic whole-brain abnormalities that differ in severity according to symptom differences and may reflect differential involvement of dysgenic and atrophic pathophysiological processes.

Two studies with related goals reported data from the epidemiologically based Roscommon Family Study. Kendler, Neale, and Walsh (1995) examined the familial aggregation and coaggregation of five hierarchically defined disorders— schizophrenia, schizoaffective disorder, schizotypal-paranoid personality disorder, other nonaffective psychoses, and psychotic affective illness—in siblings, parents, and relatives of index and comparison probands. The aim was to determine whether these patterns could be explained by a single underlying continuum of liability to the schizophrenic spectrum. Although schizophrenia and psychotic affective illness could be clearly assigned to the two extremes of the schizophrenia spectrum, the proper placement of schizoaffective disorder, schizotypal-paranoid personality disorder, and other nonaffective psychoses could not be clearly made. Nonetheless, Kendler and his colleagues considered these results to support the existence of a schizophrenic spectrum in which these five disorders manifest with varying severity an underlying vulnerability that is strongly transmitted within families. In a companion report, Kendler, McGuire, Gruenberg, and Walsh (1995) found that probands with schizoaffective disorder differed significantly from those with schizophrenia or affective illness in lifetime psychotic symptoms as well as in outcome and negative symptoms assessed at follow-up. Relatives of probands with schizoaffective disorder had significantly higher rates of schizophrenia than did relatives of probands with affective illness. These data are consistent with the hypotheses that schizoaffective disorder results from the co-occurrence of a high liability to both schizophrenia and affective illness and that DSM-III-R criteria for schizoaffective disorder define a syndrome that differs meaningfully from either schizophrenia or affective illness.

Strakowski (1994) examined antecedent, concurrent, and predictive validators of the DSM-III/DSM-III-R schizophreniform disorder diagnosis. Consistent data in support of the validity of the diagnosis as either a distinct diagnostic entity or a subtype of schizophrenia or affective illness could not be found. Instead, Strakowski concluded that these patients constitute a heterogeneous group with new-onset schizophrenia, schizoaffective disorder, and atypical affective disorder and a small subgroup with a remitting nonaffective psychosis.

In sum, this research on the schizophrenic spectrum disorders has succeeded in validating the disorders at the two ends of the continuum, but it has largely failed to identify distinct validators—either clinical or neurobiological—for those disorders between the extremes.

Depressive Disorders

Kendler and Roy (1995) explored links between two common diagnostic sources of lifetime major depression—family history and personal history—and three independent validators. Although data from personal interview and family history agreed diagnostically at only a modest level, controlling for presence or absence of a personal interview diagnosis of major depression permitted family history diagnosis of the same disorder to predict future episodes of major depression, neuroticism, and familial aggregation of major depression significantly. Kendler et al. (1996) applied latent class analysis to 14 disaggregated DSM-III-R symptoms for major depression reported over the course of a year by members of more than a thousand female-female twin pairs. Three of the seven identified classes represented clinically significant depressive syndromes: mild typical depression, atypical depression, and severe typical depression. Depression was not etiologically homogeneous in this sample of twins; instead it was composed of several syndromes at least partially distinct from clinical, longitudinal, and familial-genetic perspectives. Both studies by this group, then, showed a convergence of old and new validators of major depression. In contrast to Kendler’s research on the schizophrenic spectrum disorders, however, the validity of the major depression diagnostic syndrome was consistently supported.

Haslam and Beck (1994) tested the content and latent structure of five proposed subtypes of major depression. Analysis of self-reported symptom and personality profiles of more than 500 consecutively admitted outpatients with a primary major depressive diagnosis yielded clear evidence for discreteness only for the endogenous subtype; the other proposed forms lacked internal cohesion or were more consistent with a continuous or dimensional account of major depression.

Interface of Depression and Anxiety

Clark, Watson, and Reynolds (1995) have sought to validate co-occurring depression and anxiety by means of a tripartite model that groups symptoms of these conditions into three subtypes: (a) largely nonspecific symptoms of general distress; (b) the manifestations of somatic tension and arousal that are relatively unique to anxiety; and (c) the symptoms of anhedonia specific to depression. The validity of this model was tested in five samples—three student samples, one adult sample, and one patient sample—in two studies reported in 1995. Watson, Weber, et al. (1995) used the Mood and Anxiety Symptom Questionnaire (MASQ) along with other symptom and cognition measures to validate these hypothesized symptom groups. Consistent with the tripartite model, MASC Anxious Arousal and Anhedonic Depression scales differentiated anxiety and depression well and also showed excellent convergent validity. Watson, Clark, et al. (1995) conducted separate factor analyses of the 90 items of the MASQ. The same three factors (General Distress, Anhedonia vs. Positive Affect, Somatic Anxiety) emerged in each of the five data sets, suggesting that the symptom structure in this domain is highly convergent across diverse samples. Moreover, the factors broadly corresponded to the symptom groups proposed by the tripartite model.

Joiner and his colleagues provided additional support for the validity of the tripartite model’s portrayal of anxiety and depression. In three separate studies, they reported that the model (a) distinguished among pure forms of depression and anxiety, comorbid depression and anxiety, and mixed anxietydepression in a group of college students (Joiner & Blalock, 1995); (b) provided a good fit for data from self-report measures of depression, anxiety, self-esteem, and positive and negative affect completed by another group of undergraduates (Joiner, 1996); and (c) described validly the psychopathologic behavior of a group of child and adolescent psychiatric inpatients (Joiner, Catanzaro, & Laurent, 1996).

In an effort to validate Beck’s cognitive model of depression and anxiety (Beck, 1976, 1987), which shares important assumptions with the tripartite model, Clark, Steer, and Beck (1994) explored both common and specific symptom dimensions of anxiety and depression proposed by both models in groups of psychiatric outpatients and undergraduates. Principal-factor analyses with oblique rotations on the items of the Beck Depression Inventory and the Beck Anxiety Inventory revealed two correlated factors: depression and anxiety. Second-order factor analyses yielded a large general distress or negative affect factor underlying the relationship between the two first-order factors. These results are consistent with both the tripartite and cognitive models, with cognitive and motivational symptoms found to be specific to depression and physiological symptoms found to be unique to anxiety.

Using five standard measures of anxiety and depression, Clark, Beck, and Beck (1994) compared symptom features of four DSM-III subtypes of depressive and anxiety disorders in a group of outpatients. Depression was distinguished by anhedonia, cognitions of personal loss and failure, and dysphoric mood, whereas anxiety was characterized by specific autonomic arousal symptoms, threat-related cognitions, and subjective anxiety and tension. Major depression and panic disorder were better differentiated by specific symptom markers than were dysthymia and generalized anxiety disorder.

Zinbarg and Barlow (1996) relied on a semistructured clinical interview and a self-report battery of questionnaires to identify central features of the anxiety disorders in a large group of patients seeking outpatient treatment. Their results were consistent with both the DSM-III-R and DSM-IV hierarchical models of anxiety and the anxiety disorders, as well as with Beck’s and with Clark and Watson’s trait models.

Criticisms of DSM-III and DSM-III-R

DSM-III represented a major improvement in the diagnosis of psychopathology. Nevertheless, numerous criticisms have also been made. Discussed in the following sections are concerns regarding the propagation of diagnostic labels, definition of mental disorder, promotion of the medical model, and its atheoretical approach.

Propagation of Diagnostic Labels

The number of diagnostic labels in DSM-III totaled more than three times the number contained within DSM-I. For this proliferation of diagnostic labels, the drafters of the instrument were severely criticized. Child clinical psychologist Norman Garmezy (1978), for example, expressed great concern about the marked increase in diagnoses for childhood and adolescent conditions. He feared that this many new diagnoses would tempt clinicians to label unusual but normal behaviors of childhood as pathological, thereby stigmatizing children who were simply behaving like children. Related concerns were expressed shortly after DSM-IV appeared. Social workers Kirk and Kutchins (1994), for example, accused the developers of the instrument of “diagnostic imperialism” by inappropriately labeling “insomnia, worrying, restlessness, getting drunk, seeking approval, reacting to criticism, feeling sad, and bearing grudges . . . (as) possible signs of a psychiatric illness” (p. A12).

Definition of Mental Disorder

The definition of mental disorder developed for DSM-III and retained in DSM-III-R and DSM-IV has also been widely criticized. It has been viewed both as far too broad and encompassing of behaviors not necessarily pathological and criticized as insufficiently helpful to clinicians who must distinguish between uncommon or unusual behavior and psychopathological behavior. The definition states the following:

Each of the mental disorders is conceptualized as a clinically significant behavioral or psychological syndrome or pattern that occurs in an individual and that is typically associated with either a painful symptom (distress) or impairment in one or more important areas of functioning (disability). In addition, there is an inference that there is a behavioral, psychological, or biological dysfunction, and that the disturbance is not only in the relationship between the individual and society. (APA, 1980, p. 6)

Addressing these concerns, Spitzer and Williams (1982) defended the definition by noting their intention to construct a nomenclature that would cast as wide a clinical net as possible so that persons who were suffering from even moderately disabling or distressing conditions would receive the help they needed. The promise of alternative definitions of mental disorder proposed by Wakefield (1992, 1997) and Bergner (1997) is evaluated later in this research paper.

Promotion of the Medical Model

Schacht and Nathan (1977), Schacht (1985), and others questioned the frequent emphasis in DSM-III on disordered brain mechanisms in its discussions of etiology as well as its apparent endorsement of pharmacological treatments in preference to psychosocial treatments for many disorders. In response, Spitzer (1983) justified these endorsements by noting that the DSM-III text was intended simply to reflect the state of knowledge of etiology and treatment. Similar concerns have been voiced about DSM-IV (Nathan & Langenbucher, 1999).

Atheoretical Approach

DSM-III and its successors were also were criticized for their intentionally atheoretical, descriptive position on etiology. Critics (e.g., Klerman et al., 1984) charged that this stance ignored the contributions of psychodynamic theory toward fuller understanding of the pathogenesis of mental disorders and the relationships between emotional conflict and the ego’s mechanisms of defense. Responding to these concerns in 1984 (Klerman, Vaillant, Spitzer, & Michels, 1984), Spitzer questioned the empirical basis for the claim that psychodynamic theory had established the pathogenesis of many of the mental disorders. In the foreword to DSM-III-R (APA, 1987), Spitzer (the primary author of the foreword) noted that the descriptive, atheoretical approach of DSM-III and DSMIII-R reflects “our current ignorance of the etiology and pathogenesis of most of the mental disorders” but added that “it is not intended to inhibit or denigrate the role of etiologic theories in psychiatry” (APA, 1987, p. xxiv).

DSM-IV

The development of DSM-III and DSM-III-R was chaired by psychiatrist Robert Spitzer. The development of DSM-IV was given to another psychiatrist, Allen Frances, who was given the mandates to provide a better documentation of the empirical support for the decisions that would be made, to improve the utility of the manual for the practicing clinician, and to improve congruency with International Classification of Diseases (ICD) 10.

DSM-IV Process

The principal goal of the DSM-IV process (Frances,Widiger, & Pincus, 1989; Nathan, 1998; Nathan & Langenbucher, 1999; Widiger & Trull, 1993) was to create an empirically based nomenclature. To achieve this goal, a three-stage process was used. The process began with the appointment of 13 work groups, each consisting of four to six individuals. They covered the anxiety disorders; child and adolescent disorders; eating disorders; late luteal phase dysphoric disorder; mood disorders; the multiaxial system; delirium, dementia, amnesia, and other cognitive disorders; personality disorders; psychiatric system interface disorders; psychotic disorders; sexual disorders; sleep disorders; and substance-related disorders.

The work groups began their work by undertaking systematic literature reviews designed to address unresolved diagnostic questions. When the literature reviews failed to resolve outstanding questions, the work groups sought clinical data sets that might be capable of doing so; 36 reanalyses of existing patient data sets were ultimately completed. The work groups also designed and carried out 12 large-scale field trials involving more than 7,000 participants at more than 70 sites worldwide.

The DSM-IV process is thoroughly chronicled in four sourcebooks that archive literature reviews and summarize findings from data reanalyses and field trials. The first three (Widiger, Frances, et al., 1994; Widiger, Francis, et al., 1996; Widiger et al., 1997) include 155 detailed literature reviews commissioned by the DSM-IV work groups. Volume 4 (Widiger et al., 1998) includes summaries of the results of 36 data reanalyses completed by nine work groups, the key findings of 12 field trials, and each work group’s final report.

DSM-IV Field Trials

Although summaries of the field trials appear in Volume 4 of the sourcebooks, nine DSM-IV field trial reports have also been published in journals. Field trial reports on mixed anxietydepression (Zinbarg et al., 1994) and oppositional defiant disorder and conduct disorder in children and adolescents (Lahey, Applegate, Barkley, Garfinkel, & McBurnett, 1994) appeared in August 1994, followed later the same year by field trial reports on sleep disorders (Buysse et al., 1994) and autistic disorder (Volkmar et al., 1994). The following year, field trial reports for obsessive-compulsive disorder (Foa & Kozak, 1995), somatization disorder (Yutzy et al., 1995), the mood disorders (Keller et al., 1995), and the substance-related disorders (Cottler et al., 1995) appeared. The field trial report for antisocial personality disorder was published in 1996 (Widiger, Hare, Rutherford, Corbitt, & Hart, 1996).

Most of the field trials contrasted the diagnostic sensitivity and specificity of alternative sets of existing diagnostic criteria—including those of ICD-10, DSM-III-R, and DSM-III—with one or more sets of new criteria. Many explored the impact on diagnostic reliability of changes in the wording of criteria. Some field trials also considered the diagnostic consequences of differing criterion thresholds and assessed the need for additional diagnostic categories.

Symptom data from the field trial for schizophrenia and related psychotic disorders were recently factor analyzed (Ratakonda, Gorman, Yale, & Amador, 1998) to determine whether the three common schizophrenic symptom domains— labeled positive, negative, and disorganized—also encompass the symptoms of psychotic disorders other than schizophrenia. These domains are apparently not exclusive to schizophrenia; they also describe the behavior of patients with schizoaffective disorder and primary mood disorder.

Reliability and Validity

Most data reported to date on the reliability and validity of DSM-IV categories have come from the field trial reports. They suggest modest increments in the reliability of a few diagnostic categories (e.g., oppositional defiant disorder and conduct disorder in children and adolescents, substance abuse and dependence) and validity (e.g., autistic disorder, oppositional defiant disorder in childhood and adolescence). Unfortunately, they also report no real progress in addressing the substantial reliability problems of the personality disorders, the sleep disorders, the disorders of childhood and adolescence, and some of the disorders within the schizophrenic spectrum.

The Research Diagnostic Project (RDP) at Rutgers University has focused on diagnostic issues in substance abuse for several years. Initial RDP studies found the reliability of lifetime DSM-IV diagnoses of alcohol, cannabis, cocaine, and opiate abuse and dependence to be high (Langenbucher, Morgenstern, Labouvie, & Nathan, 1994b). The diagnostic concordance of DSM-III, DSM-IV, and ICD-10 for disorders involving alcohol, the amphetamines, cannabis, cocaine, the hallucinogens, the opiates, PCP, and the sedative-hypnotics was found to be high for severe disorders and less so for disorders barely reaching diagnostic threshold (Langenbucher, Morgenstern, Labouvie, & Nathan, 1994a). Langenbucher and his colleagues (1997) also compared the predictive validity of four sets of dependence criteria and found the DSM-IV criteria for tolerance and dependence to be less discriminating than alternative criteria were. Finally, Langenbucher and Chung (1995) traced the onset and staging of symptoms of alcohol abuse and dependence. They identified three discrete stages—alcoholabuse,alcoholdependence,andaccommodation to the illness—thereby supporting the construct validity of alcohol abuse as a discrete first illness phase and alcohol dependence as distinct from and succeeding abuse.

Three additional studies of DSM-IV reliability and validity have also appeared. The first (Eaton, Anthony, Gallo, Cai, & Tien, 1997) reported a 1-year incidence rate of 3.0 per 1,000 per year for major depression in Baltimore, diagnosed according to DSM-IV criteria; this rate was comparable to earlier estimates that used DSM-III criteria. Two other studies explored diagnostic issues raised by the personality disorders. Ball, Tennen, Poling, Kranzler, and Rounsaville (1997) examined relations between alcohol, cocaine, and opiate abusers’personality disorders and their responses to five- and seven-factor models of personality (respectively, the Neuroticism, Extraversion, Openness Personality Inventory, NEO-PI; Costa & McCrae, 1989 and the Temperament and Character Inventory, TCI; Cloninger, Svrakic, & Przybeck, 1993). NEO-PI scales were strongly linked with specific personality disorders and TCI scales somewhat less so, thereby adding to the growing literature attesting to the power of personality trait dimensions to portray personality disorder validly. On exploring differences in clinicians’uses ofAxes I and II, Westen (1997) appeared to have identified an additional source of the diagnostic unreliability of the personality disorders. Whereas clinicians tended to use questions taken directly from the operational criteria to diagnose Axis I disorders, they made Axis II diagnoses more often by “listening to patients describe interpersonal interactions and observing their behavior with the interviewer” (p. 895). The latter practice may contribute to the unreliability of personality disorder assessment.

Gender and Cultural Bias

Corbitt and Widiger (1995) lamented the paucity of empirical findings on gender prevalence rates in the personality disorders (PDs) that contributed to the controversy surrounding DSM-III-R’s estimates that more women than men merit the diagnoses of histrionic PD and dependent PD. The DSM-IV text now avoids specifying gender prevalence rates for these disorders. DSM-IV has also added three PDs (schizoid, schizotypal, and narcissistic) to the three (paranoid, antisocial, and obsessive-compulsive) disorders that DSM-III-R indicated were diagnosed more often in males than in females.

Corbitt and Widiger ask whether DSM-IV has unintentionally introduced diagnostic bias in the laudable effort to combat it, by going beyond the modest empirical data on gender prevalence rates for the histrionic and dependent PDs. Hartung and Widiger (1998) consider the question of gender prevalence rates in DSM-IV diagnoses more generally. Although conclusions about these rates were informed by data from several sources—the field trials, existing data sets, and systematic literature reviews—they could possibly have been compromised by sampling biases (e.g., disproportionate numbers of one or the other gender in sample populations) or biases within the diagnostic criteria themselves (e.g., lack of gender neutrality in criteria sets). Hartung and Widiger claim that as a result, DSM-IV may retain vestiges of the gender-based bias that characterized its predecessors.

Criticisms of DSM-IV

There is general agreement on the strong empirical base that underlies DSM-IV. Yet even persons most involved in the development of the instrument acknowledge limitations on full utilization of the extensive empirical database because of unavoidable, biased, or misleading interpretations of the data (e.g., Kendler, 1990; Widiger & Trull, 1993).

Responding to criticisms that professional issues overshadowed scientific ones in the creation of DSM-IV (e.g., Caplan, 1991; Carson, 1991), Widiger and Trull (1993) defended attention to issues of utility that sometimes preempted issues of validity, such as when a valid diagnosis is de-emphasized because so few patients meet its criteria. Nonetheless, even though these authors admitted that the DSM-IV task force had to be sensitive to a variety of forensic, social, international, and public health issues, they saw the result as largely an empirically driven instrument. They also expected many of the decisions made during development of the instrument to continue to be debated, in part because the database for these decisions was often ambiguous or inadequate.

The lead review of DSM-IV in the American Journal of Psychiatry was written by Samuel Guze (1995), a key figure in the development of DSM-III.Although it was largely positive, Guze’s review (1995) expressed concern that many DSM-IV diagnostic conditions failed to meet Robins and Guze’s (1970) criteria for diagnostic validity. He criticized the apparent proliferation of less than fully validated diagnostic entities, a theme others (e.g., Grumet, 1995; Kirk & Kutchins, 1992, 1994) have also sounded. To this concern, Pincus, First, Frances, and McQueen (1996) note that although DSM-IV contains 13 diagnoses not in DSM-III-R, it has eliminated eight DSM-III-R diagnoses, for a net gain of only five.

Continuing Diagnostic Controversies

DSM-IV is a notable improvement over the prior editions of the APA diagnostic manual. Nevertheless, many diagnostic controversies remain. Discussed in the following sections are issues concerning excessive comorbidity, biases in diagnosis, the debate over categorical versus dimensional models, and new definitions of mental disorder.

Comorbidity

In a conceptual consideration at diagnostic comorbidity, Klein and Riso (1993) revisited two fundamental issues: whether disorders are discrete and natural classes or artificial categories created by the establishment of arbitrary cutoffs on a continuum and whether categorical or dimensional models of psychopathology better capture the essence of psychopathology. (We review research on the first of these issues in this section and research on the second later.) A phenomenon that is pervasive in clinical research is that patients and community participants rarely meet diagnostic criteria for just one mental disorder, contrary to the intentions of the authors of the diagnostic manual to develop criteria sets that lead to the identification of one single pathology. Klein and Riso listed six conceptual and statistical methods that could be used to demonstrate the existence of discrete boundaries between disorders and 11 possible explanations for diagnostic co-occurrences; they ultimately concluded that even these sophisticated methods may not properly account for all instances of comorbidity.

Evaluating the impact of high rates of comorbidity on clinical practice and research design in a large sample of young adults, Newman, Moffit, Caspi, and Silva (1998) concluded that groups that underrepresent comorbidity (e.g., student samples) probably also underestimate effect sizes for relations between a disorder and its correlates (e.g., physical health problems, interference with daily living, use of treatments, etc.), whereas groups that overrepresent comorbidity (e.g., clinical samples) overestimate effect sizes.

Concerns about the nature and extent of comorbidity led to the development of the National Comorbidity Survey (NCS), a nationwide stratified multistage survey of the U.S. population from 15 through 54 years of age. In an initial NCS report, Blazer, Kessler, McGonagle, and Swartz (1994) reported higher 30-day and lifetime prevalence estimates of major depression than the estimates reported in the ECA study and confirmed the high rates of co-occurrence between major depression and a range of other mental disorders. Kessler et al. (1995) examined the prevalence and comorbidity of DSM-III-R posttraumatic stress disorder (PTSD) in a second NCS article. PTSD was strongly comorbid with other lifetime DSM-III-R disorders in both men and women—especially the affective disorders, the anxiety disorders, and the substance use disorders. In another NCS report, Magee, Eaton, Wittchen, McGonagle, and Kessler (1996) reported that lifetime phobias are highly comorbid with each other, with other anxiety disorders, and with affective disorders; they were more weakly comorbid with alcohol and drug dependence.As with major depression, comorbid phobias are generally more severe than pure phobias.

Four additional comorbidity studies all investigated the frequent co-occurrence of substance-related and other psychiatric disorders. Two (Hudziak et al., 1996; Morgenstern, Langenbucher, Labouvie, & Miller, 1997) explored links between PDs and substance abuse; a third (Brown et al., 1995) traced the clinical course of depression in alcoholics; the fourth (Fletcher, Page, Francis, Copeland, & Naus, 1996) investigated cognitive deficits associated with long-term cannabis use. All confirmed that substance abuse and the PDs—especially borderline and antisocial PD—co-occur at high frequency, as does substance abuse and mood disorder as well as long-term substance abuse and cognitive dysfunction.

Reflecting recent clinical interest in comorbid mental and physical disorders, Sherbourne, Wells, Meredith, Jackson, and Camp (1996) reported that patients with anxiety disorder in treatment for chronic medical conditions like hypertension, diabetes, or heart disease functioned at levels lower than those of medical patients without comorbid anxiety. These differences were most pronounced in mental-health-related quality-of-life measures and when anxiety rather than depression was comorbid with the chronic medical conditions. A study with related aims (Johnson, Spitzer, Williams, Kroenke, & Linzer, 1995) reported that many of the primary care medical patients they studied also suffered from alcohol abuse or dependence; nearly half also had other co-occurring mental disorders. Although the substance abusers reported poorer health and greater functional impairment than did primary care patients without any mental disorders, they were less impaired than were patients who were diagnosed with mood, anxiety, eating, or somatoform disorders.

O’Connor, McGuire, Reiss, Hethering, and Plomin (1998) attempted to fit adolescent and parent reports and observational measures of depressive symptoms and antisocial behavior from a national sample of 720 same-sex adolescents to behavioral genetic models to determine the respective genetic and environmental influences on individual differences in and co-occurrence of the two psychopathological behaviors. Approximately half the variability in the depressive symptoms and antisocial behaviors could be attributed to genetic factors, although shared and nonshared environmental influences were also significant.

Reflecting another major societal concern, a 1996 issue of Archives of General Psychiatry featured five reports on the co-occurrence of violence and mental illness (Eronen, Hakola, & Tiihonen, 1996; Hodgins, Mednick, Brennan, Schulsinger, & Engberg, 1996; Jordan, Schlenger, Fairbank, & Caddell, 1996; Teplin, Abram, & McClelland, 1996; Virkkunen, Eggert, Rawlings, & Linnoila, 1996). Summarizing the principal findings from these studies, Marzuk (1996) observed that this relationship “appears strongest for the severe mental illnesses, particularly those involving psychosis, and it is increased by the use of alcohol and other psychoactive substances” (pp. 484–485). Results from a 26-year prospective study of a 1966 Finnish birth cohort (Tiihonen, Isohanni, Rasanen, Koiranen, & Moring, 1997) supported the same conclusions: Risk for criminal behavior was significantly higher among persons with psychotic disorders, especially persons suffering from alcohol-induced psychoses or schizophrenia and coexisting substance abuse.

Overall, the extensive research on comorbidity to date has confirmed both the identity of the disorders most commonly comorbid (e.g., substance-related disorders, personality disorders, depression, anxiety) and comorbidity’s substantial adverse social, physical, psychological, and psychiatric consequences.

Diagnostic Bias

Diagnostic biases based on race and gender have recently been confirmed. Garb (1997) concluded thatAfricanAmerican and Hispanic patients are less likely than Caucasians are to be diagnosed with psychotic mood disorder and more likely to be diagnosed with schizophrenia despite comparable symptoms. Becker and Lamb (1994) reported that females are more likely to be diagnosed with histrionic personality disorder than are males, whereas males are more likely than females are to be diagnosed with antisocial personality disorder despite equivalent symptoms. Depression is also diagnosed more often in women than it is in men, even when symptoms of mood disorder are comparable across the genders (Potts, Burnam, & Wells, 1991). Gender also influences the differential diagnosis of major depression and organic mental disorder (Wrobel, 1993). Both male and female clinicians show these genderbased diagnostic biases (Adler, Drake, & Teague, 1990).

An important question for future research is the source and nature of these apparent biases. Biases can be inherent to a diagnostic nomenclature (e.g., the presence of a particular diagnosis could reflect cultural biases within the organization that constructed the nomenclature), or they could be confined to the diagnostic criteria (i.e., the disorder does in fact exist but the criteria set is biased against a particular ethnic or gender group), clinicians’ applications of the criteria set, or instruments of assessment.

The Categorical-Dimensional Debate

Categorical versus dimensional classification first became a matter of concern when DSM-III more than doubled the number of diagnoses included in its predecessors. As diagnoses proliferated with DSM-III and DSM-IV, the frequency of comorbidity substantially increased, causing clinicians to ask whether the comorbidity that resulted represented the cooccurrence of two or more separate mental disorders or a single disorder that had simply been labeled in different ways.As a consequence, the advantages and disadvantages of dimensional and categorical approaches to personality and diagnosis has begun to be debated and explored extensively (e.g., Clark, Livesley, & Morey, 1997; Clark, Watson, & Reynolds, 1995; Klein & Riso, 1993; Widiger, 1997b). The focus of these efforts has been primarily on the personality disorders, in which symptom overlap is greatest, but the issues and proposals apply to other sections of the manual as well. For example, Watson and Clark (1995), Watson, Clark, et al. (1995), and Watson, Weber, et al. (1995) have explored dimensions that underlie depression and anxiety. Some of that research has already been considered previously, in our review of efforts to validate DSM-III and DSM-III-R diagnoses. The research has also been reviewed extensively by Mineka, Watson, and Clark (1998), as well as by Clark (2000).

According to Clark (1999), dimensional approaches to personality disorder (a) are theoretically consistent with the complexity of symptom patterns that is observed clinically, (b) increase reliability, (c) are theoretically consistent with the observed lack of discrete boundaries between different types of psychopathology and between normality and psychopathology, and (d) provide a basis for understanding symptom heterogeneity within diagnoses by retaining information about component trait levels.

Clark (1999) distinguished between two different dimensional approaches to the personality disorders. The first, rooted in the traditional categorical system, conceptualizes each separate disorder as a continuum, so that in principle any patient could exhibit different levels of traits of several personality disorders. The alternative is the trait dimensional approach, in which assessment aims to create a profile of the personality traits that underlie the disorder. Although several instruments that reflect the higher order factors describing normal personality have proven useful for studying relations between personality and personality disorders (e.g., Widiger, 1993), only recently have instruments been developed specifically for the purpose of tapping into the lower order traits relevant to personality disorders. These include the 15-dimension Schedule for Nonadaptive and Adaptive Personality (SNAP; Clark, 1993) and the 18-dimension Dimensional Assessment of Personality Pathology–Basic Questionnaire (DAPP-BQ; Livesley, 1990).

In unpublished research by Clark and her colleagues relating diagnostic and trait dimensional approaches to personality disorder (Clark, 1999), two patient samples were interviewed with the Structured Interview for DSM-III-R Personality Disorders–Revised (SIDP-R; Pfohl, Blum, Zimmerman, & Stangl, 1989) and completed the SNAP. Multiple correlations between SNAP scales and diagnostic interview scores revealed a great deal of common variance: The information in a SNAP profile enabled prediction of between one quarter and three quarters of the variance in the interview-based diagnostic ratings, suggesting that the trait dimensions assessed by the SNAP underlie clinical ratings of personality pathology. These findings are especially impressive in view of data reviewed by Clark et al. (1997) to the effect that obtaining convergent validity for measures of PD assessment has been extremely difficult.

It is widely believed that categorical and dimensional models are inherently incompatible, and that one must choose between them. In actuality, however, it is more accurate to describe these models as existing in a hierarchical relation to one another, with dimensions being the blocks from which categories may be built. (Clark et al., 1997, p. 206)

O’Connor and Dyce (1998) recently reviewed the clinical data supporting the several models of PD configuration. They found moderate support for the DSM-IV dimensions and Cloninger’s (1987) tridimensional theory, and they found stronger support for the five-factor model (Widiger, Trull, Clarkin, Sanderson, & Costa, 1994) and Cloninger and Svrakic’s (1994) empirically derived seven-factor model. On balance, they concluded that four of the five factors within the five-factor model explain the bulk of the variance associated with PD. Unfortunately, these authors failed to include in their comparisons either the tripartite model or the trait dimensional approaches characterized by the SNAP and the DAPP-BQ. The integration of the SNAP and DAPP-BQ models with the five-factor model has been demonstrated in studies by Clark and Reynolds (2001) and Clark and Livesley (1994). However, although impressive progress has been made in recent years in amassing conceptual and historical support for dimensional versus categorical approaches to the personality disorders and other overlapping psychopathological entities, the ultimate utility of the trait dimensional approach will not be known until substantially more research data have been gathered that demonstrate empirically the advantages of this approach to these disorders.

New Definitions of Mental Disorder

As previously indicated, a continuing criticism of DSM-III and DSM-IV has been their definition of mental disorders, which critics have seen as so broad and all-encompassing as to include many nonpathological behaviors within its purview. As a result, alternative definitions have been proposed. Two of the most widely discussed of these are briefly reviewed here.

Wakefield’s Harmful Dysfunction

Wakefield first defined mental disorder as harmful dysfunction in 1992 and in subsequent publications (e.g., 1997, 1999a, 1999b) defended and clarified the definition.ToWakefield, whether a condition is harmful requires a value judgment as to its desirability or undesirability, and dysfunction refers to a system’s failure to function as shaped by processes of natural selection.

A condition is a mental disorder if and only if (a) the condition causes some harm or deprivation of benefit to the person as judged by the standards of the person’s culture (the value criterion), and (b) the condition results from the inability of some mental mechanism to perform its natural function, wherein a natural function is an effect that is part of the evolutionary explanation of the existence and structure of the mental mechanism (the explanatory criterion). (Wakefield, 1992, p. 385)

Bergner (1997), however, has argued that Wakefield’s harmful dysfunction conceptualization requires clinicians to make judgments about patients’ mental mechanisms and that many such judgments cannot reliably be made. Lilienfeld and Marino (1995, 1999) also take issue with Wakefield’s definition. They argue that many mental functions are not direct evolutionary adaptations but are instead neutral by-products of adaptations. They also note that the concept of the evolutionarily designed response neglects the fact that natural selection often produces extreme behavioral variability across individuals and that many disorders that have achieved consensus are best portrayed as evolutionarily adaptive responses to danger, threat, or loss.

Disagreeing, Spitzer (1997) calls Wakefield’s construct a “brilliant breakthrough” (p. 259) because it emphasizes that what is not working in the organism is the function that we expect to be present and in operation by virtue of evolution and selection. Richters and Hinshaw (1997) also laud Wakefield’s construct, even though they acknowledge that it requires a thorough knowledge of internal, neurobiological operations as well as value judgments about external, social data—both requirements that are difficult to satisfy.

Bergner and Ossorio’s Significant Restriction

Claiming that consensus on a definition of psychopathology has not been achieved despite years of trying, Bergner (1997) concluded that this situation has seriously affected efforts to study psychopathology, to treat it, and to deal with its social consequences. He endorsed a definition of psychopathology previously proposed by Ossorio (1985): Psychopathology is best defined as “significant restriction in the ability of an individual to engage in deliberate action and, equivalently, to participate in available social practices” (Bergner, 1997, p. 246). This definition “meets the intellectual criteria that an adequate definition represent a non-empirical articulation of the necessary and sufficient conditions for correct application of a concept, and that it successfully discriminate instances of a concept from non-instances.”

Comparing Bergner’s definition to his own (Wakefield, 1992), Wakefield (1997) concluded that it is neither necessary nor sufficient to define a disorder. Its most serious problem is its overinclusiveness: Many restrictions on deliberate action are imposed in normal mental functioning. By contrast, Wakefield understandably affirmed that his own harmful dysfunction analysis, criticized by Bergner (1997), adequately discriminates between disorder and nondisorder. Spitzer (1997), whose own attempt to define mental disorder is represented by DSM-III (APA, 1980) and its successors, admitted to fatigue at efforts to define psychopathology and expressed uncertainty over the value of a consensus definition. He wrote that the Bergner-Ossorio definition simply “muddles the issues,” whereas he lauded Wakefield’s harmful dysfunction conceptualization as a “brilliant breakthrough” because it clarifies important underlying issues (p. 259).Although Widiger (1997a) was pleased that Bergner addressed the fundamental issues and principal difficulties in defining mental illness, he agreed with others that the Bergner-Ossorio definition of mental disorder ultimately will not be more successful than earlier efforts were. A major reason is the absence of distinct boundaries between either physical disorders or normality for the construct proposed— an attraction for a scientist like Widiger who has espoused dimensional approaches to some forms of psychopathology. Finally, Nathan (1997) took issue with Bergner’s statement that a consensus on a definition of mental disorder does not exist, in view of the widespread acceptance of the value of DSM-IV and its predecessors by mental health professionals. Moreover, Nathan (1997) noted, however attractive Bergner’s construct may be, in the final analysis, data on utility—absent to this time—will be the ultimate arbiter of the construct’s worth.

In a subsequent expansive articulation of his position, Bergner (in press) restated and defended his conception of pathology as behavioral disability or functional impairment, concluded that it unifies theoretically divergent explanations of psychopathology, offered a consequent model of integrative psychotherapy, and weighed the considerable scientific and clinical implications of this integrative framework.

New Quantitative Methods for Diagnostic and Classification Research

The advanced quantitative techniques described in this section of the research paper increasingly have been brought to bear on problems of diagnosis and classification. The success to date of these efforts suggests even greater use of these approaches in the future.

Event-History Analysis

Event-history techniques such as survival-hazard analysis (Cox & Oakes, 1984; Singer & Willett, 1991) appear to hold great promise for all levels of nosologic analysis—subcriterion, criterion, and composite algorithm—for which longitudinal data are available. To date, survival-hazard analysis has been applied to both the onset of depression, alcoholism, marijuana use, and other mental disorders at the syndrome level (e.g., Burke, Burke, Rae, & Regier, 1991) and the individual symptom level (Langenbucher & Chung, 1995; Martin, Langenbucher, Kaczynski, & Chung, 1996; Rosenberg & Anthony, 2001).

Survival methods generate two kinds of functions. The survival function estimates the proportion of individuals (cases) at each point in time to have escaped (survived) the onset of an index event such as the emergence of a psychological symptom or syndrome. Survival curves with steep slopes represent events that tend to occur relatively early; gradual slopes indicate that the events have occurred later. The hazard function estimates the likelihood that a case that has not yet experienced an event will do so during that period. An advantage of survival-hazard analysis for nosologic research on illness onset and symptom staging is its capacity to accommodate cases that have not yet developed the problem (so-called right-censored data); other methods only use data from frankly ill or fully symptomatic subjects with (presumably) more severe illnesses. Another advantage is that survival-hazard analysis shows changes in risk and onset patterns across time rather than revealing only a cross-sectional view. The method produces a graphic plot of case survival and hazard, an intuitive and appealing mode of presenting data that often highlights relationships—such as symptom clusters or stages (e.g., Langenbucher & Chung, 1995)—that are otherwise obscure.

Item Response Theory

Item response theory (IRT) focuses on the distribution of individuals’response patterns for a given set of items and offers the researcher a choice of models for understanding item or criterion behavior (Hambleton, Swaminathan, & Rogers, 1991).

The two-parameter logistic IRT model appears most relevant to the situation in which diagnostic symptoms are assessed by structured interview. Two-parameter IRT obtains estimates of each symptom’s discrimination and threshold parameters. An item’s discrimination is its ability to distinguish subjects with levels of the latent trait (the underlying dimension of illness severity) above or below the item’s threshold. An item’s threshold is the point on the underlying dimension or trait at which 50% of respondents endorse the item (i.e., report that they have the symptom). Threshold is therefore closely related to the construct of item difficulty in classical psychometrics. Discrimination and threshold, called the a and b parameters, respectively (Hambleton et al., 1991), can be presented graphically as an item response function or IRF. Plotted for each item or symptom, the IRF is an S-shaped normal ogive function that shows the probability that the symptom will be endorsed at each level of the latent trait. IRFs express discrimination as slope (gradually sloping IRFs indicate items with low discrimination and steeply sloping IRFs indicate items with high discrimination) and express threshold as horizontal displacement on the latent trait axis (IRFs displaced far to the right indicate items with high threshold and IRFs displaced to the left indicate items with low threshold). IRFs can be used to construct a scale that provides a parsimonious assessment of individual respondents’ positions on the underlying or latent dimension. An optimally constructed scale consists of items with discrimination greater than 1.0 and thresholds relatively evenly dispersed from low to high, in order to cover the full range of the underlying dimension.

Two-parameter IRT appears especially useful for understanding patterns of responses to structured or semistructured diagnostic interviews. In this situation IRT assumes unidimensionality of the underlying dimension of psychopathology, requiring a preliminary stage of factor analysis to ensure that the diagnostic questions can be adequately modeled by a single underlying dimension—for example, depression, cognitive disorganization, and so forth. This is a limitation of the method because unidimensionality is a firm requirement for IRT. Although they are not yet widely used at this time, IRT-based analyses are powerfully heuristic and are beginning to surface in analyses of both discrete diagnostic instruments— for example, Cooke and Michie’s (1997) analysis of the Hare Psychopathy Checklist–Revised (Hare, 1991) and Kirisci, Moss, and Tarter’s (1996) analysis of the Situational Confidence Questionnaire (Annis & Graham, 1987)—and DSM-IV diagnostic criteria themselves (Langenbucher et al., 2000).

Latent Class Analysis

Latent class analysis (LCA; McCutcheon, 1987) has been used to explore naturally occurring subtypes within a previously homogeneous collection of cases. LCA has much in common with and can be viewed as a categorical form of conventional factor analysis.

Different symptom profiles exist even for members of the same diagnostic category—for example, some alcoholics complain principally of physiological symptoms, whereas others experience loss of control, just as some schizophrenics complain of dramatic positive symptoms, whereas others are withdrawn, with predominantly negative symptom profiles. LCA assumes that these different profiles are the result of the presence within the diagnostic group of a limited number of mutually exclusive subtypes or latent classes, each with its own characteristic symptom profile. LCA identifies the structure and number of these latent classes by maximizing goodness of fitacross models with different numbers and composition of latent classes. Results of LCA include membership probabilities—a statement of the expected prevalence of each latent class within the data set—and symptom endorsement probabilities (SEPs), which reflect the likelihood that a member of a particular class will endorse an item or symptom as present.

Although it is quite new, LCA has generated a great deal of excitement in subtyping research because it creates strong, falsifiable models. It has been applied successfully to studies of the latent classes of alcoholism (Bucholz et al., 1996), eating disorders (Bulik, Sullivan, & Kendler, 2000), depression (Chen, Eaton, Gallow, & Nestadt, 2000), and social phobia (Kessler, Stein, & Berglund, 1998), among other types of mental disorder. Bucholz et al.’s (1996) study underlined the centrality of substance-specific withdrawal in alcoholism and of nosologic instability—issues that have been a matter of debate for several decades. Kessler et al.’s (1998) study showed that social phobia is actually a conglomerate of two separate fear types—pure speaking fear versus more diffuse social fear—based on performance and interactional anxiety.

Taxometric Analyses

Meehl and his colleagues have developed a variety of statistical techniques, coined taxometrics, for exploring the latent structure of psychological constructs. These techniques are statistical procedures that analyze relationships within and between variables that would be uniquely indicative of latent classes, referred to as taxons (Meehl, 1995;Widiger, 2001).

One of the first taxometric techniques was maximum covariance analysis or MAXCOV (Waller & Meehl, 1998). MAXCOV permits investigation of relationships among several fallible but valid indicators of a disorder. Two indicators are correlated with one another across groups identified by the scores obtained on the indicators. For example, two of the diagnostic criteria for a major depressive disorder could be correlated with one another in persons with only one, two, three, four, or more of the remaining diagnostic criteria for depression. The analyses are repeated using two other diagnostic criteria and are repeated again and again until all possible pairs of indicators have been correlated with one another. The average distribution of correlations is then obtained. If the distribution of averaged correlations lies flat along the levels of the other indicators, then the distribution is said to be consistent with a continuous, dimensional variable; if there is a clear peak in the distribution of correlations, then the distribution is said to be consistent with a categorical variable.

Additional taxometric techniques include mean above minus mean below a cut (MAMBAC; Waller & Meehl, 1998), maximum eigenvalue (MAXEIG;Waller & Meehl, 1998), and latent mode factor analysis (L-MODE; Waller & Meehl, 1998). MAXEIG is a multivariate generalization of MAXCOV. Like MAXCOV, MAXEIG examines the degree of covariation between indicators along successive points along another indicator. However, whereas MAXCOV considers the covariance between a pair of indicators, MAXEIG considers the eigenvalue (the multivariate analogue of covariance) of the first principal factor of the matrix of all remaining indicators.

MAMBAC creates a series of cuts along one indicator and examines differences in scores on a second indicator for cases falling above and below each cut. If the latent structure is taxonic, a plot of these differences should be peaked (suggesting the presence of an optimal cutting score). If there are no underlying latent class taxons, then the plot should take on a dish-shaped curve that would be characteristic of a dimensional latent structure. L-MODE works by factor analyzing all available indicators and examining the distribution of scores on the first principal factor. If the construct is taxonic, factor scores should be bimodally distributed; if the construct is dimensional, factor scores should be unimodally distributed.

Most researchers use more than one taxometric analysis in any given study because the consistency of findings across different taxometric techniques is considered to be most informative. These techniques have been used in many diagnostic studies, including studies of the latent class structure of depression (Haslem & Beck, 1994; Ruscio & Ruscio, 2000), schizotypia (Lenzenweger & Korfine, 1992), and dissociation (Waller, Putnam, & Carlson, 1996; Waller & Ross, 1994).

Receiver-Operator Characteristic Analysis

Typically, DSM-III, DSM-III-R, or DSM-IV diagnoses are entered if a patient meets some minimum number of symptoms—three of a possible nine for a diagnosis of substance dependence, five of a possible nine for a major depressive episode. A shortcoming of the DSM tradition, however, has been the promulgation of these clinical thresholds or cut points in an essentially arbitrary way, as the result of expert consensus rather than empirical research. Receiver-operator characteristic (ROC) analysis represents an attractive alternative to this arbitrary approach because it is capable of suggesting symptom thresholds with optimum points of balance between diagnostic sensitivity and specificity (Hsiao, Bartko, & Potter, 1989)

At each possible cut point—one of seven possible symptoms, two of seven, three of seven, and so on—ROC plots diagnostic sensitivity (the proportion of ill cases diagnosed as ill) against specificity (the proportion of well cases diagnosed as well) in a simple bivariate space, so that the effect on diagnostic positivity and base rates of setting the cut point at different levels can be studied. A refinement, quality ROC (QROC; Clarke & McKenzie, 1994) compares different ROC curves for different symptom combinations, identifying threshold levels of particular symptom combinations that are more efficient than others.Although they are not yet widely used, ROC based methods have begun to show substantial promise for selecting symptom thresholds and profiles that are most sensitive to cases of true illness. For example, Cassidy, Chapman, Kwapil, Eckblad, and Zinser (2000) have validated a proposed algorithm for mixed bipolar episode that requires two of a possible six dysphoric symptoms and Mota and Schachar (2000) developed, with ROC-based methods, specific combinations of ADHD symptoms; these combinations were two to three times more efficient than were stock DSM-IV algorithms in discriminating hyperactive from normal children.

A Final Word

Although we have worked hard to ensure that this research paper accurately reflects the problems—substantial,controversial,and perplexing—that continue to burden syndromal diagnosis, we have worked just as hard to represent the extensive empirical support for the validity of DSM-IV (APA, 1994). Nevertheless, few would disagree that substantial improvement will have to be made to the processes as well as the conceptual underpinnings of syndromal diagnosis more generally. We have suggested both some of the problems that must be faced and some of the research methods that will be deployed to address these problems if such substantial improvement in the performance of our diagnostic systems is, in fact, to be gained.

We are confident that very significant improvements in the reliability, external validity, coverage, and cultural coherence of our diagnostic systems will be made in DSM-V and its successors because nosology is a maturing field widely and rightly viewed as crucial to the development of the mental health sciences generally. It is possible that DSM-IV’s successors will take a very different form and perhaps even be based on a completely different set of underlying assumptions, such as a dimensional rather than categorical view of mental illness. Nonetheless, it is also true that DSM-IV’s current categorical structure conveys in easily accessible form valuable information on etiology, epidemiology, psychopathology, associated features, and treatment of most forms of human psychological suffering.

Bibliography:

1. Adler, D. A., Drake, R. E., & Teague, G. B. (1990). Clinicians’practices in personality assessment: Does gender influence the use of DSM-III Axis II? Comprehensive Psychiatry, 31, 125–133.
2. Albee, G. W. (1970). Notes toward a position paper opposing psychodiagnosis. In A. R. Mahrer (Ed.), New approaches to personality classification (pp. 385–398). New York: Columbia University Press.
3. American Psychiatric Association. (1933). Notes and comment: Revised classified nomenclature of mental disorders. American Journal of Psychiatry, 90, 1369–1376.
4. American Psychiatric Association. (1952). Diagnostic and statistical manual of mental disorders. Washington, DC: Author.
5. (1968).Diagnosticandstatistical manual of mental disorders (2nd ed.). Washington, DC: Author.

6. (1980).Diagnosticandstatistical manual of mental disorders (3rd ed.). Washington, DC: Author.
7. American Psychiatric Association. (1987). Diagnostic and statistical manual of mental disorders (3rd ed., rev.). Washington, DC: Author.
8. (1994).Diagnosticandstatistical manual of mental disorders (4th ed.). Washington, DC: Author.
9. Andreasen, N. C. (1995). The validation of psychiatric diagnosis: New models and approaches. American Journal of Psychiatry, 152, 161–162.
10. Annis, H. M., & Graham, J. M. (1987). Situational confidence questionnaire. Toronto, Ontario, Canada: Addiction Research Foundation.
11. Ball, S. A., Tennen, H., Poling, J. C., Kranzler, H. R., & Rounsaville, B. J. (1997). Personality, temperament, and character dimensions and the DSM-IV personality disorders in substance abusers. Journal of Abnormal Psychology, 106, 545–553.
12. Beck, A. T. (1976). Cognitive therapy of the emotional disorders. New York: New American Library.
13. Beck, A. T. (1987). Cognitive models of depression. Journal of Cognitive Psychotherapy, 1, 2–27.
14. Beck, A. T., Ward, C. H., Mendelson, M., Mock, J. E., & Erbaugh, J. K. (1962). Reliability of psychiatric diagnoses: II. A study of consistency of clinical judgments and ratings. American Journal of Psychiatry, 119, 351–357.
15. Becker, D., & Lamb, S. (1994). Sex bias in the diagnosis of borderline personality disorder and post traumatic stress disorder. Professional Psychology: Research & Practice, 25, 55–61.
16. Bergner, R. M. (1997). What is psychopathology? And so what? Clinical Psychology: Science and Practice, 4, 235–248.
17. Bergner, R. M. (in press). Putting it all together: An integrative framework for psychopathology and psychotherapy. Journal of Abnormal Psychology.
18. Birley, J. L. T. (1975). The history of psychiatry as the history of an art. British Journal of Psychiatry, 127, 393–400.
19. Black, S. (1971). Labeling and psychiatry: A comment. Social Science and Medicine, 5, 391–392.
20. Blashfield, R. K. (1984). The classification of psychopathology. New York: Plenum.
21. Blashfield, R. K., & Draguns, J. G. (1976). Toward a taxonomy of psychopathology. British Journal of Psychiatry, 42, 574–583.
22. Blazer, D. G., Kessler, R. C., McGonagle, K. A., & Swartz, M. S. (1994). The prevalence and distribution of major depression in a national community sample: The National Comorbidity Survey. American Journal of Psychiatry, 151, 979–986.
23. Brown, S. A., Inaba, R. K., Gillin, M. D., Schuckit, M. A., Stewart, M. A., & Irwin, M. R. (1995). Alcoholism and affective disorder: Clinical course of depressive symptoms. American Journal of Psychiatry, 152, 435–452.
24. Bucholz, K. K., Heath, A. C., Reich, T., Hesselbrock, V. M., Kramer, R., Nurnberger, J. I., & Schuckit, M. A. (1996). Can we subtype alcoholism? A latent class analysis of data from relatives of alcoholics in a multicenter family study of alcoholism. Alcoholism: Clinical and Experimental Research, 20, 1462–1471.
25. Bulik, C. M., Sullivan, P. F., & Kendler, K. S. (2000). An empirical study of the classification of eating disorders. American Journal of Psychiatry, 157, 886–895.
26. Burke, K. C., Burke, J. D., Rae, D. S., & Regier, D. A. (1991). Comparing age of onset of major depression and other psychiatric disorders by birth cohorts in five US community populations. Archives of General Psychiatry, 48, 789–795.
27. Buysse, D. J., Reynolds, C. F., Hauri, P. J., Roth, T., Stepanski, E. J., Thorpy, M. J., Bixler, E. O., Kales,A., Manfredi, R. L.,Vgontzas, A. N., Mesiano, D., Houck, P. R., & Kupfer, D. J. (1994). Diagnostic concordance forDSM-IVsleep disorders:Areport from the APA/NIMH DSM-IV field trial. American Journal of Psychiatry, 151, 1351–1360.
28. Cantor, N., Smith, E. E., French, R. deS., & Mezzich, J. (1980). Psychiatric diagnosis as prototype categorization. Journal of Abnormal Psychology, 89, 181–193.
29. Cantwell, D. P., Russell, A. T., Mattison, R., & Will, L. (1979). A comparison of DSM-II and DSM-III in the diagnosis of childhood psychiatric disorders: I. Agreement with expected diagnosis. Archives of General Psychiatry, 36, 1208–1213.
30. Caplan, P. J. (1991). How do they decide who is normal? The bizarre, but true, tale of the DSM Canadian Psychology, 32, 162–170.
31. Carson, R. C. (1991). Dilemmas in the pathway of the DSM-IV. Journal of Abnormal Psychology, 100, 302–307.
32. Cassidy, F., Ahearn, E., Murry, E., Forest, K., & Carroll, B. J. (2000). Diagnostic depressive symptoms of the mixed bipolar episode. Psychological Medicine, 30, 403–411.
33. Chapman, L. J., Chapman, J. P., Kwapil, T. R., Eckblad, M., & Zinser, M. C. (1994). Putatively psychosis-prone subjects 10 years later. Journal of Abnormal Psychology, 103, 171–183.
34. Chen, L.-S., Eaton, W. W., Gallow, J. J., & Nestadt, G. (2000). Understanding the heterogeneity of depression through the triad of symptoms, course and risk factors: A longitudinal, populationbased study. Journal of Affective Disorders, 59, 1–11.
35. Chen,Y. R., Swann, A. C., & Burt, D. B. (1996). Stability of diagnosis in schizophrenia. American Journal of Psychiatry, 153, 682– 686.
36. Clark, L. A. (1993). Manual for the Schedule for Nonadaptive and Adaptive Personality. Minneapolis: University of Minnesota Press.
37. Clark, L. A. (1999). Dimensional approaches to personality disorder assessment and diagnosis. In C. R. Cloninger (Ed.), Personality and psychopathology. Washington, DC: American Psychiatric
38. Clark, L. A. (2000). Mood, personality, and personality disorder. In R. Davidson (Ed.), Anxiety, depression, and emotion (pp. 171– 200). NewYork: Oxford University Press.
39. Clark, L. A., Beck, A. T., & Beck, J. S. (1994). Symptom differences in major depression, dysthymia, panic disorder, and generalized anxiety disorder. American Journal of Psychiatry, 151, 205–209.
40. Clark, L. A., & Livesley, W. J. (1994). Two approaches to identifying the dimensions of personality disorder: Convergence on the fivefactor model. In P. T. Costa & T. A. Widiger (Eds.), Personality disorders and the five-factor model of personality (pp. 261– 278). Washington, DC: American Psychological Association.
41. Clark, L. A., Livesley, W. J., & Morey, L. (1997). Special feature: Personality disorder assessment: The challenge of construct validity. Journal of Personality Assessment, 11, 205–231.
42. Clark, L. A., & Reynolds, S. K. (2001). Predicting dimensions of personality disorder from domains and facets of the Five-Factor model. Journal of Personality, 69, 199–222.
43. Clark, L. A., Steer, R. A., & Beck, A. T. (1994). Common and specific dimensions of self-reported anxiety and depression: Implications for the cognitive and tripartite models. Journal of Abnormal Psychology, 103, 645–654.
44. Clark, L. A., Watson, D., & Reynolds, S. (1995). Diagnosis and classification of psychopathology: Challenges to the current system and future directions. In J. T. Spence, J. M. Darley, & D. J. Foss (Eds.), Annual review of psychology (pp. 121–153). Palo Alto, CA: Annual Reviews.
45. Clarke, D. M., & McKenzie, D. P. (1994). A caution on the use of cut-points applied to screening instruments or diagnostic criteria. Journal of Psychiatric Research, 28, 185–188.
46. Cloninger, C. R. (1987). A systematic method for clinical description and classification of personality variants. Archives of General Psychiatry, 44, 573–588.
47. Cloninger, C. R., & Svrakic, D. M. (1994). Differentiating normal and deviant personality by the seven-factor personality model. In S. Strack & M. Lorr (Eds.), Differentiating normal and abnormal personality (pp. 40–64). New York: Springer.
48. Cloninger, C. R., Svrakic, D. M., & Przybeck, T. R. (1993). A psychobiological model of temperament and character. Archives of General Psychiatry, 50, 975–990.
49. Cooke, D. J., & Michie, C. (1997). An item response theory analysis of the Hare Psychopathy Checklist–Revised. Psychological Assessment, 9, 3–14.
50. Corbitt, E. M., & Widiger, T. A. (1995). Sex differences among the personality disorders: An exploration of the data. Clinical Psychology: Science and Practice, 2, 225–238.
51. Coryell, W., Winokur, G., Shea, T., Maser, J. D., Endicott, J., & Akiskal, H. S. (1994). The long-term stability of depressive subtypes. American Journal of Psychiatry, 151, 199–204.
52. Costa, P. T., & McCrae, R. R. (1989). The NEO-PI/NEO-FFI manual supplement. Odessa, FL: Psychological Assessment Resources.
53. Cottler, L. B., Schuckit, M. A., Helzer, J. E., Crowley, T., Woody, G. E., Nathan, P. E., & Hughes, J. (1995). The DSM-IV field trial for substance use disorders: Major results. Drug and Alcohol Dependence, 38, 59–69.
54. Cox, D. R., & Oakes, D. (1984). Analysis of survival data. London: Chapman and Hall.
55. Eaton, W. W., Anthony, J. C., Gallo, J., Cai, G., & Tien, A. (1997). Natural history of Diagnostic Interview Schedule/DSM-IV major depression. Archives of General Psychiatry, 54, 993–999.
56. Eronen, M., Hakola, P., & Tiihonen, J. (1996). Mental disorders and homicidal behavior in Finland. Archives of General Psychiatry, 53, 497–501.
57. Faraone, S. V., & Tsuang, M. T. (1994). Measuring diagnostic accuracy in the absence of a “gold standard.” American Journal of Psychiatry, 151, 650–657.
58. Feighner, J. P., Robins, E., Guze, S. B., Woodruff, R. A., Winokur, G., & Munoz, R. (1972). Diagnostic criteria for use in psychiatric research. Archives of General Psychiatry, 26, 57–63.
59. Fennig, S., Kovasznay, B., Rich, C., Ram, R., Pato, C., Miller, A., Rubenstein, J., Carlson, G., Schwartz, J. E., Phelan, J., Lavelle, J., Craig, T., & Bromet, E. (1994). Six-month stability of psychiatric diagnoses in first-admission patients with psychosis. American Journal of Psychiatry, 151, 1200–1208.
60. Fletcher, J. M., Page, J. B., Francis, D. J., Copeland, K., & Naus, M. J. (1996). Cognitive correlates of long-term cannabis use in Costa Rican men. Archives of General Psychiatry, 53, 1051– 1057.
61. Foa, E. B., & Kozak, M. J. (1995). DSM-IV field trial: Obsessivecompulsive disorder. American Journal of Psychiatry, 152, 90–96.
62. Frances, A. J., Widiger, T. A., & Pincus, H. A. (1989). The development of DSM-IV. Archives of General Psychiatry, 46, 373–375.
63. Garmezy, N. (1978). Never mind the psychologists: Is it good for the children? The Clinical Psychologist, 31, 1–6.
64. Grumet, G. W. (1995). What hath APA wrought? American Journal of Psychiatry, 152, 651–652.
65. Gur, R. E., Mozley, P. D., Shtasel, D. L., Cannon, T. D., & Gallacher, F. (1994). Clinical subtypes of schizophrenia: Differences in brain and CSF volume. American Journal of Psychiatry, 151, 343–350.
66. Guze, S. B. (1995). Review of American Psychiatric Association Diagnostic and Statistical Manual of Mental Disorders–Fourth Edition. American Journal of Psychiatry, 152,
67. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of Item Response Theory. Newbury Park, CA: Sage.
68. Hanada,K.,&Takahashi,S.(1983).Multi-institutionalcollaborative studies of diagnostic reliability of DSM-III and ICD-9 in Japan. In R. L. Spitzer, J. B. W. Williams, & A. E. Skodal (Eds.), International perspectives on DSM-III (pp. 273–290). Washington, DC:American PsychiatricAssociation.
69. Hare, R. D. (1991). The Hare Psychopathy Checklist–Revised. Toronto, Ontario, Canada: Multi-Health Systems.
70. Hartung, C. M., & Widiger, T. A. (1998). Gender differences in the diagnosis of mental disorders: Conclusions and controversies of DSM-IV. Psychological Bulletin, 123, 260–278.
71. Haslam, N., & Beck, A. T. (1994). Subtyping major depression: A taxometric analysis. Journal of Abnormal Psychology, 103, 686–692.
72. Helzer, J. E., Clayton, P. J., Pambakian, R., Reich, T., Woodruff, R. A ., & Reveley, M. A. (1977). Reliability of psychiatric diagnosis: II. The test/retest reliability of diagnostic classification. Archives of General Psychiatry, 34, 136–141.
73. Helzer, J. E., Robins, L. N., Taibleson, M., Woodruff, R. A., Reich, T., & Wish, E. D. (1977). Reliability of psychiatric diagnosis: I. Methodological review. Archives of General Psychiatry, 34, 12(Suppl. 328), 18–21.
74. Hodgins, S., Mednick, S. A., Brennan, P. A., Schulsinger, F., & Engberg, M. (1996). Mental disorder and crime: Evidence from a Danish birth cohort. Archives of General Psychiatry, 53, 489– 496.
75. Hsiao, J. K., Bartko, J. J., & Potter, W. Z. (1989). Diagnosing diagnoses: Receiver operating characteristic methods and psychiatry. Archives of General Psychiatry, 46, 664–667.
76. Hudziak, J. J., Boffeli, T. J., Kriesman, J. J., Battaglia, M. M., Stanger, C., & Guze, S. B. (1996). Clinical study of the relation of borderline personality disorder to Briquet’s syndrome (hysteria), somatization disorder, antisocial personality disorder, and substance abuse disorders. American Journal of Psychiatry, 153, 1598–1606.
77. Johnson, J. G., Spitzer, R. L., Williams, J. B. W., Kroenke, K., & Linzer, M. (1995). Psychiatric comorbidity, health status, and functional impairment associated with alcohol abuse and dependence in primary care patients: Findings of the PRIME MD1000 study. Journal of Consulting & Clinical Psychology, 63, 133–140.
78. Joiner, T. E. (1996). A confirmatory factor-analytic investigation of the tripartite model of depression and anxiety in college students. Cognitive Therapy and Research, 20, 521–539.
79. Joiner, T. E., & Blalock, J. A. (1995). Gender differences in depression: The role of anxiety and generalized negative affect. Sex Roles, 33, 91–108.
80. Joiner, T. E., Catanzaro, S. J., & Laurent, J. (1996). Tripartite structure of positive and negative affect, depression, and anxiety in child and adolescent psychiatric inpatients. Journal of Abnormal Psychology, 105, 401–409.
81. Jordan, B. K., Schlenger, W. E., Fairbank, J. A., & Caddell, J. M. (1996). Prevalence of psychiatric disorders among incarcerated women: II. Convicted felons entering prison. Archives of General Psychiatry, 53, 513–519.
82. Kanfer, F. H., & Saslow, G. (1965). Behavioral analysis: An alternative to diagnostic classification. Archives of General Psychiatry, 12, 529–538.
83. Keller, M. B., Klein, D. N., Hirschfeld, R. M. A., Kocsis, J. H., McCullough, J. P., Miller, I., First, M. B., Holzer, C. P., Keitner, G. I., Marin, D. B., & Shea, M. T. (1995). Results of the DSM-IV mood disorders field trial. American Journal of Psychiatry, 152, 843–849.
84. Kendler, K. S. (1990). Toward a scientific psychiatric nosology: Strengths and limitations. Archives of General Psychiatry, 47, 969–973.
85. Kendler, K. S., Eaves, L. J., Walters, E. E., Neale, M. C., Heath, A. C., & Kessler, R. C. (1996). The identification and validation of distinct depressive syndromes in a population-based sample of female twins. Archives of General Psychiatry, 53, 391–399.
86. Kendler, K. S., McGuire, M., Gruenberg, A. M., & Walsh, D. (1995). Examining the validity of DSM-III-R schizoaffective disorder and its putative subtypes in the Roscommon Family Study. American Journal of Psychiatry, 152, 755–764.
87. Kendler, K. S., Neale, M. C., & Walsh, D. (1995). Evaluating the spectrum concept of schizophrenia in the Roscommon Family Study. American Journal of Psychiatry, 152, 749–754.
88. Kendler, K. S., & Roy, M.-A. (1995). Validity of a diagnosis of lifetime major depression obtained by personal interview versus family history. American Journal of Psychiatry, 152, 1608–1614.
89. Kessler,R. C., Sonnega,A.,Bromet, E., Hughes,M., & Nelson, C. B. (1995).PosttraumaticstressdisorderintheNationalComorbidity Survey. Archives of General Psychiatry, 52, 1048–1060.
90. Kessler, R. C., Stein, M. B., & Berglund, P. (1998). Social phobia subtypes in the National Comorbidity Survey. American Journal of Psychiatry, 155, 613–619.
91. Kirisci, L., Moss, H. B., & Tarter, R. E. (1996). Psychometric evaluation of the Situational Confidence Questionnaire in adolescents:Fitting a graded item response model.Addictive Behaviors, 21, 303–317.
92. Kirk, S. A., & Kutchins, H. (1992). The selling of DSM: The rhetoric of science in psychiatry. Hawthorne, NY: Walter de Gruyter.
93. Kirk, S. A., & Kutchins, H. (1994, May 28). Is bad writing a mental disorder? New York Times, A12.
94. Klein, D. N., Ouimette, P. C., Kelly, H. S., Ferro, T., & Riso, L. P. (1994). Test-retest reliability of team consensus best-estimate diagnoses of Axis I and Axis II disorders in a family study. American Journal of Psychiatry, 151, 1043–1047.
95. Klein, D. N., & Riso, L. P. (1993). Psychiatric disorders: Problems of boundaries and comorbidity. In C. G. Costello (Ed.), Basic issues in psychopathology (pp. 19–66). New York: Guilford Press.
96. Klerman, G. L., Vaillant, G. E., Spitzer, R. L., & Michels, R. (1984). A debate on DSM-III. American Journal of Psychiatry, 141, 539–553.
97. Kraepelin, E. (1907). Clinical psychiatry: A textbook for students and physicians (A. R. Diefendorf, Trans.). London: Macmillan.
98. Lahey, B. B., Applegate, B., Barkley, R. A., Garfinkel, B., & McBurnett, K. (1994). DSM-IV field trials for oppositional defiant disorder and conduct disorder in children and adolescents. American Journal of Psychiatry, 151, 1163–1171.
99. Langenbucher, J. W., & Chung, T. (1995). Onset and staging of DSM-IV alcohol dependence using mean age and survival/hazard methods. Journal of Abnormal Psychology, 104, 346–354.
100. Langenbucher, J. W., Chung, T., Morgenstern, J., Labouvie, E., Nathan, P. E., & Bavly, L. (1997). Physiological alcohol dependence as a “specifier” of risk for medical problems and relapse liability in DSM-IV. Journal of Studies on Alcohol, 58, 341–350.
101. Langenbucher, J. W., Labouvie, E., Sanjuan, P., Kirisci, L., Bavly, L., Martin, C., & Chung, T. (2000). Item Response Theory analysis of alcohol, cannabis and cocaine criteria in DSM-IV. Unpublished manuscript, Rutgers University.
102. Langenbucher, J. W., Morgenstern, J., Labouvie, E., & Nathan, P. E. (1994a). Diagnostic concordance of substance use disorders in DSM-III, DSM-IV and ICD-10. Drug and Alcohol Dependence, 36, 193–203.
103. Langenbucher, J. W., Morgenstern, J., Labouvie, E., & Nathan, P. E. (1994b). Lifetime DSM-IV diagnosis of alcohol, cannabis, cocaine and opiate dependence: Six-month reliability in a multisite clinical sample. Addiction, 89, 1115–1127.
104. Larsen, F., & Vaglum, S. (1986). Clinical experiences with the DSM-III system of classification: Interrater reliability of the DSM-III diagnoses in two Norwegian studies on psychiatric and super-obese patients. Acta Psychiatrica Scandinavia, 73, 18–21.
105. Lenzenweger, M. F., & Korfine, L. (1992). Confirming the latent structure and base rate of schizotypy: A taxometric analysis. Journal of Abnormal Psychology, 101, 567–571.
106. Lilienfeld,S.O., &Marino, L.(1995). Mental disorderas a Roschian concept: A critique of Wakefield’s “harmful dysfunction” analysis. Journal of Abnormal Psychology, 104, 411–420.
107. Lilienfeld, S. O., & Marino, L. (1999). Essentialism revisited: Evolutionary theory and the concept of mental disorder. Journal of Abnormal Psychology, 198, 400–411.
108. Livesley, W. J. (1990). Dimensional Assessment of Personality Pathology: Basic Questionnaire. Unpublished manuscript, University of British Columbia.
109. Magee, W. J., Eaton, W. W., Wittchen, H.-U., McGonagle, K. A., & Kessler, R. C. (1996). Agoraphobia, simply phobia, and social phobia in the National Comorbidity Survey. Archives of General Psychiatry, 53, 159–168.
110. Martin, C. S., Langenbucher, J. W., Kaczynski, N. A., & Chung, T. (1996). Staging in the onset of DSM-IV alcohol symptoms in adolescents: Survival/hazard analyses. Journal of Studies on Alcohol, 57, 549–558.
111. Marzuk, P. M. (1996). Violence, crime, and mental illness: How strong a link? Archives of General Psychiatry, 53, 481–486.
112. Mattanah, J. J. F., Becker, D. F., Levy, K. N., Edell, W. S., & McGlashan, T. H. (1995). Diagnostic stability in adolescents followed up 2 years after hospitalization. American Journal of Psychiatry, 152, 889–894.
113. Mattison, R., Cantwell, D. P., Russell, A. T., & Will, L. (1979). A comparison of DSM-II and DSM-III in the diagnosis of childhood psychiatric disorders: II. Interrater agreement. Archives of General Psychiatry, 36, 1217–1222.
114. McCutcheon, A. L. (1987). Latent class analysis. Newbury Park, CA: Sage.
115. Meehl, P. E. (1995). Bootstrap taxometrics: Solving the classification problem in psychopathology. American Psychologist, 50, 266–274.
116. Mineka, S., Watson, D., & Clark, L. A. (1998). Comorbidity of anxiety and unipolar mood disorders. In J. T. Spence, J. M. Darley, & D. J. Foss (Eds.), Annual review of psychology (pp. 377–412). Palo Alto, CA: Annual Reviews.
117. Morgenstern, J., Langenbucher, J., Labouvie, E., & Miller, K. J. (1997). The comorbidity of alcoholism and personality disorders in a clinical population: Prevalence rates and relation to alcohol typology variables. Journal ofAbnormal Psychology, 106, 74–84.
118. Mota, V. L., & Schachar, R. J. (2000). Reformulating attentiondeficit/hyperactivity disorder according to signal detection theory. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 1144–1151.
119. Nathan, P. E. (1967). Cues, decisions, and diagnoses. New York: Academic Press.
120. Nathan, P. E. (1997). In the final analysis, it’s the data that count. Clinical Psychology: Science and Practice, 4, 281–284.
121. Nathan, P. E. (1998). DSM-IV and its antecedents: Enhancing syndromal diagnosis. In J. W. Barron (Ed.), Making diagnosis meaningful: New psychological perspectives (pp. 3–27). Washington, DC: APA Books.
122. Nathan, P. E., Andberg, M. M., Behan, P. O., & Patch, V. D. (1969). Thirty-two observers and one patient: A study of diagnostic reliability. Journal of Clinical Psychology, 25, 9–15.
123. Nathan, P. E., & Harris, S. L. (1980). Psychopathology and society (2nd ed.). New York: McGraw-Hill.
124. Nathan, P. E., & Langenbucher, J. W. (1999). Psychopathology: Description and classification. In J. T. Spence (Ed.), Annual review of psychology (Vol. 50, pp. 79–107). Palo Alto, CA: Annual Reviews.
125. Nelson, E., & Rice, J. (1997). Stability of diagnosis of obsessivecompulsive disorder in the Epidemiologic Catchment Area study. American Journal of Psychiatry, 154, 826–831.
126. Newman, D. L., Moffit, T. E., Caspi, A., & Silva, P. A. (1998). Comorbid mental disorders: Implications for clinical practice and psychopathology research. Journal of Abnormal Psychology, 107, 305–311.
127. O’Connor, B. P., & Dyce, J. A. (1998). A test of models of personality disorder configuration. Journal of Abnormal Psychology, 107, 3–16.
128. O’Connor, T. G., McGuire, S., Reiss, D., Hethering, E. M., & Plomin, R. (1998). Co-occurrence of depressive symptoms and antisocial behavior in adolescence: A common genetic liability. Journal of Abnormal Psychology, 107, 27–37.
129. Ossorio, P. (1985). Pathology. In K. Davis & T. Mitchell (Eds.), Advances in descriptive psychology (pp. 151–202). Greenwich, CT: JAI Press.
130. Pfohl, B., Blum, N., Zimmerman, M., & Stangl, D. (1989). Structured Interview for DSM-III-R Personality. Iowa City: University of Iowa.
131. Pincus, H. A., First, M., Frances, A. J., & McQueen, L. (1996). Reviewing DSM-IV. American Journal of Psychiatry, 153,
132. Potts, M. K., Burnam, M. A., & Wells, K. B. (1991). Gender differences in depression detection: Acomparison of clinician diagnosis and standardized assessment. Psychological Assessment, 3, 609–615.
133. Ratakonda, S., Gorman, J. M., Yale, S. A., & Amador, X. F. (1998). Characterization of psychotic conditions. Archives of General Psychiatry, 55, 75–81.
134. Raven, P. H., Berlin, B., & Breedlove, D. E. (1971). The origins of taxonomy. Science, 174, 1210–1213.
135. Regier, D. A., Myers, J. K., Kramer, M., Robins, L. N., Blazer, D. G., Hough, R. L., Eaton, W. W., & Locke, B. Z. (1984). The NIMH Epidemiologic Catchment Area program: Historical context, major objectives and study population characteristics. Archives of General Psychiatry, 41, 934–941.
136. Richters, J. E., & Hinshaw, S. (1997). Psychiatry’s turbid solution. Clinical Psychology: Science and Practice, 4, 276–280.
137. Robins, E., & Guze, S. (1970). Establishment of diagnostic validity in psychiatric illnesses: Its application to schizophrenia. American Journal of Psychiatry, 126, 983–987.
138. Robins, L. N., Helzer, J. E., Croughan, J., & Ratcliff, K. S. (1981). National Institute of Mental Health Diagnostic Interview Schedule: Its history, characteristics, and validity. Archives of General Psychiatry, 38, 381–389.
139. Rosch, E. R. (1973). Natural categories. Cognitive Psychology, 4, 328–350.
140. Rosenberg, M. F., & Anthony, J. C. (2001). Early clinical manifestations of cannabis dependence in a community sample. Drug and Alcohol Dependence, 64, 123–131.
141. Rosenhan, D. L. (1973). On being sane in insane places. Science, 179, 250–258.
142. Ruscio, J., & Ruscio, A. M. (2000). Informing the continuity controversy: A taxometric analysis of depression. Journal of Abnormal Psychology, 109, 473–487.
143. Sandifer, M. G., Pettus, C., & Quade, D. (1964). A study of psychiatric diagnosis. Journal of Nervous and Mental Disease, 139, 350–356.
144. Schacht, T. E. (1985). DSM-III and the politics of truth. American Psychologist, 40, 513–526.
145. Schacht, T. E., & Nathan, P. E. (1977). But is it good for the psychologists? Appraisal and status of DSM-III. American Psychologist, 32, 1017–1025.
146. Sherbourne, C. D., Wells, K. B., Meredith, L. S., Jackson, C. A., & Camp, P. (1996). Comorbid anxiety disorder and the functioning and well-being of chronically ill patients of general medical providers. Archives of General Psychiatry, 53, 889–895.
147. Singer, J., & Willett, J. (1991). Modeling the days of our lives: Using survival analysis when designing and analyzing longitudinal studies of duration and timing of events. Psychological Bulletin, 110, 268–290.
148. Skodol, A. E., & Spitzer, R. L. (1987). An annotated bibliography of DSM-III. Washington, DC: American Psychiatric Press.
149. Spitzer, R. L. (1983). Psychiatric diagnosis: Are clinicians still necessary? Comprehensive Psychiatry, 24, 399–411.
150. Spitzer, R. L. (1997). Brief comments from a psychiatric nosologist weary from his own attempts to define mental disorder: Why Ossorio’s definition muddles and Wakefield’s “harmful dysfunction” illuminates the issues. Clinical Psychology: Science and Practice, 4, 259–266.
151. Spitzer, R. L., & Endicott, J. (1968). DIAGNO: A computer program for psychiatric diagnosis utilizing the differential diagnostic procedure. Archives of General Psychiatry, 18, 746–756.
152. Spitzer, R. L., & Endicott, J. (1969). DIAGNO II: Further developments in a computer program for psychiatric diagnosis. American Journal of Psychiatry, 125, 12–21.
153. Spitzer, R. L., Endicott, J., Fleiss, J. L., & Cohen, J. (1970). The Psychiatric Status Schedule: A technique for evaluating psychopathology and impairment of role functioning. Archives of General Psychiatry, 23, 41–55.
154. Spitzer, R. L., Endicott, J., & Robins, E. (1975). Research Diagnostic Criteria (RDC) for a selected group of functional disorders. New York: New York State Psychiatric Institute.
155. Spitzer, R. L., Fleiss, J. L., Endicott, J., & Cohen, J. (1967). Mental Status Schedule: Properties of a factor-analytically derived scale. Archives of General Psychiatry, 16, 479–493.
156. Spitzer, R. L., Forman, J. B., & Nee, J. (1979). DSM-III field trials: I. Initial interrater diagnostic reliability. American Journal of Psychiatry, 136, 815–817.
157. Spitzer, R. L., & Williams, J. B. W. (1982). The definition and diagnosis of mental disorders. Sage Annual Reviews of Studies in Deviance, 6, 15–31.
158. Spitzer, R. L., & Williams, J. B. W. (1986). Structured Clinical Interview for DSM-III. New York: New York State Psychiatric Institute, Biometrics Research Department.
159. Spitzer, R. L., Williams, J. B., & Skodol, A. E. (1980). DSM-III: The major achievements and an overview. American Journal of Psychiatry, 137, 151–164.
160. Strakowski, S. M. (1994). Diagnostic validity of schizophreniform disorder. American Journal of Psychiatry, 151, 815–824.
161. Szasz, T. S. (1960). The myth of mental illness. American Psychologist, 15, 113–118.
162. Teplin,L.A.,Abram,K.M.,&McClelland,G.M.(1996).Prevalence of psychiatric disorders among incarcerated women: I. Pretrial jail detainees. Archives of General Psychiatry, 53, 505–512.
163. Tiihonen, J., Isohanni, M., Rasanen, P., Koiranen, M., & Moring, J. (1997). Specific major mental disorders and criminality: A 26-year prospective study of the 1966 Northern Finland birth cohort. American Journal of Psychiatry, 154, 840–845.
164. Virkkunen, M., Eggert, M., Rawlings, R., & Linnoila, M. (1996). A prospective follow-up study of alcoholic violent offenders and fire setters. Archives of General Psychiatry, 53, 523–529.
165. Volkmar, F. R., Klin, A., Siegel, B., Szatmari, P., Lord, C., Campbell, M., Freeman, B. J., Cicchetti, D. V., Rutter, M., Kline, W., Buitelaar, J., Hattab, Y., Fombonne, E., Fuentes, J., Werry, J., Stone, W., Kerbeshian, J., Hoshino, Y., Bregman, J., Loveland, K., Szymanski, L., & Towbin, K. (1994). Field trial for autistic disorder in DSM-IV. American Journal of Psychiatry, 151, 1361–1367.
166. Wakefield, J. C. (1992). The concept of mental disorder: On the boundary between biological facts and social values. American Psychologist, 47, 373–388.
167. Wakefield, J. C. (1997). Normal inability versus pathological disability: Why Ossorio’s definition of mental disorder is not sufficient. Clinical Psychology: Science and Practice, 4, 249–258.
168. Wakefield, J. C. (1999a). Evolutionary versus prototype analysis of the concept of disorder. Journal of Abnormal Psychology, 108, 374–399.
169. Wakefield, J. C. (1999b). Mental disorder as a black box essentialist concept. Journal of Abnormal Psychology, 108, 465–472.
170. Waller, N. G., & Meehl, P. E. (1998). Multivariate taxonomic procedures: Distinguishing types from continua. Newbury Park, CA: Sage.
171. Waller, N. G., Putnam, F. W., & Carlson, E. B. (1996). Types of dissociation and dissociative types: A taxometric analysis of dissociative experiences. Psychological Methods, 1, 300–321.
172. Waller, N. G., & Ross, C. A. (1997). The prevalence and biometric structure of pathological dissociation in the general population: Taxometric and behavior genetic findings. Journal of Abnormal Psychology, 106, 499–510.
173. Watson, D., & Clark, L. A. (1995). Depression and the melancholic temperament. European Journal of Personality, 9, 351–366.
174. Watson, D., Clark, L. A., Weber, K., Assenheimer, J. S., Strauss, M. E., & McCormick R. A. (1995). Testing a tripartite model: II. Exploring the symptom structure of anxiety and depression in student, adult, and patient samples. Journal of Abnormal Psychology, 104, 15–25.
175. Watson, D., Weber, K., Assenheimer, J. S., Clark, L. A., Strauss, M. E., & McCormick, R. A. (1995). Testing a tripartite model: I. Evaluating the convergent and discriminant validity of anxiety and depression symptom scales. Journal of Abnormal Psychology, 104, 3–14.
176. Westen, D. (1997). Divergences between clinical and research methods for assessing personality disorders: Implications for research and the evolution of Axis II. American Journal of Psychiatry, 154, 895–903.
177. Widiger, T. A. (1993). The DSM-III-R categorical personality disorder diagnoses: A critique and an alternative. Psychological Inquiry, 4, 75–90.
178. Widiger, T. A. (1997a). The construct of mental disorder. Clinical Psychology: Science and Practice, 4, 262–266.
179. Widiger, T. A. (1997b). Mental disorders as discrete clinical conditions: Dimensional versus categorical classification. In S. M. Turner & M. Hersen (Eds.), Adult psychopathology and diagnosis (pp. 3–23). New York: Wiley.
180. Widiger, T. A. (2001). What can we learn from taxometric analyses? Clinical Psychology: Science and Practice.
181. Widiger, T. A., Frances, A. J., Pincus, H. A., First, M. B., Ross, R., & Davis,W.(Eds).(1994).DSM-IVsourcebook(Vol.1).Washington, DC:American PsychiatricAssociation.
182. Widiger, T. A., Frances, A. J., Pincus, H. A., Ross, R., First, M. B., & Davis, W. (Eds.). (1996). DSM-IV sourcebook (Vol. 2). Washington, DC: American Psychiatric Association.
183. Widiger, T. A., Frances, A. J., Pincus, H. A., Ross, R., First, M. B., & Davis, W. (Eds.). (1997). DSM-IV sourcebook (Vol. 3). Washington, DC: American Psychiatric Association.
184. Widiger, T. A., Frances, A. J., Pincus, H. A., Ross, R., First, M. B., Davis, W., & Kline, M. (Eds.). (1998). DSM-IV sourcebook (Vol. 4). Washington, DC:American Psychiatric Association.
185. Widiger, T. A., Hare, R., Rutherford, M., Corbitt, E., & Hart, S. (1996). DSM-IV antisocial personality disorder field trial. Journal of Abnormal Psychology, 105, 3–16.
186. Widiger, T. A., & Trull, T. J. (1993). The scholarly development of DSM-IV. In J. A. C. de Silva & C. C. Nadelson (Eds.), International review of psychiatry (pp. 59–78). Washington, DC: American Psychiatric Press.
187. Widiger, T. A., Trull, T. J., Clarkin, J. F., Sanderson, C., & Costa, P. T. (1994). A description of the DSM-III and DSM-IV personality disorders with the five-factor model of personality. In P. T. Costa & T. A. Widiger (Eds.), Personality disorders and the Five-Factor model of personality (pp. 41–57). Washington, DC: American Psychological Association.
188. Williams, J. B. W., Hyler, S. E., & Spitzer, R. L. (1982). Reliability in the DSM-III field trials: Interview versus case summary. Archives of General Psychiatry, 39, 1275–1278.
189. Wrobel, N. H. (1993). Effect of patient age and gender on clinical decisions. Professional Psychology: Research & Practice, 24, 206–212.
190. Yutzy, S. H., Cloninger, C. R., Guze, S. B., Pribor, E. F., Martin, R. L., Kathol, R. G., Smith, G. R., & Strain, J. J. (1995). DSM-IV field trial: Testing a new proposal for somatization disorder. American Journal of Psychiatry, 152, 97–101.
191. Zigler, E., & Phillips, L. (1961). Psychiatric diagnosis: A critique. Journal of Abnormal Psychology, 63, 607–618.
192. Zilboorg, G. (1941). A history of medical psychology. New York: W. W. Norton.
193. Zinbarg, R. E., & Barlow, D. H. (1996). Structure of anxiety and the anxiety disorders. Journal of Abnormal Psychology, 105, 181–193.
194. Zinbarg, R. E., Barlow, D. H., Liebowitz, M., Street, L., Broadhead, E., Katon, W., Roy-Byrne, P., Lepine, J.-P., Teherani, M., Richards, J., Brantley, P. J., & Kraemenr, H. (1994). The DSM-IV field trial for mixed anxiety-depression. American Journal of Psychiatry, 151, 1153–1162.
195. Zubin,J.(1967).Classificationofbehaviordisorders.AnnualReview of Psychology, 18, 373–406.