Longitudinal Data Research Paper

The purpose of this research paper is to describe some personcentered methods that respond to the challenge of characterizing life histories, ascertained with longitudinal data, placing emphasis on the effective utilization of both cross-time and multiple-life-domain information. We motivate the methods and illustrate their use in the context of explaining how a variety of health outcomes occur. However, the methodological issues are generic and transfer to a wide range of problems in the behavioral and social sciences.

1. Motivation

Extensive empirical evidence documents associations between early and mid-life risk factors and later-life chronic disease and disability (Barker and Osmond 1986, Wadsworth and Kuh 1997). These findings generate considerable interest in understanding the subtle nuances of life histories that might explain how such outcomes occur. Broad-gauged interest in successful aging (Rowe and Kahn 1998), with particular emphasis on how some individuals manage to adjudicate effectively multiple difficult challenges, also suggests in-depth comparative studies of life histories. Detailed accounts of whole lives have previously been studied primarily as individual cases, with evidence presented in forms ranging from rich ethnographies to focused interviews and, in a medical context, from open-ended responses to structured inquiries by clinicians.

Complementing such narrative accounts of life histories are structured longitudinal surveys that contain—for each individual—hundreds, or even thousands, of responses to questions about experiences over time and across multiple life domains (Hauser et al. 1993, Power et al. 1991, Wadsworth and Kuh 1997). In principle, those interested in the impact of multiple facets of life histories on a given outcome(s) should be able to construct representations of whole lives from the survey responses. Despite considerable interest in this task across multiple disciplines (Elder 1974, Vaillant 1983), there has been a dearth of analytic strategies for producing representations of whole lives and for aggregating them into meaningful taxonomies that facilitate understanding of how given outcomes occur.

With these observations at hand, our primary objective is to describe top-down and bottom-up methods for constructing taxonomies of life history representations and use them as the basis for explaining how given outcomes occur. We will also emphasize the construction of narratives from numerical survey responses and their utilization as part of an analytical strategy to understand ‘whole lives.’ We illustrate the ideas and methods in the context of health outcomes. However, the methodological issues are quite general and by no means restricted to this class of outcomes.

2. Classes of Methods

The techniques discussed in this research paper are called person-centered in the sense that the individual life history is the unit of analysis. Within this broad class of methods are two categories of techniques called ‘bottom-up’ and ‘top-down.’ The bottom-up strategies begin with highly idiosyncratic individual histories, and the analytical steps then identify important commonalities and differences across lives, leading to aggregation of histories into relatively homogeneous groups. Optimal matching (Abbott and Hrycak 1990), p-technique integrated with dynamic factor analysis (Shifren et al. 1997), and the integrated qualitative– quantitative aggregation strategy of Singer et al. (1998) are examples of bottom-up strategies.

Top-down techniques sequentially partition a heterogeneous population into progressively more homogeneous sub-groups. Recursive partitioning (Zhang and Singer 1999), Grade-of-Membership models (Berkman et al. 1989), and the clustering methods of Bergman (1998) are generic examples of these methods.

3. A Bottom-up Example: Understanding Psychological Resilience

3.1 The Problem

Among those who experience major episodes of depression in adult life, or earlier, some manage to subsequently regain high levels of well-being—that is, they demonstrate resilience. We ask whether those who show recovery have particular features that distinguish them from those who continue to suffer depression. An affirmative answer could suggest prevention strategies which, if applied early in life, might serve to bypass episodes of depression, or minimize their severity and facilitate recovery from them.

3.2 Longitudinal Data

The Wisconsin Longitudinal Study ( WLS) began with a random sample of 10,317 men and women who graduated from Wisconsin high schools in 1957. Survey data were collected from the original respondents in 1957, 1975, and 1992 93. Telephone interviews were conducted in 1992 93 with 8,020 of the 1975 respondents, 6,535 of whom also responded to a mail survey. Data have been collected on respondents’ family background, starting resources, academic abilities, youthful aspirations, social support, social comparisons, and the timing and sequencing of adult educational and occupational achievements, work events and conditions, family events, and physical and mental health. We focus on a subset of the 5,009 primary respondents who participated in all three waves of the WLS and who responded to questions assessing affective disorder and well-being in the most recent survey. The affective questions were administered to a randomly selected 80 percent of respondents. For each individual, there were responses on over one thousand variables covering all three waves of the survey and information across the multiple life domains indicated above.

The resilience criteria (see Singer et al. 1998) were satisfied by 168 women. There were 59 women who satisfied the depressed unwell criteria. The analytical task is the development of a taxonomy of life histories for each group such that the characterization of any history is sufficiently nuanced to capture the richness of individual experience, but not so detailed as to represent only very few women. For a more broadbased discussion of resilience, using both narrative and numerical data on individual histories to the study of racial and ethnic inequalities in health, see Singer and Ryff (1997).

3.3 A Person-centered Strategy

Construction of an empirical taxonomy of life histories requires a set of organizing principles—based on prior literature—that constrain the distillation of over one thousand responses for each individual into composite variables. Such composites are the basis for the ultimate representations of categories of histories. The following five principles were used to specify combinations of individual characteristics essential to explain how the outcomes come about.

(a) Adversity and its cumulation over time has negative mental health consequences.

(b) Advantage and its cumulation over time has positive mental health consequences.

(c) Reactions to adversity or advantage influence the impact of life experiences.

(d ) Position in social hierarchies has consequences for mental health.

(e) Social relationships influence the impact of life experiences.

These propositions are not new to the social sciences, mental health, or aging fields. What is new is the integration of them into an analytical strategy to guide life-history analysis. The methodology is usefully decomposed into five principle steps:

(1) guided by the organizing principles, life history narratives are constructed for a sub-sample of cases from each outcome group (i.e., resilient and depressed unwell);

(2) abstracted chronological charts, each one representing the synthesis of several narratives, were constructed;

(3) reduced response vectors, b, consisting of composite variables suggested by the chronological charts and individual variables whose rationale derives from the organizing principles are specified;

(4) aggregates (categories) of histories, in terms of response vectors judged to be close to one another, are specified; and

(5) tests of distinguishability are conducted.

Step (5) reduces to ascertaining whether or not the categories of histories characterizing the resilient women occur with low frequency in the depressed unwell group and, conversely, whether histories characterizing the depressed unwell women occur with low frequency among the resilient women. If the taxonomies of histories are to be a basis for explanation of the mental health outcomes, there should be distinguishability of categories of histories between the two groups.

3.4 Narrati es

A small random sample ( 10 cases) is selected from each outcome group, and a narrative biographical story is written about each individual. We adopt Stone’s (1979) conception of narrative, defined as ‘the organization of material in a chronological sequential order and the focusing of the content into a single coherent story, albeit with subplots.’ Our rationale for writing narratives based on survey data (with over one thousand responses per individual) is that the human mind has difficulty processing enormous amounts of information in parallel (i.e., long lists of responses representing a single life), although it is well suited to process a coherent story (i.e., a written narrative about a life) (Turner 1996). The use of narratives here is qualitatively different from their role when they are part of the raw data. Specifically, the construction of narratives—by the analyst and starting with numerical response data—is part of an analytical strategy. The narratives do not represent empirical data derived from focused interviews, recorded conversations, or text written by the respondents, but stories written by investigators to describe ‘whole lives through time.’ In generating the narratives, responses selected for inclusion were those expected to impact mental health, based on prior theoretical and empirical literature. The goal is to tell a life story in such a way that omitted information would make little or no difference for understanding the relevant life experiences preceding and predicting midlife mental health outcomes.

3.5 Abstract Chronological Charts

Two life histories can be aggregated and called similar, or even approximately indistinguishable, when the response vectors representing them are similar. At the level of individual cases, represented by raw response data on more than one thousand variables or by narratives, there is so much idiosyncratic detail that meaningful aggregation seems almost hopeless. However, via the organizing principles, it is possible to start with three or four narratives and begin specifying composite variables that ultimately lead to the specification of response vectors, some of which are sufficiently similar that they can be aggregated into a common category.

The composite variables are the initial basis for revised response vectors for all individuals that need to satisfy two criteria. First, the revised response vector for a given individual should provide for a nuanced summary of the person’s life history as it relates to mental health outcomes. Second, the response vector should not be too idiosyncratic. The latter condition is essential if we are to aggregate life histories— represented as reduced response vectors—so that individuals with ‘similar’ response vectors can be grouped into common categories of histories.

3.6 Reduced Response Vectors

Guided by the abstract chronological charts, we specify a provisional reduced vector of variables, whose values for a single individual define the person’s reduced response vector. The provisional reduced vector of variables will, clearly, represent all persons for whom we have constructed narratives. However, this is only a small sub-sample of a much larger population. We augment this group with 10–15 new cases for whom we do not have narratives. Their provisional reduced response vectors augmented by responses on additional variables that are clearly linked to the organizing principles are displayed. Visual examination of these augmented response vectors for the new cases AND those for which there are narratives serves to suggest new composite variables and revisions of those specified for the provisional reduced vector of variables. The result of this visual inspection and judgment is a new second-stage reduced vector of variables. We continue bringing in blocks of 10–15 new cases at a time, repeating the above step, and thereby iteratively refining the reduced vector of response variables until all cases for a given mental health outcome group have been examined. The final reduced vector of response variables, b, is the basis for initiating the formal specification of a taxonomy of histories.

3.7 Aggregation of Reduced Response Vectors: Pathway Specification

All variables are identified by binary codes in b. Next, we identify the longest AND statement common to at least 20 individuals. This identifies a set of response vectors—equivalently, a set of life histories—that exactly match one another on the components of the AND statement. If the components of the AND statement are distributed across the whole lives and they are interpretable as meaningful pathways to a given mental health outcome, then we have succeeded in aggregating a set of life histories into a well-defined group. For the resilient women in WLS, the binary coded version of b had 17 components, and the longest AND statement common to at least 20 individuals had seven components.

With an initial aggregate of histories at hand, we eliminate this group from further consideration and identify the longest possible AND statement characterizing at least 20—and hopefully more—of the remaining men. In the WLS resilient women, this did not lead to a long enough AND statement that was sufficiently nuanced to characterize possible routes into episodes of depression, as well as the cumulation of advantage that would promote recovery. This led to specification of more complex Boolean statements that were both consistent with our organizing principles and represented the life histories of a substantial number of women. At least two strategies can be used for this specification. First, visual inspection of the 17-component response vectors revealed a group of 48 women, all of whom grew up in a household with at least one alcoholic parent AND who had a diversity of different forms of compensating advantage in adulthood (represented by logical OR statements). For a narrative description of the resulting Boolean statements, see Singer et al. (1998).

A more formally mechanized approach to this step—using, for example, Ragin’s QCA software (1987)—would be to minimize the Boolean expression representing all women who are not in the first group identified via the ‘longest possible AND statement criterion.’ The resulting minimal Boolean expression would, hopefully, have clearly interpretable components that would represent distinct pathways—hence, additional members of the desired taxonomy of life histories—to given mental health outcomes. Although the analysis in Singer et al. (1998) proceeded beyond the first specification of an initial group by inspection and judgment, linked to organizing principals, we recommend application of both strategies en route to a final taxonomy. Indeed, as illustrated in Ragin (1987), minimal Boolean expressions that are not readily interpretable on a first pass through reduced response vectors often suggest recoding and construction of alternative composite variables that ultimately lead to a useful taxonomy.

3.8 Tests of Distinguishability

Having constructed a taxonomy of pathways for each outcome category—resilient and depressed unwell in the present example—it is necessary to demonstrate that the routes to any one outcome differ in important ways from the set of routes to a different outcome. To this end, we first ask whether the aggregate pathways for the resilient women differ from the pathways for the depressed unwell population. In particular, we tabulate the frequency with which the resilient pathways occur in the depressed unwell population. Conversely, we take the aggregate pathways representing the depressed unwell women and tabulate the frequency with which those pathways actually occur among the resilient women. We expect to find that the resilient pathways are rare in the depressed unwell population and also that the depressed unwell pathways are rare in the resilient population. For example, 16 percent of the resilient women are on the pathway indicated above by the seven-component logical AND statement. Only 8 percent of the depressed unwell have the same pathway ( p .10). Twenty percent of the resilient women are on the pathway where at least one parent is a chronic alcoholic AND there is a diversity of compensating positive experiences in adulthood. This pathway only occurs for 5 percent of the depressed unwell population ( p .05). There are persistently negative-experience pathways in the depressed unwell population that have no counterpart in the resilient group. Sharper discrimination occurs depending upon the particular application (see e.g., Zhao et al. 2000). The central point, however, is that assessment of distinguishability is important for making the case that there are, as hypothesized, qualitatively different routes to distinct mental health outcomes.

4. An Alternati e Bottom-up Strategy

An analytical strategy that might serve as an alternative to what is put forth in the above example is optimal matching (Abbott and Hrycak 1990, Abbott and Barman 1997). Optimal matching derives from the literature on comparison of DNA sequences. It takes univariate time series for individuals and groups them together into sets of what are regarded as equivalent histories. The grouping requires the availability of a metric on the space of possible sequences (histories); and two sequences are defined to be members of the same group if their distance apart—in terms of the metric—is sufficiently small. To apply the automated optimal matching technique requires that input data be in the form of what we have called ‘reduced response vectors.’ The analogue of steps leading from more complex longitudinal data to this stage, as we have described above, must still be carried out for univariate time series prior to carrying out the formal aggregation step (Abbott and Hrycak 1990). Although optimal matching is a natural way to proceed in the construction of a taxonomy of histories, the problem with its implementation on cross-time, multi-life-domain data is that there is no obvious metric yet developed that is suited to the complex multidimensional time series corresponding to individual histories, as discussed in the above example. Furthermore, a single measure of distance between two multidimensional histories seems too coarse for decisions about what constitute equivalent histories. The strategy for aggregation of reduced response vectors—in the above example—uses information about multiple features of life histories. This suggests that any automated aggregation technique, analogous to optimal matching, should be based on a multidimensional measure on the space of histories.

5. Top-down Methods

There is a considerable diversity of top-down, personcentered analytical strategies. A valuable review of both concepts and methods is contained in Cairns et al. (1998). One common strategy focuses on clustering techniques, where a critical input to the algorithms is a metric on the space of histories. Not surprisingly, applications of these techniques, which partition a heterogeneous population into relatively homogeneous clusters, have been restricted to lowdimensional data and rather simple summaries of histories. To reach the level of nuance and multidomain representation of the kind we have discussed above requires a quite different set of measures of similarity of histories to be constructed than heretofore. This is an important research topic for the future.

Recursive partitioning (RP) classification (Breiman et al. 1984, Zhang and Singer 1999) is an automated procedure that fits trees to multi-dimensional longitudinal data such that pathways down the tree to a given outcome (terminal node) identify homogeneous sub-populations with common histories. Trees are constructed with the objective of maximizing ability to predict membership in the outcome categories. Each pathway down the tree ending in a given terminal node (identified, for example, with a health outcome) corresponds to a logical AND statement incorporating information across multiple life domains. Two or more pathways into the same terminal node are identified with a logical OR statement describing how that particular outcome occurred. The judgment of the investigator is required to determine whether all pathways to the nodes identified with a given outcome should be summarized by a single complex Boolean expression, or partitioned into two or more sets of pathways defining qualitatively different kinds of histories. Recursive partitioning (RP) algorithms could accept either the original raw response vectors or reduced response vectors, as described in the above example.

From our perspective, a limitation of RP for generating a taxonomy of life histories is the fact that the performance criteria and the splitting rules for tree construction are not constrained by substantive knowledge from any field of scientific inquiry (e.g., a set of organizing principles). Thus, pathways down the best predictive tree are not always interpretable within the knowledge of a given field. This disjunction between statistical performance criteria, based solely on numerical goodness-of-fit, and constraints on tree structure and splitting rules derived from extant scientific knowledge have proven problematic in past applications of RP (Levy et al. 1981, 1985), for example, applied RP to neurological assessments of comatose patients to predict which patients were likely to recover with moderate disabilities as opposed to remaining vegetative or dying within one year of hospital admission. The numerically best predictive trees were virtually never neurologically interpretable. Rather, inspection of 20 nearly optimal trees led to selection, based on scientific judgment, of a prognostic tree that had interpretable pathways. For a start at more interactive—i.e., allowing the investigator to modify trees during the construction process on the basis of organizing principles—and flexible strategies based on RP technology, the reader should consult Zhang and Singer (1999).

6. Conclusions

We have described a variety of person-centered methods for the analysis of longitudinal data. Particular emphasis has been given to a relatively new strategy—the bottom-up example—that takes a full complex individual history as the unit of analysis. The principal and novel features of this strategy are:

(a) It includes the writing of narratives at the beginning and terminal stages of the analysis of survey data.

(b) It is personnot variable-centered.

(c) It produces pathways that represent aggregates of individual life histories.

(d ) It works simultaneously across multiple life domains.

(e) It is capable of identifying multiple pathways to a given outcome.

( f ) It systematizes the decision making en route to delineation of the pathways.

(g) It tests hypotheses about whole lives that are the integration of what is typically studied as separate pieces.

Point (a) underscores the importance of examining whole lives in story form as a means of comprehending the volume of detail that necessarily comprises human lives. Although point (b) is a clear objective in much of the life-history literature (e.g., Elder 1974, Magnusson and Bergman 1990), the lack of person-centered methodological tools has prevented investigators from proceeding to point (c). Attempts to get at aggregates have typically involved the use of variable-centered techniques, since these were available. However, the investigators mentioned above and many others were well aware that the lack of preservation of individuals as units of analysis severely limited the inferences they could draw about the dynamics of individual (or aggregates of individuals) change. Point (d) is conventionally dealt with, if at all, by variable-centered techniques. The bottom-up example strategy is, to the best of our knowledge, the first incorporation of multiple life domain information in a process of aggregation of whole life representations. Many investigators (e.g., Harris et al. 1990, Magnusson and Bergman 1990) have struggled with point (e); however, the absence of tools for aggregating whole life representations has prevented their delineating full pathways. Harris et al. (1990) set forth a template of several pathways to depression, and Magnusson and Bergman (1990) looked for ‘patterns’ connecting childhood problems with adult maladjustment. Nevertheless, their variable-centered empirical analyses (using logistic regression with interaction terms in the former, and cluster analysis in the latter) did not achieve the desired objective. What was needed was a personcentered methodology. Point (f ) emphasizes the critical role of scientific judgment embedded in the progression of methodological steps, something that is an essential part of any data analysis, but is infrequently reported. Finally, point (g) carries hypothesis testing into the uncharted territory of whole lives.

Recommended reading for more detail on the person-centered methods emphasized in this research paper and on the integration of numbers and narratives to study whole lives, including narratives as data, see Singer et al. (1998), Singer and Ryff, (2001), and Sampson and Laub (1993). An in-depth exposition of recursive partitioning, including online software, is contained in Zhang and Singer (1999). An instructive example of formal modeling of multiple-episode event history data is given in Heckman and Walker (1998) and a useful illustration of modeling of multivariate event history data, with both observed and unobserved covariates, is contained in Manton et al. (1987).

Bibliography:

1. Abbott A, Barman E 1997 Sequence comparison via alignment and Gibbs sampling: A formal analysis of the emergence of the modern sociological article. In: Raftery A (ed.) Sociological Methodology, 1997. American Sociological Association, Washington, DC, pp. 47–87
2. Abbott A, Hrycak A 1990 Measuring resemblance in sequence data: An optimal matching analysis of musician’s careers. American Journal of Sociology 96: 144–85
3. Barker D, Osmond C 1986 Infant mortality, childhood nutrition, and ischaemic heart disease in England and Wales. Lancet I: 1077–81
4. Bergman L 1998 A pattern-oriented approach to studying individual development: Snapshots and processes. In: Cairns R, Bergman L, Kagan J (eds.) Methods and Models for Studying the Indi idual. Sage Publications, Thousand Oaks, CA, pp. 83–122
5. Berkman L, Singer B, Manton K 1989 Black white differences in health status and mortality among the elderly. Demography 26(4): 661–78
6. Breiman L, Friedman J, Olshen R, Stone C 1984 Classification and Regression Trees. Wadsworth, Belmont, CA
7. Cairns R, Bergman L, Kagan J (eds.) 1998 Methods and Models for Studying the Indi idual. Sage Publications, Thousand Oaks, CA
8. Elder G 1974 Children of the Great Depression: Social Change in Life Experience. University of Chicago Press, Chicago
9. Harris T, Brown G, Bifulco A 1990 Loss of parent in childhood and adult psychiatric disorder: A tentative overall model. Development and Psychopathology 2: 311–28
10. Hauser R, Carr D, Hauser T, Hayes J, Krecker M, Kuo H, Magee W, Presti J, Shinberg, D, Sweeney M, ThompsonColon T, Uhrig S Warren J 1993 The Class of 1957 After 35 years: O er iew and Preliminary Findings. Working paper 93-17, Center for Demography and Ecology, University of Wisconsin-Madison, Madison, WI
11. Heckman J, Walker J 1998 Using goodness of fit and other criteria to choose among competing duration models: A case study of Hutterite data. In: Cloog C (ed.) Sociological Methodology, 1987. American Sociological Association, Washington, DC, pp. 247–307
12. Levy D, Bates D, Caronna J, Cartlidge N, Knill-Jones R, Lapinski R, Singer B, Shaw D, Plum F 1981 Prognosis in nontraumatic coma. Annals of Internal Medicine 94: 293–301
13. Levy D, Caronna J, Singer B, Lapinski R, Frydman H, Plum F 1985 Predicting outcome from hypoxic-ischemic coma. Journal of American Medical Association 253: 1420–26
14. Magnusson D, Bergman L 1990 A pattern approach to the study of pathways from childhood to adulthood. In: Robins L, Rutter M (eds.) Straight and De ious Pathways from Childhood to Adulthood. Cambridge University Press, Cambridge, UK, pp. 101–15
15. Manton K, Stallard E, Woodbury M, Tolley D, Yashin A 1987 Grade-of-membership techniques for studying complex event history processes with unobserved covariates. In: Clogg C (ed.) Sociological Methodology, 1987. American Sociological Association, Washington, DC, pp. 309–46
16. Power C, Manor O, Fox A 1991 Health and Class: The Early Years. Chapman and Hall, London
17. Ragin C 1987 The Comparati e Method: Mo ing Beyond Qualitati e and Quantitati e Strategies. University of California Press, Berkeley, CA
18. Rowe J, Kahn R 1998 Successful Aging. Pantheon Books, New York
19. Shifren K, Hooker K, Wood P, Nesselroade J 1997 Structure and variation of mood in individuals with Parkinson’s disease: A dynamic factor analysis. Psychology and Aging 12(2): 328–39
20. Singer B, Ryff C D 1997 Racial and ethnic inequalities in health: Environmental, psychosocial, and physiological pathways. In: Devlin B, Fienberg S, Resnick D, Roeder K (eds.) Intelligence, Genes, and Success: Scientists Respond to the Bell Cur e, Springer-Verlag (Copernicus), New York, pp. 89–122
21. Singer B, Ryff C D 2001 Person-centered methods for understanding aging: The integration of numbers and narratives. In: Binstock R, George L K (eds.) Handbook of Aging and the Social Sciences. Academic Press, San Diego, CA, pp. 44–65
22. Singer B, Ryff C, Carr D, Magee W 1998 Linking life histories and mental health: A person-centered strategy. In: Raftery A (ed.) Sociological Methodology, 1998. American Sociological Association, Washington, DC, pp. 1–51
23. Stone L 1979 The revival of narrative: Reflections on a new old story. Past and Present 85: 3–24
24. Turner M 1996 The Literary Mind. Oxford University Press, New York
25. Vaillant G 1983 The Natural History of Alcoholism. Harvard University Press, Cambridge, MA
26. Wadsworth M, Kuh D 1997 Childhood influences on adult health: A review of recent work from the British 1946 national birth cohort study, the MRC national survey of health and development. Pediatric Perinatal Epidemiology 11: 2–20
27. Zhang H, Singer B 1999 Recursi e Partitioning in the Health Sciences. Springer-Verlag, New York
28. Zhao H, McLanahan S, Brooks-Gunn J, Singer B 2000 Studying the real child rather than the ideal child: Bringing the person into developmental studies. In: Bergman L, Cairns R, Nilsson L-G, Nystedt L (eds.) Developmental Science and the Holistic Approach. Lawrence Erlbaum, Mahwah, NJ, pp. 393–419