Methods Of Studying Change Research Paper

1. Concepts Of Change And A System Of Categories To Differentiate Them

1.1 Basic Concepts Of Change

Traditionally, there are at least two important basic concepts of stability and change: absolute and normative stability (Baltes et al. 1977). Whereas normative stability is only defined at the group level, absolute stability can be defined at the individual and the group level. A definition of absolute stability for an individual is that the value of a variable for this individual does not change between occasions of measurement. Absolute stability at the group level is defined as the mean level of the group for a variable does not change between observations. Figure 1(c) shows absolute stability at the individual level for individual 3 but not for individual 1. At the same time there is stability at the group level, because the mean of the whole sample does not change. Figure 1(a) describes change in absolute stability for each individual and at the group level.

Normative stability is given if the rank order with respect to a variable does not change between two occasions of measurement. Imagine, for instance, that the variable control belief is measured on two occasions. Each individual starts from a different level of control belief and then shows an increase by the same amount (cf., Fig. 1(a)). The rank correlation is 1 and, therefore, normative stability is given. Figure 1(c) illustrates change with respect to normative or rankorder stability: the rank order is reversed from time 1 to time 2. In this sample, the stability coefficient measured as rank correlation between occasions is r = –1.

Another concept of stability change is stability of variability. Stability of variability requires that the variance of a variable does not change between two occasions of measurement. Figure 1(b) shows an increase in the variability across situations. The example in Fig. 1(b) demonstrates that change in the variability across occasions can occur despite absolute and normative stability. To conclude, these examples demonstrate that there exist different concepts of stability change and that stability can be given for one concept, but not necessarily for the others.

1.2 A Classification System For Stability Change Concepts

To gain an even fuller picture of change concepts, a classification system of change concepts will be outlined, which takes into account the following:

 (a) individual change or change within groups;

(b) number of variables: univariate or multivariate change;

(c) number of and time distance between occasions: two or multiple measurements;

(d) scaling of the variables: categorial, ordinal, interval scale.

For reasons of economy, other important dimensions are not (explicitly) included within this classification:

(e) change can be located at the surface or the construct level;

(f) (measured variable or latent construct; phenotype or genotype);

(g) trend, growth, and rhythm;

(h) change of the (dimensional) structure;

(i) change in measurement error;

(j) synchronicity, asynchronicity;

(k) change and causality.

2. Examples Of Important Methods For Studying Change

When outlining a system of categories for change concepts, it becomes evident that by combining the different dimensions of the system a large number of change concepts could be derived. Within this short description, however, only a subset can be explained in more detail.

2.1 Analyzing Change By Using Differences Between Two Occasions

This way of studying change is frequently used in research. The way of measuring change shown in Fig. 1(a) is based on comparing measures of one variable for a sample of individuals for two occasions. The question is whether there is a change in the mean of the variable. To analyze these questions, one computes differences between time-2 and time-1 measures. Critiques of this approach have pointed to various problems:

(a) differences are dependent on the level of the first measurement;

(b) differences are not reliable (not as much as the original variable);

(c) regression to the mean leads to systematic biases.

(a) The law of initial values states that high initial values correlate with small differences between second and first measurement and vice versa, which means that there is a negative correlation between initial status and change.

(b) From the assumptions of classical test theory, e.g., Lord (1963) derived an equation for the reliability of differences. He argued that if there are two fallible measures, the resulting measure must be worse. This means that if there is a high correlation between the first and second measurements and the reliabilities of the single measurements are medium or high, then the reliability of the difference is low (numerical example: if r₁₁= r₂₂= 0.84 and r₁₂= 0.83, then it follows that r_dd = 0.06).

(c) Regression to the mean says that extreme values for the first occasion will probably be followed by more average values in the second measurement. The mathematical formulation for regression to the mean is s(y_pred) < s(x) (y_pred are the predicted values). It can be shown easily that regression to the mean is a simple mathematical tautology depending on the assumption of equal variances for preand post-test.

For each of these points of critique, Rogosa (1995) gives examples that they do not hold in general and that they can be subsumed under the myths of longitudinal research. Therefore, difference scores can be used if there are only two occasion measurements.

2.2 Cross-Lagged-Panel Analysis

After studying change using difference scores, we turn to the study of causal relations using change measurements. Consider, for instance, the theoretical question of whether there exists a causal relationship between control belief and academic performance and vice versa. The empirical basis for computing estimates of cross-lagged-panel relationships is that a sample of individuals is measured (at least) twice with respect to (at least) two variables.

The cross-lagged-panel model contains two important cross-lagged paths (a) between control belief (time 1) and performance (time 2) and (b) between performance (time 1) and control belief (time 2). These models can be estimated and tested for significance, yielding possible causal (in the sense of time-lagged relationships; see Schmitz 1990) relationships: no causality, unidirectional causality, or bidirectional causality (feedback).

2.3 Analysis Of Time Series

The data basis for analysis of time series is that at least one individual is studied for at least one variable, using many occasions of measurement. Whereas the methods in Sect. 2.1 require only a small database (two occasions measurements), for time series analysis one often needs more than 20 or even 50 data points. One might think that it would be difficult to collect such data, but in psychophysiology, behavioral observations, and diary approaches, it is feasible to collect this kind of data. One of the great advantages of the time-series approach is that one can test hypotheses also for a single individual. A typical research question may be to study the effect of an intervention (e.g., does a training have an effect on control belief?) and the identification of trends and rhythms. A variable shows a linear trend if the increase from one occasion to the next is constant. An example of a variable which shows a simple rhythm is workload measured daily: usually, the workload is high during workdays whereas at weekends people will not work. In the bior multivariate case one can test synchronous or asynchronous relationships (see Schmitz and Skinner 1993). Synchronous relationships occur if variables show similar patterns over time (e.g., days); they show asynchronous relationships, e.g., if one variable lags behind another variable (shows a similar pattern just one day later). One can analyze, for instance, intraindi idual causal relationships between variables. If there are more individuals, one can perform intraindividual time-series analyzes for each individual and then combine the individual parameters at the aggregate level, as is done in hierarchical linear modeling (HLM) (Bryk and Raudenbush 1987) or in the approach proposed by Schmitz et al. (1996). Both methods can be regarded as solutions to the idiographic–nomothetic controversy. (See Fig. 2.)

3. Other Methods And Outlook

A special case of the general time-series models are chaos models, which are nonlinear.

One characteristic of chaos is the severe dependence on the initial conditions, which is often referred to as the butterfly effect. There are simple chaotic systems which can be described by only one so-called order parameter. These systems converge to homeostasis if the order parameter stays within certain limits. However, if the parameter changes slightly the system shows chaotic behavior (Alligood et al. 1997).

Sometimes the assumption of interval data is not fulfilled, and in these cases synchronicity and asynchronicity can be studied using Markov models (see Gottman and Roy 1990). If information is given on how long it takes until an event occurs, methods of event-history analyzes can be applied (Willett and Singer 1995).

In sum, methods of studying change based on poor information—as in two-occasion measurements—can only lead to poor results (one cannot derive conclusions that are valid for individuals), whereas methods which use the information contained in rich (e.g., time-series) data can lead to rich conclusions (e.g., which are also valid for individuals). If someone is really interested in stability and change (although it requires a great deal of effort to conduct a longitudinal study), one should not always apply only simple methods (such as differences) and small data basis (such as two measurement points) for economical reasons as there are alternative methods which may provide more differentiated results. To conclude, it is time to avoid shortcomings of studying change or, as Rogosa (1995 p. 8) put it, ‘Myth 1: two observations a longitudinal study make.’

Bibliography:

1. Alligood K T, Sauer T D, Yorke J A 1997 Chaos. An Introduction to Dynamical Systems. Springer, New York
2. Baltes P B, Reese H W, Nesselroade J R 1977 Life-span Developmental Psychology: Introduction to Research Methods. Brooks Cole, Monterey, CA
3. Bryk A S, Raudenbush S W 1987 Application of hierarchical linear models to assessing change. Psychological Bulletin 101: 147–58
4. Gottman J M, Roy A K 1990 Sequential Analysis. A Guide for Behavioral Researchers. Cambridge University Press, Cambridge, UK
5. Lord F M 1963 Elementary models for measuring change. In: Harris C W (ed.) Problems in Measuring Change. University of Wisconsin Press, Madison, WI, pp. 21–38
6. Rogosa D R 1995 Myths and methods: ‘myths about longitudinal research’ plus supplemental questions. In: Gottman J M (ed.) The Analysis of Change. Erlbaum, Mahwah, NJ, pp. 3–66
7. Schmitz B 1990 Univariate and multivariate time-series models: the analysis of intraindividual variability and intraindividual relationships. In: von Eye A (ed.) Statistical Methods in Longitudinal Research, Volume II: Time Series and Categorical Longitudinal Data. Academic Press, Boston, pp. 351–86
8. Schmitz B, Skinner E 1993 Perceived control, effort, and academic performance: interindividual, intraindividual, and multivariate time-series analyses. Journal of Personality and Social Psychology 64: 1010–28
9. Schmitz B, Stanat P, Sang F, Tasche K G 1996 Reactive effects of a survey on the television viewing behavior of a telemetric television audience panel: a combined time-series and control group analysis. Evaluation Review 20: 204–29
10. Willett J B, Singer J D 1995 Investigating onset, cessation, relapse, and recovery: using discrete-time survival analysis to examine the occurrence and timing of critical events. In: Gottman J M (ed.) The Analysis of Change. Erlbaum, Mahwah, NJ, pp. 203–59