Controlled Clinical Trials In Psychology Research Paper

1. Development Of Controlled Clinical Trials In Psychology

Parallel developments in two clinical disciplines, medicine and psychology, beginning in the 1930s, edged both fields toward the use of controlled trials. In an early study spanning both medicine and psychology (Chappell and Stephenson 1936), the authors demonstrated that an educational group program resulted in greater improvement and less relapse in a group of peptic ulcer patients than the control group, both groups having received the same medical care. Another study, the Cambridge-Somerville Youth Project, begun in the late 1930s and continuing with a long-term follow-up into the 1950s, was one of the earliest trials of a psychological treatment (Teuber and Powers 1953). This large-scale, well-designed study of 650 boys thought to be at high risk of developing delinquency was aimed at evaluating a prevention program. The outcome, as these individuals were followed over time, demonstrated that the treatment group fared less well than the control group in terms of a range of delinquent Behaviors. Such negative results, while disappointing, are important because they allow ineffective approaches to treatment to be abandoned. These early controlled trials were the exception rather than the rule. Most studies of the effectiveness of psychological therapies consisted of treating a series of patients without a control group. Hence, it was impossible to be sure whether the results were due to spontaneous recovery or to the treatment, let alone specific aspects of the treatment rather than a nonspecific (placebo) effect. Two developments catalyzed the use of the controlled trial. Hans Eysenck, Professor of Psychology at London University, reviewed the psychotherapy treatment literature in 1952 and again in 1966 (see Eysenck 1952, 1966). On both occasions Eysenck concluded that there was little evidence that psychotherapy was effective, that is, that the results did not exceed the expected rate of spontaneous recovery. While this opinion resulted in a vigorous debate and defense of psychotherapy, including the addition of a few treatment trials that demonstrated some success for various forms of psychotherapy that Eysenck had not considered, Eysenck’s conclusions were widely accepted, reaffirming the need for well-conducted controlled trials. Some, however, argued that the effects of psychological therapies, because they were aimed at ‘inner’ essentially unmeasurable changes, could never be adequately assessed in a controlled trial. This problem was largely overcome by the next development.

The second, and most important, catalyst for the use of controlled trials of psychological treatments, was the emergence of Behavior therapy (see Eysenck 1966, p. 39) had concluded that ‘With the single exception of the psychotherapeutic methods based on learning theory, results of published research … suggest that the therapeutic effects of psychotherapy are small or nonexistent … .’ Because Behavior therapy was based on learning theory, the focus of treatment was directly on the problem Behaviors comprising the clinical syndrome. This allowed for the development of objective measures of treatment outcome. In addition, because treatment was scientifically based, its practitioners were more likely to use scientific methods, including the randomized controlled trial, to assess the effectiveness of these new treatments.

2. The Flow Of Research

Although the controlled trial is the definitive method of ascertaining the effectiveness of a treatment, it is not the only method used in the research process aimed at the development of a new therapy. The first step in the development of a new therapy, the initial formulation of its conceptual basis and procedures, may stem from basic psychological research or theory, and/or from clinical observation. The first step in the development of a new therapy is usually an uncontrolled observational study, namely treatment of a series of patients. If a sufficient number of patients respond favorably to the new treatment, then the next step is to document the treatment procedures in a manual so that it can be replicated by others. The first controlled study may compare treatment to no treatment, a step that is relatively easy to carry out with a small number of patients, providing information as to whether the new treatment is better than doing nothing, and allowing the results to be roughly compared with those of standard therapy for the same condition, if such a therapy exists. One objective method of comparing the effects of different therapies is to calculate the effect size (the change from pretreatment to post-treatment in a group of patients divided by the standard deviation of the change). An effect size of 0.3 would be regarded as small, one of 0.5 as moderate, and 1.0 as large.

The next step is to ascertain whether the promising new treatment has specific effects over and above what is usually termed the placebo or nonspecific effect (see Sect. 3). Hence, it is important to compare the new treatment to a second treatment containing nonspecific features common to all therapies, but not containing the specific procedures aimed at ameliorating the Behaviors being studied. Single case controlled studies may also be useful at this stage of development to more precisely delineate the active components of treatment.

Once the new treatment has been demonstrated to have specific effects on a particular disorder, the next step will probably consist of a comparison with the standard therapy for the condition. At this stage related studies are likely to be carried out at different centers. The results of such studies can be compared using a procedure known as meta-analysis in which effect sizes are compared across studies. It may be clear from the results of such an analysis that the new treatment is better or worse than standard treatment, or that it is useful for a particular subgroup of individuals, or that it has advantages such as being less costly but similarly effective. On the other hand the results of such comparisons may be unclear. Some studies may demonstrate positive effects while others may show no effect. Such conflicting results are often due to the initial studies having too few patients enrolled to determine accurately the effectiveness of the new treatment as compared with other treatments. In this case a study involving several clinical sites may be needed to recruit a sufficient number of patients in a short enough time, to obtain more definitive results.

2.1 Multisite Studies

Multisite studies are necessarily more complex to carry out than a study at a single site. They are also more expensive than single-site studies and for that reason should only be entered upon when the preliminary work described above has been completed, and the need for such a study has been precisely formulated. The investigators involved must agree to a common experimental protocol. The treatment(s) being studied must be well documented in a treatment manual and therapists at each of the sites trained in the delivery of the treatments. Moreover, the therapists should be carefully supervised throughout the study to ensure that they are delivering the treatment(s) in accord with the protocol. Tape recordings of treatment sessions may also be audited to ensure fidelity to the treatment. In addition, the assessments used must be carried out in a standard manner and done comparably between sites. This may mean that assessors will also be trained and supervised across sites in the particular assessment methods. One advantage of a multisite over a single-site study is that the effects of treatment are examined in different sites that may have unique characteristics in terms of patient population, therapist characteristics, and so on. Hence, generalizability of the results of treatment to the clinical world is greater than in the single-site study.

Another reason to conduct a multisite study is when treatment for a relatively rare disorder is being studied. If the study were to be done at only one site it might take many years before sufficient participants entered the study. A multisite study would provide more rapid results. Another reason for considering a multisite study is when the question of interest shifts to what treatment or treatments might benefit those who do not respond to an initial effective treatment. Because only treatment failures will be entered into such a study, a large number of individuals must be treated in order to generate a sufficiently large sample for the study.

2.2 Effectiveness Research

As noted above, a distinction has arisen between efficacy and effectiveness controlled trials. Trials to establish efficacy are considered to be carried out in refined settings with carefully chosen patients and expert therapists. Effectiveness studies on the other hand are considered to be carried out in clinical practice settings with ‘real life’ patients and possibly less expert therapists. Some would consider efficacy studies to be of limited value in assessing the outcome of psychological therapies in contrast to effectiveness studies. However, if the flow of research described above is taken into consideration it is evident that efficacy and effectiveness studies lie on a continuum rather than being different types of studies, and that research along the entire continuum is needed. Moreover, the scientific questions being studied differ along the continuum that constitutes the flow of research.

Once a psychological treatment has been established as useful for a particular disorder in a number of single-site studies (efficacy studies) and, if needed, a multisite study, research as to how to transfer the treatment to other settings may be needed (effectiveness studies). This step in the development of a treatment may involve controlled studies of modifications of the treatment. Such modifications may include simplification of the treatment for wider dissemination to less skilled therapists and investigation of the necessary clinic environment to support the therapy. For example, in most controlled trials therapists are carefully supervised, and the question arises whether such supervision is essential to the effectiveness of the therapy. Additionally, the type of patient encountered in different clinical settings may differ, again requiring modification of the therapeutic procedures. Once these questions have been resolved, often using controlled trials, an effectiveness study can be designed in one or more clinical settings using the staff of such clinics as the therapists, and enrolling all the patients with the particular disorder into the study as they appear in the clinic. A control condition in such cases might be the usual care that participants would receive in such a clinic.

3. Essential Elements Of A Controlled Trial

While the basic design of two groups differing only in A (the components of a new treatment) suffices for early stages of research, other questions will need to be addressed. Among the most problematic is how to control for nonspecific elements of psychological treatments. Treatments can be regarded as a package containing both specific procedures aimed at the particular problem, together with elements common to all therapies, such as therapist attention, and therapeutic instructions that include the rationale for therapy together with engendering an expectancy that treatment is likely to be successful.

Research has demonstrated that nonspecific elements lead to significant Behavior change. Hence, it is important to know to what extent procedures specific to the therapy contribute to a successful outcome over and above nonspecific elements. This raises problems for psychological therapies because there is no placebo pill. This problem has been solved in several ways. The first solution is to use a nonspecific therapy, that is, one without the specific therapeutic procedures being examined but which also has face validity as a therapy with which to compare the active treatment. Some controlled trials may compare a psychotherapeutic procedure with medication and with a pill placebo as a control condition, thus solving the nonspecific effect question in a different manner. Finally, it is possible to compare a new therapy with a standard therapy for the condition which has already been demonstrated more successful than a nonspecific therapy.

3.1 The Use Of Treatment Manuals

The reports of early studies of psychological therapies usually described the treatment in a paragraph or two. Such descriptions are obviously insufficient to allow other investigators or therapists to provide the same treatment to their patients. The treatment manual was developed to solve this problem. Such manuals, now regarded as an essential element of a controlled trial of psychological therapies, describe the treatment procedures in sufficient detail for others to replicate them. Therapists are carefully trained in the procedures, which the manual facilitates.

There is some controversy regarding the clinical use of treatment manuals, which have now become a common method of disseminating new therapies to clinicians. Some argue that manuals lock the therapist into a rigid method of providing treatment, thus eliminating the opportunity to individualize the treatment. Others suggest that manuals provide essential guidelines for therapists and that the experienced therapist will be able to individualize treatment to the extent necessary. There is relatively little experimental data on the effectiveness of the use of manuals in the clinic; however existing data suggests that manuals are likely to enhance therapy probably by ensuring that the therapist stays on track during the treatment process.

3.2 Controlling The Fidelity Of Treatment

A further essential element of a clinical trial is a method to ensure that the therapy provided is faithful to the treatment manual. This is usually ensured in two ways. First, therapists are trained in the use of the manual and practice the treatment on several patients before the controlled trial begins. During this period the therapists are supervised by an expert. Second, once the trial begins, therapy sessions are audiotaped, and a random sample of such tapes is audited. The auditor will use a checklist to ensure that all the elements of treatment are being provided, and that no foreign elements have crept in. Regular feedback is provided to the therapists, and the degree of therapist compliance with the treatment protocol should be reported in the paper describing the study.

4. Outcomes Of Controlled Trials

There are several potential outcomes from a randomized clinical trial. First and foremost is whether the treatment is effective as compared to the control group; second, whether there are specific predictors as to which patients do best with the treatment under study; third, whether variables that mediate the outcome of treatment can be identified. Such mediators may throw light on how a given therapy works.

4.1 Assessment Of The Primary Outcome

The primary outcome for a controlled trial is the main indicator or indicators that characterize the syndrome. For example, for panic disorder, the number and severity of panic attacks would comprise the main outcome; for depression a measure of the severity of depression; for bulimia nervosa, a measure of binge eating and purging. Because these measures test the primary hypothesis addressed, that is, the null effect of the treatment in question as compared with a control group, the study should stand or fall depending on whether the primary outcome is positive or not. If the primary outcome is positive then secondary measures—usually associated psychopathology—are also examined for the effect of treatment. Because some symptoms such as mild anxiety or depression are normative in a community, the extent to which the treated population now approximates community norms may also be reported.

4.2 Examination Of Predictors

It is usual in a controlled trial to assess a series of variables that are hypothesized to affect the outcome of treatment. Examples of potential predictors include: the duration and severity of the problem, the age and social class of the patient, and associated psychopathology. If predictors of sufficient strength are found they may be used in future trials to select patients most likely to succeed with the treatment, thus refining the indications for treatment. In general, the search for such predictors has not been very successful. Usually such predictors are sufficient for a general description of the patient most likely to improve, but not specific enough to identify such patients before treatment begins. An alternative method of prediction is to examine the progress of patients during treatment. In many psychological treatments patients obtain significant benefit early in therapy. Patients who do not demonstrate this early response tend to do poorly. Hence, it may be possible to identify those most likely to do badly with a particular psychotherapy early in treatment and to use a different treatment approach with such patients.

4.3 Identifying Treatment Mediators

A mediator is a variable that is responsible for an effective treatment outcome. Hence, it is a result of a therapeutic procedure that occurs prior to the benefits of treatment and that is significantly related to the outcome. Rather than measuring such variables before treatment begins, as is the case with predictors, potential mediators are measured early in the course of treatment. Potential mediators are usually selected based on precise hypotheses about how the treatment works. The identification of mediators can throw important light on the mechanisms underlying the effectiveness of a given psychological treatment and may provide insight as to how to improve the treatment.

5. Are Controlled Psychological Treatment Trials Useful?

While controversy over the use of controlled trials to evaluate psychological treatments continues, the argument has shifted as to whether such trials can reliably inform clinical work and social policy. Some argue that uncontrolled trials conducted in ‘real’ clinical settings with real patients would be more informative. This argument ignores the necessity for a sequenced flow of research in the development of a new psychological therapy. Moreover, without a control group, it is impossible to determine whether it is the treatment or some other variable that is responsible for the outcome. Despite this ongoing controversy, the controlled trial has become the accepted method of evaluating the effectiveness of psychotherapy. Most individuals would prefer that their medical treatment was informed by controlled therapeutic trials; similarly, consumers of psychotherapy should demand that these procedures are well tested.

6. Future Developments

It is likely that the use of randomized controlled trials to evaluate the effectiveness of psychological treatments will accelerate. At present a relatively large number of treatment procedures have been identified as effective for specific psychological problems. Many, but not all, such therapies fall under the rubric of Behavior therapy. These research efforts have now generated a small, but growing number of multisite trials in which the treatment is put to a more definitive test, compared with another effective psychotherapy, or with medication either separately or in combination with psychotherapy. As is the case in medical research, it would be expected that this trend will continue, and that multisite trials will be increasingly used in psychotherapy research.

Because no one psychotherapy is effective with all those suffering from a particular psychological disorder, research is beginning to examine the efficacy of a second treatment approach for those who fail initial therapy. Moreover, there is much interest in the effectiveness of therapist-supported bibliotherapy. Such developments may ultimately lead to algorithms guiding the manner in which psychological treatments should be sequenced for the treatment of a given condition. For example, treatment might begin with the least costly treatment followed by increasingly expensive (and probably more effective) treatments for those who fail one or more initial treatment.

In the somewhat further future the biological mechanisms through which psychotherapies work will become a focus of research, ultimately bringing together psychological and biological mechanisms underlying Behavior change.

Bibliography:

1. Beutler L E, Crago M 1991 An International Review of Programmatic Studies. American Psychological Association, Washington, DC
2. Campbell D T, Stanley J C 1963 Experimental and Quasi- Experimental Designs for Research. McNally, Chicago, IL
3. Chappell M N, Stephenson I T 1936 Group psychological training in some organic conditions. Mental Hygiene, 20: 588–97
4. Eysenck H J 1952 The effects of psychotherapy: An evaluation. Journal of Consulting and Clinical Psychology 16: 319–24
5. Eysenck H J 1966 The Effects of Psychotherapy. International Science Press, New York
6. Goldfried M R, Wolfe B E 1998 Toward a more clinically valid approach to therapy research. Journal of Consulting and Clinical Psychology 66: 143–50
7. Kazdin A E 1998 Research Design in Clinical Psychology, 3^rd Allyn and Bacon, Boston, MA
8. Kraemer H C, Pruyn J P 1990 The evaluation of different approaches to randomized clinical trials. Archives of General Psychiatry 47: 1163–9
9. Persons J B, Silberschatz G 1998 Are results of randomized clinical trials useful to psychotherapists? Journal of Consulting and Clinical Psychology 66: 126–35
10. Russell R L, Orlinsky D E 1996 Psychotherapy research in historical perspective: Implications for mental health care policy. Archives of General Psychiatry 53: 708–15
11. Teuber N L, Powers E 1953 Evaluating therapy in a delinquency prevention program. Proceedings of the Association for Research in Nervous & Mental Disease 31: 138–47
12. Ullman L P, Krasner L 1965 Case Studies in Behavior Modification. Holt, Rinehart and Winston, New York
13. Wilson G T 1998 The clinical utility of randomized controlled trials. International Journal of Eating Disorders 24: 13–29