At the end of Part I of this post of September 12, 2012, I mentioned that doing psychotherapy outcome studies is diabolically difficult, and promised to discuss the reasons why in this post. In fact, because of the literally infinite numbers of uncontrolled variables in studies of human beings both individually and in relationship to other human beings, psychotherapy outcome research can never be the only standard by which the "science" of human behavior-change technology should be measured. In fact, it's not even the gold standard.
In order to better understand ourselves and what leads us to change our behavior, we must use ALL available sources of information. We have to look at the widespread clinical experience of psychotherapists who use a variety of techniques and theories with a variety of clinical populations. We have to look for potential biases in both clinical studies and within an individual therapist's anecdotes and the conclusions that we draw from them. In forming conclusions about both anecdotal and controlled-trials data, we have to look at a wide variety of possible explanations, as well as for any information and experiences that would seem to contradict those explanations.
We also have to look at the experience of marketers who have been able to induce a large numbers of people to buy a product or vote for a politician (many of whose very powerful techniques have been discussed in previous posts on this blog). We have to look at the social psychology literature, which shows that people behave very differently when interacting with different groups of people and with different individuals than they might do in a therapist's office.
We have to look at historical and sociological trends. We have to look at new knowledge from the neurosciences that might account for findings that are difficult to explain or reconcile with other beliefs. We must look at evolutionary biology. We must examine our own beliefs for logical inconsistencies. We have to be honest about thinking about ourselves what makes us tick personally.
What follows is a list of some of the difficulties psychotherapy outcome researchers encounter, as well as the reasons they may overstate their results. A fuller discussion of each of them can be found in my book, How Dysfunctional Families Spur Mental Disorders.
1. The Problem of the “False Self”
People do not act the same way in all social contexts. They do not act or speak the same way around a boss that they do when they are alone with a lover. A man’s behavior in a strip club is very different than his behavior when he is playing with his children. We have different “faces” or masks which we apply to ourselves in different environments. Not infrequently, these masks are meant to manipulate others to get them to do what we want them to do.
I never cease to be amazed at how mental health professionals and researchers seem to believe that they really know what is going on in a patient’s or a research subject’s life based solely on the self report of the patient, or solely on the reports of the patient’s intimates, or even on the reports of people like teachers who observe the behavior of children in only one context that includes thirty other distracting students. If these professionals were asked if they believe that people often act differently in public than they do behind closed doors, they would of course say yes, but they seem to develop amnesia for this fact in discussions and in studies.
A patient’s family members may be just as motivated to give a distorted view of a patient as is a patient. Parents, for example, may prefer to believe that their child has some sort of mental defect, so as not to experience as much of their own covert guilt about their parenting skills. Conversely, some may actually prefer to blame the child’s behavior completely on themselves, in order to let their “perfect” child off the hook. Most mental health practitioners do not make home visits to watch patients and family members interact in their natural environment. Even if they did, unless they had a camera operating twenty four hours a day as in the movie The Truman Show, they could still be easily deceived.
2. Double Blinding
When it comes to psychotherapy treatment outcome studies, we cannot do double blind placebo controlled comparisons of two different types of psychotherapy treatments. This is true because, in a sense, the therapist – or more correctly the relationship between the patient and the therapist – is the treatment. If the study were to meet the criteria for being double blind, that would mean that the therapists who administer the treatment would have to not know what they were doing.
Of course, they cannot administer psychotherapy without being aware of what techniques they are using. If they could, that would mean that they were incompetent. Not a fair test of a treatment! The fact that the therapy relationship is one of the basic aspects of the treatment also makes placebo or “sham” treatments difficult, because any relationship has some effect on an individual. Does this mean we should give up on evaluating psychotherapy treatment scientifically and rely exclusively on clinical anecdotes? Of course not. Studies are still important. We just have to understand their limitations.
3. Randomized Clinical Psychotherapy Trials
One strategy used in therapy studies is to compare the outcomes of two different, presumably active types of psychotherapy treatment techniques with each other, rather than with placebo, on similar groups of patients, without the benefit of double blinding. These studies are referred to as randomized clinical trials, or RCT’s.
A large number of different schools of psychotherapy have different approaches to the understanding of and methodology for changing a patient’s repetitive dysfunctional habits. Most of these therapy schools were designed by charismatic and creative individuals who based their ideas on clinical anecdotes. These innovators are highly invested emotionally in their own personal theories, and want them to look good in comparative psychotherapy outcome studies.
This leads to a so-called allegiance effect in RCT’s. The preferred psychotherapy school of the researcher is likely to be delivered more enthusiastically and with more rigor to subjects in the study than is the competing therapy treatment. One survey study examined 29 RCT outcome studies that compared one type of therapy to another and found a correlation of .85 between researchers’ therapy allegiance and outcome. That is, the researcher’s preferred treatment came out ahead 85% of the time. Just as in sponsored drug studies, this number is too way high to discount the presence of a significant bias in the studies.
When differences are found between two therapies, they are often statistically but not clinically significant. They show that one therapy is slightly more advantageous than the other, but that the actual improvement of the subjects was so minimal as to be inconsequential.
When both groups of therapists who are providing the treatment in the study are equally committed to the paradigms they are delivering, comparative psychotherapy outcome studies almost always result in a tie. In psychotherapy research circles, it is known as the “dodo bird verdict.” This refers to a character from Alice in Wonderland, the dodo bird, who in one passage said “Everybody has won, and all must have prizes.”
4. Treatment as Usual
Lately, many researchers engaged in psychotherapy RCT’s have employed a control group called treatment as usual (TAU), which really stacks the deck in favor of their pet psychotherapy school. Lining up practitioners of a different school from the researcher’s to act as therapists for a comparison group is often difficult. Additionally, if the researcher were able to do so, the study might not show his or her therapy to be superior to another type. For these reasons, the use of TAU control groups has become almost epidemic in psychotherapy RCT’s.
Subjects randomly assigned to the TAU condition, which serves as a comparison group to the group of subjects receiving the researcher’s therapy model, are simply released back into the community to get whatever other treatments are already out there. Some may see practitioners from other therapy schools, some may get medications, some may get both, and many others may get neither. Both the TAU group and the experimental group are followed up at equal time intervals and given all the same outcome measures. The psychotherapy methodology that serves as the investigated treatment always seems to beat TAU.
The reader should understand that within any widely-practiced therapy model, both good therapists and bad therapists can be found, just as practicing physicians can be either good or bad psychopharmacologists. For the subjects in the TAU condition who receive treatment, the results of the good clinicians in the community are probably cancelled out by those of the bad ones. TAU subjects may also be seen less frequently. Some, as I mentioned, are getting no treatment at all. Meanwhile, the experimenter’s group is usually getting a lot more individualized attention by therapists who are highly committed to the particular treatment model. These factors may be the real reasons their patients do better than those receiving TAU.
The experimenter’s therapy is provided by uniformly well-trained and enthusiastic therapists under very well controlled conditions. The psychotherapy that is provided is applied with rigor and consistency, and is scrutinized by other observers through the use of videotapes of the sessions. Therapists who make errors are supervised almost immediately. On top of this, research therapists often have caseloads that are very much smaller than those of folks out in practice, allowing them to spend more time deciding how to approach the clinical issues they face. I cannot recall a single instance of a study in which TAU beat another therapy delivered in such a manner. If it did, I would have to wonder how the experimenter could have possibly accomplished such an unlikely feat.
5. The Choice of Outcome Measures
Another huge controversy in psychotherapy RCT’s is the question of which outcome measures should be used, because the different psychotherapy schools target different aspects of the patient's problems. What should we measure, exactly? Symptom relief? Whether or not the patient continues to meet criteria for a specific DSM disorder? Whether they stop engaging in chronic repetitive dysfunctional behavior? Whether they experience fewer conflicted emotions? Whether they have improved and more satisfying relationships? Whether they can hold a good job?
In their November 1995 issue, Consumer Reports published a large survey on the public’s experience with psychotherapy. In response to the survey, satisfaction with their experience was reported widely by those who had been in therapy. Furthermore, longer term therapy was rated higher than therapy limited in duration by managed care, and satisfaction with psychotherapy was rated about equally by those who had therapy alone and those who were given a combination of therapy and medication.
In surveys of this sort, the question of selection bias arises - whether or not the sample of people polled is representative of the general therapy population. The respondents in this case were almost all middle class and educated, so we have to limit our conclusions to that population rather than the population of the United States as a whole. Nonetheless, the study was the most extensive study of its type on record. Should opinions expressed by patients evaluating psychotherapy outcomes for themselves count as science? Many in the field would say no.
But one must consider, however, that psychotherapy target's a patient’s subjective sense of personal well being. That is, we want to know whether patients feel better mentally about their lives. By this measure, assuming that the survey respondents were being somewhat honest with their answers, we would have to say it should count. A paradox about the “science” of psychotherapy is that we are attempting to be objective about subjectivity, two concepts which are usually defined as antonyms.
6. Funding Issues
Another big issue for the field concerns which psychotherapy treatment outcome research studies get funded. If a scientist cannot get money for a project, it will rarely get off the ground, because these types of studies are very expensive to mount. Most successful psychotherapy RCT’s have employed CBT because of the predominance of this model in professional psychology training programs, and because most CBT treatments aim primarily for symptom reduction, which is relatively easy to measure, rather than personality change, which is not.
7. Problems with Generalizability of Study Results
Another big problem with all psychotherapy RCT’s is that most studies require that the subject population be homogeneous, meaning that the subjects in the study must be very similar in the nature of the disorder they exhibit, and in how severe it is. This requirement means that patients who have more than one DSM disorder or psychological problem are often excluded from studies. In contrast, most patients seen in practice, at least by psychiatrists, have more than one disorder (co-morbidity). This fact alone limits what is termed the generalizabilityof the study’s finding. We do not know from a study using patients who have only one disorder if the treatment employed in the study would work as well with patients who have multiple problems.
Because studies need subjects who will stick with the treatment until the end of the research project, some subjects who have certain characteristics that are common in clinical practice tend to be excluded. This problem further limits the generalizability of the study. For instance, most studies of treatments for depression exclude patients who are suicidal! Another problem with RCT’s is that many subjects drop out of a study or are removed from a study as it proceeds because they do not completely cooperate with the treatment in some way. CBT studies, as do many others, tend to have a fairly high dropout rate. The subjects who end up completing the study are usually the most motivated to change, and would therefore be expected to do better than those who lack this motivation. This all makes the treatment method look much better than it would be if it were employed in a typical clinical practice setting.
8. Treatment Manuals
Another big issue in psychotherapy outcome studies is that, in comparing different treatments for different behavioral problems, researchers need to have some way of knowing that all the therapists employed in the study are doing mostly the same things. In early psychotherapy studies, the methods section of a journal report might say that such and such number of patients was “given psychoanalytic therapy,” with no description of what the therapists actually did. This was clearly unacceptable.
The solution to this problem that has been employed most frequently is the use of treatment manuals, as well as manuals that measure the study therapists’ competence in and their adherence to the treatment paradigm. Treatment and adherence manuals specify exactly what the therapist should be doing under a variety of circumstances. Tapes of psychotherapy sessions of patients in the studies are reviewed to make sure that the therapists are employing the same interventions. A “rater” will count the times the therapist did the right thing and the times the therapist strayed from the procedures specified in the manual. This process assures that all patients are getting approximately the same treatment.
Unfortunately, a good therapist has to be flexible and employ a variety of different strategies in ways that are tailored to the proclivities and sensitivities of the patient in front of them. Generic interventions may not only fail to work, they may backfire and make matters worse. Verbal interventions must often be phrased differently to different patients in order to reduce the patient’s defensiveness. Treatment and adherence manuals take away a lot of this flexibility, so that the therapy as performed in an RCT is not always similar to the way that clinicians out in the field practice it. Creating treatment manuals can itself be a daunting task. I heard a respected researcher tell a group of other researchers that his team was having trouble designing such a manual because the founder of the treatment they wanted to study was observed to perform psychotherapy completely differently than his own wife, who supposedly was a practitioner of the same model of therapy.
Tidak ada komentar:
Posting Komentar