Hi Mike W. - Not dismissing the defects (not sure where you got this
impression)- again, as I said there are plenty of important questions to be
raised here and I'm hardly an apologist for the current state of research on
psychotherapy outcome. I was merely pointing out that what you wrote in your
email, namely, that all of the dependent measures in psychotherapy outcome
research come from either the client or the investigator, is factually
incorrect. Do you still stand by your original assertion?
I agree fully with Mike W. that it's technically impossible to ever
conduct a genuine double-blind study of psychotherapy; I don't know any
(reasonable) person in the field who would take issue with that assertion. I
do not agree that this renders all psychotherapy outcome studies categorically
"invalid," as you have stated in your messages, any more than it means that we
should dismiss all twin studies because the equal environments assumption
hasn't been tested for all phenotypes or that we should dismiss all studies on
the construct validation of clinical assessment measures because we cannot rule
out with 100% confidence the possibility of response biases. What is does
mean, and I think we agree here, is that we need to bear this internal validity
threat in mind when evaluating the results of psychotherapy outcome studies.
All the best...Scott
Scott O. Lilienfeld, Ph.D.
Department of Psychology, Room 473
Emory University
36 Eagle Row,
Atlanta, Georgia 30322
slil...@emory.edu; 404-727-1125
-----Original Message-----
From: Brandon, Paul K [mailto:paul.bran...@mnsu.edu]
Sent: Wednesday, September 14, 2011 10:45 AM
To: Teaching in the Psychological Sciences (TIPS)
Subject: Re: [tips] How to blind a treatment study of psychotropic meds or
psychotherapy?
Good points!
There's a lot of faith based practice out there ('the problem is serious,
therefore the treatment must be effective')
On Sep 14, 2011, at 12:50 AM, Mike Wiliams wrote:
> Hello All,
>
> Its interesting how Scott and Mike P. dismiss the threat to internal
> validity as if a meta-analysis balances out the defects. All a meta-analysis
> does is add up the defects.
>
> The Meta-analyses can only present the data that is collected in the
> individual studies. As far as I know, no blinded subject has ever been asked
> whether they were in the treatment condition or not. No blinded investigator
> has ever been asked if they could identify the treated subjects. I
> definitely could identify them from their report of side effects. All the
> investigators could have done the same by examining the adverse event reports.
>
> The observer ratings that Scott refers to can all be influenced by the
> same internal defect. All the observers, including parents, know that the
> children are taking a medication because of the side effects. All these same
> people know when a child is being treated with a behavior intervention
> because it appears very different than a waiting list control or other
> control conditions.
>
> All the drug companies and the psychotherapy outcome investigators
> need to do is survey the subjects and the investigators to verify the
> blinding. They don't do this because they know that these studies can never
> be blinded. They interpret the results as if they are.
>
> Until a genuinely blinded treatment study is conducted, all the effect
> sizes in all these studies could be the result of the internal bias that
> Campbell, Stanley and Shadish so eloquently present.
>
> If anyone can present a study that was correctly blinded, or even
> present a way this could be done, it would advance the field 100% since all
> the studies done up to this point have presented noise.
>
> The research defect I described doesn't exist among studies in which
> the blinding isn't threatened by side effects and other clear indications of
> the treatment condition.
>
> Until a genuinely blind treatment study is conducted, these drugs and
> psychotherapy interventions have no empirical validation.
>
> No insurance company should pay for these treatments until they are
> empirically validated.
>
> Isn't anyone but me curious about why placebos are sugar pills? Why
> not try a salt pill? The control condition must be similar to the treatment
> condition for humans or they quickly figure out which condition they are in
> and they are very influenced by the social setting of research.
>
> Mike Williams
>
>
>
> On 9/14/11 1:00 AM, Teaching in the Psychological Sciences (TIPS) digest
> wrote:
>> Mike W. is right to raise useful questions regarding the internal
>> validity of psychotherapy designs, but I agree with Mike P. that he is wrong
>> to categorically dismiss all of them simply as "invalid." Surely, no study
>> is perfect, but many of them yield highly useful inferences. In addition to
>> Mike P.'s endorsement of Don Campbell's writings on internal validity, I'd
>> like to add Campbell's helpful principle of the heterogeneity of
>> irrelevancies. The most helpful inferences derive the consilience of
>> multiple independent studies, all with largely offsetting flaws.
>
>
> ---
> You are currently subscribed to tips as: paul.bran...@mnsu.edu.
> To unsubscribe click here:
> http://fsulist.frostburg.edu/u?id=13438.3b5166ef147b143fedd04b1c4a6490
> 0b&n=T&l=tips&o=12633 or send a blank email to
> leave-12633-13438.3b5166ef147b143fedd04b1c4a64900b@fsulist.frostburg.e
> du
---
You are currently subscribed to tips as: slil...@emory.edu.
To unsubscribe click here:
http://fsulist.frostburg.edu/u?id=13509.d0999cebc8f4ed4eb54d5317367e9b2f&n=T&l=tips&o=12650
or send a blank email to
leave-12650-13509.d0999cebc8f4ed4eb54d5317367e9...@fsulist.frostburg.edu
________________________________
This e-mail message (including any attachments) is for the sole use of
the intended recipient(s) and may contain confidential and privileged
information. If the reader of this message is not the intended
recipient, you are hereby notified that any dissemination, distribution
or copying of this message (including any attachments) is strictly
prohibited.
If you have received this message in error, please contact
the sender by reply e-mail message and destroy all copies of the
original message (including attachments).
---
You are currently subscribed to tips as: arch...@jab.org.
To unsubscribe click here:
http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=12651
or send a blank email to
leave-12651-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
