The conventional wisdom: STEP-BD for bipolar depression
CATIE was the largest (about 1500 patients) and most ambitious of the three canons of psychopharmacology; it failed to show that most patients even stay on their medications for a year in schizophrenia. STAR*D was about half the size of CATIE (around 750 patients), but still large, and showed benefit acutely but again most patients did not stay well with their medications for a year with depression. STEP*BD was smaller again, about half the size of STAR*D, but very different in its goals. The efficacy of lithium and other mood stabilizers for long-term treatment was well-established. What was controversial was short-term treatment, specifically with antidepressants.
Thus, a 6 month acute trial of 366 patients was designed comparing bupropion versus paroxetine versus placebo for acute bipolar depression. Both of those agents had been studied previously and found to have low mania switch rates, which is why they were chosen. The result of the main study was that both agents were equivalent to placebo
The simple conclusion was that antidepressants were proven equal to placebo and thus ineffective in bipolar depression. A secondary outcome was that mania switch rates also were equal in all three arms, and thus these antidepressants were shown not to cause mania any more than placebo.
In short, antidepressants didn’t help, but they also didn’t hurt. Since most clinicians used antidepressants in most patients with bipolar illness, the researchers didn’t argue against that practice, and in fact antidepressant use in bipolar illness has not declined, and in fact has increased somewhat since STEP-BD was published.
Alternative interpretations of STEP-BD: Bayesian aspects
The main result of STEP-BD would seem very straightforward. When a drug is equivalent to placebo, the standard interpretation is that the drug doesn’t work, assuming adequate statistical power and other clinical trial methods, which was the case in STEP-BD. But it’s one thing to say a drug doesn’t work before it comes to the commercial market, as is the case before FDA indications. When drug companies have results of a new medication that are equivalent to placebo, they won’t even bother taking it to the FDA. They know it will be rejected. If a drug isn’t being used, then equivalence to placebo means it won’t be used.
But what happens when a drug already is in use, and commonly so, as with antidepressants in bipolar depression. Despite absence of FDA indication for that purpose, antidepressants are on the commercial market for other depressive conditions, and clinicians believe they work for bipolar depression also. The STEP-BD results arrive, therefore, in a context that is quite different than when a drug is being developed by a pharmaceutical company before it is taken to the FDA to possibly enter the market. The issue now with antidepressants in bipolar depression is that the results of STEP-BD, showing inefficacy, would have to convince clinicians to stop using those agents, as opposed to not letting them do so at all. This is a very different scenario.
The situation can be understood through the lens of Bayesian statistics. The same study result can mean very different things in different settings. Where no one knows anything about a drug, a study showing placebo equivalence would mean that no one would believe in it. When people already believe in a drug, a study showing placebo equivalence might influence some fence-straddlers, but to the majority who strongly believe in drug efficacy, reasons will be found to keep doing what they’re doing.
In the case of antidepressants in bipolar depression, the reason was the second outcome: the drugs didn’t cause acute manic switch more than placebo. So if they don’t hurt, and I believe they work, I’ll keep using them, reasoned many a clinician.
A Bayesian analysis could get into the other harmful effects of antidepressants not measured in STEP-BD, such as their long-term mood-destabilizing effects causing rapid-cycling in about1/4 of patients. But the STEP-BD researchers did not address those issues, and many clinicians came away with a Bayesian interpretation which they didn’t realize was Bayesian: they allowed a partial assessment of the scientific literature, excluding other known harms, to allow them to downplay the implications of the inefficacy shown with antidepressants in STEP-BD.
The methodological take-home point is that Bayesian analysis can be distorted if it isn’t as complete and objective as possible.
Alternative interpretations of STEP-BD: Concomitant medications
As with CATIE, we see a misuse of p-values in STEP-BD’s analysis of acute manic switch, to claim a false equivalence between drugs and placebo. STEP-BD was not powered to assess whether bupropion or placebo cause manic switch. Thus, the absence of a statistical difference cannot be claimed to show no difference. Again p-values are misused, and effect sizes and confidence intervals should have been used, as shown previously with CATIE. 10.1% (18/179) of the antidepressant group had manic switch versus 10.7% (20/187) of the placebo group. In this case the direction of effect, if any, is slightly in the direction of placebo. Of course there is no biological or clinical rationale for antidepressants reducing the risk of manic switch, so this result likely is not meaningful. More importantly, both groups also were taking baseline mood stabilizers, which reduce the risk of manic switch. Thus, the key factor here may not be that misuse of p-values obscured a real effect, but the effect of concomitant medications, namely treatment with anti-manic agents that may have prevented the observation of any potential risk.
An analogy would be as follows: Suppose you were doing a study of a drug which might cause fever. But all patients were treated with anti-fever drugs like aspirin at the same time. If that drug had similar fever rates as placebo, in patients treated with anti-fever treatment with aspirin, would you then conclude that the drug inherently doesn’t cause fever?
