Journal of Psychosocial Nursing and Mental Health Services

The articles prior to January 2012 are part of the back file collection and are not available with a current paid subscription. To access the article, you may purchase it or purchase the complete back file collection here

Psychopharmacology 

Publication Bias and Outcome Reporting Bias: Agomelatine as a Case Example

Robert H. Howland, MD

Abstract

Publication bias and outcome reporting bias contribute to distorted perceptions of drug efficacy and the underreporting of adverse events. To demonstrate these biases, this article describes how the clinical profile of the antidepressant agent agomelatine (Valdoxan®) has been presented in the literature. Agomelatine has been systematically assessed in 10 short-term placebo-controlled studies and three long-term placebo-controlled relapse prevention studies. Five published trials demonstrated clinically modest but statistically significant benefits over placebo. Five unpublished trials did not find agomelatine more effective than placebo, but in two of these studies the active comparison drug (fluoxetine [Prozac®] or paroxetine [Paxil®]) was more effective than placebo. Agomelatine was more effective than placebo in one of three relapse prevention studies, but only the positive study was published. Based on what is evident in the entire published and unpublished dataset, agomelatine does not have a tremendously superior sleep and sexual effects profile. The risk of liver toxicity is also not prominently highlighted in the published literature.

Abstract

Publication bias and outcome reporting bias contribute to distorted perceptions of drug efficacy and the underreporting of adverse events. To demonstrate these biases, this article describes how the clinical profile of the antidepressant agent agomelatine (Valdoxan®) has been presented in the literature. Agomelatine has been systematically assessed in 10 short-term placebo-controlled studies and three long-term placebo-controlled relapse prevention studies. Five published trials demonstrated clinically modest but statistically significant benefits over placebo. Five unpublished trials did not find agomelatine more effective than placebo, but in two of these studies the active comparison drug (fluoxetine [Prozac®] or paroxetine [Paxil®]) was more effective than placebo. Agomelatine was more effective than placebo in one of three relapse prevention studies, but only the positive study was published. Based on what is evident in the entire published and unpublished dataset, agomelatine does not have a tremendously superior sleep and sexual effects profile. The risk of liver toxicity is also not prominently highlighted in the published literature.

Dr. Howland is Associate Professor of Psychiatry, University of Pittsburgh School of Medicine, Western Psychiatric Institute and Clinic, Pittsburgh, Pennsylvania.

The author discloses that he has no significant financial interests in any product or class of products discussed directly or indirectly in this activity. The author discloses past grant support from Novartis.

Address correspondence to Robert H. Howland, MD, Associate Professor of Psychiatry, University of Pittsburgh School of Medicine, Western Psychiatric Institute and Clinic, 3811 O’Hara Street, Pittsburgh, PA 15213; e-mail: HowlandRH@upmc.edu.

Posted Online: August 17, 2011

Last month, I described the concepts of publication bias and outcome reporting bias, which contribute to distorted perceptions of drug efficacy and the underreporting of adverse events (Howland, 2011b). As an example to demonstrate these biases, in this article I will describe how the clinical profile of the antidepressant agent agomelatine (Valdoxan®) has been presented in the literature.

Agomelatine in 2007

Agomelatine is a synthetic analog of the hormone melatonin. It stimulates melatonin MT1 and MT2 receptors and inhibits serotonin 5HT-2C receptors. These pharmacological effects distinguish it from other antidepressant drugs.

I reviewed agomelatine in this column in 2007. At the time, it was being actively investigated in Europe and in the United States but had not yet been approved by the European Medicines Agency (EMA) or by the U.S. Food and Drug Administration (FDA). Three published short-term placebo-controlled studies (one included paroxetine [Paxil®] as an active control) and one long-term placebo-controlled study (published only in abstract form) all reported significantly positive efficacy findings for agomelatine (paroxetine was also effective). Agomelatine appeared to be nontoxic, except for the rare adverse effect of increased liver function tests. I suggested that agomelatine, based on its pharmacology, should improve the quality of sleep, although studies of sleep effects compared with other antidepressant drugs were not available. I also stated that comparative studies had shown that agomelatine was much less likely to cause adverse sexual effects than venlafaxine (Effexor®) or paroxetine. This statement was based on unpublished findings from two studies alluded to by Rouillon (2006).

Agomelatine in 2009

Agomelatine was first recommended for European approval by the EMA in November 2008. In 2009, I published an expanded review of agomelatine that included data from unpublished studies in Europe available from the EMA’s website. At the time of my 2009 review, the efficacy of agomelatine had been systematically assessed in six short-term placebo-controlled studies (including two studies using fluoxetine (Prozac®) and one study using paroxetine as active controls) and two long-term placebo-controlled relapse prevention studies. Other controlled clinical trials had been conducted in the United States, but the findings from these studies had not been released. The three published short-term trials (which I had previously reviewed in 2007) demonstrated clinically modest but statistically significant benefits over placebo. None of the three unpublished short-term trials found agomelatine more effective than placebo. The active control fluoxetine (Prozac®) was more effective than placebo in one of these unpublished negative trials. Similar to agomelatine, however, neither fluoxetine nor paroxetine were more effective than placebo in the other two unpublished negative trials. A meta-analysis of these six short-term trials demonstrated a small, statistically significant, marginally clinically relevant difference between agomelatine and placebo. Agomelatine was more effective than placebo in the one published relapse prevention study but was ineffective compared with placebo in the other unpublished relapse prevention study.

In addition to the above studies, one unpublished controlled study conducted in older adult patients did not demonstrate a significant benefit for agomelatine compared with placebo. Interestingly, during the peer review of my 2009 review paper, one of the reviewers commented that “an unpublished study in elderly is mentioned, but not a positive study published in severely depressed elderly” patients. My response to this reviewer comment was that the unpublished study was a specifically designed placebo-controlled efficacy and safety study of agomelatine in elderly patients. The “positive study” referred to by the peer reviewer was an unplanned secondary post hoc subgroup data analysis of a selected subgroup of elderly patients (Montgomery & Kasper, 2007). The negative findings from the main outcome study were not published, whereas the post hoc positive findings (derived from their data-mining efforts) were published. This was a clear example not only of publication bias but also outcome reporting bias.

A second peer reviewer of my 2009 paper commented that “the author reviews all the clinical trials performed (pivotal and non pivotal; published and unpublished) with this new drug; surprisingly, he avoids to mention some recent published reviews.” The reviewer provided the references to these six review papers. My response to this reviewer comment was that three of the papers were published in journal supplements as part of a collection of invited papers, which were based on two industry-supported symposia. The objective of these two symposia was to lay the groundwork for and to introduce the “novel” antidepressant agent agomelatine. As expected, these papers simply provided a theoretical rationale and positive need for a new and novel drug in the antidepressant market. Four of the papers suggested by the reviewer included only the same three published positive trials. All of these “review” papers are generally positive, favorable, and uncritical about the efficacy and tolerability of agomelatine (based on the three published trials), and none of them include negative (unpublished) findings.

In my 2009 review, I discovered that the putative favorable effects of agomelatine on sexual function in the two unpublished studies alluded to by Rouillon (2006) were more mixed when the complete findings from both studies were published (Kennedy, Rizvi, Fulton, & Rasmussen, 2008; Montejo et al., 2010). In addition, the entire dataset of published and unpublished placebo-controlled studies, including those studies using fluoxetine and paroxetine as controls, does not support prominent differences in the reported rate of adverse sexual effects among the active drugs.

Similarly, the favorable effects of agomelatine on sleep are more modest than previously suggested, based on two published studies (Kasper et al., 2010; Lemoine, Guilleminault, & Alvarez, 2007) as well as the entire dataset of published and unpublished short-term and long-term studies.

Liver toxicity is a finding not highlighted in published reports but is evident in reviewing all of the published and unpublished data. In the clinical trials, significant elevations of liver enzymes (i.e., increases greater than three times the upper limit of normal) occurred in 1.39% of patients taking agomelatine 50 mg per day, 1.04% taking 25 mg per day, and 0.72% taking placebo. Repeated dose toxicity studies in rats and monkeys indicated that the liver is the target organ of toxicity. Agomelatine causes hepatic enzyme induction in these animals, and they consequently showed enlarged livers or hepatocellular hypertrophy. Significant elevations of liver enzymes in patients were sometimes very serious and included rare cases of hepatitis. Because these hepatic reactions were not predictable based on clinical symptoms or the duration of treatment, the EMA recommended monitoring of liver enzyme levels of all patients before starting treatment; after 6, 12, and 24 weeks of treatment; and then thereafter when clinically indicated based on the judgment of the treating physician. Agomelatine is contraindicated in patients with any degree of liver impairment, such as cirrhosis or other active liver disease. The liver precautions and the need for laboratory monitoring are a distinct disadvantage for the use of agomelatine compared with many other antidepressant drugs.

Agomelatine in 2011

I recently completed an updated review of agomelatine that included data from the three short-term trials (two published and one unpublished) and one long-term relapse prevention trial (unpublished) conducted in the United States (Howland, 2011a). The unpublished study results are available in the Novartis Clinical Trial Results Database ( http://www.novctrd.com). One short-term published study found that agomelatine 50 mg (but not 25 mg) was superior to placebo, whereas the other published short-term study found that 25 mg (but not 50 mg) was superior to placebo. The unpublished short-term study included paroxetine as a control: Paroxetine was superior to placebo, but agomelatine was not. The unpublished long-term relapse prevention study also found no difference between agomelatine and placebo.

A recent review of melatonin-based therapies published in a prominent medical journal highlighted agomelatine (Hickie & Rogers, 2011). The paper abstract states:

In the short-term, agomelatine has similar antidepressant efficacy to venlafaxine, fluoxetine, and sertraline and, in the longer term, fewer patients on agomelatine relapse…than do those receiving placebo…. Patients with depression treated with agomelatine report improved sleep quality and reduced waking after sleep onset. As agomelatine does not raise serotonin levels, it has less potential for the common gastrointestinal, sexual, or metabolic side-effects that characterise many other antidepressant compounds.

Hickie and Rogers (2011) include a table of all published and unpublished studies, but in the text the authors highlight the positive findings from the three published short-term placebo-controlled trials and the one published long-term placebo-controlled relapse prevention study. Rather than critically evaluating the overall efficacy of agomelatine based on all studies, the authors normalize the negative findings by stating that “however, the wider set of studies of agomelatine now available highlights the more general problem encountered in the evaluation of new antidepressants—that separation from placebo is an inconsistent finding” (p. 8).

Hickie and Rogers (2011) correctly noted that comparator antidepressant agents included fluoxetine, paroxetine, sertraline, and venlafaxine, but they do not distinguish those studies whose primary outcome objective was antidepressant efficacy from the studies that had other primary objectives. Among the active comparator trials, they state that “agomelatine had similar efficacy to venlafaxine and was more efficacious than fluoxetine and sertraline” (p. 8). The two comparator trials with venlafaxine were designed with separate primary objectives of assessing sleep and assessing sexual function, and the comparator trial with sertraline was designed with a primary objective of assessing circadian rest-activity cycles. Antidepressant efficacy was only a secondary outcome for all three studies. The comparator trial with fluoxetine was a nonplacebo-controlled double-blind randomized study. Agomelatine was superior to fluoxetine on the change in the depression rating scale score by Week 8, but response rates and remission rates at this time point were not significantly different between the drugs. Surprisingly, Hickie and Rogers (2011) did not highlight the three unpublished placebo-controlled studies demonstrating that fluoxetine and paroxetine were significantly superior to placebo but agomelatine was not.

Contrary to what is evident in the entire published and unpublished dataset, Hickie and Rogers (2011) did not question the modest sleep and sexual effects profile of agomelatine and they made no mention of the adverse liver effects of the drug.

Conclusion

A critical evaluation of published and unpublished agomelatine studies confirms that publication bias and outcome reporting bias contribute to a distorted perception of drug effects. Nurses searching the literature for drug information should be aware of the possibility of bias.

References

  • Hickie, I.B. & Rogers, N.L. (2011). Novel melatonin-based therapies: Potential advances in the treatment of major depression. Lancet. Advance online publication. doi:10.1016/S0140-6736(11)60095-0 [CrossRef]
  • Howland, R.H. (2007). Agomelatine: A novel atypical antidepressant. Journal of Psychosocial Nursing and Mental Health Services, 45(12), 13–17.
  • Howland, R.H. (2009). Critical appraisal and update on the clinical utility of agomelatine, a melatonergic agonist, for the treatment of major depressive disease in adults. Neuropsychiatric Disease and Treatment, 5, 563–576. doi:10.2147/NDT.S5453 [CrossRef]
  • Howland, R.H. (2011a). A risk-benefit assessment of agomelatine in the treatment of major depression. Drug Safety, 34, 709–731.
  • Howland, R.H. (2011b). What you see depends on where you’re looking and how you look at it: Publication bias and outcome reporting bias. Journal of Psychosocial Nursing and Mental Health Services, 49(8), 13–15. doi:10.3928/02793695-20110705-06 [CrossRef]
  • Kasper, S., Hajak, G., Wulff, K., Hoogendijk, W.J., Montejo, A.L., Smeraldi, E. & Baylé, F.J.,… (2010). Efficacy of the novel antidepressant agomelatine on the circadian rest-activity cycle and depressive and anxiety symptoms in patients with major depressive disorder: A randomized double-blind comparison with sertraline. Journal of Clinical Psychiatry, 71, 109–120. doi:10.4088/JCP.09m05347blu [CrossRef]
  • Kennedy, S.H., Rizvi, S., Fulton, K. & Rasmussen, J. (2008). A double-blind comparison of sexual functioning, antidepressant efficacy, and tolerability between agomelatine and venlafaxine XR. Journal of Clinical Psychopharmacology, 28, 329–333. doi:10.1097/JCP.0b013e318172b48c [CrossRef]
  • Lemoine, P., Guilleminault, C. & Alvarez, E. (2007). Improvement in subjective sleep in major depressive disorder with a novel antidepressant, agomelatine: Randomized double-blind comparison with venlafaxine. Journal of Clinical Psychiatry, 68, 1723–1732. doi:10.4088/JCP.v68n1112 [CrossRef]
  • Montejo, A.L., Prieto, N., Terleira, A., Matias, J., Alonso, S., Paniagua, G. & Portolés, A.,… (2010). Better sexual acceptability of agomelatine (25 and 50 mg) compared with paroxetine (20 mg) in healthy male volunteers: An 8-week placebo-controlled study using the PRSEXDQ-SALSEX scale. Journal of Psychopharmacology, 24, 111–120. doi:10.1177/0269881108096507 [CrossRef]
  • Montgomery, S.A. & Kasper, S. (2007). Severe depression and antidepressants: Focus on a pooled analysis of placebo-controlled studies on agomelatine. International Clinical Psychopharmacology, 22, 283–291. doi:10.1097/YIC.0b013e3280c56b13 [CrossRef]
  • Rouillon, F. (2006). Efficacy and tolerance profile of agomelatine and practical use in depressed patients. International Clinical Psychopharmacology, 21(Suppl. 1), S31–S35. doi:10.1097/01.yic.0000195659.99148.09 [CrossRef]
Authors

Dr. Howland is Associate Professor of Psychiatry, University of Pittsburgh School of Medicine, Western Psychiatric Institute and Clinic, Pittsburgh, Pennsylvania.

The author discloses that he has no significant financial interests in any product or class of products discussed directly or indirectly in this activity. The author discloses past grant support from Novartis.

Address correspondence to Robert H. Howland, MD, Associate Professor of Psychiatry, University of Pittsburgh School of Medicine, Western Psychiatric Institute and Clinic, 3811 O’Hara Street, Pittsburgh, PA 15213; e-mail: .HowlandRH@upmc.edu

10.3928/02793695-20110809-01

Sign up to receive

Journal E-contents