ADHD and homeopathy – a new review. What’s in it?

A few days ago, a thread by Canadian health scientist and sceptic Timothy Caulfield made waves on Twitter. Caulfield was referring to a systematic review of ADHD treatment with individual homeopathy, specifically that a journal belonging to the Springer Nature Publishing Group (Pediatric Research) had published it. Caulfield stated that homeopathy is a method without any positive evidence (“scientifically absurd”) and expressed incomprehension about how such a topic could appear in such a renowned journal. Where had the Board of Editors been, how had it got through peer review?

This is about the work of Gaertner, K., Teut, M. & Walach, H. Is homeopathy effective for attention deficit and hyperactivity disorder? A meta-analysis. Pediatr Res (2022). https://doi.org/10.1038/s41390-022-02127-3

In addition, Caulfield’s criticism was also directed at the team of authors, all proven members of the homeopathic scene and with the not unknown Prof. Harald Walach as senior author, a personality whose scientific reputation has not exactly grown in the last pandemic years. But we don’t want to put an ad hominem in the foreground – even if the authors demonstrate their “bias” by setting the mood for reading by presenting the efficacy of homeopathy for a whole series of diseases as a fact. And the falsely stated fact [1][2] that the prevalence of ADHD (the frequency of the disease in relation to the population as a whole) is steadily increasing, does not necessarily raise expectations.

And not to forget: anyone who recognises that the basic assumptions of homeopathy, such as a general principle of similarity in nature and an amplification of effect through shaking, cannot have any validity, can also only assess empirical results on this basis as false-positive. Especially since Mr. Walach has repeatedly described the “effect” of homeopathy as “non-material” – in other words, he has actually removed it from the object of scientific research. But here he nevertheless tries – repeatedly – to provide empirical evidence in favour of homeopathy.

Well, we can certainly understand Caulfield’s criticism of the publication as such. Of course, he reflects on the fact that “Nature” in particular is to be criticised because of its sceptical-critical tradition. Nature was made great by the legendary editor John Maddox, who debunked Benveniste’s “water memory” experiments in his time and after whom an important prize for advocacy against pseudoscience is named today. And yes, it is right to criticise homeopathy in principle and also to include the fundamental implausibility of the basic homeopathic assumptions in an overall scientific assessment via the medical statistical (non-) evidence. We remember a similar process in which the mother journal had also intervened itself and it then came to a retraction of the work (“The legend of the five times eight rats“).

Now, a small or even medium Twitter storm will have little effect on the fact that this review is published and exists in reality and will undoubtedly serve as another poster child for the homeopathic scene. The only way to “get to the bottom of it” is through a more detailed analysis of this review.

Such an analysis has already been published on Prof. Edzard Ernst’s blog – and makes the question of where the Editors Board had its head and the reviewers their five senses appear to be highly justified.

We resume:

It should be mentioned at the outset that the evidence to date on the homeopathic treatment of ADHD has been more than meagre. The negative result of a review published in the Cochrane Library (Coulter and Dean 2007) was the state of affairs until now. There they came to the conclusion:

The forms of homeopathy evaluated to date do not suggest significant treatment effects for the global symptoms, core symptoms of inattention, hyperactivity or impulsivity, or related outcomes such as anxiety in Attention Deficit/Hyperactivity Disorder.

However, let us take a critical but unbiased look at how Gaertner et al. arrive at their conclusion that “individualized homeopathy showed a clinically relevant and statistically robust effect in the treatment of ADHD.”.

However, let us take a critical but unbiased look at how Gaertner et al. arrive at their conclusion that “individualised homeopathy showed a clinically relevant and statistically robust effect in the treatment of ADHD”.

The review, i.e. the evaluation and assessment of relevant studies, comprised six papers, one of which was (correctly) excluded in advance due to a lack of data reporting, because an effect size could not be calculated. As is well known, the object of a systematic review is the critical appraisal of the included studies and the conclusion to be drawn from them. Have the included studies by Gaertner et al. been evaluated critically enough? The authors themselves state that “all included studies used individualised homeopathy and were of comparable, solid quality, so that a lack of methodological rigour is unlikely to be the reason for the difference between homeopathy and controls…”. So one wants to prevent methodological criticism right from the start.

But is that so?

Perhaps not.

• • The authors’ own classification sees three of the six included papers as having a high risk of bias and another as “unclear” in terms of bias risk (medium risk of bias). This does not really fit in with the claimed “comparable solid quality” of all studies.
  • The authors included two active controlled trials, both of which did not include information on the type of treatment of the control group (Filbert et al. 2016 and 2019). In other words, these studies failed to report important data, which was a declared exclusion criterion. Thus, according to the authors’ own criteria, they should not have been included at all. Moreover, it is reasonable to assume that the study Filbert et al. 2019 also may only have the status of a pilot study, as their authors themselves point out that it is about a “feasibility-testing of two treatments and a methodology”.
  • One of the papers (Jacobs et al. 2005) was only a pilot study, which does not belong into a systematic review, and also produced a negative result. The statement in the review that the comparison between verum and placebo was positive for the homeopathy group cannot be understood according to the data in the original study.
  • Jones (2009) is a dissertation that cannot be considered a study and was published without any peer review.
  • Frei et al. (2005) is well known and much discussed,[3][4] a crossover study (change of therapy in the groups), from which evidence could only be read with a lot of effort and a homeopathic view and which could only fulfil the methodological requirements of an RCT in the crossover phase (one of four).[5] In part, its results even speak against homeopathy. This work was also evaluated accordingly in the aforementioned review by Coulter & Dean (2007); in contrast, here Gaertner et al. award Frei’s work the highest quality level in all criteria defined by them.

What remains? The work of Oberai et al. from 2013. Indeed, at first glance, this one shows the greatest significance (p-value 0.000, so something like the statistical equivalent of the real truth …) and a relevant clinical effect of homeopathic therapy. And indeed, Oberai is the “cash cow” of the review, because this study alone is causal for the fact that “overall” a “positive result” was reported.

Therefore, a closer look at this.

• • Also in the case of Oberai et al. it was only a (nine year old) pilot study that was never followed by a main study (or one never got published).
  • The publication was in a somewhat dubious journal that is also not associated with the major medical databases such as MedLine.
  • The reporting on the study is very meagre. For example, it is not clear how the ADHD diagnoses were verified in the patients. (Comparable would only be cases in which the diagnoses were made strictly according to the criteria of the ICD / DSM, the international diagnostic classifications).
  • The control group received only placebo for one year. If this is not an ethical problem … at least in the case of a confirmed ADHD diagnosis, one has to ask.
    Strange is the authors’ statement that “acute symptoms” were also treated with homeopathy in the control group during the study period. This simply nullifies any claim that a “placebo-controlled study” was carried out.
  • Only patients were blinded, not the study staff. So it is not a “double-blind” study either – with all the risks that one tries to avoid with the “gold standard” of the controlled, randomised double-blind study (RCT).

And THAT is the “hard core” of the review by Gaertner et al. ? Around which the work has built a “framing”, so to speak, from a handful of other, even less valid works? Really?

Edzard Ernst points out that even according to the review authors’ own exclusion criteria, the study by Oberai et al. should not have been included. Why was it included anyway? Perhaps because there would have been no more “fuel” for a positive result of any kind? We are only asking …

The result of a closer look at the included studies and their evaluation is that this review does not justify the conclusion formulated by the authors that an individual homeopathic treatment of ADHD has a clinically relevant and statistically robust effect in the treatment of ADHD.

And this does not even address further questions. For example, the study selection covered the period from 1980 onwards – since then a lot has changed in the diagnostic classifications as well as in the standard therapies for ADHD. So it would be quite questionable what was actually compared with what. But let’s leave it at that – the conclusion remains that this review should not have taken the obstacles for publication in a renowned journal. Whether this could be communicated to the journal would be a question that still needs to be clarified.

[1] Sciutto, M. J.; Eisenberg, M. (2007): Evaluating the evidence for and against the overdiagnosis of ADHD. In: Journal of attention disorders. Bd. 11, Nr. 2, Sept. 2007, S. 106–113.

[2]  https://www.rki.de/DE/Content/Gesundheitsmonitoring/Studien/Kiggs/Basiserhebung/GPA_Daten/ADHS.pdf (in German)

[3] http://www.beweisaufnahme-homoeopathie.de/?p=110 (in German)

[4] http://www.beweisaufnahme-homoeopathie.de/?p=1297 (in German)

[5] https://www.homöopedia.eu/index.php?Title=Artikel:Oft_gehörte_Argumente_-_Verweise_auf_konkrete_Studien_und_Experimente#ADHS-Studie_von_Heiner_Frei (in German)

Picture: Screenshot Twitter (Timothy Caulfield)