Homeopathic study: The legend of the five times eight rats

Scientific Reports, quite a medical journal of good reputation, published on 10.09.2018 a paper entitled “Ultra-diluted Toxicodendron pubescens attenuates pro-inflammatory cytokines and ROS- mediated neuropathic pain [1] in rats“. The paper concludes that homeopathic high potencies based on Rhus toxicondendron (poison ivy, a traditional homeopathic remedy for pain symptoms) are at least as effective as Gabapentin in reducing pain and its molecular correlates (i.e. changes in the cellular signalling pathway), both in cell experiments (in vitro) and in animal experiments with rats.

Once again, parts of the homeopathic scene (associated with the exaggeration that the article appeared directly in “Nature“) announced the ultimate breakthrough in the effort to prove homeopathy “scientifically”. The Italian daily La Repubblica drew more attention to the matter with an article that was entranced by the statement:

“The studies published in Scientific Reports confirm the effectiveness of homeopathic ultra-thin dilutions: Homeopathic medicine works and has statistically significant biological effects in alleviating neuropathic pain in rats. The effect of the dynamised dilutions was similar to that of Gabapentin […]”.

Note the lack of subjunctive, also the uncritical use of the term “statistically significant”, which as a statistical probability value does not allow conclusions to be drawn about the actual effect size, but which is suggested by the formulation. No reproach against a daily newspaper, but a suggestion for the public that is welcome in homeopathy and never corrected.

Proof of the efficacy of homeopathic high potencies – a bold statement, which must be considered in view of the contradiction to the homeopathic study situation according to the postulate “extraordinary claims require extraordinary proof”.

First of all, it is a completely unscientific conclusion that the paper “confirms the effectiveness of homeopathic ultra-thin dilutions”. A single study has no evidential value, but at most an indicative value, especially if it produces a highly unlikely result. Such a result must be replicated independently – preferably several times – before the described effect can be assumed with sufficient probability to be actually present and further scientific discussion can be initiated. All this is missing here.

A first glance shows that the in vivo tests on rats involved a total of 40 animals, which were divided into five test groups (n=8). This is astonishing with regard to the small number of test subjects. Should the “groundbreaking” result of the study be seriously based on the data of a verum group of eight animals? It can’t be inferred from the study that case number planning was carried out, highly required for intervention studies and representing a factor for the validity of the later results. Above all, if differences – as here – are to be determined in five differently parameterized groups (not only verum versus placebo or standard means), the careful determination of a sufficient and meaningful number of test objects is anything but trivial.

The publication also shows relatively easily identifiable uncleanliness / abnormalities, which the portal RESIS – Research Integrity Solutions – explains in a contribution by Enrico Bucci from Temple University in Philadelphia.

This starts with the fact that graphical representations of the in vitro results are partly incorrectly labeled, partly the same graphics were published for different parameters. Even more important is the fact that different effect/time diagrams (one for a cold test, one for a heat test in animal experiments) in all (!) displayed data points (“every experimental point”) correspond 1 : 1 (which is not immediately visible due to the differently selected axis sizes), so that RESIS draws the conclusion that something elementary cannot be correct here – random chance excluded. The raw data are not published – of course not. RESIS states: If there are really two different experiments, the shown coincidence of the data is simply impossible.

RESIS points out that these are not the only statistical-numerological inconsistencies and rightly believes that this is enough to discredit the validity of the study. Other portals and forums have also already taken a critical look at the work, e.g. PubPeer, where the panelists go into even more critical detail.

A particularly strange circumstance, in which journals usually tend to react very sensitively, is added. One of the authors of the Indian research group names a mail address that was demonstrably used in another work by an author of another name. This is not only completely unusual, but also doesn’t correspond in any way to good practice and integrity in publishing.

The pharmacology professor Silvio Garattini, who has discovered further methodological shortcomings on the sanita24 portal, concludes: “Homeopathy remains a method without scientific proof, and there is no change of that because of the alleged efficacy claimed by the studies published in Scientific Reports”.

We agree – and ask ourselves together with RESIS, how such a work can survive a peer review for a journal, also in view of the responsibility a journal has towards the scientific community and the public. This does not mean, however, that there can be no mistakes in scientific publishing, not at all. However, one can rely on the self-control of the scientific community, as in this case.

After all, the publication of Scientific Reports now carries an editor’s note that refers to the objections against the work and announces “appropriate editorial measures” “as soon as this matter has been clarified”. The quick-shot statement of the authors of the study that this may be all, but does not change the result seems to no longer satisfy the editors of Scientific Reports.

Supplement, 11.10.2018, 15:00 o’clock:

In the meantime, “Nature”, the “mother journal” of Scientific Reports, also reports on the matter. They are striving for neutrality, but with a rather critical undertone.

Supplement, 11.06.2019:

As Retraction Watch reports, the study has just been retracted by Scientific Reports. Scientific Reports published a comprehensive retraction note today (11.06.2019). It says:

“Following publication, the journal received criticisms regarding the rationale of this study and the plausibility of its central conclusions. Expert advice was obtained, and the following issues were determined to undermine confidence in the reliability of the study.

The in vitro model does not support the main conclusion of the paper that Rhus Tox reduces pain. The qualitative and quantitative composition of the Rhus Tox extract is unknown. Figures 1G and 1H are duplicates; and figures 1I and 1J are duplicates. The majority of experimental points reported in figure 3 panel A are duplicated in figure 3 panel B. The collection, description, analysis and presentation of the behavioural data in Figure 3 is inadequate and cannot be relied upon.

As a result the editors are retracting the Article. The authors do not agree with the retraction.“

In other words, everything that was already the main object of criticism eight months ago bears out. Nature resp. Scientific Reports must not be criticized for the period of time that has elapsed until the retraction, here is carefully examined and external neutral expert knowledge is used. All the more important is a complete retract. It is noteworthy that the main author of the study does not agree at all with the withdrawal and repeats his indications that it is a matter of “typos” and perhaps a little laxity in the publication. This can probably be seen as a mixture between massive confirmation bias and defensive strategy. More interesting, the question remains how such a work could “slip” through the peer review of a highly respected scientific publication.

Our conclusion:

Another case from the colourful wreath of the famous “many studies” that “undoubtedly prove the effectiveness of homeopathy”, as it is heard every day. The special thing about this case compared to many others is its deep fall: After all, it was loudly exclaimed as the ultimate proof of the “efficacy of homeopathic ultra-dilutions” and thus settled even a few floors above the well-known “results” of Jacques Benveniste and Luc Montagnier – these two had “only” postulated that they had found “differences” between pure solvent and homeopathic high potencies, not more. Here in the case of the “rat study” it was even up to “efficacy of homeopathy”…

Failed, once more. The lack of evidence of homeopathy is a fact; it has not been able to prove such evidence either for individual indications or as a method. With studies such as this, she at least continues to contribute to its discrediting. We rest our case.

1] ROS: Reactive Oxygene Species, known as “free radicals”. A mediating role of ROS in intra- and intercellular signalling processes is considered to be assured

Picture credits: Screenshot Scientific Reports online