The Central Association of Homeopathic Physicians (DZVhÄ) has recently published an article on its website under the heading “Encouraging Evidence on Homeopathy” (in German) and also sent out several press releases.
The largest (but by no means the only) review of homeopathic studies currently available is the 2015 “NHMRC” review, published by the Australian Health Authority in a summary information paper, which concluded that no homeopathically treated condition has been shown to be effective (beyond placebo and contextual effects). Of course, this did not suit homeopath’s ideas and they tried for years to obtain a former version of the review, which was claimed to have originally spoken for homeopathy, but was then “suppressed”. Under “Release the first Report” homoeopaths from all over the world kept on raging until the Australian health authorities agreed to publish the “first version” from 2012. The homeopathic fraction is now trying to capitalize on this release and derive “encouraging evidence” from it.
This is astonishing, because the own homeopathic research of the highest evidence class (among others four systematic reviews by R.T. Mathie on behalf of the Homeopathic Research Institute) corresponds exactly with the findings of the NHMRC review of 2015: There is no reliable evidence that homeopathy is specifically effective in any clinical picture.
From the preliminary draft of a major review, one suddenly wants to derive results that not even one’s own homeopathic research was able to achieve with years of effort?
A review of the “prehistory”
In a press release dated 12.04.2017, the DZVhÄ reported on a complaint against the review of the Australian Health Authority NHMRC of 2015, which found no reliable evidence for homeopathy for any indication. The DVZhÄ coined the term “deception of the public” by allegedly serious failures of the NHMRC.
They announced that the English Homeopathy Research Institute (HRI) had “filed a complaint with an official Commonwealth office and published the first results of its research”. This “official Commonwealth office” was the Ombudsman of Her Majesty in Australia, who – to this day – was obviously somewhat astounded being expected to make a decision on a scientific question. This is probably the reason why such a decision – now after several years – does not exist. The existence of the complaint was confirmed after a long back and forth, but neither the HRI was the complainant nor was the text of the complaint ever disclosed. By the way, just as little as the “results of the research” by the HRI, nothing ever published did go beyond the enumeration of several points of a rather general nature (there was always talk of a “200-page paper”, which at any rate nobody outside the HRI got to see).
One, if not the central point, of the excitement was the allegation that there had been a first version of the NHMRC review, which, since it had allegedly been positive for homeopathy, had then been “suppressed” and replaced by the version published in 2015. Even petitions in Australia and England called for their disclosure. At the end of August then somewhat unexpected happened – the disclosure of the draft by the Australian health authority.
Now nobody could imagine, apart from the homoeopaths, who in the meantime had gotten stuck into this matter, that a state medical research centre had “suppressed” a draft of a major review, which – contrary to all previous investigations – would have come to a positive result for homoeopathy. And this is exactly what the NHMRC explained on the occasion of the publication of the draft: they wanted to “take the wind out of the sails” of the circulating rumours by the best of all possibilities, namely transparency. One may assume that this was done on the recommendation of Her Majesty’s Ombudsman in Australia, who has been sitting there with the complaint for several years now and obviously can’t do anything with it.
We will let representatives of the NHMRC have their say on the classification of the publication. First of all, the Executive Director Research Translation Branch , Alan Singh. Among other things, he writes:
“I would like to inform you that I am releasing an unfinished draft report, in an annotated form, of an overview of systematic reviews of homeopathy that was started in 2012 but never completed (frequently referred to as the 2012 draft report). There have been a number of enquiries about the 2012 draft report and requests for its release. I am aware of ongoing community interest in the content of this draft report, and that a substantial body of misinformation has grown up about it.
To address this misinformation, an annotated version of the 2012 draft report will be made available on the NHMRC website. The annotations provide context as to why the report was not developed further for use in NHMRC’s Homeopathy Review that produced the NHMRC Statement on Homeopathy and the NHMRC Information Paper: Evidence of the effectiveness of homeopathy for treating health conditions (2015). The 2012 draft report is simply an initial summary of the authors’ findings of systematic reviews on homeopathy: it was never completed and has not undergone the usual NHMRC quality assurance steps (for publication). (…)
NHMRC strongly encourages interested members of the community to refer to the 2015 NHMRC Information Paper: Evidence of the effectiveness of homeopathy for treating health conditions. This states that ‘based on the assessment of the evidence of effectiveness of homeopathy, NHMRC concludes that there are no health conditions for which there is reliable evidence that homeopathy is effective.”
So the draft was the first – deficient – collection of material done by an external contractor – and no longer. The numerous comments of the NHMRC in the now published draft explain these deficiencies in detail. Ultimately, the accumulation of these shortcomings has been the deciding factor in abandoning and restarting the project. As you can see, it is basically an exaggeration to speak of a draft report at all.
This is simply the background to the “secret first draft/report”.
Science does not mean cherrypicking
What makes the DZVhÄ think to draw conclusions on homeopathic evidence from such a disordered and unevaluated conglomerate? Very simple – or even not. Just take the draft and use the proven method of “cherrypicking” to find passages from the preliminary assessments by the first contractor that seem to fit best. In addition, ignore the NHMRC’s numerous marginal comments, which explain why these classifications were not useful. The following illustration from a “homeopathic” tweet, which claimed to have “convicted” the NHMRC and discovered “evidence”, shows this visually in all clarity. One refers to the original statements in the first material collection and completely ignores (or sees in it a “suppression of the facts”) what the NHMRC noted in detail thereon. Color lottery: Black is valid, blue not, red is the color for cherrypicking?
So you reach into the trash of the NHMRC, pull out a crumpled note and think you can find in the original text “evidence” for misconduct, a “suppression” of valid results? To top it all off, you also ignore the accompanying remarks, which prove why the original statements were not usable? By the way, the systematic reviews compiled by the first contractor had all been known for a long time – nobody would have had to wait for the NHMRC to “suppress” them. Let’s remember: the draft (this time the right one) of the 2015 review was pre-published with the request to report publications not yet covered in it. This has indeed been done, the final review took into account a number of “post-notified” publications. More transparency? Nearly impossible.
One thing should be noted at this point. A systematic review is an overall view of many studies or – as here – of many previous reviews under methodical and statistically comparative evaluation of the work involved. The fact that studies / reviews with different results being included there is inherent in the system and precisely intended – it is a matter of the overall result. To select and uphold individual results with positive evidence, even if they are real, from the preparation of a review is an absurd reversal of the meaning of a review / a meta-analysis – namely the methodical determination of the “over-all evidence” just without “cherrypicking” – into its complete opposite. A review / a meta-analysis can only be viewed and criticized from the point of view of the final result. Even if one finds problematic evaluations of the studies / reviews included, the first question to be clarified is always whether they have any effect at all on the final result.
Solid evidence versus “encouraging evidence”
Here we come up to the term “encouraging evidence” again.To be seen in the screenshoat, the NHMRC rightly notes that the term “encouraging evidence” does not exist in the context of this study. Why not? Well, it does not exist at all as a term, it is simply not a scientific category, but a euphemism that the first contractor had inserted by “free fantasy”, so to speak. It is highly significant that precisely this passage, which convincingly proves the lack of qualification of the first contractors, is now also used as a positive term by the DDZVhÄ and widely quoted.
Prof. Anne Kelso, the general director of the NHMRC, becomes even clearer in her official statement on the publication of the draft (after all, this is the statement of the head of a high-ranking institution of the Australian government):
“As part of the process to develop the NHMRC Statement on Homeopathy, an initial contractor was engaged to review the scientific evidence on the effectiveness of homeopathy in treating a variety of clinical conditions with the aim of providing Australians with reliable information about its use. Thecontract was terminated in August 2012 with the mutual agreement of NHMRC and the contractor. The draft of the unfinished report, titled The Effectiveness of Homeopathy: an overview review of secondary evidence (i.e. the 2012 draft report), was not endorsed by the expert Homeopathy Working Committee.
A second reviewer was contracted and provided an overview of published systematic reviews and a review of submitted literature. The second contractor developed the NHMRC Information Paper: Evidence of the effectiveness of homeopathy for treating health conditions (published in March 2015 at the same time as the NHMRC Statement on Homeopathy).
The document I am releasing comprises the 2012 draft report in its entirety, with annotations that have been reviewed by the previous Chair of the Homeopathy Working Committee. These annotations have been made for context and to avoid misunderstanding of the draft report. ”
Truth or oppression?
Now one might still think that all this is just the next step in the suppression and manipulation of an alleged “truth”. One should only be aware that the threshold for the establishing of a veritable conspiracy theory would be finally crossed by that. What if the whole NHMRC review didn’t exist – what would that change? For the lack of evidence for homeopathy: nothing at all. There would be ten instead of eleven major systematic reviews to prove this.
But this is how one abuses the openness and transparency of the NHMRC: instead of disproving the conclusions of the 2015 review in a scientifically sound and reliable way (there has been sufficient time to do so), poking through the waste paper basket and searching for anything seeming useful für alleged evidence – for promoting a desired but not factual opinion.
It is significant that of all things the scientifically nonsensical term of an “encouraging evidence” is used for this as well. What this has to do with a scientific discussion about a medical question remains the secret of the DZVhÄ. Well, perhaps that’s the only way to keep in the debate a question being decided long ago at all – and to call for “more research” after more than 200 years without proof of efficacy. Isn’t that more in the direction of conspiracy theory, away from serious scientific discourse? We wish a good journey.
Previous publications of the INH on the NHMRC report:
Statement of the Homoeopathy Information Network on the HRI’s complaint about the Australian Ministry of Health (NHMRC) Review (in English available soon)
Open letter of the INH to the interview with Dr. Tournier (HRI) on “Homöopathie online”
Infinite Stories – once again to the Homeopathy Review of the NHMRC
5 Replies to “The “Suppressed First Report” of the NHMRC – Source of “Encouraging Evidence”?”
Comments are closed.