Many people have internalised a basic distrust of “scientific studies”. They consider that study results are worthless because they depend solely on the interests of the person who conducted and/or funded the study. However, very few people have an idea of the many and sophisticated instruments, criteria and processes in science – and thus also in medicine – by which it is judged what good and what bad studies are, where sources of error may lie or where actual errors may occur or studies only confirm a “self-fulfilling prophecy”.
Drug approval on a scientific basis
The statement “We know that drug XY works” is practically synonymous with the statement “There are scientific studies that show an effect (above placebo level)”. Science is a method by which such statements can be substantiated and the means to an end are studies.
Before a normal drug is approved for use on patients, it is first subjected to “theoretical” studies in pre-clinical studies to find out whether there are reasonable basic assumptions for a possible efficacy of the planned drug. Thus, it is first examined for “plausibility” according to chemical and physiological criteria. On the one hand, there are economic reasons (avoidance of bad investments), but on the other hand, the ethical aspect also weighs heavily here. Because the ethical principles in medicine demand that no people in clinical studies should be exposed to imprudent “quick shots”, according to the “try-and-error method” so to speak. Only then do the practical clinical studies follow. The first step is to find out how the drug to be investigated acts on the body (pharmacodynamics) and how it is distributed in the body and then metabolized (pharmacokinetics). It is already about side effects and above all about the right dosage. If results are obtained, it must be confirmed beyond doubt that the relationship between the administration of the drug and the therapeutic change is significant, i.e. not purely coincidental. It is usually compared with a placebo or another already established drug (the “standard treatment”). Only then may a drug be approved and used, whereby it continues to be observed “in the open field” whether and which side effects occur when used in large numbers of patients. Only in approx. 8% of cases (source: FDA 2004) does a drug overcome this process and may be used.
So much for the standard procedure within medicine. It is a highly complex and time-consuming procedure to approve a drug. But not all studies are of a similar nature. There are quite different varieties and not all allow the same statements. But everyone can have mistakes. Fortunately, in medicine, you usually know what they can be. A good guide through the study jungle can be found here.
And how’s homeopathy going?
Homeopaths report of “countless” studies that are supposed to reliably prove the superiority of homeopathy over placebo. How is that to be classified now?
“Successful” homeopathic studies are usually on the verge of significance (scientifically this means: on the verge of expected random results). Their validity is strongly dependent on the extent of their methodological-systemic deficiencies in the study design, but above all on the statistical peculiarities of the investigated facts. Furthermore, they often only show statistical artefacts (sometimes not so easily recognizable), i.e. banal, non-reproducible random results, are based on more or less skilful data manipulations or are simply methodically poor.
In pharmaceutical approval procedures, in addition to the scientific community itself, internal control institutions of evidence-based medicine and much other public and private institutions monitor the integrity of submitted study results, complain about misconduct and in some cases make this public. In the world of homeopathy, one looks in vain for control bodies that strictly question the effectiveness of homeopathic remedies according to the rules of evidence-based medicine. This makes it easy for the homeopathic scene to disseminate a certain view of the “study situation”, thereby ignoring negative results and – apparently – overinterpreting positive ones. There is also no question of mutual critical control within the homeopathic research community.
The sometimes difficult task of homeopathic criticism is to search for errors – in study design, in data evaluation, in interpretation (or all at once) – or, in the worst case, to prove conscious data manipulation. There is no doubt that these errors are there because so far every successful homeopathic study has proved to be faulty and untenable in some way. Why homeopaths nevertheless do not (do not want to) see these mistakes is incomprehensible to us; after all, they constantly claim “scientificness” for their method. Are they just not familiar with study interpretation? Are they so attached to their method that they are blind to the missing or erroneous evidence? How can they tell wrong things to their patients and followers without a guilty conscience?
There is a whole series of systematic reviews and meta-analyses of homeopathy, i.e. studies that systematically summarise individual studies. By systematizing individual results – and critically previewing their validity – these studies arrive at results that are inevitably much more valid than those of individual studies. This is why these reviews/meta-analyses also have the highest degree of evidence, the highest degree of scientific relevance, so to speak. Of the ten major reviews carried out since 1991, not one (!) comes to a robust proof of superiority over placebo in any homeopathic treatment. Neither are the reviews by homeopathic researchers. There is hardly any greater discrepancy between these results and the promises of homeopaths that homeopathy is a medically effective method with great practical benefit (“clinical relevance”) for the patient. Because what cannot prove superiority beyond placebo (placebo is everywhere) cannot claim to be recognized as “medicine”.
As a general rule, the following applies to all research: Individual positive studies or results (and even more so from research groups that have a great interest in positive results, i.e. are not independent) are not meaningful. It is important and crucial that results can be independently repeated (“reproduced”) and that they occur in a frequency that excludes random results.
All of this is critically focused every day in science – and even more so in medicine with its high relevance for human well-being. Science is constant criticism and self-criticism, that is part of its essence.
after all we haved learned here:
Who is reluctant to talk about negative study results, palliates by euphemistic representations of selectively selected study results the proven fact that homeopathic remedies are just placebos? Who lives from the alpha error or simply denies that homeopathy has not managed to prove superiority over placebo in 200 years? Right, the homeopaths.
Who criticizes at the same time with a great attitude that there are faults in studies of scientific medicine? Yes, also homeopaths. But they do not look for and criticize faults in themselves – they hold all this against the “orthodox medicine” they defame as such. Such is called whataboutism, also called the “hold the thief” principle. But this doesn’t change the state of the homeopathic studies in any way. In scientific medicine, there are the strictest mechanisms and methods to ensure the integrity of study results in order to avoid and correct errors. In homeopathy, none of that exists. And why isn’t there such a thing? Because the legislator – an absurd thing – has exempted homeopathy from a proof of efficacy based on approved scientific methods and studies …