Homeopathy and the curse of the scientific method Pt1

I really wish I could find a free version of this journal article so that I could share it with you all – it is single the best piece of writing I have ever seen concerning homeopathy and the placebo effect.

Buried in the pages of The Veterinary Journal is an absolute treasure known as Overall (2009). In this article the authors first outline something very important: how to understand the role of bias in selecting populations for use in drug or medication trials. If a researcher approaches an individual to ask them to take part in a trial of a homeopathic medication, there are three main types of people a researcher can encounter, with respect to belief about the efficacy of homeopathy:

1. People who believe homeopathy is effective.
2. People who are unsure.
3. People who believe homeopathy is ineffective.

Of these three people, type 1 is the most likely type to agree to participate in a trial, especially one that is time consuming or involves a lot of work or effort on the part of the participant. When randomizing such a pool, it is highly unlikely that individuals in each respective pool will think (in this case) that homeopathy is ineffective. They are much more likely to believe homeopathy is effective. If they do not know much about homeopathy, they may assume that since this is not a trial to see if homeopathy works, but a trial to see if a particular homeopathic medication works, that homeopathy is therefore evidence based, and researchers are trying to prove that a particular remedy is effective. Either way, most people will be hoping and looking for positive results.

In animal trials in which owners are taking part in the evaluation of the behaviors of their own pets, they are likely to be quite biased or report different results based on their belief in the efficacy of the treatment. This is especially true if the changes in animal behavior are more subjective rather than objective. An owner who believes that a remedy is working may notice things like panting, drooling, or pacing less than an owner who has no such belief.

People often say that animals do not suffer from a placebo effect – that is, if you treat an animal with a medication it does not really know you are giving it something that should make it feel better. If the animal is in pain, it is not likely to feel less pain if you give it a sugar pill. This is basically true, but completely irrelevant, especially when observations of changes in animal behavior (indicative of an effect on the animal) are reported by people not trained in objectivity.

There is a problem in medical literature known as inter-rater reliability. Inter-rater reliability is the measure of different observers’ ability to report similar findings using the same measurement tool. A measurement tool is said to have good inter-rater reliability when several different individuals report very similar results. Inter-rater reliability relies on training in how to use the measurement tool and how to report findings objectively.

For example, several individuals grading a true or false quiz would have very high inter-rater reliability; that is several individuals, if given a key with the answers, will very likely give the same person the same grade. Inter-rater reliability may be reduced if the graders aren’t given an answer key and have to decide if the answers are correct based on their own knowledge. Inter-rater reliability could be extremely low if the tests are given to graders who have no answer key and varying levels of knowledge of the test material.

Similarly, trained investigators using a measure to observe the behaviors of animals may have high inter-rater reliability, whereas a trained investigator observing little Rover’s behaviors versus Mrs. MyDogIsMyChild observing Rover’s behaviors may have very low inter-rater reliability.

So, this is why we must use double blind placebo controlled testing for animals. The owners of animals are very likely to report their findings based on how much they believe the remedy will work. Just think about all of the times a dog owner reported that their dog “looked depressed because daddy went on a business trip”, or “pooped on the floor out of spite”. We’re very good at projecting emotions onto our animals that are not there.

The pet may not know that they are participating in an experiment, but the pet owner knows, and the pet owner is the one reporting perceived effects of treatment. This is why placebo-controls in veterinary medicine are as essential as they are in human medicine.

A placebo is pretty well defined by the Oxford English Dictionary: “A drug, medicine, therapy, etc., prescribed more for the psychological benefit to the patient of being given treatment than for any direct physiological effect; esp. one with no specific therapeutic effect on a patient’s condition, but believed by the patient to be therapeutic (and sometimes therefore effective). Also: a substance with no therapeutic effect used as a control in testing new drugs, etc.; a blank sample in a test”

Obviously, since the placebo effect can be so powerful, especially with respect to pain, anxiety, and other subjective issues, placebo trials must be conducted in order to separate remedies or medicines which affect physiology directly from remedies or medications which affect someone psychologically – it does mot matter if the pill is a sugar pill, a starch pill, a lactose pill or – dare I say – a homeopathic pill chemically identical to a sugar pill – the individual took a pill, and their belief that the pill will relieve their symptoms causes their symptoms to be relived. That’s a placebo.

Now, if a homeopathic, herbal, or X-type of medicine remedy has the same effects as a sugar pill, one can reasonably conclude that the remedy is inert physiologically. That is, it has no effect beyond the same effects seen in a sugar pill.

So, when companies market their remedies as being “80% effective”, but don’t tell you that the placebo arm of their study was also “80% effective”, people are being misled. Yet this is what homeopathic practitioners often do. In conventional medicine, if there is no difference between the effects of a placebo and the effects of a new type of medicine, then that medicine is discarded in favor of medicines in which there is an actual effect beyond the placebo effect. Complimentary and Alternative Medicine, on the other hand, seems to absolutely rely on the placebo effect to the point at which we may as well rename it to, “Complementary and Alternative Placebos” save for the fact that their effect will be diminished if people are aware they are being treated with placebo medicine.

Tomorrow, we discuss the importance of effect size!

Overall, K., Dunham, A., Homeopathy and the curse of the scientific method. The Veterinary Journal(2009)  180: 141-148
Online abstract