“They couldn’t just say, this is nonsense, this is not the case,” he told Laura Kuenssberg in July 2021, and he wasn’t wrong. The claim can’t be debunked because it’s true. It is also misleading, because that truth comes with a boat load of caveats and context that take a long time to explain and understand. These explanations lose many listeners once they hear, ‘That’s technically true, but…’

Readers may be familiar with my views on the placebo effect. I’ve spent many years on my podcast Skeptics with a K, and within the pages of The Skeptic, trying to explain why claims like ‘the placebo effect is real and powerful’ don’t hold up. But it is also difficult to explain, because of the required nuance and context. Which ‘placebo effect’? For what condition? What does ‘real’ mean? These distinctions need to be unpacked and understood.

So let’s go back to basics.

Testing treatments

If we have a new drug we want to test, a simple approach might be: give it to a patient and see if they get better. This is flawed, of course, because they may have gotten better for some other reason. So instead of recruiting just one person for the test, maybe we recruit a few. We also don’t give all of them the new drug, just half, so we can compare how many in each half get better.

If we only involve very small numbers of people, maybe just a few in each group, we are still vulnerable to one or two patients recovering by fluke and making the new drug look effective when it is not. So, ideally, we would have lots of people in each group, the idea being that if we have enough people, fluke recoveries will be distributed between the groups and average out.

If we can choose which patients go into which group, then we might be tempted to put particular people into the test group, because we’re convinced the drug will help them. But if we put just very ill people in one group and not so ill people in the other, then again it might look like the drug works better or worse than it actually does. So we should assign patients to the groups randomly.

And if the patients and researchers know which group is getting the real drug, that can change the results too. Researchers might be more or less likely to report changes because they know what should be happening. Patients might stick more or less rigidly to the treatment plan.

And so we should also ensure that no one knows which group is which until after the study is finished. One way we can do this is with placebos, fake pills which look like the real thing, but which have no known effect on the disease or condition we are studying.

In the ideal case then, we would have many patients, a control group, patients randomly assigned to the groups, and no one knows which group is which. This is the randomised placebo-controlled trial, or RCT, widely considered the gold standard of medical research. We work hard to ensure both groups do the same things, except for that one thing we’re trying to measure, because then we can be confident that any differences between the groups are caused by the new treatment.

What’s fascinating, and superficially unexpected, is that patients in both groups will improve. Patients who do not get the real drug really do get better. And this can result in some observers forgetting that the purpose of the control group is to give us a standard for comparison. Instead they come to believe that maybe something about the very idea of a treatment makes patients better.

So let’s take a moment to understand the reasons why someone in the control group, someone who does not get any real medicine, might record an improvement in their condition regardless.

Why do symptoms change?

How the disease works

In the first case, we have the natural history of the disease. Many conditions are self-limiting and clear up by themselves eventually. If we were to design a study to test the effects of Caramac bars on a cold, and we did a clinical trial, after two weeks, everyone in the Caramac group would have recovered. Not because of anything to do with the Caramacs, just because colds get better after a week or so. So, if we imagine a pie chart where the total area represents the improvement observed in the control group, a slice of that pie is going to be the natural history of the disease.

It’s not about placebos, or belief, or expectation. It’s just how some diseases work.

Conditions even out over time

Then we have regression toward the mean. Some of the improvement observed will be the result of the natural fluctuation of symptoms, especially in chronic conditions. Particularly extreme observations, whether bad or good, are likely to be followed by an observation closer to the average, because extremes are rare by definition.

Crucially, patients will most commonly seek medical care during a flare-up, when their symptoms are at their peak. Which means that, in all likelihood, the very next thing to happen will be an improvement in their symptoms, simply due to regression. Again, this has nothing to do with placebos or the expectations of the patient, so it is another slice we can take from our pie chart.

People just get better

There is spontaneous improvement: sometimes patients just get better. The patient’s own immune system is a popular candidate for why, but sometimes this even happens in conditions where we don’t expect it to. Another slice to take out of the pie chart.

Simultaneous treatments

Then we have parallel interventions. One famous study, Beard 2018, investigated the effects of the removal of bone spurs and soft tissue, commonly promoted as a treatment for shoulder pain. Beard found the procedure was no more effective than placebo – but remarkably, the placebo (in this case, a fake operation) was superior to no treatment at all. This turns out to be because even the patients getting the fake operation were also given physiotherapy to help them recover from surgery, which could have improved their shoulder pain.

There are also unrecorded parallel interventions, which are far more insidious. This is where patients get some other treatment, but either don’t report it to the doctors running their trial, or if they do no one writes it down. It is much harder to know when this happens because, by definition, it is unrecorded – but one example might be Daniel Moerman’s 1983 paper on gastric ulcers.

Moerman reported that some studies showed a 90% cure rate for ulcers after the administration of a placebo, but other studies showed just a 10% cure rate. One possible cause for this wide variability might be because patients were inadvertently curing their ulcers with antibiotics they were taking for some unrelated reason. At the time, it was not understood that almost all gastric ulcers were caused by H. pylori bacteria, so patients’ antibiotic use was not recorded. Recorded and unrecorded parallel interventions are another slice we can take from the placebo effect pie.

We’re all biased

Then there are psychological effects. There are observer-expectancy effects, where desires and opinions of the researchers influence the recorded outcomes. Researchers are more likely to record the effects they expect to see and less likely to record effects they do not. Similarly, subject-expectancy effects are the same thing from the other side. Patients are more likely to notice and report effects they expect to see and are less likely to record effects they do not.

One example here is Blackwell 1972, where medical students were given pink or blue placebo pills and a list of effects they could expect to experience after taking them. So, of course, some reported those same effects! But would they have said anything if they hadn’t been told what to look for?

Crucially, these effects only change how the data is recorded, they do not change the actual condition of the patient. Another two slices to take out of the placebo pie.

Human behaviour is complex

Even just knowing they are taking part in a study is enough to change a patient’s behaviour. This is sometimes known as the Hawthorne Effect. Participants are aware that the things they do are being studied and recorded, so they may comply more tightly with standard care than they were before. They may eat a little better, or take a little more exercise, because they know that’s what they should be doing and the researchers are now checking up on them. One can easily imagine, for example, a patient in a study on asthma who is supposed to take a steroid-based inhaler daily, but is a little lax in taking it. Upon starting an asthma trial, they start taking it reliably because they know they should, and this can change their health outcomes. Another slice of the placebo pie gone.

Reporting biases mean that patients might be selective in what information they give to clinicians, perhaps through social pressure or embarrassment. A patient who regularly smokes ten cigarettes per day might report smoking only two or three because they are embarrassed by the true number. Over the course of the study, they cut down their smoking to match what they had initially reported. Patients may not regard this as a big deal, but any improvements resulting from that change in behaviour would appear in the data to be apparently spontaneous. The fact they changed their smoking habits during the study is not recorded anywhere, and if that patient happens to be in the placebo wing of a trial, this becomes data supporting a powerful placebo narrative. In fact, it was just a patient who was ashamed of their smoking. Another slice gone.

Some patients may exaggerate their symptoms at the start of the study, even unconsciously. Perhaps they’re at the end of their tether and trying to impress upon the doctors how difficult they’re finding things. There are also patients who will under-report their symptoms at the end of the study, because they think they should be better. They’ve taken the medicine, they don’t want to upset or annoy their doctor, so they report what they think they should.

That can even be in a small, subtle, way like reporting a five on a pain scale instead of a six. They may not even realise they’re doing it. But it’s another effect that makes it appear like a fake treatment is improving the patient’s condition.

Our memories are bad

Following on from this, there is a bias of recall. Patients who mis-remember how bad they felt at the start of the study, may report an exaggerated improvement at the end of the study. The patient may remember they had scored themselves as a six on the pain scale at the start, and under the belief they now feel better, report a five at the end. Except they actually don’t feel better, they’re just misremembering how they felt before. So now they’re recorded in the data as an improvement, when actually nothing has changed.

Along similar lines is enrollment bias, where doctors themselves exaggerate or misrepresent a patient’s condition to ensure they qualify for a study. In one trial investigating benign prostatic hyperplasia, researchers determined that a minimum prostate volume of 30cc was required for enrollment. However, patients recorded by their doctor as having 30cc prostates were discovered to have volumes of 27-29cc when the first follow-up measurements were taken. This created an apparent placebo effect, as patients without the active treatment seemed to immediately improve at the start of the trial, before starting to slowly deteriorate. The improvement was entirely artificial, created by the difference between exaggerated baseline measurements and accurate follow-up.

In later studies, researchers introduced a one-month lead-in period for all patients before taking baseline measurements. This eliminated the enrollment bias, and this apparent placebo effect vanished. Patients never were getting better in the placebo group, the baseline data was simply wrong. But without understanding this bias the improvements appear to be evidence of a powerful placebo.

So where is the placebo effect?

None of these effects are actually clinically relevant. None of them represent a patient actually getting any better. They are unreliable, non-specific, and often illusory effects which we should be carefully controlling for, not celebrating as ‘the amazing power of placebo’. These effects fool with our data and interpreting them improperly will lead us to inaccurate conclusions.

If biases, measurement errors, conditioning, and other confounding effects were able to be independently controlled for, if we were able to measure and subtract all of them from our placebo pie chart… how much pie is left? That’s the million-dollar question. How much effect is there after we have taken out the boilerplate and biases and cruft?

How large is this ‘true’ placebo effect, the seemingly magical effect that results from your body reacting to your mind’s insistence that you have taken some medicine.

My guess is that the answer is zero. My guess is there’s no placebo pie left and those positing a true powerful placebo effect are effectively making a ‘placebo of the gaps’ argument. So when I say ‘the placebo effect is not real’, that is what I mean and I’ve yet to see any data that persuades me otherwise.

I would just find it hard to fit all that on the side of a bus.

This article was updated on October 1st 2025 to improve the explanation of regression toward the mean.

The post If we take away the statistical quirks and biases, is there any placebo effect left? appeared first on The Skeptic.

Proponents of this narrative are often light on detail for how this might actually work, invoking instead the wishy-washy sounding ‘mind-body healing process’, or something similar. On rare occasions they might invoke endorphins or dopamine but, while these chemicals can be produced in response to psychological changes, they have a limited range of effects.

An alternative interpretation of the same observations says that placebo effects are mostly made up of statistical effects and biased reporting. Convince a patient they have taken a drug, and they tell you they are experiencing the effects of that drug, which is not the same thing as actually experiencing it. What happened and what the patient says happened are obviously related, but patient reports are also influenced by the biases and opinions of the clinician and the patient themselves.

Psychological factors like the Subject Expectancy Effect can mean that patients report what they think should be happening, rather than what is happening. Demand Characteristics can mean that patients report what they think their doctor wants to hear. Even simple politeness can result in misleading answers coming from otherwise well-intentioned patients, who don’t want to upset or disappoint the researchers.

Beyond this, we can also recognise that some fraction of patients will see an improvement anyway, no matter what you do. Many medical conditions will run their course and spontaneously resolve. Other conditions wax and wane, and patients who join a trial when their symptoms are at their worst will naturally improve regardless of the intervention.

One researcher even contacted The Skeptic to highlight cases where doctors have exaggerated the severity of their patient’s condition to ensure they meet the eligibility criteria of a trial. Which means those patients appear to make an immediate and miraculous improvement, regardless of whether they get real medicine or a sham control.

If we view the placebo effect as the bucket into which we toss our biases and other contextual effects, there is no need to invoke a mysterious mind-body healing process, or decree that the placebo effect is one of the strongest medical responses there is.

Over the past decade or so, numerous media reports have outlined how the placebo effect is somehow getting more powerful. For example, the gap between the effectiveness of painkillers and placebos in clinical trials has narrowed significantly since the 1990s. In one report from 2015, a research team led by Alexander Tuttle found that the ‘treatment advantage’ (the improvement from the active treatment over and above the placebo group) had diminished from 27% in 1996, to just 9% by 2013. This reduction was driven by an increase in the placebo response; as the mysterious placebo effect grows in strength, drugs are struggling to compete.

The phenomenon itself is real enough. Tuttle is not the only researcher to have documented that placebo responses in trials appear to be increasing over time, particularly for subjective outcomes. The question is not whether this trend exists, but how we interpret it.

All in the interpretation

Under the standard mind-over-matter narrative, this is a strange and mysterious thing. Why should placebos work better today than they did 20 years ago? Is it because we have greater faith in doctors and medical science? Is it because of television advertising, promoting how powerful and effective drugs are? Somehow, this makes dummy pills more effective painkillers than they used to be?

Perhaps a more parsimonious explanation is this: placebo responses have not increased because of any therapeutic effect, but because trials have become better at isolating treatment effects from noise. As methodological standards improve, non-specific effects that previously leaked into the treatment arm are more accurately contained within the control group.

Or to put it another way, the treatment advantage was always 9% and when we measured it at 27% in the past we were in error. Our trials at time were not sufficiently well designed or conducted to offer accurate results.

These competing interpretations are not merely academic, since each generates distinct empirical predictions. If placebo effects were genuinely becoming more powerful, we would expect to see gains not only in control groups but also in groups receiving the active treatment. Since any drug effect adds to the placebo effect, both arms should benefit from a growing placebo response. The rising tide lifts all boats, as it were.

However, if placebo responses are increasing due to improved trial methodology, the overall effect would remain unchanged and the gap between treatment and placebo would narrow. Non-specific effects that had previously inflated the apparent treatment effect are now correctly controlled for.

When researchers have examined how placebo and treatment responses have changed over time, the results align better with this second explanation. Tuttle analysed US trials of neuropathic pain and found that, while placebo responses increased, outcomes in active treatment arms did not. A similar pattern emerged in antidepressant trials. These findings are difficult to reconcile with the idea of an increasingly powerful placebo effect.

The geographic distribution of this effect provides further evidence. Tuttle reported that the increased placebo response is most pronounced in trials conducted in the United States, while trials elsewhere have shown no comparable trend. Crucially, these US-based trials also tend to have longer durations and larger sample sizes, features associated with greater methodological rigour.

Proponents of the powerful placebo hypothesis have attributed the US-specific increase to cultural factors, such as direct-to-consumer drug advertising. But this explanation falls apart when we consider that New Zealand (the only other country permitting such advertising) has not reported similar increases in placebo responses.

What’s the harm?

Modern trials are longer, larger, better blinded, and more rigorously monitored than their predecessors. They employ more sophisticated randomisation techniques, standardised outcome measures, and stricter protocols for handling dropouts and protocol violations. These improvements serve to isolate the specific effects of interventions from the many confounding factors that can masquerade as treatment benefits.

When the gap between treatment and control narrows in modern trials, the correct interpretation is not that placebos have grown stronger, but that we have grown better at distinguishing signal from noise.

The misinterpretation of rising placebo responses as evidence of therapeutic potential carries real risks. Some trial sponsors have begun relocating studies to regions where placebo responses tend to be lower. Ostensibly this is for economic reasons, but it also serves to effectively avoid the methodological rigour that higher placebo responses represent. 

This trend should be deeply troubling. We do not obtain better evidence by weakening our controls to maximise the chance of a statistically significant result. Rather than celebrating improved methodology that better isolates true treatment effects, the pharmaceutical industry risks undermining the very advances that make modern trials more reliable than their predecessors.

The post No, placebos probably aren’t getting stronger over time appeared first on The Skeptic.

Naturally, this surge in illness leads to renewed interest in how to treat a cold effectively, but despite advances in many fields of medical care over recent centuries, the common cold remains stubbornly incurable. Instead of a definitive cure, we’re left managing symptoms to ease discomfort.

So as we leave winter behind, I want to revisit a BBC Future article that was updated and republished at the start of the season, examining the evidence behind common home remedies for treating colds.

BBC Future is part of the BBC’s international online service, covering science, technology, environment, and health. They position themselves as a source of truth, facts, and science – an approach I fully support. So, what do they have to say about the evidence behind home remedies?

Immune supplements

The article begins by noting that many home remedies focus on the idea of boosting the immune system, also noting that for otherwise healthy individuals, immune function is only impaired when there’s a deficiency in essential vitamins or minerals. If your diet is already well balanced, supplements offer little benefit. It’s a valid point – despite the claims of supplement pedlars, supplements won’t supercharge an already healthy immune system. It then goes on to discuss a specific piece of research on this, a pilot study published in PLoS One in 2020.

The study involved 259 participants who were randomly assigned to receive either a supplement (containing vitamins A, D, C, E, B6, B12, folic acid, zinc, selenium, copper, and iron) or a placebo. Over 12 weeks, participants completed weekly surveys tracking any cold symptoms. The results indicated fewer runny noses and fewer coughs among those taking supplements, concluding that this low-cost intervention merits further investigation.

Given the context already discussed (supplements aren’t expected to benefit otherwise healthy individuals without a vitamin or mineral deficiency) one assumes that some fraction of the cohort was mildly deficient in some nutrients, so the supplements here brought their immune function back up to par, reducing the incidence of colds. It’s an interesting finding, but there are a few issues.

The study had a high drop-out rate, with nearly 50% of participants failing to complete the weekly surveys. More concerningly, it did not account for multiple comparisons – a crucial flaw in scientific research. 

Studies commonly use p-values to assess whether results are statistically meaningful or just due to chance. The typical threshold is p<0.05, meaning there’s only a 5% chance of observing results like these (or better) if there’s no real effect. However, when multiple outcomes are tested, the likelihood of finding at least one significant result by chance increases. It’s like playing dice: the probability of rolling a six on one attempt is low, but if you roll 20 times, the odds of getting at least one six rise to 97%.

Many studies fail to properly adjust for this, and this pilot study is no exception. The reported improvements for runny noses (p=0.01) and coughs (p=0.04) only hold for a single comparison, but the study also examined the incidence, duration, and severity of headaches, sore throats, congestion, aches, and fever. With so many comparisons, the probability of finding a significant result purely by chance increases dramatically. Just as you can’t roll 20 dice, pick up a six and demand an extra turn, you can’t do 20 comparisons in your study and then talk about how there is only a 1% chance of getting these findings if the supplement does nothing.

While there are a handful of significant effects here, they may well just be noise in the data given the number of comparisons made, and a simple statistical correction for the multiple comparisons eliminates these findings entirely.

To their credit, the authors of the paper acknowledge that the results are not conclusive, calling for more rigorous research. However, these larger, more robust trials have yet to be conducted. Given this, one could argue that it’s premature for BBC Future to reference this paper in an article on the effectiveness of supplements.

Garlic

Next their attention turns to garlic, referencing a 2001 study in which 146 volunteers took either a garlic supplement or a placebo for 12 weeks. The results were striking: the garlic group reported significantly fewer colds; just 24 cases compared to 65 in the placebo group. Cold duration was also notably shorter, averaging just over one day in the garlic group vs five days in the placebo group. The paper even goes so far as to claim that ‘the supplement studied may represent a cure for the common cold.’

These results are so extreme as to be absurd. Are we really expected to believe that participants taking garlic experienced colds lasting only one and a half days? If garlic genuinely cured colds, wouldn’t we have recognised its effects long before 2001? After all, people were chewing willow bark for pain relief for thousands of years before salicin was refined into aspirin. And if we really had discovered that garlic was the cure for the common cold in 2001, where are the Nobel Prizes? Why are we all still getting colds, a quarter of a century later?

Beyond the implausibility of its results, the study also had serious design flaws. The placebo was not taste-matched to the supplement, meaning participants could likely tell which group they were in. This introduces a major source of bias. Another red flag is the author’s claim that garlic ‘may represent a cure’ for colds, since this study wasn’t designed to test garlic as a cure but rather as a preventative, a fundamental difference.

Perhaps the most immediate explanation for these extreme findings lies in the study’s authorship. The lead researcher is the owner of The Garlic Centre, a business that sells garlic supplements – a clear conflict of interest. A 2014 Cochrane review found no reliable support for garlic’s effectiveness against colds, and characterised this paper as ‘poor quality.’ Yet, BBC Future chose to highlight this flawed study as evidence.

Vitamin C and Zinc

In the next section, BBC Future discusses vitamin C and zinc as potential ways to shorten the duration of colds. For vitamin C, they cite two meta-analyses: one that found vitamin C reduces the severity of cold symptoms by around 15%, and a second which they claim suggests that vitamin C supplements are low risk, so there’s no harm in trying them. 

However, they fail to mention that this second analysis also found no positive effect for vitamin C. While it does conclude that vitamin C is generally safe, it does not support the claim that it reduces the duration or severity of colds – an omission that I would argue misrepresents the evidence.

The pattern repeats with zinc. BBC Future cites one review which suggests that zinc shortens the duration of runny and blocked noses while also reducing coughing and sneezing. However, a second paper they reference found no such benefit. In fact, some measures in this second paper showed participants in the placebo group do better than those taking zinc, suggesting that in some cases zinc may even be counterproductive. In fairness, this time BBC Future does point this out in their coverage.

Also to their credit, they highlight a key limitation in this type of research: studies rarely, if ever, test whether participants are deficient in vitamin C or zinc before supplementation begins. Any observed benefit could simply be due to correcting an undiagnosed deficiency rather than proving that these supplements provide a meaningful advantage for already healthy individuals. So fair play to BBC Future for recognising that nuance.

What really bothers me is that nearly all the papers on zinc and vitamin C cited by BBC Future come from the same author. Harri Hemilä, a professor at the University of Helsinki, appears to be a strong advocate for vitamin C megadosing and zinc supplementation. Nearly all of his recent publications focus on the supposed benefits of these supplements – not just for colds, but also for Covid, pneumonia, cardiovascular disease, sepsis, asthma, and more.

When he’s not promoting vitamin C and zinc, he’s criticising studies that fail to find positive results. He even authored a paper accusing mainstream medicine of bias against vitamin C, insisting that the evidence in its favour is unambiguously positive but unfairly dismissed. Given this, relying so heavily on his work without acknowledging his fringe stance on the subject is a significant oversight from BBC Future.

Skeptics often criticise false balance in journalism, where fringe and mainstream science are presented as if they hold equal weight. But here, we see almost the opposite problem. Even if we accept that there is some scientific debate about the efficacy of vitamin C megadosing, BBC Future leans overwhelmingly on data from a single researcher – one who openly acknowledges that mainstream science does not support megadosing as an effective intervention.

The placebo effect

Perhaps the most frustrating part of the article is its discussion of the placebo effect. As I’ve outlined in detail, the evidence for any real therapeutic placebo effect is scant at best. The placebo effect excels at manipulating people into reporting large health improvements, but in cases where we can objectively measure purported improvements, we typically find no actual benefit.

With respect to the common cold, BBC Future cites a 2011 study in which 719 participants who had only just caught a cold were randomly assigned to one of four groups. The first group received echinacea, a popular remedy claimed to help with colds, and knew that’s what it was. The second group also received echinacea, but they were not told it was echinacea, making this a blinded echinacea group. The third group received a placebo but believed they were taking echinacea. The final group received no treatment at all.

The primary outcome measures in the study were illness duration and illness severity. Duration was assessed by asking participants if they believe they still have a cold. The number of days they answered ‘yes’ determined the total duration. Illness severity was measured using a standardised questionnaire – a self-reported and subjective assessment completed by the participants twice a day.

Beyond these, the study collected a range of other data points, including stress levels, general health, and an open-ended question allowing participants to report side effects such as diarrhoea, headaches, nausea, rash, and upset stomach. There were also two objective measures: interleukin-8 concentration, which indicates immune response, and neutrophil counts, which reflect inflammation levels. Finally, researchers asked participants whether they believed echinacea works or not.

For readers who may be unfamiliar, echinacea is a popular alternative remedy derived from a type of daisy. While it has been used for centuries to treat colds, there is no good evidence to support its effectiveness in any medical context. However, it is known to interact dangerously with some medications, making its use potentially risky.

BBC Future reported this study as showing a placebo effect, where participants who believed in echinacea experienced shorter and milder colds than those who did not. They further note that this pattern held regardless of whether participants had actually taken any echinacea at all.

However, these findings rely entirely on self-reported data, where patient bias can easily influence results. People who believe they have received an effective treatment are more likely to report an improvement, even if no real improvement has occurred. This is a well-documented issue with subjective measures in clinical research and often overlooked in placebo effect research.

When looking at the objective data – the inflammation and immune response markers – there was no effect at all, either from echinacea itself or from belief in echinacea. Even the subjective findings disappear once a statistical adjustment is made for the large number of comparisons in the study, further undermining the claim that belief in echinacea had any measurable impact.

Nevertheless, BBC Future goes on to cite a second study in support of the first. This second paper, published in 2010, shows that patients who are unaware if they are getting echinacea do not report improved cold symptoms. This may appear to reinforce the placebo effect narrative: participants in the first study who believed in echinacea reported shorter and less severe colds, but those in the second study, who lacked that belief, did not report the same.

However, upon reading this second paper, I experienced a strong sense of déjà vu. It involved 719 participants who had only just caught a cold being randomly assigned to one of four groups – echinacea, blinded echinacea, placebo, and no treatment. It also shared almost all the same authors as the first study. 

Because it turns out that this paper is actually the same dataset. The same patients, documenting the same colds – just published in a different journal eight months earlier. In this earlier version, they found no effect. This earlier paper concluded that illness duration and severity were not significantly affected by echinacea compared to the control groups.

Dissatisfied with these findings, the authors appear to have revisited the raw data and conducted subgroup analyses based on whether participants believed in echinacea. They then published these results in a different journal, presenting them as a separate study all about the placebo effect. Notably, the question about belief in echinacea does not get mentioned in the earlier paper, nor is it mentioned in the trial registration.

On the face of it, this appears to be a case of p-hacking, where researchers manipulate data analysis to produce statistically significant results. This is often done by testing multiple hypotheses, selectively reporting favourable results, or adjusting statistical methods until a desired finding emerges. While usually done with the best of intentions, p-hacking undermines the reliability of scientific research by increasing the likelihood of false positives.

Trial registration is a tool which is designed to combat this, keeping researchers honest about what questions they will ask, and what analyses they will perform. In this case, however, it appears to have fallen down; the second paper with the modified analysis simply does not mention that the registration exists.

BBC Future appears to be a genuine effort to present good science, but in this case, it fell short. Rather than offering a careful, critical analysis, the article relied on flawed studies, overlooked biases, and misrepresented key concepts. 

Science journalism should do more than just report findings – it should question them. A more skeptical approach would have provided a clearer, more accurate picture of the evidence. Perhaps next time, BBC Future will live up to its stated mission and truly delve deeper.

The post Supplements and garlic won’t cure the common cold, despite what the BBC say appeared first on The Skeptic.

According to the story, while working at an army field hospital, Beecher faced a critical shortage of morphine for wounded soldiers in need of urgent pain relief. In desperation, he administered injections of saline (salt water) instead. To his astonishment, the soldiers responded as though they had received actual morphine. This remarkable observation sparked Beecher’s interest in the placebo effect and initiated decades of research into its extraordinary power.

The most extraordinary thing about this story, however, is that it probably never happened.

In 2015, the writer Shannon Harvey spent some time reading through Beecher’s published work and even contacted the library at Harvard, which holds his private archives. Harvey discovered that, while Beecher wrote extensively about the placebo effect, the story about running out of morphine does not appear in any of Beecher’s public or private writings. Nor does it appear in his 1976 obituary in the New York Times.

Several years later, science communicator Jonathan Jarry and I tried tracing the origins of this tale. Jarry documented the numerous variations he encountered. In some versions, the events are set in North Africa; in others, they occur on a Pacific island or in Italy. Some accounts involve a nurse mistakenly administering saline, with the observant and guileful Beecher taking note. Others depict Beecher himself administering the saline. In one telling, Beecher even hands out cigarettes instead of salt water.

Curiously, the earliest version of the story we found wasn’t even about Henry Beecher. It appeared in a 1978 episode of the TV series M*A*S*H, in which wounded soldiers are given powdered sugar cribbed from the tops of doughnuts after a supply of morphine is accidentally contaminated and fresh stocks won’t arrive until morning.

When Jarry spoke with the writer of that episode, he learned that the plot had been provided by the series producer, Gene Reynolds. Where Reynolds got the idea from remains a mystery. He passed away in 2020.

The closest parallel we found in Beecher’s work comes from a 1946 paper, titled ‘Pain in Men Wounded in Battle’. In it, Beecher criticises the rote use of morphine on the battlefield and argues that other factors, such as anxiety, traumatic shock, or even simple thirst, are often the immediate cause of a wounded soldier’s distress. He recounts the case of a 19-year-old soldier who was severely injured and ‘wild with pain.’ The soldier was mistakenly given a sedative instead of morphine, which it turned out was sufficient to calm him down and allow him to sleep, without the need for pain relief. Beecher notes, ‘the dose [of sedative] given would not have controlled pain’ and concludes ‘his manic state was not due to pain.’ 

Regardless of whether the legend is true, Beecher played a pivotal role in highlighting the importance of the placebo effect in clinical research. This year marks the 70th anniversary of the publication of his landmark paper, ‘The Powerful Placebo,’ which found that 35% of patients in clinical trials improve after the administration of a placebo alone.

This appears to be the origin of the widely circulated medical truism that 30% of the effect of any drug is placebo, but these are fundamentally different claims. Saying that ‘30% of patients improve after a placebo’ is not the same as stating that ‘30% of a drug’s effect is attributable to placebo.’ This distinction is overlooked, and the latter interpretation has become the commonly cited version.

Beecher derived this statistic by examining fifteen studies which reported placebo effects, but the specifics of these studies raise questions about his methods and the legitimacy of this figure. For one thing, Beecher claims that the fifteen studies were ‘chosen at random’, but almost half of them were authored or co-authored by Beecher himself. This somewhat undermines the claim of random selection and suggests the possibility of a selection bias.

Also, none of the fifteen studies Beecher analysed were explicitly designed to investigate the placebo effect. Consequently, Beecher attributes all improvements in the control groups to the placebo effect, ignoring other possible explanations.

For example, in a 1933 study on the common cold, patients are reported as showing an improvement a couple of days after receiving a placebo, but six days after the onset of symptoms. Beecher attributes this improvement to the placebo effect, disregarding the fact that many colds will naturally improve within that timeframe — a fact even noted in the original paper! This failure to account for the natural course of an illness will have inflated the apparent placebo effect.

He also overlooks the influence of parallel interventions, where patients receive some additional treatment alongside the placebo that influences their outcomes. A patient who is coincidentally taking penicillin for an unrelated condition might see their improvement attributed to the placebo when the penicillin was actually responsible. Despite some studies explicitly mentioning such additional treatments, Beecher still credits all observed improvements to the placebo effect.

Another oversight was conditional switching of treatments. In this 1933 paper on angina, which is referenced in ‘The Powerful Placebo,’ patients in the placebo group were switched to the treatment group if their condition deteriorated. They were switched back to placebo once they were stable again. This practice exaggerates the apparent placebo effect, as patients are only permitted to remain in the placebo group for as long as they are improving.

In 1997, researchers Kienle & Kiene revisited the original fifteen papers cited in ‘The Powerful Placebo,’ and concluded that not a single one of them presented any compelling evidence for a real, therapeutic, placebo effect. They identified numerous unaddressed factors that could create the illusion of a placebo effect, including the natural progression of illness, parallel interventions, and conditional switching of treatment, as well as regression to the mean, observer bias, and answers of politeness. Perhaps the most damaging, however, is Beecher’s frequent misquoting of data. Reporting on a 1954 study on coughs by Gravenstein, Beecher claims that 36% of 22 patients showed an improvement with a placebo. However, Gravenstein does not have a placebo group of 22 patients, or any figure reported as 36%. Gravenstein even remarks that it was ‘not possible to answer the question’ of placebo effectiveness, as patients could not be studied without medication for any extended period.

This is not the only example of misquotation or even the most egregious. According to Kienle & Kiene, Beecher also ‘cited as a percentage of patients what in the original publications is referred to as something completely different, such as the number of pills given, the percentage of days treated, the amount of gas applied in an experimental setting, or the frequency of coughs after irritating a patient.’

Despite its reputation, nothing in ‘The Powerful Placebo’ demonstrates a convincing effect which could only be explained by a real, therapeutic placebo effect. Yet it remains one of, if not the, most cited paper in the placebo effect literature. This prompted Kienle & Kiene to comment that something about the placebo topic invites ‘sloppy methodological thinking.’

The same errors made by Beecher are still being made in the modern placebo effect literature. Correlation (the patient took a fake pill and improved) is mistaken for causation (the improvement was prompted by the fake pill.) Confounding effects like disease progression, parallel interventions, and regression to the mean are ignored.

Placebos have a crucial and perhaps irreplaceable role in medical research as a control, but that doesn’t mean they have a role in clinical care. After 70 years, perhaps it is past time we put the myth of the Powerful Placebo behind us.

The post The Beecher story, the origin of the placebo effect myth, likely didn’t happen appeared first on The Skeptic.

This isn’t to suggest that the mind holds no power over the body. Our minds constantly influence our physical state; I can stand up and move simply by thinking about it. However, this does not imply that the mind’s ability to affect the body is boundless or largely unexplored. The contexts in which our minds exert control over our bodies are limited and typically mediated through nervous signalling or hormonal control.

When discussing the nocebo effect, the same handful of studies are frequently mentioned. One of the most prominent is the case study of the ‘placebo overdose’, published in General Hospital Psychiatry in 2007. In this paper, Reeves et al report on a 26-year-old man (referred to as Mr A) who arrived at the emergency room exclaiming, “Help me! I took all my pills”, before collapsing. Reeves notes that Mr A exhibited ‘rapid respirations’, appeared ‘drowsy and lethargic’, had a high heart rate (110 bpm), and very low blood pressure (80/40).

Doctors found an empty bottle of pills on him, labelled as part of a clinical trial for an experimental antidepressant. These were the pills that Mr A had taken; he denied taking anything else. Blood tests were conducted, revealing no evidence of paracetamol or aspirin overdose, and drug screening was negative. All other lab results were within normal range, including blood urea, nitrogen, creatinine, and electrolytes. The ER doctors gave intravenous saline to try and stabilise his blood pressure and, although it helped somewhat, his blood pressure remained low.

Four hours later, a doctor who was administering the drug trial arrived. He informed the attending physicians that Mr A was in the placebo group, and that the pills he had taken were fakes. Upon hearing this news, Mr A expressed surprise and ‘almost tearful’ relief. Within 15 minutes, he was fully alert, his blood pressure had stabilised at 126/80, and his pulse was 80.

For those worried for Mr A’s wellbeing, he was admitted to the psychiatric unit, and diagnosed with depression and dependent personality disorder. He was prescribed sertraline and psychotherapy, to which he responded well.

At first glance, the story of the ‘placebo overdose’ seems compelling. It also appears to directly contradict my view that placebo effects are generally limited to subjective reports, largely driven by things like confirmation bias and the subject expectancy effect. The Cochrane Review on placebo effects shares this stance: placebo effects do not manifest with objective symptoms.

In Mr A’s case, however, his symptoms were undeniably objective. His blood pressure and heart rate were measured, not reported. So, is this really a case of ‘placebos making you sick’?

Don’t panic

The term ‘nocebo effect’ in this context is nebulous and mysterious: Mr A thought he was going to die, and so his body started to. When you die in the Matrix you die in real life; a demonstration of how the mind can affect the body. The mystery rather vanishes, however, when we simply reframe Mr A’s experience as a panic attack.

Mr A had a history with depression, having been prescribed amitriptyline when he was 22 years old. He stopped taking the medication shortly afterward, as he found the side effects intolerable.

In the weeks leading up to his admission to the ER, Mr A was again struggling with depression. During a particularly low and hopeless ebb, Mr A saw an advertisement for a clinical trial and decided to enrol, hopeful (one supposes) that this new antidepressant would result in fewer side effects than the amitriptyline.

During the first month of the trial, Mr A’s condition improved, but at the start of the second month, following an argument, he impulsively took all 29 remaining pills. He quickly regretted this and begged a neighbour to take him to hospital.

As panic set in, Mr A would start to hyperventilate, described by Reeves as ‘rapid respirations’. Hyperventilation increases blood oxygen but reduces blood carbon dioxide, leading to respiratory alkalosis, where blood pH rises. (This is also why cartoon characters breathe into a paper bag to calm down; it is a simple mechanism to prevent respiratory alkalosis by breathing more carbon dioxide!)

Respiratory alkalosis will cause peripheral vasodilation, an expansion of the blood vessels outside the heart, leading to a significant drop in blood pressure. Essentially, the same blood volume is now within a larger space, lowering the pressure. This also explains why Mr A’s lab results were normal, including for markers like creatinine and blood urea nitrogen. If, for example, the low blood pressure was caused by dehydration, these markers would be elevated. The intravenous fluids also had limited effect as the underlying issue was not blood volume depletion.

In response to the drop in blood pressure, Mr A’s heart would start to beat rapidly to try and compensate. This also then accounts for Mr A’s rapid heart rate, described by Reeves. Respiratory alkalosis will also induce constriction of the cerebral blood vessels, accounting for the reported drowsiness and lethargy, as well as his collapse. As Mr A experienced these symptoms, he likely interpreted them as signs of impending death, further reinforcing and exacerbating his state of panic.

Upon hearing the news that he had only taken placebos, Mr A would have calmed down. Once he stopped hyperventilating, his blood pH and blood pressure would quickly have returned to normal.

Perhaps it is reasonable to characterise this as a ‘nocebo effect’ or a ‘placebo overdose’, but I would argue that to do so is to lend undeserved credence to the power of the placebo. Mr A’s condition wasn’t the result of placebo administration. The objective symptoms reported were the result of hyperventilation, through well-understood and documented mechanisms. Scratching our chins and nodding sagely about the amazing power of the mind doesn’t explain anything, doesn’t reveal anything, and certainly doesn’t help people like Mr A.

Maybe a more valuable lesson we can take away from this case study is that panic attacks can be far more serious that we think, and those suffering them don’t just need to ‘pull themselves together’. And perhaps if Mr A’s doctors had recognised his condition for what it was sooner, he would have been spared those hours of pain, distress, and discomfort.

The post An overdose on placebo pills can cause adverse reactions… but not because of the nocebo effect appeared first on The Skeptic.

I remain unconvinced that there is any real, meaningful, clinical effect that can be described as a ‘placebo effect’. The improvements observed in the placebo groups of clinical trials can be sufficiently explained by several known factors, including statistical effects, psychological influences, normal immune responses, external factors (like taking an unrelated medication at the same time), and even just straightforward mistakes.

In 2018, the BBC’s Horizon series aired a documentary titled The Placebo Experiment: Can My Brain Cure My Body? Presented by the late Michael Mosley, the programme made several claims of seemingly miraculous improvements attributed to the placebo effect. For this article, we are concerned with just one of these claims: the effects of placebo surgery.

In a drug trial, we could typically take a group of patients, randomly assign each patient to get either the drug or a placebo, and then examine the differences in health outcomes between the two groups. In a placebo-controlled surgical trial, the process is much the same. Patients suffering with some condition (for example, osteoarthritis) are recruited and randomly assigned to get either surgery or sham surgery.

Patients in the surgery group are given the real surgical procedure, as per usual practice. Patients in the sham group are prepped as normal, taken to theatre as normal, and anaesthetised or sedated as normal. Incisions are still made and for an arthroscopic (key hole) procedure, the scope is even inserted as normal. After a simulated surgery, patients are sewn up and sent to recover, without any meaningful surgical intervention having taken place.

For Horizon, Michael Mosley spoke with a surgeon named Andrew Carr, attached to Oxford University, who took part in such a study. The operation, known as an acromioplasty, involved the removal of soft tissue and bone spurs from the shoulder, in the expectation that it would relieve pain. In the sham condition, incisions were made and a scope was inserted, but no material was removed.

This study was published by Beard et al in The Lancet in 2018, and ultimately involved over 300 patients and 51 surgeons. It concluded that there was no significant difference in pain relief or functional improvement between the surgery and sham groups.

Several years earlier, the New England Journal of Medicine had published a similar study. Moseley et al (no relation) took 180 patients suffering with osteoarthritis of the knee and randomised them to get either arthroscopic débridement (the removal of damaged tissue), arthroscopic lavage (flushing with water), or sham surgery.

Moseley reported: “at no point did either of the intervention groups report less pain or better function than the placebo group […] the outcomes after arthroscopic lavage or arthroscopic débridement were no better than those after a placebo procedure.”

The straightforward interpretation of these studies is that, since surgery failed to outperform sham in both cases, the procedures are ineffective. Arthroscopic débridement and lavage do not treat osteoarthritis of the knee. Acromioplasty does not alleviate shoulder pain. These operations should therefore be discontinued as they do not provide a meaningful therapeutic benefit.

Instead, due to the peculiar influence the placebo topic has on scientific rigour, there came calls for sham surgeries to replace real ones. Rather than viewing these studies as evidence that the surgeries themselves are ineffective, some interpret the findings as proof of the placebo’s effectiveness.

The Canadian science communicator Jonathan Jarry coined a term for this convoluted interpretation: the ShamWow Fallacy. This is when experts, invested in the power of the placebo, interpret negative outcomes as evidence of the power of placebos.

A systematic review in the British Medical Journal in 2014 found that, in about half of the studies, surgery failed to outperform placebo. Interestingly, it also found that in 74% of trials there was a beneficial therapeutic effect within the placebo group. On the face of it, this could present a problem. Shamwow effect aside, if it is the case (as I contend) that the placebo effect is an illusion, how do we account for the improvements observed in the placebo groups of surgical trials?

Beard illustrates this nicely. In contrast to many studies, Beard included three groups: surgery, sham surgery, and no treatment. Although surgery performed no better than sham, both surgical groups performed better than no treatment. So, does that prove that placebo surgery really does work?

Sadly, no. For one thing, any no-treatment control is necessarily unblinded. Patients who do not get surgery are aware they are getting no surgery, and this is likely to influence any patient-reported outcomes. Patients are less likely to report an improvement when they are aware they have undergone no intervention; this is why medical studies use blinding in the first place.

Beard also notes that while the difference between surgery and no treatment was statistically significant, it was not clinically important. That is to say, although there is an improvement on paper, it does not translate to any meaningful change in the quality of life of the patient.

Perhaps most important is the fact that, in both Beard’s and Moseley’s studies, patients receiving surgery were also given post-operative physiotherapy to help support their recovery. Since physiotherapy is also a treatment for both shoulder pain and osteoarthritis, this likely contributed to the observed improvements. Patients were, in fact, being given a second treatment after the first, a parallel intervention. This is exactly the sort of external factor which can cause an illusion of a therapeutic placebo effect, if we are not careful in how we interpret our data.

As for Andrew Carr, according to Horizon he discontinued acromioplasty operations after the study he was part of demonstrated they were ineffective. His current recommendation? Physiotherapy.

The post Placebo surgery: why performing fake operations doesn’t actually help anyone appeared first on The Skeptic.

One stark illustration of this was a study published in the New England Journal of Medicine in 2011, examining the placebo effect in asthma. This study has been discussed in some detail in The Skeptic before, but we can reiterate briefly.

Forty-six asthmatic patients were randomised to receive either a real inhaler, a fake inhaler, sham acupuncture, or no treatment. The real inhaler improved lung function by 20%; the other groups only showed around a 7% improvement. There was no placebo effect here, the fake interventions had the same effect as no treatment. The marginal improvements in those groups can be attributed to effects like regression to the mean.

However, when asked how much better they felt they were, patients told a different story. Recipients of both real and fake inhalers reported around a 50% improvement, despite the lung function tests showing there were no meaningful improvements for any of the sham groups. The discrepancy highlights the powerful role of bias: patients were reporting improvements they didn’t really have. Without the objective measurements for comparison, we might have mistakenly concluded that inhalers are a waste of time, since their sham counterparts are just as effective.

Another commonly cited benefit of the placebo effect is the effect of branding. One study used as a basis for claims like this was published in the British Medical Journal in 1981, under the title ‘Analgesic Effects of Branding in the Treatment of Headaches’, and authored by Branthwaite & Cooper.

Researchers recruited 835 women who reported using painkillers at least once a month and split them into four groups. The first group were given 50 aspirin in the packaging of a recognisable aspirin brand. The second group were given 50 aspirin in plain packaging labelled ‘analgesic tablets’. The third group were given 50 dummy pills, in the branded packaging. The final group were given 50 dummy pills labelled ‘analgesic tablets’.

Over a two-week period, the women were told to take two tablets from the box any time they have a headache. They should record how much better they felt after 30 minutes and then again after one hour. Scores were recorded on a six point scale: ‘worse’, ‘the same’, ‘a little better’, ‘a lot better’, ‘considerably better’, and ‘completely better.’

Branthwaite found that after one hour users of the unbranded dummy pills reported a mean pain relief of 1.78 while users of branded dummy pills reported a mean pain relief of 2.18. Unbranded aspirin scored 2.48 and branded aspirin scored 2.7. They conclude that branded tablets were significantly more effective than unbranded in relieving headaches, and that these effects were due to the increased confidence in obtaining relief from a well known brand.

I believe this conclusion to be overstated.

There are some methodological limitations in the study. Participants were recruited by going door-to-door, which means some participants may have known each other and been able to communicate about the study design. Branthwaite tries to control for this by requiring recruiters to skip ten houses after every successful sign-up, but the possibility remains.

The number of participants involved is good, but the duration of the study is relatively short. Moreover, when some women had not recorded any headaches over the two weeks, they were permitted to extend the trial for a further two weeks. The dummy pills were also not taste-matched to the real aspirin, so anyone familiar with the taste of aspirin, especially users of that brand, would be able to identify they were getting fakes.

The largest issue, however, is the conflation of an effect with a reported effect, a problem I understand to be common in the medical literature at large, not just in placebo effect research. While it may genuinely be the case that taking pills from a branded packet truly affords greater analgesia than an equivalent pill from an unbranded packet, it could equally be the case that taking a pill from a branding packet merely makes one more likely to report greater analgesia regardless of any true change in physiological pain.

The data in Branthwaite cannot tease these scenarios apart. Think back to the lung function tests: patients reported huge improvements in their lungs, apparently on the basis of bias alone, when actually there was no meaningful improvement. In Branthwaite, we have no objective measure to check against so we cannot be sure whether the reported differences between branded and unbranded pills reflect real pain relief.

It is fair to say that pain represents perhaps a unique case in placebo research, as there are completely fair and reasonable questions to be asked about whether a change in pain and a change in the perception of pain are meaningfully different. However, even the perception of pain is still different to reported pain. Patients could experience identical levels of perceived pain, and yet still report them differently because of the role of bias.

While this may seem like a trivial distinction, it has real-world implications. In 2016, Australian regulators fined Reckitt Benckiser, the makers of Neurofen, several million Australian dollars for selling identical painkillers branded and priced differently to target specific types of pain. Consumers were being charged a premium for products like Neurofen Tension Headache or Neurofen Period Pain, when the active ingredient and dosage were actually the same across variants.

Despite this, some advertising agencies continue to advocate for selling such products at inflated prices. They cite studies like Branthwaite to argue that Neurofen Period Pain would genuinely work better than regular Neurofen or generic ibuprofen for period pain, because of how it is branded. The placebo effect validates the claim, and so justifies the premium.

Many claims about the placebo effect assume that the placebo itself is responsible for the observed outcomes but, as these examples show, the effects we attribute to placebos are often a mix of statistical effects, patient bias, and other artefacts of the research process. For this reason, we should be both vigilant and cautious when evaluating the clinical relevance of placebo interventions.

The post Does pill packet branding change the placebo response, or is this just another placebo myth? appeared first on The Skeptic.

One particularly vivid recollection is of Ben Goldacre, author of the excellent book Bad Science, bounding on stage with that terrific, restless enthusiasm, and giving a whistle-stop tour of the amazing power of the placebo effect and its evil twin, the nocebo.

‘Pacemakers,’ he spilled into the microphone, ‘improve congestive cardiac failure after they’ve been put in, but before they’ve been switched on!’

The comment was clearly crafted and timed to elicit the laugh it earned, but it was also a comment which made me sit up and pay attention. Pacemakers improve congestive cardiac failure after they’ve been put in, but before they’ve been switched on? As Goldare quickly commented, this is a ‘properly outrageous’ finding. And it’s one which piqued my interest.

The study behind this claim was published in the American Journal of Cardiology in 1999. Linde et al. recruited 81 patients with obstructive hypertrophic cardiomyopathy (HOCM, a thickening of the heart wall) and fitted them with pacemakers. The patients were randomly assigned to one of two pacemaker settings: ‘atrioventricular synchronous pacing’ or ‘atrial inhibited mode’ at 30 beats per minute.

In AV synchronous pacing, the pacemaker coordinates the electrical activity between the atria and the ventricles. It detects the natural electrical signals of the atria and delivers a pacing pulse to the ventricles with an appropriate delay, mimicking the heart’s normal conduction pathway and ensuring proper timing between contractions. In other words, the pacemaker is switched on.

In contrast, the atrial inhibited mode only stimulates the heart if it fails to beat normally. In this study, the inhibited pacemakers were set to 30 beats per minute, meaning the heart would have to pause for two full seconds before a pulse was triggered. Since this won’t typically happen, these pacemakers were effectively placebos. They are fully implanted, but do not stimulate the heart.

Several key metrics were recorded at baseline. The New York Heart Association (NYHA) functional classification is a scale to measure the extent of the patient’s heart failure, ranging from one (you have no symptoms and no limitation on physical activity) to four (severe limitation, symptoms even at rest, usually bedridden).

Another metric was left ventricular outflow tract (LVOT) gradient, a measurement of the pressure difference between the aorta and the left ventricle during blood ejection. In patients with HOCM, higher pressure is expected because the thickened heart muscle forces the same blood volume through a narrower passage.

Linde also recorded the time in minutes each patient could tolerate exercise, their peak oxygen uptake, their peak heart rate during exercise, and systolic anterior motion, an abnormal movement of the mitral valve.

Three months after implantation, the patients were brought back and the measurements were repeated. Linde reported that patients with inactive pacemakers saw a statistically significant decrease in their LVOT gradient.

Pacemakers improve congestive cardiac failure after they’ve been put in, but before they’ve been switched on!

Of course, there is far more nuance to this study than that.

As we have touched upon in previous articles, statistical significance is typically measured using a p-value. This is a number between zero and one which expresses the probability of getting these results, or more extreme results, even if there is no true effect.

By convention, scientists have collectively agreed that the threshold for what is considered ‘significant’ should be 0.05, or 5%. While this is a widely accepted convention, it is ultimately arbitrary. Some researchers advocate for a stricter threshold, moving it from 0.05 (5%) to 0.01 (1%) or even to 0.005 (0.5%).

The LVOT gradient change reached statistical significance with a p-value of 0.04. Although this meets the conventional threshold, it is marginal and would not qualify as significant under stricter criteria. By contrast, the change in LVOT with the active pacemaker had a significance level of < 0.0001 – below one tenth of one percent!

Furthermore, the other measurements don’t support the claim that this was a real clinical improvement. Exercise tolerance and peak heart rate significantly improved in the active pacemaker group but showed no improvement in the placebo group. There was also no change in the NYHA functional class for the placebo group, meaning these patients did not experience an improvement in their overall heart failure symptoms. In contrast, the active pacemaker group improved by a full class, on average, from 2.6 to 1.7.

In total, of the six objective measures assessed, five showed no significant change for the placebo pacemaker group. The only measure that showed an effect, LVOT gradient, was borderline significant, and is likely spurious.

A p-value is meaningful for an individual outcome measure, but every additional measure you make is another opportunity to record a fluke finding by chance. Measure two things, you’re twice as likely to find something significant. Three things, and you’re three times as likely, and so on. If Linde’s figures were correctly adjusted to account for the many different outcomes recorded, the LVOT gradient change would be exposed as random noise.

Alongside the objective measurements recorded so far, Linde also records several subjective measurements, such as chest pain, dizziness, and reported palpitations. As we have discussed elsewhere, subjective measurements must be interpreted cautiously because of potential for them to be modified by bias. Subject expectancy effects, answers of politeness, and other forms of response bias can lead to patients reporting changes which don’t actually reflect any real world difference. Bearing this in mind, it is notable that of the 14 subjective outcomes which were measured, only five showed a significant effect in the placebo group, and all but one of these (palpitations) disappear when an adjustment is made for the number of outcomes measured in this study.

Another problematic aspect of this study was that three patients had their placebo pacemakers reconfigured for ‘active’ pacing part way through the study, because they complained to their doctors that the treatment wasn’t working. The paper isn’t clear what happened to the data from these patients. Was it removed from the analysis altogether? It doesn’t appear to have been, as the size of the inactive pacing group is unchanged at the end. Were the patients assessed early and their data included anyway? No such early assessment is mentioned. Unfortunately, either approach risks skewing the data in favour of the placebo effect by deemphasising the patients who failed to respond to placebo.

In short, while Linde does not provide strong evidence for a real therapeutic placebo effect, it is the kind of research that continues to be cited as evidence for the power of placebo. The reported effects are vanishingly small, and the clinical utility is dubious. I remain skeptical.

The post Pacemakers don’t work when they’re switched off – we should doubt studies that say otherwise appeared first on The Skeptic.

The review comes to a cautiously positive, if slightly hedged, conclusion: ‘[Colours] seem to influence the effectiveness of a drug.’ Since publication, the idea that different coloured placebo pills can produce different effects has become a frequently cited example whenever ‘the power of placebo’ is discussed in science communication or popular media, with this review often cited in support.

A closer look at the included studies, however, uncovers significant problems. The BMJ references six studies on the impact of drug colour on effectiveness, conducted between 1968 and 1978, all of which are impacted by some methodological or statistical issue.

Blackwell 1972

The first is Blackwell 1972, which was reviewed recently for The Skeptic. Fifty-six medical students were given either blue or pink placebo pills and asked to report which effects they felt. Blackwell claimed to show that blue pills made students feel ‘more drowsy’ and ‘less alert’, compared to pink pills.

Unfortunately this study is small, only single blind, and is based on self-reported effects – a combination which opens the door to all sorts of uncontrolled biases. Worse still, the students were primed with a list of possible effects, making it more likely they would report something even if nothing had changed. There are good reasons to doubt that the findings from this paper represent a real effect.

Schapira 1970

The next paper in the BMJ review is Schapira 1970. Forty-eight patients suffering with anxiety were treated with the anti-anxiety drug oxazepam over the course of several weeks. Although the dose was identical in each case, pills were presented in a range of colours. Patients self-reported their condition and were also evaluated by a clinician. Since the active drug was the same in all cases, Schapira suggests that any differences would be the result of the colour of the pills alone.

While the paper initially reports that anxiety was ‘most improved with green [pills],’ and depression ‘appeared to respond best to yellow,’ (and these are the findings cited by both the BMJ review and Ben Goldacre’s popular science book Bad Science), in fact neither of these findings reaches statistical significance. Schapira reports only one statistically significant finding with respect to pill colour, and that is for the effect of green pills on phobias. Even this finding is questionable, however, since it is based on the clinician reports only (no effect is found when considering the patient reports), and it involved just 17 of the 48 patients. The remaining 31 were not suffering with phobias, since participants were recruited for their anxiety symptoms. The fact that 17 of them also struggled with phobias was unintended and the analysis performed post-hoc.

This effect in phobias also disappears when the figures are properly adjusted to account for the fact that Schapira makes many different comparisons, across pill colour, symptom, and rating type.

Cattaneo 1970

Cattaneo 1970 examined 120 patients awaiting surgery on their varicose veins. The patients were randomly given either an orange or a blue pill and told they were sedatives to help them sleep. In fact, the pills had no drug in them at all. The next morning, patients were asked how long it took them to fall asleep, how long they slept, whether they felt rested, and which of the two pills they preferred.

In an odd leap of logic, Cattaneo asserts that whichever pill patients said they preferred must be the one which best helped them sleep, but it should be immediately obvious why this doesn’t necessarily follow. Perhaps orange is simply their favourite colour? Maybe they’re big Everton supporters and will pick a blue anything regardless of how well it does?

You may also reasonably ask why they are giving fake sleeping pills to patients who are awaiting varicose vein surgery? The paper claims that since they are awaiting surgery, therefore patients must be experiencing mild-to-moderate anxiety. Since they are experiencing mild-to-moderate anxiety, therefore they must have trouble sleeping. And since they cannot sleep, they must need a sedative. 

No effort is made to establish whether the patients actually are suffering with anxiety, nor is any effort made to establish whether they are struggling to sleep; these claims are just nakedly asserted.

In fairness, the paper does claim that there is correlation between the ‘favourite pill’ and the other self-reported sleep quality scores gathered from the patients, but it makes no effort to quantify this statistically. The primary findings of the paper are then based on this self-reported preference, not the sleep quality scores.

Cattaneo reports that 41% of patients preferred the blue pill and 39% preferred orange. The remainder expressed no preference. Astute readers will doubtlessly have noticed that there is only a very small difference between 39% and 41%. In fact, if patients who expressed no preference are excluded, this is a 51/49 split. This is not a significant effect that can be generalised to the wider population, it is a coin flip.

With no meaningful effect in the overall analysis, the paper then breaks the results down by sex, claiming that men prefer the orange pills and women prefer the blue. This comparison just makes it to the common threshold for a significant finding, with a p-value of 0.042. As discussed in a previous article, the p-value represents the probability of obtaining these results even when there is no true effect. In this case, there is a 4.2% chance we would see these results even if there were no effect from pill colour by sex.

However, we should still be skeptical of the data, which appears to be the product of p-hacking. P-hacking refers to the practice of intentionally or unintentionally manipulating your analysis until you find some significant result and then reporting on that. For example, performing a subgroup analysis by sex when the overall analysis finds nothing. Even if we leave Cattaneo’s bizarre methodology aside, p-hacked data does not lead us to reliable conclusions.

Luchelli 1978

Luchelli 1978 took a similar approach, which is perhaps no surprise given that Luchelli is a co-author on the Cattaneo paper, and Cattaneo is a co-author on Luchelli. This time, 96 patients awaiting unspecified elective surgeries were recruited. The paper reports that all participants had significant sleep problems, including difficulty falling asleep, disturbed sleep, and an average sleep duration of ‘five hours or less.’

Patients were given either an orange-coloured sedative (heptabarbital), a blue-coloured sedative (also heptabarbital, in the same dose), an orange placebo, or a blue placebo. The following morning, they were interviewed to determine how long it took them to fall asleep, how long they slept, the quality of their sleep, and whether they woke feeling rested or groggy.

Unlike the earlier ‘pill preference’ metric, Luchelli used sleep onset and duration data directly in the analysis, which found two significant findings concerning pill colour. Patients who took blue pills fell asleep 32 minutes faster and slept 33 minutes longer on average. No statistically significant effect was observed for pill colour on sleep quality or grogginess.

Despite these results, there are significant issues with relying on self-reported data for metrics like sleep onset time. Be honest, can you recall the exact time you fell asleep last night or how long you slept? Such figures are difficult to report accurately, and there is substantial room for biases to affect these kinds of subjective measurements. Even if pill colour had no real impact, the effect of bias may result in an illusory effect being recorded in the data. Luchelli acknowledges these limitations but defends the approach, claiming ‘sound results have been obtained based on subjective assessments.’

Unfortunately, the paper doesn’t provide enough data to verify whether the statistical analysis was conducted correctly, but the results presented for the effect of blue pills as sedatives are marginal and would likely disappear if properly adjusted to account for the large number of subgroup comparisons made in the study.

Moreover, these subgroup findings contradict the overall analysis. For example, men taking orange capsules fell asleep faster and slept longer on the first night compared to those taking blue capsules, but on the second night, the opposite effect was observed. In women, the effect of blue capsules remained consistent across both nights, but orange capsules were more effective on the second night than the first.

Inconsistencies like these make it clear that, if pill colour has any true effect, it is variable and unpredictable. Given the variability in the data and the small sample size, it wouldn’t be surprising if just one or two outliers were skewing these results.

Nagao 1968

Nagao 1968 is a paper we unfortunately cannot examine in any great detail (despite my best efforts) as I’ve been unable to obtain a copy of the original text. But since it was published in Japanese, I’m unlikely to have understood it anyway. The BMJ does, however, outline the main findings: ‘79% of patients reported adequate pain relief with red pills, compared to 73% with white pills.’ We don’t know if this was a significant finding or not, since we can’t see the data, how the analyses were performed, which other comparisons were made, or the trial methodology.

One observation we can make is that these findings will be based on self-reported data, as there is no real alternative when measuring pain. Self-reported outcomes are especially susceptible to bias, and while it is unlikely that patients are being deliberately deceptive when reporting how much pain they are in, there are several psychological effects which can distort those reports. The subject-expectancy effect can result in patients reporting what they think should be happening, rather than what is actually happening. Social-desirability bias can result in patients reporting what they think is the most pleasing or acceptable answer.

Importantly, these sorts of effects (and dozens of others like them) can modify what is recorded in the data without necessarily changing anything about the patient. For this reason, it can be difficult to disentangle self-reported data from simple bias in studies like Nagao.

Huskisson 1974

Finally, Huskisson 1974 conducted a study on 24 patients with rheumatoid arthritis, each of whom required on-demand pain relief (in addition to their standard care) at least once per day. In a somewhat complicated design, patients were randomised to receive one of three active painkillers or a placebo, with the pills presented in pairs and in various colours, across several days. Patients self-reported their pain relief on a scale from 0 (no relief) to 3 (complete relief), and their responses were recorded hourly for six hours after taking the medication.

Huskisson found that pill colour did not significantly alter the effectiveness of the active painkillers, but patient-reported pain relief did vary by pill colour when administering placebos. No difference between the colours was found one hour after administration, but differences appeared at two hours (p < 0.02), three hours (p < 0.05), four hours (p < 0.02), five hours (p < 0.05), and six hours (p < 0.05).

Despite this, we should still interpret the data cautiously. First, the sample size was small, with at most six patients receiving each coloured placebo. Two patients dropped out of the study, but the paper does not specify why or which groups they left. Second, like Nagao, the results are likely to be influenced by reporting biases, as all the data gathered was self-reported by patients. Finally, even the statistically significant results were relatively marginal, with p-values ranging from < 0.02 to < 0.05. Given the number of statistical comparisons performed, it is likely that even these results would not remain significant if adjusted for the false discovery rate.

Interestingly, the red-coloured pills did not outperform the other colours when active painkillers were administered. This effect, if it represents a real phenomenon, was limited to the placebo pills. However, studies like Huskisson are often used to support the idea that the colour of medication can enhance its placebo effect, even while this study’s findings do not support such claims for active drugs.

Returning to the BMJ review itself, each of the six papers referenced is graded for its methodological quality. Blackwell, which claimed that blue pills made students less alert, scores 6.5/10. Schapira, which claimed yellow pills were better for depression and green are best for anxiety, scores 7/10, as does Cattaneo, with patients waiting for their varicose vein surgery. The remaining three studies, Luchelli, Nagao, and Huskisson, all score less than 5/10.

Taken together, the studies from the BMJ paint a picture not of a meaningful placebo effect tied to pill colour, but of random noise in poorly designed research. Perhaps not a surprise, given that most of it was conducted 20 years before the BMJ review, which itself is rapidly approaching 30 years old. The most impressive effects are found in the studies judged to have the poorest methodological quality, while the better-designed trials show little to no real effect.

This is a pattern we see frequently in pseudoscience: acupuncture, homeopathy, reiki, and similarly implausible therapies tend to show the biggest effects in poorly controlled studies. When proper controls are enforced, the effects disappear.

The BMJ tries to walk a middle ground. The review acknowledges that the evidence is inconsistent while still suggesting that colour might influence the effectiveness of a drug. It ends with a call for more research. And while it’s plausible that colour could change how patients perceive a treatment, the data don’t show a reliable and meaningful clinical effect. The studies that seem to support the idea are small, weak, and flawed. The more robust trials find little of interest.

So should we start colour-coding pills, based on what we think patients will respond to best? I’ve seen more than one commentator make this very suggestion, but absent any robust, reliable, and reproducible data showing that pill colour has a measurable impact on clinical outcomes, I would suggest we focus on what we can be sure actually works: proper treatment, solid evidence, and good science.

The post The evidence for pill colour impacting placebo effects gets flimsier the more you examine it appeared first on The Skeptic.

While homeopathy is naked pseudoscience, it nevertheless yields occasional positive results in clinical trials. For scientists, skeptics, and those with an interest in the philosophy of science, this serves as an example of how poor trial design can lead to unreliable conclusions. Homeopathy, like acupuncture, has a habit of showing strong effects in badly designed trials, and small or no effect in well-conducted ones. If the heyday of homeopathy is over, perhaps we need a new paradigm to illustrate how weak design can mislead us?

I propose: the powerful placebo.

Over the coming months, I hope to explore some of what has been described as the crème de la crème of placebo effect research, scrutinising the biases, flaws, and weaknesses that challenge the widely accepted view of placebos as having real, powerful therapeutic effects. We will examine the primary literature to see if the work is really as compelling as some would have us believe.

Blackwell 1972, originally published in The Lancet, is a favourite of science communicators, who breathlessly claim it as a demonstration of how the colour of a placebo pill changes its effect. Blackwell took 56 medical students at the University of Cincinnati and randomised them to receive either blue or pink placebo pills. A further 44 students declined to take part, after learning what the study involved.

The students were told that they would receive either a sedative or a stimulant drug, but that they would not know which. They were also given a list of effects and side effects they could expect to experience as a result of taking the drugs. Six effects and six side effects for each, for a total of 24 expected effects.

The pill colour was assigned randomly, resulting in 29 of the students being given blue pills, and the remaining 27 receiving pink pills. In fact, all of the pills were placebos; there were no real drugs involved. Blackwell reports:

“There were two significant differences due to colour, both indicative of the blue capsules producing more sedative effects than pink capsules. 66% of subjects on blue capsules felt less alert compared with 26% on pink; 72% on blue capsules felt more drowsy compared to 37% on pink.”

Aside from the fact that ‘less alert’ and ‘more drowsy’ are arguably the same thing, there are more serious concerns with the study’s methodology if we are using it to support such extraordinary claims.

For one thing, it is only single-blind. Reading the paper, this seems to have in fact been an exercise to teach a group of medical students about placebos that their professor decided to write up for The Lancet after the fact. As a class exercise, of course the professor knew that the pills were placebos, and knew what the expected effects were. In an illustration for students, that is to be expected. But today this paper is used to support the purported effects of pill colour generally.

Moreover, the professor provided the students with a list of effects they can expect to experience after taking the pills. They were told that stimulants would make them more cheerful, more talkative, more alert, less drowsy, less sluggish, and less tired. They were also warned the stimulants would make them more tense, more jittery, more irritable, less relaxed, less calm and less easy going. Sedatives were said to cause the opposite effects, so less cheerful, more sluggish, and so on.

Since the students were asked to self report their condition after taking the pills, it’s hardly surprising they reported some of the many effects they were told to expect. Confirmation bias, if nothing else, would lead them to notice and report the changes highlighted by their professor, even if they may have ignored those same changes had they not first been prompted to look for them. In psychology, this is referred to as priming.

Researchers frequently overlook this and fail to acknowledge that there is a distinction between the reported effect of an intervention and the true effect, because of the role of bias. In fact, a 2010 review on placebo effects for Cochrane concluded that it is ‘difficult to distinguish patient-reported effects of placebo from biased reporting’.

It is also notable that, despite the potential confounding role of bias, only two of the 24 effects that students were asked to look out for rose to the level of statistical significance. The paper does not give specific p-values for those two findings, but notes that findings were reported as ‘significant’ if p<0.05.

For those who aren’t statistical experts, the p or probability value tells us how likely it is that we would see these results if the null hypothesis is true – in this case, if there is no actual effect of the pill colour. A p-value of 0.05 means there’s a 5% chance of observing these results purely by chance, assuming no real effect. However, each comparison made is another roll of the dice, another opportunity to obtain a false positive by statistical fluke, and Blackwell tests against many different outcomes.

One common technique used to account for this is the Bonferroni Correction, which adjusts the p-values to account for the number of comparisons being made, reducing the chance of a false positive.

While the paper does not present the raw data, sufficient data is provided for us to work backward the raw figures, at least for the two significant findings. We know how many participants there were in total, and we know how many were given each colour. The paper reports that 66% of people who had blue pills reported less alertness, which equates to nineteen people (65.51%). Applying the same methods to the other groups, we can determine the raw figures for the two ‘significant’ colour findings.

From here, we can compute our own p-values, using a chi-squared test, and apply a Bonferroni correction. This reveals that, once adjusted for multiple comparisons, neither alertness (p = 0.072) or drowsiness (p = 0.19) are actually significant findings.

While this study may have originally been designed as an educational tool for medical students, its citation by science communicators today as robust evidence of a powerful placebo effect invites a higher level of scrutiny. It is perhaps no surprise it fails to hold up when evaluated against more rigorous standards than it may have originally been designed to bear.

This is a small, single-blind study, based on self-reported data – a potent combination that opens the door to all sorts of biases. Worse still, the students were primed with a list of possible effects, making it more likely they would report something, even if they hadn’t felt much. This isn’t good science, it’s a recipe for false positives.

Of course, Blackwell was not the final word on pill colour and placebo, there are other studies that claim to find an effect, but those are stories for another day.

The post Does the colour of a pill really influence what kind of placebo effect you’ll experience? appeared first on The Skeptic.