I wanted an article to list some of my grievances with “Science Based Medicine”
I’m a skeptic. I read “skeptic” blogs sometimes because many times they have valid points and I like to see a diversity of opinions. Sometimes SBM has valid points and can be a source of information, but people need to be skeptical of SBM as well.
- About SBM
- True Skepticism vs Selective Skepticism
- On Implausibility
- Skepticism as an Attitude
- SBM Doctrines and Group Think
- What is Considered Evidence?
- National Guidelines
- “Science” Based Evidence is a Misnomer
- The Faith of SBM
- Skepticism is not the Same as Rationalism or Better Health
- Selective Skepticism is Fine if You’re Not Forced to Choose
- Why do I Think that Self-hacking is More Reliable in the Long Run?
- Assumptions Made By SBM
- Fallacious Thinking Committed By SBM
- Examples of SBM Not Being Skeptical
- Mouse Studies Aren’t Good For Demonstrating Safety
- Other SBM Grievances
- SBM Resorts to Ad Hominems When I Show Them Reason
- What is the Incentive of SBM?
- Supplements Don’t Contain Active Ingredients?
- The Placebo Effect: It’s Complex
- Response by Science Based Medicine
SBM is a cult organization that is highly dogmatic. However, much of what they write I agree with such as when they debunk stuff that is just loaded with crap.
They will take a “quack” that is completely ridiculous and debunk him. But in the process, they often throw the baby out with the bathwater since even quacks often have some truthful aspect.
True Skepticism vs Selective Skepticism
Science-Based Medicine is an organization that tries to portray itself as skeptical, but I say it’s selectively skeptical.
This is because they are not willing to entertain certain ideas and they form strong beliefs that certain treatments have no value outside of what your doctor recommends.
With regard to health, true skepticism means to be doubtful in a belief that some treatment works, but at the same time also be doubtful in forming opinions that something doesn’t work.
This means you shouldn’t form beliefs one way or the other. Your attitude should be one of not knowing if you want to be truly skeptical.
If you read SBM posts, you will see they are skeptical of ‘alternative’ remedies, but they believe that these remedies don’t work.
If you ask them about any remedy that isn’t proven by science, they will tell you the chance is close to zero that it works. They believe the people who feel an effect from them are just experiencing the placebo effect.
This is a belief that that goes against true skepticism.
True skepticism would be not forming an opinion that they work or that they don’t work. ‘I don’t know’ should form the basis of true skepticism, not that XYZ treatments don’t work because it doesn’t fit with our knowledge paradigm or because there’re no large clinical trials.
If there’s no evidence that a treatment works, there’s also usually no evidence that the treatment doesn’t work. So forming an opinion one way or another is not true skepticism.
‘Skeptics’, SBM and company need to realize that the absence of scientific evidence isn’t evidence of absence.
Or how about before we knew that too much sugar is bad for you or that lead is bad for you.
Even just 5 years ago, many people were laughing at me for staying away from added sugars because THERE’S NO SCIENTIFIC EVIDENCE that the quantities we consume are bad. In reality, there has been evidence, but it’s been confined to small trials, animal studies, etc…
I think the atmosphere is changing with regard to various healthy behaviors and all of a sudden it doesn’t sound crazy that I’ve been trying to stay away from added sugars when there was no strong scientific evidence that it was bad.
Some ‘skeptics’ will argue that there’s still no evidence because we all view the word evidence in a different way.
Skepticism falls along a continuum. I would say that SBM and co. are certainly more skeptical than the alternative health movement, but I can’t consider them true skeptics.
There is something to be said about things being implausible. Homeopathy, for example, is implausible.
I can’t say for certain anything is impossible, but homeopathy is one of those things that has such a small chance of doing anything that it’s not even worth it to try it out.
Any given religion is based on many assumptions, and they generally contain doctrines that go against information that we do have a good idea to be true.
For example, believing in Judaism requires the belief in a Bible that contains hundreds of statements that are highly implausible (giants, people living to 1000, the world being 6000 yrs old, etc…), without any evidence to balance out the implausible parts.
But when we get to herbs and supplements, in general, the same case can’t be made. Every supplement has ingredients that interact with the body in some way. They are not inert.
The question isn’t if they do something because 99% of them do something. The exception would be things that can’t cross your gut barrier, but even then they can have a pretty strong effect.
All 300 or so supplements/interventions that I’ve tried had a noticeable effect. The question isn’t if they have an effect, but what that effect is and if it benefits you more than it harms you.
Sometimes, equipment doesn’t have any noticeable effect, and you can’t increase the dosage (for example, grounding).
I know supplements have an effect because I keep upping the dosage until I’m certain I feel an effect.
The normal dosage is one pill. However, when I took one pill I didn’t notice anything. I had to take 4 pills to notice an effect. I guarantee you that if you keep upping the dosage you will feel an effect, and it will likely make you somewhat more relaxed.
From my perspective, it makes no difference whether the clinical evidence shows it’s effective for depression. I know what it feels like and if I ever have a use for it, I’ll take it. It happens to be that I don’t take it because I find other supplements to be more useful for my specific needs.
On the other hand, if the science shows overwhelming evidence for a given anti-depressant, but it doesn’t work for me, then I’m going to stop it, no matter how many placebo controlled trials were done.
Skepticism as an Attitude
I view skepticism as more of an attitude rather than what you say directly.
You can talk the talk of skepticism, but not walk the walk. SBM uses the right skeptical linguistics, but they form deep and strong opinions and bonds to their ideas.
To behave like a skeptic, it requires not forming strong beliefs about anything (unless there is strong evidence for that belief).
Developing a skeptical attitude is not merely a result of learning cognitive pitfalls and logical reasoning. That’s just the beginning.
I’ve found that developing a skeptical attitude required practice in cultivating awareness and letting go. You need to let go of your convictions.
We all build up strong opinions and convictions over time, but the true skeptics practice letting go of their ideas every so often.
SBM uses skeptical lingo, but they have very strong beliefs. Anyone who looks at their site will see this.
I can predict their opinion about a subject before they write about it. All of their writers have the same opinions about everything, it seems.
However, you can try to predict my opinion about subjects and you’ll be surprised often because I don’t follow any dogma.
SBM Doctrines and Group Think
Whenever a group bands together for a common purpose, strong beliefs will form. This includes Atheists, “skeptics”, etc…
I don’t believe in God, but I also don’t belong to these atheist groups, as I find group think takes place there as well.
I would caution the reader to be careful about listening to any ‘expert’ when they are highly predictable in their opinion about a subject.
I actually don’t find SBM writers to be very knowledgeable when they write about certain subjects. It seems like they suffer from tunnel vision.
Here’s how to know SBMs opinion before you read any article. If you find that even one of their thousands of articles don’t fit this paradigm, please post it.
- If it’s natural, it’s not effective.
- If a study has shown a natural product effective, it’s flawed.
- If the study isn’t flawed, it’s clinically insignificant.
- If the FDA claims something is safe, SBM will never disagree.
- If a drug is FDA approved, it’s effective. SBM will never question its efficacy or ‘clinical significance’.
What is Considered Evidence?
Science is a method of investigation, with the goal of teasing out a cause.
The type of evidence that SBM advocates is a random threshold of evidence. There’s no reason it can’t be lower.
There’s no rational basis to say that animal studies, our subjective evaluation, anecdotes, tradition or in-vitro studies don’t constitute evidence.
I think there are flaws with all kinds of evidence, but I don’t think animal studies, anecdotes or traditional knowledge are irrelevant. I think they can be highly useful and add pieces to the evidence puzzle, albeit there are better forms of evidence.
SBM assumes that all of this evidence is worthless or close to it. They try to support this by pointing out flaws to these types of evidence.
I agree that these kinds of evidence have flaws, but you can poke holes in any type of evidence.
For example, the majority of medical science can’t be replicated. (R)
Over the past decade, Amgen’s oncology and haematology researchers could not replicate 47 of 53 highly promising results they examined. (R) Let that sit with you for a bit…
There’s a good reason not to believe any study done by a pharmaceutical company because they have billions to gain from it, yet I don’t disregard their studies completely.
The take away is that every form of evidence has flaws, some significantly more than others.
To blindly follow one source of evidence and completely discard the rest is faulty and, I would say, stupid.
I am more likely to agree with SBM’s threshold of evidence when it comes to forming national guidelines, but not as a guide to individual choices.
Meaning, my experiences might be good evidence for ME, but they don’t say what a random sampling of the population is like.
We don’t want to base national guidelines on animal studies, anecdotes, tradition or other weak forms of evidence.
If a recommendation is to be for a population as a whole, it should be based on large and rigorous clinical trials.
“Science” Based Evidence is a Misnomer
I really don’t like the term science-based medicine, because it paints an inaccurate picture.
Much of the scientific knowledge we have is based on animal studies, yet we call it science.
I include scientific information in my experiments and I use it as the framework for how I understand the body.
However, SBM is not referring to this type of science with regard to medicine. They are referring to large, replicated, double-blind, placebo-controlled trials, published in the top 3 journals.
This method of investigating is a branch of science, but this definition confuses things by making you believe that nothing else can be considered science.
SBM should be called large-replicated- double-blind-placebo-controlled trials published by the top 3 journals-Based Medicine
The Faith of SBM
SBM worships double-blind, placebo-controlled trials by the top 3 journals and every other form of evidence is pretty much meaningless. This is a faith.
They like Cochrane reviews, but only when it supports their prior beliefs.
When a study is not published in the top 2 or 3 journals, they will only believe it if it conforms to their prior beliefs.
If a study is published in the top journals, but it goes against their ideology, they will discard it.
SBM isn’t skeptical of the studies done by Big Pharma. Even if they say they are, they aren’t in attitude.
Most of the best clinical trials are conducted by Big Pharma. All you need to know is rudimentary psychology to realize that when you have a billion dollars to gain from a drug, you’re going to pull all kinds of ‘shtick’ to make a study say what you want it to say.
I believe the best form of evidence for YOU is to experiment with something and see how you feel.
You obviously want to start experimenting with things that have the most evidence, but you should keep experimenting with interventions that have less evidence if you aren’t cured by the intervention with the most evidence.
Skepticism is not the Same as Rationalism or Better Health
I would like to point out that skepticism is not the same as rationalism.
Skepticism says we should be doubtful of everything (my definition).
Rationalism says we should make decisions based on reason and logic, not emotion (my definition).
Skepticism and rationalism often go to together, but they don’t necessarily go together.
Being skeptical doesn’t necessarily result in better outcomes, but a rational strategy almost always does.
If someone is truly skeptical, they still don’t have a strategy or direction to achieve optimal health.
If you don’t know if a treatment works or not, it doesn’t mean you should refrain from trying it out. SBM would say to not try it out.
My opinion and the approach that worked for me was to try things out. Discoveries are often made not by a directed top-down approach, but rather by tinkering and seeing what works.
SBM favors a very top-down approach rather than tinkering.
This is not an approach that will achieve the best health outcome for you as an individual, even if this approach makes more sense with regard to national recommendations.
Selective Skepticism is Fine if You’re Not Forced to Choose
Again, skepticism is an attitude, not necessarily a prescription for a better life.
For example, if you have a health issue, you are forced to act in one way or another. You can do nothing, try conventional treatments or alternative treatments, or a combination of these.
I am defining the alternative as anything not embraced by the mainstream medical establishment. This surely includes a wide-ranging spectrum that includes silly quackery to scientifically supported treatments.
Say we have a skeptical attitude of not believing a treatment works or doesn’t work, we are still forced to choose whether to take the treatment or not. To do nothing is also a choice.
SBM editors would say to refrain from treatments that don’t have evidence, but this is not in my experience an optimal approach to health.
I say even if something only has anecdotal support, it’s rational to experiment with it – if it’s safe and relatively cheap. This doesn’t mean that if you have a health issue, you’re going to start out trying hyped up, expensive ‘proprietary formulas’ that only has anecdotes behind it. I actually don’t like these because you don’t know what is or isn’t working. Maybe one ingredient is making you better and another worse.
Think about it this way: If there’s even only a 10% chance some treatment with only anecdotal support will work, it would still make sense to try it out. This is because you’ve now found a treatment that you can use for the rest of your life and over your lifetime this can produce a lot of value for you.
The same calculation doesn’t hold for everyone. Someone without any health issues should seek out a higher level of evidence for benefit because they have more to lose than gain. There’s always a risk with every treatment, even if it’s very small (with some exceptions).
Everyone has to make their own calculations and risk/benefit analyses, but the equation will be different for everyone.
SBM is too naive and simplistic to take this into account. They have one equation for everyone.
If SBM were playing the Monty Hall game in medicine, they would say we’re not choosing! There’s no evidence to pick one door over another! True, but even so, there’s a rational strategy to achieve an optimal result.
Why do I Think that Self-hacking is More Reliable in the Long Run?
Imagine the scenario where the best clinical trial came back as proving that a drug was helpful for a condition, but through your own experimentation, you found it didn’t work for you. You’d stop taking it, or at least I would.
Now imagine a scenario where said drug came back as ineffective, but you took it and felt amazing. You wouldn’t give a hoot what some study says, no matter how rigorous it was.
Studies can only tease out effects in a given population. For example, studies often deal with a given age, gender or people with a specific genetic makeup.
Other times, a study will not select a homogenous group of participants, in which case it won’t be large enough to detect whether certain individuals (say males who are 30 years old with a given lifestyle, diet and genetic makeup) will benefit from the treatment. So it’s possible that a study will come back as not statistically significant, but in reality, it could completely cure one individual in 100 without any side effects.
Therefore, it’s impossible for any study to tell you that some substance, drug or food does or doesn’t work for you.
These studies are good in helping you figure out what to experiment with first, but they don’t replace good old self-hacking.
Assumptions Made By SBM
These assumptions aren’t always explicitly stated by SBM, but if you read their stuff, you’ll see that they believe these implicit assumptions.
SBM tends to assume that if some natural treatment will work, then it can always be patented and made better by a pharmaceutical company.
SBM believes that if something hasn’t been proven by double-blind, placebo-controlled trials and published in the top journals and replicated, then it has no value, especially if it’s a natural product or approach.
SBM assumes that synergies don’t play a role in the action of drugs (it’s their attitude, at least). They believe if a herb works, then you can purify the chemical that works. This is often true, but not always.
SBM assumes that if there’s no strong scientific evidence for something being harmful, then it’s probably safe (think Aspartame, BPA, Phthalates). This means they believe in the power of science to unearth negative effects in a given time frame.
I believe in the power of science to do this, but it could take 100 years. It took 100 years of widespread use to realize that Saccharin may cause insulin resistance. See my article on artificial sweeteners and recent studies showing that BPA and Phthalates are not safe.
SBM also believes that if some treatment works, a company can make enough money from it to conduct rigorous clinical trials.
I disagree with all of these assumptions made by SBM.
Fallacious Thinking Committed By SBM
1) SBM believes that natural products are just as likely to be harmful as synthetic ones.
They cite the naturalistic fallacy, which is if it’s natural then it must be safe. This is indeed a fallacy. There are many natural chemicals that are deadly (ricin, etc..).
However, the naturalistic fallacy doesn’t deal with probability. I think if something is natural and has a long history of safe usage, then it’s very likely to be safe. The naturalistic fallacy doesn’t deal with this scenario.
2) SBM believes that correlation doesn’t provide any evidence that something can be effective.
They cite the post hoc ergo propter hoc fallacy. This fallacy essentially says that correlation doesn’t prove causation.
Correlation certainly doesn’t prove or equal causation, but if something is correlated then I believe it’s more likely to work compared to a treatment that’s not correlated. So correlation might be evidence for something, but it’s certainly not proof.
Science is actually based on correlation, if you think about it. You introduce a variable and there is change. So, a study will find that X drug correlates with Y improvement. However, this is a better form of evidence in the scientific context because they are introducing a new variable and then correlating, and they do this many times. In addition, there is a control group where a inert variable is introduced, and the effect is comparatively lacking.
Examples of SBM Not Being Skeptical
Phthalates, BPA, Artificial Sweeteners
SBM thinks to be skeptical means to take the position that something is safe because there’s no evidence of harm.
But if you click on the links, you’ll see I cite evidence that has only recently come out that shows that these aren’t as safe as we thought.
I don’t think science has the necessary tools to demonstrate that something is safe in all respects and for 100% of the population. It would take too long to do this. It’s impossible to take into account how other environmental factors combine (negative synergies).
There’s hundreds of examples in history when we thought something was safe given animal studies, until further evidence revealed that it wasn’t.
To naively think that we can demonstrate something is safe is anti-skeptical in my view.
For example, SBM destroys a quack that talks about oxygen. I agree with 95% of what this SBM author is saying here, but when you read the article you think oxygen has no use to promote health.
I use an Oxygen concentrator and it certainly has an effect on mood and calmness.
Again, you’d think after reading this article that unless your doctor recommends oxygen, you won’t be helped by it. False.
So while SBM is good at debunking complete shit by quacks, they never give you a complete picture. It’s always narrow and simplistic.
One editor, Scott Gavura, wrote a whole post about how it’s implausible that Collagen works for joint pain.
Whether it works or not is beside the point. To call something “implausible” based on his knowledge is lacking skepticism. He could say he doesn’t understand how it would work, but saying it’s implausible is anti-skeptical.
Basically, the argument is that we digest the collagen proteins and therefore the only benefit we get out of it would be the amino acids found in there. Since we consume adequate protein and arthritis is caused by inflammation, this shouldn’t help us.
A scientist in the comments section completely debunks his “implausible” assertion, showing that it is possible for collagen to cross our gut and deposit itself in our joints.
I have a different take. I think the collagen might be useful for building oral tolerance. I bring down many scientific studies that support oral tolerance.
Other than oral tolerance, collagen is an extremely rich source of glycine, which functions as an anti-oxidant and anti-inflammatory in the gut and has many other healthful properties. Many people would benefit from more glycine, I believe.
Whether it will be effective or not can’t be known, but I doubt Scott has done any reading on Oral Tolerance, the studies on glycine or the information presented by the scientist in the comments section. He came to the “Implausible” conclusion only based on his incomplete knowledge of how the body works.
What he’s basically doing is taking his current state of knowledge and coming to firm conclusions based on it, without being skeptical that in reality, we know very little about the universe as a whole.
To assume he knows so much that he can claim confidently that it’s “implausible” is a type of arrogance, and it shows a lack of skepticism.
Mouse Studies Aren’t Good For Demonstrating Safety
The studies used to demonstrate safety for drugs and chemicals use lab rats that are of a specific and tame breed.
It’s hard to see if something is wrong with the mice unless they’re really fucked up. Imagine if mice experienced an average 10 point IQ loss from some drug (assuming they had an IQ). There’s no way you’ll be able to see this relatively small change.
For example, mice studies didn’t show any negative effect on the offspring of pregnant mice taking Paxil. However, human studies have been able to detect deleterious effects on children.
When researchers studied mice in an evolutionary model, they were able to detect differences. (R)
The bottom line is that mouse studies are used to demonstrate the safety of chemicals and these are inadequate. SBM has no problem relying on these highly problematic ways of demonstrating safety.
Other SBM Grievances
When a natural or alternative product is shown to be effective in clinical trials, SBM all of a sudden starts complaining about “clinical significance.” Two topics that come to mind are acupuncture and EDTA.
I don’t hear them talking about clinical significance when it comes to drugs that barely work. And who the hell are they to decide what’s significant for us? It could help each individual differently.
A study that shows a small significant effect could mean some people benefit a lot but the vast majority of people don’t benefit at all.
Also, SBM is highly skeptical of animal studies that show the effectiveness of a natural product. This is fine.
My problem is that they aren’t skeptical of animal studies that attempt to demonstrate the safety of some chemical. The FDA declares a chemical safe based on animal studies. SBM has no problem relying on these when declaring some substance safe.
This is a contradictory attitude and demonstrates their biased thinking. I don’t think animal studies are reliable to prove a supplement as effective, nor do I believe that animal studies can prove something to be safe.
To come to any safety conclusion based on animal studies demonstrates a profoundly unskeptical attitude.
SBM Resorts to Ad Hominems When I Show Them Reason
If you see their patterns, they are constantly engaging in ad hominems. When I presented evidence against their position, I was met with the response “you don’t have the necessary authority to investigate these issues.” Not one of them dealt with my argument.
Since I didn’t have a PhD in a certain topic, it was the end of the debate. They’ve argued that you need a formal education about a topic to give an intelligent position. I disagree. You need to be educated, but not necessarily formally educated.
What is the Incentive of SBM?
Some people think that SBM is a corrupt group paid by big pharma or have some other incentive. I don’t think this is the case.
First, SBM does create a valuable service in debunking some things that are clearly scams. They infuse a dose of skepticism (albeit selective) that is often lacking.
Some of their ire is well placed and makes sense. However, there are other factors at play.
I think they are a group of highly square and dogmatic people, who subconsciously feel that their power is being taken away from them by people without degrees.
People like authority and degrees confer authority. All of the SBM writers have spent years attaining degrees. This gives them respect.
This is why people want you to call them Dr so and so. They spent time getting the degree and they crave the respect and authority.
Then comes the alternative health movement and starts telling people there are other options. This strips SBM of their power (and respect).
I also think that the whole skepticism movement is composed of people who try to feel superior in some way, specifically in trying to demonstrate superior intellect (although I am not impressed at all). Often, they are trying to super-compensate for their insecurities.
I think these are the psychological underpinnings as to why they have such a hatred for anything ‘alternative’.
Supplements Don’t Contain Active Ingredients?
There has been a spate of articles about the study that showed most supplements tested don’t contain the primary ingredient on the label. Read:
“Amazingly, 79% of the supplements tested did not contain the primary ingredient listed on the label.” A Really Bad Week For The Supplements Industry.
SBM and co have this to say:
“Supplements have no ‘active’ ingredient. Just like every CAM…..A brouhaha the last few weeks has been the realization that there is a disconnect between the label on a supplement bottle and the actual ingredients. Often the supplements contain no ‘active’ ingredients, and I put active in quotes as these substances really do nothing. Supplements producers were substituting one inert substance for others.”
This strikes me as a case of unsophisticated thinking.
The author of the article assumes that herbal supplements don’t have the stated chemicals because they don’t contain the DNA of the plant. He implicitly assumes, therefore, that they are just placebos with rice in them.
Many of the articles you’ll see will also bunch all supplements together and infer that ALL supplements aren’t reliable.
I always prefer to buy from reliable companies that have a track record of passing third party testing from Consumer Labs. Generally, all of the big companies do well.
I try to buy standardized supplements so that I know what I’m getting.
If they do a study that shows that big companies don’t have the chemicals in them that are standardized, then that’s when my ears will perk up.
Sometimes, I’ll buy the whole herb if I feel a good effect from it. Generally, you can see if you’re getting the herb in question in the same way you know that you’re buying oregano when you go to the store. It has a certain smell, look, taste and effect.
I take supplements for the effects of it, not for the DNA that they contain.
It happens to be that I’ve never really had a very powerful effect from the supplements that they tested, especially ginseng. I’ve tried many types of ginseng and have been disappointed. I don’t think we’re getting the quality stuff.
What I find fascinating is that those who believe supplements are inert would be scared shitless to ingest them.
The Placebo Effect: It’s Complex
SBM talks much about the placebo and the nocebo effect.
Obviously, people experience a placebo effect and this is a confounding variable. However, SBM really overdoes this.
First, a placebo effect is a good thing. The outcome is to feel your best one way or another.
Now, for national guidelines, the placebo effect should be discounted. But for individuals, even if we feel better because of a placebo, that’s a good thing.
Second, placebos do not work for everyone or work to varying degrees based on the situation and genetics.
Henry K. Beecher, in a paper in 1955, suggested placebo effects occurred in about 35% of people, although this ranges based on the exact situation (from 0-100%). (R)
Children seem to have a greater response than adults to placebos.
I’m willing to bet the time period also matters. You can feel better at first from a placebo response, but in the long term, it will dissipate.
I also notice that people differ in how self-aware and introspective they are.
Next, there are two famous SNPs that influence how much we are affected by the placebo response.)
A variant of the gene for tryptophan hydroxylase 2 (an enzyme that synthesizes the neurotransmitter serotonin) is linked to reduced amygdala activity and greater susceptibility to the placebo effect. This likely means that lower levels of serotonin can enhance the placebo response. (rs4570625 GG is associated with a greater placebo response.)
People with a variation of the COMT V158M gene (rs4680 AA), which increases dopamine in the prefrontal cortex, are more likely to respond to the placebo treatment. Dopamine is thought to link to reward and ‘confirmation bias’ which enhance the sense that the treatment is working. This is more prominent in studies with subjective conditions such as pain and fatigue rather than objective physiological measurements. (R)
Response by Science Based Medicine
I have invited Steven Novella to a podcast to debate these ideas, and we will see if he accepts. If they have, I will update this post with a link to it.
HOW WOULD YOU RATE THIS ARTICLE?