Claims, Studies, and Nonsense: Teasing it all Apart

Imagine for a moment that I want to convince you of something. I’m going to do my best. You have the obvious advantage of knowing walking into this post that it’s nonsense: ridiculous and silly.  The point isn’t the specifics; the purpose is to practice picking apart the process and the logic (or lack thereof).

Evaluate the following claim, based on the information I provide: Hearing loss in young adults is caused by brushing your teeth.

In a (pretend) study of 358 young adults who report hearing loss, 100% reported that they brushed their teeth daily. Every single one of the people in our study reported that they brushed their teeth every day, and every single one of them sustained clinically significant hearing loss before the age of 30. See? Brushing your teeth causes hearing loss.

Not convinced? I agree; that’s a very small sample size and thus our results might not be generalizable to a larger population. Let’s repeat the pilot study with a much larger sample of people under age 30 who have documented hearing loss. Let’s say 10,000 people participate in our survey. And … lo and behold, our correlation holds. 100% of these people (all of them under age 30 and have documented hearing loss) report brushing their teeth every day. It’s obvious something is going on here.

Next step – let’s suggest a mechanism. Let’s add a diagram of the ear and mouth, showing the opening of the Eustachian tube – right there in the nether regions of the mouth (well pharynx/nose but close enough)! There’s an obvious link between teeth and hearing if you look at the anatomy.

Now, if I add some science-y language about why brushing teeth creates an inflammation of the proximal end of the tube and that leads to migration of pathological particles into the middle ear and thus hearing loss, we’re looking better and better… right? Have I shoved you from NO WAY to no shit?

Don’t worry. I created a special product to solve this issue: a toothpaste formulated without the pathological particles that cause hearing loss. In a follow-up study, I give this toothpaste to 10,000 people under age 30 who do not have hearing loss and … in 3 months when we check their hearing – none have developed hearing loss at all. Proof positive that my toothpaste prevents hearing loss due to tooth brushing.


Totally ridiculous, right?

But let’s say that you really don’t know what causes hearing loss, which is totally reasonable (anyone who posts the number 1 cause of hearing loss in young people in the comments gets a virtual smooch from Peeka!). My explanation uses multi-syllabic science words and fancy diagrams. How would you know how to separate the wheat from the chaff?

You need to ask the “converse” question. In the original correlation, we stated that of all the folks with hearing loss we surveyed, all of them brushed their teeth. We need to ask “how many people brushed their teeth and did not develop hearing loss?” (Or if you say vaccines cause seizures in dogs and then show that 100% of dogs with seizures in your study were vaccinated, you must also ask – how many vaccinated dogs did NOT have seizures?) You need to looks at ALL the folks involved, not hand pick the ones that support your claim.

To establish causation (and not simply correlation) we need to rule out ALL other possible causes. That’s not easy to do, and that’s why you have to go to school for a lot of years and study like a maniac to become a doctoral level researcher. Could the toothbrushing be correlated but irrelevant? Could something totally different be causing the hearing loss? Since it is unethical to actually cause hearing loss in an experimental setting, you’d have to devise a study that didn’t aim to do that, yet would be able to establish what causes hearing loss. One way to do this would be forensic examination of the medical records of the people under 30 with hearing loss (hint – check out their employment and or hobbies).

Let’s say instead of preventing hearing loss, I claimed that my new product cured the common cold. I made the claim that if you took my new snake oil supplement, your symptoms would be gone in 7 -10 days. You see what I’m getting at: many (but not all) maladies resolve fully on their own in time. No matter what you do for the common cold, it lasts about 7-10 days. You can ease symptoms, and maybe reduce severity but in general, no matter what you do or don’t do, most people will see a complete recovery in that amount of time. If you propose a medication or supplement gets rid of a noxious symptom, you must ask – if I did nothing, would that symptom be gone in the same amount of time? Way too often, the answer is yes.

Same with injuries – remember time is a factor. I destroyed my knee falling on an icy rock about 5 years ago. For a bunch of reasons, I never went to a doctor. I never did anything to address it. And guess what? It eventually resolved completely. I can’t even remember which knee it was, which is impressive because I couldn’t kneel, and I limped for about 18 months. Had I been told some homeopathic remedy would heal it up nicely in 12 – 18 months… well, you see the issue. Time heals.

The time factor underlines the need for a control group. If you want to demonstrate that A causes B, you need to set up a double blind placebo-controlled experiment. You have to show that the same kind of people or dogs with the same ailments do not get better in the same amount of time without the remedy, medication, or other treatment.

Would you take a “drug” (or medication, or pharmaceutical – whatever you want to call it), or give such a substance to your child, or your pet, if there was no evidence that it worked better than doing nothing? No way, right? That seems obvious and silly – there’s no way you would waste the money and effort taking any substance that is not proven to be effective and safe. I use this same yardstick for ALL substances – medications, herbs, all remedies of any kind – they have to be proven (via real science) to be more effective than doing nothing. Any claims made by marketing efforts have to be evaluated. Heck, even food – claims that one way of eating is better than another has to be evaluated as a claim before I’ll make any changes to my diet or my dogs’ diet. Why? Because there’s an enormous amount of junk science and baseless claims out there. Every time I see it, I think “brushing your teeth causes hearing loss.” It might be easy to “prove” using junk science, but that doesn’t make it factual.

Folks working to convince us of these baseless claims might be dishonest and out to separate us from our cash, or they might be innocent and clueless, or simply operating from a place of belief rather than fact. I don’t need to take away anyone’s belief system. For some folks, that’s all they have. But I am short on cash. I only buy (literally and figuratively) stuff that is proven and has some shred of decent science behind it. If you ever want to evaluate a claim with me, I’d be happy to go through my thought process with you. I might not have a clear answer, but I am happy to share how I think about evaluating claims and making these sorts of decisions.

This entry was posted in food, writer's life. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s