Bias and Fact Checking

David Whitton

What role does bias play in science and writing about science and how can we know when something I’m reading is credible. How do we fact-check what we read?

1. The Rattlesnake King

These days Clark Stanley is a Wikipedia entry; he’s a photo of a guy with a big hat and an old-fashioned mustache. But in the 1880s (1), he was or anyway claimed to be, a cowboy.

In the course of his travels throughout the American Old West—so the story goes—Mr. Stanley, the wandering cowboy, made the acquaintance of a Native American doctor, from whom he learned the secrets of indigenous medicine.

Armed with these secrets, Clark Stanley transformed himself into The Rattlesnake King, a salesman who travelled town to town throughout the U.S. selling his special “snake oil liniment.” Mr. Stanley claimed to all who would listen that his liniment—or lotion, as we call it these days—would, when rubbed upon the skin, cure any number of problems, including

“rheumatism, neuralgia, sciatica, sprains, bunions, and sore throat, for bites of animals and reptiles, for all pains and aches in flesh, muscle and joints, as a relief for tic douloureux, and as a cure for partial paralysis of the arms and of the lower limbs, and as a remedy for paralysis and effective to reduce enlarged joints to their natural size . . .” (2)

In 1916, the U.S. government's Bureau of Chemistry did a chemical analysis of Mr. Stanley's liniment, and found it to contain no snake oil of any kind, but rather a mixture of “mineral oil, a fatty oil believed to be beef fat, red pepper, and turpentine”—a concoction that would have no effect whatsoever on that long list of complaints. (3)

Have you heard the term “snake oil salesman”? It’s another way of saying “scam artist”—someone who sells you fake cures in exchange for your hard-earned cash—and Clark Stanley was the original.

After its investigation, the Bureau of Chemistry found Clark Stanley to be guilty of false advertising and fined him $20 U.S. (just under $450 in 2021 dollars).

This punishment, however, would not stop people from making false claims or spreading bad information—far from it. Since Clark Stanley’s time, many, many others—from Instagram influencers to huge tech companies—have made lots of money selling us useless products, or false science, that they claimed would fix all of our problems, from bad hair to bad skin to bad moods.

A hundred years ago, a guy like Clark Stanley would have had to pack up his bottles and travel around the country to sell his fake snake oil, but these days anyone who wants to can jump online and sell stuff from the comfort of their living room.

2. Quick fixes and wishful thinking

We all want to believe in miracle cures, or quick fixes—simple solutions to all of life’s problems. It’s so much a part of who we are that entire multi-billion dollar industries have sprung up to take advantage.

And this desire, for the easy solution, can be as true in the world of science as it is in any other part of our society. Scientists are human, after all, and as prone to biases and wishful thinking as you or I.

Truth is, the history of science is littered with examples of bias—in medicine, or chemistry, or any other kind of research, really. And because of it, data from scientific studies have been misunderstood, warped, or even made up completely. (4)

As a matter of fact, there are so many biases in scientific research that they’ve been studied and given names.

Confirmation bias is one of them.

Confirmation bias is a mistake that happens when a scientist starts her research with a theory about how something works and looks only for data that support that theory. Like for instance, say a scientist has a hunch: that rattlesnake oil can cure the covid-19 virus. When that scientist does her research, she might, if she’s suffering from confirmation bias, ignore any data that suggest her hunch is bogus. Or she might look only for data that support her hunch. Or both. She’s not doing it on purpose; it’s just how our brains work sometimes. (5)

Another bias is called publication bias. Scientists like to be published in journals related to their fields. It makes them look good, and it helps them to fund their research. But science journals tend to prefer exciting or newsworthy stories. They might publish a study that shows that rattlesnake oil cures covid-19 but ignore a study that shows that it does not. So scientists, if they’re eager to get recognized in a journal, might look only for data that allows for their studies to be published.

There are all sorts of other biases too that can affect how data is understood or analyzed or presented to us non-scientists. Google them if you’re interested. It’s a comfort to know that scientists can mess up as badly as the rest of us.

3. As if that weren’t confusing enough . . .

By the time all of this scientific knowledge makes its way down to us non-scientists, often in the form of newspaper headlines or magazine pieces or YouTube explainers, it's possible that it's been distorted even further, by the writer or journalist or random internet person who brought it to our attention.

Think of that snake oil cure for covid-19.

What if a journalist came across that study, and decided to write an article about it? What if he wrote a glowing review of its findings, and interviewed a bunch of people who said great things about the scientist and her methods? What if this article was published on the splash page of the newspaper’s website? We could believe it then, right?

Well . . . maybe. Or maybe not.

Just like in science, there are a ton of biases in the media.

Some of these biases might arise from the journalist himself, who, after all, is human, like you and me and the scientist and the snake oil salesman, full of his own likes and dislikes.

Think about it. Maybe this journalist had money invested in a snake oil company. In that case, would he not be more likely to write something positive about snake oil? Or maybe just the opposite: maybe his mom made him drink snake oil when he was a kid, and he always hated the taste. Might he not then be more inclined to write something negative? (6)

Some other biases might come from the media outlet that employs the writer—the TV station, website, or newspaper—which, after all, makes its money by selling ads. Sensational or controversial stories—like the ones about the covid-19 vaccine, or anti-masking protests—get more clicks from readers. And more clicks equal more money from advertisers. So the people who run these media outlets have a reason to tell certain stories and ignore others. (7)

Always remember, the stories you read are the result of many decisions, made by humans, who make a lot of mistakes.

4. So how can we know what’s real and what’s not?

That’s the million-dollar question.

What would you do if someone you know, or someone you know of, like a musician or actor, ’grammed a photo of Clark Stanley's Snake Oil Liniment, and in the caption below claimed that rubbing it on your forehead would give you 95% immunity to the covid-19 virus?

What could you do to confirm or deny such a claim?

Here’s a suggestion. Think “Snake Oil Salesman.” Take the first letter from each of those words. What do you get? S.O.S.

This stands for Search, Observe, Substantiate—and it’s a pretty good way to get to the truth.

Well, the first thing you can do is to SEARCH the claim. Open up a new tab on your browser and hit up your favourite search engine. “Clark Stanley's Snake Oil Liniment kills covid-19.” What kind of links do you find? Are there multiple, reliable sources, like a government or university website, that agree or that disagree? (8)

Next, OBSERVE the person making the claim. It’s important always to try to keep in mind who is talking and why they’re talking. Do they seem super-sketchy? Way too slick? Are they selling something? A product, or a political point of view? (9)

If you’re still in doubt, try to SUBSTANTIATE the claim. We can't always research every little statement or claim that we come across on the internet. Who has the time, when there's so much Animal Crossing to play?

That's where mainstream news outlets can be your friend. Some, although certainly not all, employ fact checkers, people whose days are devoted to figuring out whether claims made by one person or another are true. Unlike, say, some random dude on TikTok.

Or, if you'd prefer, you could go to a fact-checking site like snopes.com or factcheck.org, whose entire purpose is to look into stories you see on the internet.

5. The cure for the Rattlesnake King

If he were alive today, Clark Stanley, the Rattlesnake King, would for sure be selling his fake cures on the internet. After all, it’s as easy as it’s ever been to part a fool from his money—and maybe even easier. With all the information available on our phones and laptops, it can be a huge challenge to separate fact from fiction. But if we remember a few simple strategies, we can make it a whole lot harder for all the other snake oil salesmen.

Footnotes

  • 9 CrashCourse. (2019, January 29). Who Can You Trust? Crash Course Navigating Digital Information #4 [Video]. YouTube. https://youtu.be/o93pM-b97HI

Bibliography

  • CrashCourse. (2019, January 29). Who Can You Trust? Crash Course Navigating Digital Information #4 [Video]. YouTube. https://youtu.be/o93pM-b97HI