While “Don’t knock it until you try it” is the cliché, skeptic leader Brian Dunning thinks a better suggestion is, “Don’t try it until you knock it.” He was being somewhat sarcastic, as no opinion should be formed until all available evidence is considered. But his point was that personal experiences are inferior to data.

When it comes to favoring personal experience, this mistake is most frequently committed with regard to alternative medicine therapies and products. People often trust their perceptions more than any other source. But clinical test results provide a much better assessment of efficiency than someone’s word that it worked.

Our senses are prone to error and not everyone’s are as pronounced as the next person’s. Further, we all carry preconceived notions, biases, and expectations. Then there are mood swings, good days, bad days, and medium days. Hence, the assessment of a person grabbed off the street will be filtered through his or her prejudices, biases, preconceptions, preferences, and forgetfulness. It is impractical that their anecdote will be proof that the product or procedure will work (or fail) for everyone.

That’s why scientists use controlled, randomized trials. These will overcome the biases and other weaknesses addressed in the previous paragraph. As Dunning explained, “If you want to know whether listening to a binaural beat will make you fall asleep, a science fan knows not to try it to find out. She knows her sleepiness varies throughout every day and she knows that the expectation that it’s supposed to make her sleepy skews her perception. Instead, she looks at properly controlled testing that’s been done. Those subjects didn’t know what they were listening to, they didn’t know what it was supposed to do to them, and some of them unknowingly listened to a placebo recording. She knows the difference between real, statistically-sound data and one person’s anecdotal experience.”

Trying an untested product compromises a person’s ability to objectively analyze testing data about that product. This is also true in areas beyond alternative medicine. It can come into play while reading a horoscope, seeing an alleged ghost, or attending a psychic seminar. A cousin of mine did the latter and afterward, she excitedly posted there was “NO WAY” the psychic have known what she did, save an esoteric ability.

This is known as subjective validation, where an experience being personally impactful is considered evidence that the phenomenon is authentic. But with psychics, there are issues regarding cold reading, selective memory by audience members, and the lack of confirmatory testing. In my cousin’s case, the experience resonated with her because she had an intense experience, but that is not controlled data. A test could be designed, and in fact have been carried out, and no medium has ever consistently performed better than chance.

Still, persons will insist they know something works because it did for them. But this is not necessarily what happened. During the 2016 Olympics, athletes tried cupping and elastic kinesio tape, two alternative therapies completely lacking in evidence and with no plausible working mechanism behind them. Desperate for the extra edge against fellow world-class athletes, Olympians tried them and their personal experiences convinced them it worked. Yet these swimmers, runners, and gymnasts also had access to personal trainers, excellent nutrition, regimented rest periods, massage, icing, and other attentiveness that guaranteed they would perform at their peak. Giving the credit for victory to cupping wins the post hoc reasoning Gold Medal. Michael Phelps, after all, had collected plenty of first-place finishes before he started overheating, misshaping, and discoloring his back. There’s also the issue of those who tried these techniques and came in 17^th.  

Now we will examine another instance in which personal experience is treated as preferable to tested evidence. An Answers in Genesis chestnut is “Were you there,” which they genuinely consider a solid retort to proof about the age of the universe, Earth’s earliest days, and the development of homo sapiens. This is a vacuous, absurd reply. No one questions if Thomas Jefferson wrote the Declaration of Independence by asking historians if they were there when he dipped his quill into ink.

This supposed “gotcha” question reveals Young Earth Creationists’ substantial ignorance about how science works. As Dunning explained, “Scientific conclusions are never based simply on personal reports, but upon direct measurements of testable evidence. Nobody’s been to the sun, either, but we know a great deal about it because we can directly measure and analyze the various types of radiation it puts out.”

Likewise, chemists can’t see quarks and astronomers can’t see dark matter, but these entities can be measured and their attributes analyzed. The answer to a time-honored riddle is that a tree falling in forest does indeed emit soundwaves whether anyone is there to perceive them. Likewise, Earth formed, heated, and cooled regardless of whether this was being witnessed. Researchers understand the science behind the various dating methods that are used to determine this. In the same way that DNA is preferable to a witness at trial, radiometric dating, carbon 14 dating, and the speed of light are more important than the fact that no one from Earth’s earliest days is alive to recall it.

From those that deny something has happened, we move to those who assert that something has happened despite lacking concrete evidence for this claim. Specifically, some persons will wonder what’s the harm in using a product or technique if it makes a person feel better.

The harm can come in such forms as using Therapeutic Touch instead of antibiotics. Such methods not only waste time and money, but the patient may bypass legitimate medicine that would work. And in certain cases, such as with colloidal silver, black salve, and some essential oils, active ingredients are being ingested and overuse can be dangerous.

Another way in which personal experiences are trusted over clinical evidence is to claim, “I know what I saw.” Yet senses are prone errors, deficiencies, and bias. A popular video asks viewers to count the number of times a basketball is passed between a group of persons. When most respondents are asked to give that number, they usually give the correct response. But they also fumble when asked the follow up question, “What walked through the group while the ball was being passed?” It was a man in a gorilla suit ambling by, yet most viewers missed it because they were so consumed with keeping track of the number of tosses.

“Our memories change dramatically over time and were incomplete to begin with,” Dunning wrote. “And who knows how good was the data that your brain had to work with was to begin with? Lighting conditions can come into play, as can movement, distractions, backgrounds, and expectations of what should be seen. Possible misidentifications and perceptual errors all had a part in building your brain’s experience.”

That’s why Bigfoot sightings are not considered to be “case closed” proof of the beast’s existence. Anthropologists would need to look at testable evidence, which in the case of Sasquatch, is utterly lacking.

Out of frustration, aficionados of alternative medicine, conspiracy theories, cryptozoological critters, and a Young Earth will sometimes label scientists and skeptics “closed-minded.” But closed-mindedness includes refusing to change ideas no matter how much contrary evidence one is presented with. Since phone calls from 9/11 hijack victims described Islamic terrorists, Truthers concocted an evidence-free ad hoc assertion that those victims and the family members they telephoned were in on the government’s plot.

Meanwhile, being open-minded means changing your position when you discover you’ve been mistaken. I balked when I first heard that race was a social construct instead of a biological one. Using some of the reasoning addressed earlier, “I knew what I saw,” and clearly race had to be biological since I could see the difference between someone from Canada and someone from Nigeria. But as I learned about alleles, gene frequency, migratory routes, blood types, and the Human Genome Project, I changed my mind.

There are mounds of evidence that disprove such notions as chemtrails, chiropractic, a flat Earth, vaccine shedding being the cause of disease, and the first man being spoken into sudden existence 5,000 years ago. Yet hardcore adherents to these ideas consider the skeptic or scientist to be the closed-minded one.

People who assert this think of science as an unbending set of dictates from dour men in crisp lab coats or arrogant academics perched in ivory towers. However, science is a process that continually adapts, refines, improves, adds to, subtracts from, and alters data, according to where the evidence leads. And that refinement is subject to still further peer review, examination, and testing. That is why scientifically controlled data on the ability of a eucalyptus rub to cure rheumatism will always be preferable to what Aunt Tillie says.