The December 13th, 2010 Issue of The New Yorker has an article that all UFO mavens would do well to absorb.
It’s “The Truth Wears Off” by Jonah Lehrer [Page 52 ff.].
Lehrer recounts how science (and researchers) get snookered by “The Decline Effect” –the measurable drop off of one-time observations and data that, when first obtained, seemed invincible as proof of various phenomenon, but end up on second (or third) observations to be considerably less that originally measured.
Rhine’s studies, at Duke, of ESP are mentioned, as are pharmacological, ecological, psychological, biological studies.
The gist of the article is that studies are intrinsically flawed but why is still open to question.
Some relevant quotes from the piece, which UFO debaters should heed, include:
“Most of the time, scientists know what results they want and that can influence the results they get.” [Page 52]
“Asking people to put their perceptions into words led to dramatic decreases in performance.” [Page 53]
(Re: Rhine’s studies/experiments) “What he wanted to know was whether the images [from Zener card experiments] that got a second showing were more likely to have been identified the first time around. Could subsequent exposure have somehow influenced the initial results? Could the effect be the cause?” [Page 54]
“The extrasensory powers of…subjects didn’t decline – they were simply an illusion that vanished over time.” [Page 54]
“…the ‘decline effect’ deserves more attention: its ubiquity seems to violate the laws of statistics.” [Page 54]
“…[an] author might publish several critical papers, which distort his analysis.” [Page 54]
“…some – perhaps many – cherished generalities are at best exaggerated in their…significance and at worst a collective illusion nurtured by strong a-priori beliefs [are] often repeated.” [Page 55]
“…the problem seems to be one of subtle omissions and unconscious misperceptions, as researchers struggle to make sense of their results.” [Page 55]
“…act[s] of measurement [are] going to be vulnerable to all sorts of perception biases.” [Page 56]
“The problem of selective reporting is rooted in a fundamental cognitive flaw, which is that we like proving ourselves right an hate being wrong.” [Page 56]
“…after a claim has been systematically disproven…you still see some stubborn researchers citing the first few studies that show a strong effect. They really want to believe that it’s true.” [Page 56]
“Every researcher should have to spell out, in advance, how many subjects they’re going to use, and what exactly they’re testing, and what constitutes a sufficient level of proof. We have the tools to be much more transparent about our experiments.” [Page 56]
“Although…reforms would mitigate the dangers of publication bias and selective reporting, they still wouldn’t erase the decline effect. This is largely because scientific research will always be shadowed by a force that can’t be curbed, only contained: sheer randomness.” [Page 56]
“…a lot of extraordinary scientific data are nothing but noise.” [Page 57]
“…dramatic findings are also the most likely to get published in prestigious journals…” [Page 57]
“…the decline effect is actually a decline of illusion.” [Page 57]
“While Karl Popper imagined falsification occurring with a single, definitive experiment – Galileo refuted Aristotelian mechanics in an afternoon – the process turns out to be much messier than that. Many scientific theories continue to be considered true even after failing numerous experimental tests.” [Page 57]
“Even the law of gravity hasn’t been perfect at predicting real-world phenomena. In one test, physicists measuring gravity…in the Nevada desert found a two-and-a-half-per-cent
discrepancy between the theoretical predictions and the actual data. Despite these findings…The law of gravity remains the same.” [Page 57]
“Such anomalies demonstrate the slipperiness of empiricism.” [Page 57]
“Although many scientific ideas generate conflicting results and suffer from falling effect sizes, they continue to get cited in textbooks…Why? Because these ideas seem true. Because they make sense. Because we can’t bear to let them go. And this is why the decline effect is so troubling. Not because it reveals the human fallibility of science, in which data are tweaked and beliefs shape perceptions…And not because it reveals that many of our most exciting theories are fleeting fads and soon will be rejected.” [Page 57]
The decline effect is troubling because it reminds us how difficult it is to prove anything. We like to pretend that our experiments define the truth for us. But that is often not the case. Just because an idea is true doesn’t mean it can be proved. An just because an idea can be proved doesn’t mean it’s true.” [Page 57]
“When he experiments are done, we still have to choose what to believe.” [End of article]
Scientists and (some) “ufologists” like to think they have a lock on truth. But as Mr. Lehrer’s New Yorker article shows, truth is as elusive as ever, and nothing is certain.
No comments:
Post a Comment