Why is anecdotal evidence unreliable

However, when you apply the same logic to the claim that the drug chloroquine prevents or treats COVID, a drug used to treat malaria and rheumatoid conditions, such as arthritis, with its potentially serious side effects, the stakes become higher 6. To examine the experiences of people of different ages, gender and ethnicities to see what effect, if any, dock leaves had on their nettle stings.

We call this process a systematic review of the evidence 7. A systematic review looks at all the current evidence available to answer a question. Combining all the evidence, systematically, is likely to provide a more reliable, robust answer on which to base your healthcare decisions. It is unwise to base healthcare decisions on personal experiences — or even a series of personal experiences.

Personal experiences or anecdotes do not provide robust evidence. Systematic reviews of all the available evidence on an intervention help minimise bias and produce more reliable evidence to support informed healthcare decisions. Please note, we cannot give medical advice and we will not publish comments that link to commercial sites or appear to endorse commercial products.

References pdf. Visit the iHealthFacts website to submit a health claim to be fact checked, or search for previously answered questions. Visit the Teachers of Evidence-Based Health Care website , where you can find resources which explain and illustrate why anecdotes are unreliable evidence.

Elaine is working on a variety of clinical trials. Her current projects include iHealthFacts, an online resource where the public can quickly and easily check the reliability of a health claim circulated by social media.

We again confirmed that participants were more likely to mention prior beliefs and personal experience as the basis for their decision to implement high plausible interventions than low plausible interventions; additionally, participants were more likely to reference prior beliefs than the study itself as the basis for their decision, but only for the high plausible interventions.

One important difference between the two experiments was that while evidence strength ratings were lower for the more plausible intervention in Experiment 1, evidence strength ratings were higher for the more plausible interventions in Experiment 2. Additionally, whereas participants were more likely to identify specific methodological flaws in the more plausible study in Experiment 1, there were no differences in flaw detection rates for high versus low plausible interventions in Experiment 2.

Thus, it is unclear why participants rated the high plausible interventions as having greater evidence quality in Experiment 2. One possibility is that they did not evaluate the evidence as critically because of their prior beliefs e. Additionally, the participant samples differed between the two experiments, with an undergraduate sample in Experiment 1 and Prolific participants in Experiment 2; thus, there may be baseline differences in propensities to critically evaluate scientific evidence between these two samples.

However, the fact that we still observed a dissociation between perceived evidence quality and decisions to implement the learning interventions for both low and high plausible interventions in Experiment 2 suggests that evidence quality was again underweighed as a decision factor.

The choice of an educational context for the present studies was deliberate. As Halpern argues, much of the fault lies in science communication. A better understanding of how best to communicate the science of education, such that stakeholders will consider and also critically evaluate science-based recommendations, is crucial.

However, increasing critical evaluation of evidence alone may not be sufficient, as our studies suggest that making decisions about highly plausible learning interventions overrides low-quality evidence as a factor in hypothetical implementation decisions.

Here, we presented participants with conclusions that were not supported by the evidence. For example, many educators continue to incorporate the learning styles theory into their pedagogy, despite the consistent lack of evidence that teaching students in their preferred learning style improves learning e. However, further research is also necessary to address the issue highlighted by Halpern and Seidenberg —how to convince stakeholders to rely on high quality evidence in the face of personal beliefs supporting a view not consistent with the science.

People might recognize flaws in a study and nonetheless choose to implement the recommendations based on the study—particularly if those recommendations are consistent with their prior beliefs. People may also struggle to critically evaluate education studies in particular because of strong prior beliefs about the effectiveness of certain learning interventions.

Consistent with our own findings, the findings of Newton and Miah imply that, for many educators, there is a disconnect between their belief about the scientific support for a learning theory and their practice. Educators may persist in using the learning styles theory despite their awareness of the strong body of evidence against the effectiveness of learning style interventions, possibly due to positive personal experiences with implementing the learning styles theory.

The present results are limited by use of only four exemplar scenarios in a single context—educational achievement. Future research should systematically consider the conditions under which flawed evidence is nonetheless considered to support implementation decisions in different domains. Our focus in the present studies was to examine how the plausibility of an intervention influenced evaluation of flawed evidence and, ultimately, implementation likelihood. However, we did not explicitly test how other baseline conditions affect implementation likelihood, such as high quality science evidence in the context of low or high plausibility , or no evidence i.

Including the full set of possible conditions under a variety of controlled contexts is necessary for a more complete understanding of how scientific evidence and prior beliefs influence decision-making. Another limitation is that participants in Experiment 2 may have been biased by our initial questions asking about their beliefs about the plausibility and practicality of the learning interventions.

Further work is necessary to test the extent to which prior belief assessments affect later critical analysis of evidence as well as evidence-based decisions.

A final limitation is that the implementation judgments used in our studies were hypothetical and perhaps not relevant to the participants in our study. To what extent might implementation decisions be influenced by anecdotes and prior beliefs when making actual decisions or at least hypothetical decisions that might be more relevant to the participants? It is possible that individuals with more domain knowledge are generally more critical of evidence regardless of the presence of anecdotes; for example, teachers might be more likely to consider the possibility that coming from a math class to take a math test could present a confound, and they could weigh the flaws more heavily in their implementation judgments.

On the other hand, given the findings of Newton and Miah and our own findings, teachers might persist in implementing an intervention even if they acknowledge that it is backed by flawed science, particularly if the intervention jibes with their own personal experience or the experiences of other instructors.

In conclusion, our studies show that decisions to implement interventions backed by flawed scientific evidence are strongly influenced by prior beliefs about the intervention, particularly in regards to personal experience and plausibility.

Moreover, identifying the flawed evidence behind the interventions was not enough to dissuade participants from implementing the interventions. American Association of Poison Control Centers.

Track emerging hazards. Beck, D. The appeal of the brain in the popular press. Perspectives on Psychological Science, 5 6 , — Article PubMed Google Scholar. Blackman, H. Center for Research Use in Education.

Borgida, E. The differential impact of abstract vs. Journal of Applied Social Psychology, 7 3 , — Article Google Scholar. Boser, U. What do people know about excellent teaching and learning? Center for American Progress. Bromme, R. Educational Psychologist, 49 2 , 59— Burrage, M. Cabral, T. Post-class naps boost declarative learning in a naturalistic school setting. NPJ Science of Learning. Champely, S. R package version 1. Cousineau, D. Tutorials in Quantitative Methods for Psychology, 1 , 42— Ecker, U.

Do people keep believing because they want to? Preexisting attitudes and the continued influence of misinformation. Eggertson, L. Lancet retracts year-old article linking autism to MMR vaccines.

Enkin, M. Using anecdotal information in evidence-based health care: Heresy or necessity? Annals of Oncology, 9 9 , — Fagerlin, A. Fernandez-Duque, D. Superfluous neuroscience information makes explanations of psychological phenomena more appealing. Journal of Cognitive Neuroscience, 27 5 , — Garcia-Retamero, R. The power of causal beliefs and conflicting evidence on causal judgments and decision making. Learning and Motivation, 40 3 , — Gharpure, R.

Morbidity and Mortality Weekly Report, 69 23 , — Halpern, D. Chapter 3: dissing science—selling scientifically based educational practices to a nation that distrusts science. Phye, D. Levin Eds. Academic Press. Chapter Google Scholar. Herr, P. Effects of word-of-mouth and product-attribute information on persuasion: An accessibility-diagnosticity perspective. Journal of Consumer Research, 17 4 , — Hopkins, E. The seductive allure is a reductive allure: People prefer scientific explanations that contain logically irrelevant reductive information.

Cognition, , 67— Hornikx, J. A review of experimental research on the relative persuasiveness of anecdotal, statistical, causal, and expert evidence. Studies in Communication Sciences, 5 1 , — Google Scholar. Combining anecdotal and statistical evidence in real-life discourse: Comprehension and persuasiveness. Discourse Processes, 55 3 , — Im, S. Extending the seductive allure of neuroscience explanations effect to popular articles about educational topics. British Journal of Educational Psychology, 87 4 , — Jaramillo, S.

The impact of anecdotal information on medical decision-making. Kazoleas, D. A comparison of the persuasive effectiveness of qualitative versus quantitative evidence: A test of explanatory hypotheses. Communication Quarterly, 41 1 , 40— Kirschner, P.

Do learners really know best? Urban legends in education. Educational Psychologist, 48 3 , — Klaczynski, P. Motivated scientific reasoning biases, epistemological beliefs, and theory polarization: A two-process approach to adolescent cognition. Child Development, 71 , — Koballa, T. Persuading teachers to reexamine the innovative elementary science programs of yesterday: The effect of anecdotal versus data-summary communications. Journal of Research in Science Teaching, 23 5 , — Koehler, J.

The influence of prior beliefs on scientific judgments of evidence quality. Organizational Behavior and Human Decision Processes, 56 1 , 28— Kouzy, R. Kunda, Z. The case for motivated reasoning. Psychological Bulletin, , — Lewandowsky, S. Misinformation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest, 13 3 , — Motivated rejection of science.

Current Directions in Psychological Science, 25 4 , — Liu, F. It takes biking to learn: Physical activity improves learning a second language. Lomas, J. Opinion leaders vs audit and feedback to implement practice guidelines: Delivery after previous cesarean section. JAMA, 17 , — Lord, C. Biased assimilation and attitude polarization: The effects of prior theories on subsequently considered evidence.

Journal of Personality and Social Psychology, 37 11 , — Luo, M. Credibility perceptions and detection accuracy of fake news headlines on social media: Effects of truth-bias and endorsement cues.

Communication Research. Macpherson, R. Cognitive ability, thinking dispositions, and instructional set as predictors of critical thinking. Learning and Individual Differences, 17 2 , — Matute, H. Illusions of causality at the heart of pseudoscience. British Journal of Psychology, 3 , — McCauley, M. Exploring the choice to refuse or delay vaccines: A national survey of parents of 6- through month-olds. Academic Pediatrics, 12 5 , — Merchant, R. Nancekivell, S. Journal of Educational Psychology, 2 , — Newton, P.

Frontiers in Psychology. Parong, J. Learning science in immersive virtual reality. Journal of Educational Psychology, 6 , — Pashler, H.

Learning styles: Concepts and evidence. Psychological Science in the Public Interest, 9 3 , — Pennycook, G. See Subscription Options. Discover World-Changing Science. Get smart. Sign up for our email newsletter. Sign Up. Support science journalism. Knowledge awaits. See Subscription Options Already a subscriber? Create Account See Subscription Options. Continue reading with a Scientific American subscription.