beware anecdotal evidence
Anecdotal evidence comes in many forms, and sounds most believable.
Anecdotal evidence is evidence from anecdotes: evidence collected in a casual or informal manner and relying heavily or entirely on personal testimony.
Anecdotal evidence takes different forms. I’ve encountered many team members who are vivid storytellers, drawing from previous experiences with phrases like:
- “this is how I’ve seen it done before”
- “at my previous job we did…”
- “a former mentor of mine said to me…
The problem with anecdotes is that they cannot be assessed. This opens them up to a range of issues and biases, that it’s not possible as the person listening to the anecdote to correctly evaluate. Memories are not objective representations of the situation, and will suffer from selection and survivorship bias (the belief that because something went well, it was the right thing to do). Anecdotes suffer from availability bias, the more recent the information consumed, the more weight and relevance associated to it.
Taking the concrete examples;
- This is how I’ve seen it done before: there’s no evaluation of if “it” went well or poorly, and no objective analysis of the outcome, or believability of the decision makers. An anecdote containing an observation pretends to draw from “experience” but cannot be objectively evaluated. The same applies to “at my previous job”..
- A previous mentor of mine said to me: there’s no evaluation of if the situation in which the mentor said something is comparable to this situation. It’s also not possible to know if that mentor was a good or bad mentor, or if the advice was useful. A statement of fact without the surrounding information is an anecdote that cannot be evaluated.
It’s often easier to interpret personal experience and to understand the analogies and anecdotes of good storytellers than it is to interpret complex data, even if the data is available. This leads us to want to make decisions based on the anecdotes and not the data or evidence.
Beware anecdotal evidence, objectively evaluate the situation.