We all know the quote: “There are three types of lies: lies, damned lies, and statistics.”
But like many things that are short enough to tweet, this statement is misleading.
Statistics don’t lie. People do, generally by omitting key pieces of information. Any statistic worth repeating should include two other numbers. If these are missing, the whole truth is being kept from you.
The two numbers to look for are:
- The sample size. This is the number of cases examined. If you base a conclusion on one example, you’re a politician or a pundit, relying on anecdotal evidence and shouting instead of facts and logic. While a large sample won’t guarantee reliability, a small sample will almost always be untrustworthy.
- The p-value. This is a little more complicated, but roughly speaking, it measures how convinced you should be. A small p-value (0.05 or less) means the evidence is very convincing.
I’ll talk more about these later. Until then, remember: if someone doesn’t give you these values, they’re not telling you the whole truth.