How Not To Be Wrong: The Power of Mathematical Thinking

Jordan Ellenberg, “How Not To Be Wrong: The Power of Mathematical Thinking” The Penguin Press, 2014, 437 pp.

The math I learned in school consisted of a dull set of rules.  Later in life I found I had a great curiosity about Math, albeit no noticeable talent.  In any event, it is useful to consider math, as the author states, as ‘an extension of common sense by other means.’  Most people just do not think probabilistically and often lack common sense when statistical reasoning is involved.

For instance, the author offers the story about a person receiving a newsletter from a stockbroker predicting that a certain stock is about to rise—a week later, this prediction turns out true.  This is followed by nine more newsletters, one each week, successfully predicting the rise or fall of a particular stock. Clearly, when this apparently insightful stockbroker eventually asks you to invest money with his or her firm, you might be inclined to agree. After all, 10 successes in a row cannot be merely accidental, can they?

From a mathematical perspective, however, this series of events looks entirely different. The stockbroker could have started by mailing the first newsletter to 10,240 people. Half of those newsletters would have predicted the stock to rise and half to fall. The following week's newsletter would have been sent only to those 5,120 people that had received the successful prediction. Continuing with this scheme, after the 10th week there would have been 10 people who had received a string of 10 winning predictions, independently of the actual behavior of the market.  

(I believe this is a true story—I have read it before.  Additionally, when I was in college I was aware of other students involved in sending out chain letters.  I was too involved in playing poker at the time to get involved in the chain letters!).

This book, in spite of its title, does not teach you how never to be wrong.  

As a matter of fact, it is fine to be wrong if your mistakes are the consequence of taking calculated risks.

My Notes:

Pg. 8:  The U.S Army, during WWII, presented Mathematician Abraham Wald at Columbia University a problem to determine how to better protect aircraft to survive an attack.  The army provided a scatter-gram type display of damage to areas of the returning planes to Wald.  He noticed that very few of the damaged areas displayed engine damage.  He ultimately determined this was because the engine was the most sensitive area and bullet damage to this area would result in catastrophic failure.  This is why little damage to this area was in the scatter-gram displays.  To a mathematician, the structure underlying the bullet-hole problem is a phenomenon called survivorship bias.  It arises again and again in all kinds of contexts.  (Mutual funds is another: judging a decade’s worth of mutual funds by the ones that still exist, at the end of the ten years is like judging pilots’ evasive maneuvers by counting the bullet holes in the planes that come back).

Pg. 28:  Subsequent history has failed to confirm Laffer’s conjecture that lower tax rates would raise tax revenue.  When Reagan cut taxes the result was less tax revenue, not more.  Revenue from personal income taxes (per person, adjusted for inflation) fell by 9 percent from 1980 to 1984, even though average income (per person, adjusted for inflation) grew by 4 percent over this period. 

Pg. 112:  The null hypothesis is the hypothesis that the intervention you’re studying has no effect.  The test to determine that the intervention you’re studying has no effect is called the null hypothesis significance test.  It was developed in its most commonly used form by R.A. Fisher, the founder of the modern practice of statistics, in the early twentieth century. 

Null Hypothesis procedure example:

1.       Run an experiment

2.      Suppose the null hypothesis is true (i.e., intervention has no effect) and let p be the probability (under that hypothesis) of getting results as extreme as those observed.

3.      The number p is called the p-value.  If it is very small, rejoice; you get to say your results are statistically significant.  If it is large, concede that the null hypothesis has not been ruled out.  (The p-value of .05, 1/20, is commonly used)

Pg. 121:  If only we could go back in time and declare that a result passing Fisher’s test with a p-value of less than 0.05 was ‘statistically noticeable’ or ‘statistically detectable’ instead of ‘statistically significant’.  That would be truer to the actual meaning of the method, which merely counsels us about the existence of an effect but is silent about its size or importance.   

Pg. 147:  John Ioannidis 2005 paper “Why Most Published Research Findings Are False” touched off a fierce bout of self-criticism in the clinical sciences.  It can be proven, he writes, that most claimed research findings are false. 

Pg. 150:  The crisis of replicability is real.  In a 2012 study, scientists set out to replicate some of the most famous experimental results in the biology of cancer, fifty-three studies in all.  In their independent trials, they were able to reproduce only six.  (The .05 test of significance guarantees that on average you will falsely report success 5% (1 in 20) of the time.  Also, failures do not get published.)

Pg. 236:  The WSJ reported on June 24, 2013 that “The Social Security Administration’s inspector general said the agency improperly paid $31 million in benefits to 1,546 Americans believed to be deceased.  To make matters worse, the Administration had death certificate information on each person filed, suggesting it should have known the Americans had died and halted payments. 

Note: this $31 million represents .004% of the benefits disbursed annually by the SSA.  So the agency is already extremely good at knowing who’s alive and dead.  Getting even better at that distinction, in order to eliminate those last few mistakes, would be expensive. (Journalists don’t present evidence using mathematical common-sense.  This story would not even get presented if it was fairly reported, it is not news of a sensational nature.

Pg. 239:  Pascal’s wager:  It is better to believe in God than to not believe.  What if he really exists and you were not a believer?  So he believed and warned others against non-belief.  (Apparently God either doesn’t notice or doesn’t care if you are faking it).