the signal and the noise

Nate Silver, “the signal and the noise: why so many predictions fail—but some don’t” Penguin Press, 2012, 454 pp.

The Signal and the Noise was named Amazon's #1 Best Non-Fiction Book for 2012 and by the Wall Street Journal as one of the ten best books of nonfiction published in 2012.  This book is less about what we know than about the difference between what we know and what we think we know. It recommends a strategy so that we might close that gap.   The strategy involves becoming familiar with the Bayesian way of thinking about prediction and probability.  We need to distinguish a true signal from a universe of noise data.  This is not a book for those who believe good ideas and analysis of that good idea can fit on a bumper sticker.  As I read this book, two ideas came to me: I may be getting too-late smart, and I need a new blogsite category label for books I judge necessary to read for any educated person—this book deserves that yet-to-be-determined label (which I have designated "Education").

In keeping with his own aim to seek truth from data, Silver visits the most successful forecasters in a range of areas, from hurricanes to baseball, from the poker table to the stock market, from Capitol Hill to the NBA. He explains and evaluates how these forecasters think and what bonds they share. What lies behind their success? Are they good—or just lucky?

My Notes:

Pg 1:  The original revolution in information technology came with the printing press invention by Johannes Gutenberg in 1440 which made information available to the masses.  It was a spark for the Industrial Revolution in 1775.  The printing press set in motion the events that would produce the European Enlightenment and the founding of the American Republic.   

Pg. 109:  The National Hurricane Center nailed its forecast of Katrina; it anticipated a potential it on the city almost five days before the levees were breached, and concluded that some version of the nightmare scenario was probable more than forty-eight hours away.  The tragedy is that so many did not heed the warning.

Weather forecasting is one of the success stories in this book, a case of man and machine joining forecast to understand and sometimes anticipate the complexities of nature.  Just twenty-five years ago, the forecast of where a hurricane would hit three days in advance of landfall missed by an average of 350 miles.  Today the average miss is only about one hundred miles.  However, judging how forceful hurricane will be is still not very accurate. (footnote pg. 127)

Pg. 152:  Tehran, Iran has a population of about 13 million with infrastructure and poverty similar to Haiti.  Forecasters believe a 7.0 or greater magnitude earthquake (the same magnitude that struck Haiti) will strike the area every 300 years.  When it does they will have between two and four million fatalities—Haiti had 316,000 fatalities. 

Pg. 168:  Our tendency to mistake noise for signal can occasionally produce some dire real-world consequences.  Japan, despite being extremely seismically active, was largely unprepared for its devastating 2011 earthquake.  The Fukushima nuclear reactor was built to withstand a magnitude 8.6 earthquake, but not a 9.1.  Archaeological evidence is suggestive of historic tsunamis on the scale of the 130-foot waves that the 2011 earthquake produced, but these cases were apparently forgotten or ignored.

Pg. 182:  When a group of economists give their GDP forecast, the true 90 percent prediction interval---based on how these forecasts have actually performed—spans about 6.4 points of GDP (equivalent to a margin of error of plus or minus 3.2 percent.  So when you hear on the news that GDP will grow by 2.3 percent next year that means it could quite easily grow at a spectacular rate of 5.7 percent or it could fall by 0.7 percent.  (In other words, a spectacular boom or a severe recession).  Economists aren’t unique in this regard.  Results like these are the rule; experts either aren’t very good at providing an honest description of the uncertainty in their forecasts, or they aren’t interested in doing so.  This trait has been identified in many other fields, including medical research, political science, finance, and psychology.

 Pg. 195:  Weather prediction, however, is one of the real success stories in this book.  Forecasts of everything from hurricane trajectories to daytime high temperatures have gotten much better than they were even ten or twenty years ago, thanks to a combination of improved computer power, better data-collection methods, and old-fashioned hard work. 

Pg. 245:  Studies show that if a woman does not have cancer, a mammogram will incorrectly claim that she does about 10 percent of the time.  If she does have cancer, on the other hand, they will detect it about 75 percent of the time.  But if you apply Bayes’s theorem to these numbers, you’ll come to a different conclusion: the chance that a woman in her forties has breast cancer given that she’s had a positive mammogram is still only about 10 percent.  These false positives dominate the equation because very few young women have breast cancer to begin with.  For this reason many doctors recommend that women do not begin getting regular mammograms until they are in their fifties and the prior probability of having breast cancer is higher.

Consider September 11 attacks.  Our estimate of the possibility, could be, 1 chance in chance in 20,000, or 0.005 percent.  Also, we would have assigned a very low probability to a plane hitting the World Trade center by accident empirically based on the previous 25,000 days of aviation over Manhattan prior to September 11, there had been two such accidents: one involving the Empire State Building in 1945 and another at 40 Wall Street in 1946.  That would make the possibility of such an accident about 1 chance in 12,500 on any given day or 0.008%.

Bayes’s Theorem—Terror Attack Example:

Prior Probability
Initial estimate of how likely it is that terrorists would crash planes into Manhattan skyscrapers	x	0.005%
A New Event Occurs: First Plane Hits WTC
Probability of a plane hitting if terrorists are attacking Manhattan skyscrapers	y	100%
Probability of plane hitting if terrorists are not attacking Manhattan skyscrapers (i.e. an accident).	z	0.008%
Posterior Probability
Revised estimate of probability of terror attack, given first plane hitting WTC		38%

At this point, the Bayesian estimate that this is a terror attack is 38% certain.

The idea behind Bayes’s theorem, however, is not that we update our probability estimates just once.  Instead, we do so continuously as new evidence presents itself Thus, our posterior probability of a terror attack after the first plane hit, 38 percent, becomes our prior possibility before the second one did.  And if you go through the calculation again to reflect the second pane hitting the WTC, the probability that we were under attack becomes a near certainty: 99.99 percent.

Bayes’s Theorem—Terror Attack Example cont…:

Prior Probability
Revised estimate of probability of terror attack, given first plane has hit the WTC	x	38%
A New Event Occurs: First Plane Hits WTC
Probability of a plane hitting if terrorists are attacking Manhattan skyscrapers	y	100%
Probability of plane hitting if terrorists are not attacking Manhattan skyscrapers (i.e. an accident).	z	0.008%
Posterior Probability
Revised estimate of probability of terror attack, given second plane hitting WTC		99.99%

Bayes’s Theorem—Global Warming Example.  How is the 95% current prediction that global warming is occurring affected if no net warming occurs for 10 years. (p. 407

Prior Probability
Initial estimate of how likely it is that global temperatures are increasing	x	95%
A New Event Occurs: No Net Warming Over 10 years
Probability of no net warming over 10 years if global warming hypothesis is correct	y	15%
Probability of no net warming over 10 years if global warming hyphthesis is false	z	50%
Posterior Probability
Revised estimate of how likely it is that global warming is occurring, given no net temperature increase over 10 years		85%

Bayes’s theorem does require us to think probabilistically about the world.  As a matter of fact, some well-respected statisticians have begun to argue that frequentist statistics should no longer be taught to undergraduates.  And some professions have considered banning Fisher’s hypothesis test from their journals.  In fact, if you read what’s been written in the past ten years, it’s hard to find anything that doesn’t advocate a Bayesian approach (pg. 260).

Bayes was a contemporary of Adam Smith.  Smith’s invisible hand’ might be thought of as a Bayesian process, in which prices are gradually updated in response to changes in supply and demand, eventually reaching some equilibrium. (p. 332)

Pg. 329:  In the 1950s, the average share of common stock in an American company was held for about six years before being traded.  By the 2000s, the velocity of trading had increased roughly twelve-fold and was now traded every six months.

Pg. 357:  Irrational behavior in the markets may result precisely because individuals are responding rationally according to their incentives.  So long as most traders are judged on the basis of short-term performance, bubbles involving large deviations of stock prices from their long term values are possible—and perhaps even inevitable.

Pg. 375:  The greenhouse effect was first proposed by the French physicist Joseph Fourier in 1824 and is usually regarded as having been proved by the Irish physicist John Tyndall in 1859.

Pg. 378:  Healthy skepticism needs to weigh the strength of new evidence against the overall strength of the theory, rather than rummaging through fact and theory alike for argumentative and ideological convenience, as is the cynical practice when debates become partisan and politicized.

Pg. 385:  A 2008 survey of climate scientists found that 94 percent were agreed that climate change is occurring now, and 84 percent were persuaded that it was the result of human activity.  But there was much less agreement about the accuracy of climate computer models.  Just 19 percent, for instance, thought they did a good job of modeling what sea-rise levels will look like fifty years hence.

Pg. 392:  El Nino is a cycle which evolves over intervals of about three years at a time.  It is instigated by temperature shifts in the waters of the tropical Pacific.  El Nino years, when the cycle is in full furce, produce warmer weather in much of the Northern Hemisphere, and probably reduce hurricane activity in the Gulf of Mexico.  La Nina years, when the Pacific is cool, do just the opposite.