I’ve finished reading the book The Signal and the Noise: Why So Many Predictions Fail but Some Don’t by Nate Silver.
For those of you who have not heard about him, he is the mind behind the FiveThirtyEight blog, where he correctly predicted the winner of all 50 states and the District of Columbia for the US 2012 Presidential Election. He became a celebrity by basing his predictions on multiple poll data sources and the use of bayesian statistics. His methodological process performed much better than the predictions by the political pundits.
The book itself is a light read. It does not delve into deep statistical topics but provides a broad vision over a range of fields where statistics and big data have (or are poised to have) a big impact.
If you want to read a pair of reviews and a commentary from a reputed statistician, you can take a look at Andrew Gelman’s take on the matter: Two reviews of Nate Silver’s new book, from Kaiser Fung and Cathy O’Neil.
My review is based on some quotes from the book that I consider to be good food for thought.
One of the most important feedbacks in the market is between what he calls fear and greed. Some investors have little appetite for risk and some have plenty, but their preferences balance out: if the price of a stock goes down because a company’s financial position deteriorates, the fearful investor sells his shares to a greedy one who is hoping to bottom-feed. Greed and fear are volatile quantities, however, and the balance can get out of whack. When there is an excess of greed in the system, there is a bubble. When there is an excess of fear, there is a panic.
I found this description of the origins of bubbles and panics extremely easy to understand.
Political news, and especially the important news that really affects the campaign, proceeds at an irregular pace. But news coverage is produced every day. Most of it is filler, packaged in the form of stories that are designed to obscure its unimportance. Not only does political coverage often lose the signal - it frequently accentuates the noise.
This is something we tend to forget. We live in a world were information travels faster than ever and where the need for news agencies and media to provide breaking news all the time is creating a race to the bottom in the quality they offer. Additionally, the barrier of entry for creation of media that can have a global reach is extremely low. Moreover, nowadays, anyone can create content and can share their opinion disguised as facts. These unimportant news and the bogus online content are indeed accentuating the noise, making the act of finding useful information more difficult than it should.
Ultimately, the right attitude is that you should make the best forecast possible today - regardless of what you said last week, last month, or last year. Making a new forecast does not mean that the old forecast just disappears. (Ideally, you should keep a record of it and let people evaluate how well you did over the whole course of predicting an event.) But if you have reason to think that yesterday’s forecast was wrong, there is no glory in sticking to it. “When the facts change, I change my mind,” the economist John Maynard Keynes famously said. “What do you do, sir?”
And this is the beauty of Bayesian statistics, new facts help us refine our predictions.
Goodhart’s law, after the London School of Economics professor who proposed it, holds that once policy makers begin to target a particular variable, it may begin to lose its value as an economic indicator. For instance, if the government artificially takes steps to inflate housing prices, they might well increase, but they will no longer be good measures of overall economic health. At its logical extreme, this is a bit like the observer effect (often mistaken for a related concept, the Heisenberg uncertainty principle): once we begin to measure something, its behavior starts to change.
Not much to add. I particularly like when ideas from one field find a similar formulation in another one.
… as a default, just as we perceive more signal than there really is when we make predictions, we also tend to attribute more skill than is warranted to successful predictions when we assess them later. Part of the solution is to apply more rigor in how we evaluate predictions. The question of how skillful a forecast is can often be addressed through empirical methods; the long run is achieved more quickly in some fields than in others. But another part of the solution - and sometimes the only solution when the data is very noisy - is to focus more on process than on results. If the sample of predictions is too noisy to determine whether a forecaster is much good, we can instead ask whether he is applying the attitudes and aptitudes that we know are correlated with forecasting success over the long run. (In a sense, we’ll be predicting how good his predictions will be.)
For lack of better ways to assess performance, at least we should make sure that we are doing things right.
All in all, I found it a very enjoyable book accessible to the layman to get a different perspective of today’s world. If instead, you are looking for deep statistical analysis this is not the book you are looking for.