There are books and there are books, and then there is The Black Swan, by Nassim Nicholas Taleb It happens to be interesting, infuriating, and quite a must-read.
Some of the ideas in the book are iconoclastic, yet ones that I violently concur with, and wish more people would pay more attention to. And yet, there is much to be said in disagreement with it as well, but more on that later.
Before they knew better, once upon a time, the western world believed that all swans were white, with what they thought was ample empirical justification. And then, cygnus atratus was discovered in the southern hemisphere. Taleb uses the Black Swan as a metaphor for the unexpected.
Taleb's basic proposition can be summarized thus: Critical events in human history have been, and continue to be shaped by, Black Swan events - events that are considered improbable outliers, but whose effects are quite significant. Black Swans are never predicted, and yet we convince ourselves in hindsight that they were predictable, that we just didn't do the prediction bit right.
Describing himself as an 'empirical skeptic', Taleb begins by recounting his adolescence in Lebanon, dominated by a war which everyone believed, for 'good' reasons, would never happen, and once started, nobody believed would last much long. Yet it happened, and is still on, and turned young Nassim Nicholas into a lifelong skeptic of theories.
Taleb divides the world into two: mediocristan (where things are nice and Gaussian), and extremistan (where they're not). He speculates that most of the environment we encountered during our evolutionary history was mediocristan-like, characterized by relatively few outliers, and no outlier being able to produce a dominant effect. Modern reality is more like extremistan,
where outliers are large enough to produce a dominant effect. Contrast the weight of individuals, which is in mediocristan territory, with their wealth which belongs to extremistan. No viable individual can skew the average weight of a group of hundred in any significant fashion. Add Bill Gates to any similar group, and he would dominate the average wealth metric. Add J K Rowling to a group of a hundred authors, and she would dominate the average number of books sold metric. Taleb speculates that our troubles arise from using mental tools learnt in mediocristan to make sense of extremistan.
Our basic world view, Taleb would have it, suffers from fatal flaws. We are suckers for the narrative fallacy: we tend to fit facts into seemingly logical narratives, even when we shouldn't.
Complicated explanations for why the Levant (the area containing Lebanon) was stable for centuries are a good example: they were widely believed, and yet evaporated the moment conflict broke out. The physiology of our brains drives us into constructing elaborate theories to fit our facts, because that's precisely the most efficient way of storing those facts. Often, these theories are inaccurate, and cannot predict rare outliers. In mediocristan, where we evolved, outliers wouldn't have that much of an effect in any case.
We suffer from a series of mental imperfections that make us, among other things, blind to the possibility of black swans. We are optimistic about "confirmatory" evidence, which makes us reason inductively in a very optimistic fashion. As the examples of Russell's chicken (or Taleb's turkey) and the joke about the moron who fell of a skyscraper show, a hundred instances are not sufficient confirmation of a theory, but one counter-example is ample disproof. We see patterns where there is only randomness . We see evidence of "luck" where we really have self-selection from an ensemble: drowned pilgrims do not give evidence, unlucky beginners tend to quit gambling.
Taleb uses the metaphor of the "anti-library" or the large number of unread books in Umberto Eco's collection to represent the problem of limited knowledge. We focus too much on what we know (or have read) already, rather than what we do not know (or have not read).
Taleb also mentions what he calls the ludic fallacy: confusing real-world randomness with the sterile randomness of artificial models. Real-world randomness is much more Black-Swan like, unpredictable and not easy to model. Confusing models with reality may be a minor sin in mediocristan, but it happens to be quite a major one in extremistan.
Taleb goes on to demolish the idea of "experts" being better at prediction, especially in the social sciences and financial markets. Our funamental intellectual inadequacies, born of our mediocristani heritage are to blame. Emprical studies indicate that, in the social and financial spheres, alleged experts would be equally good or bad as a random amateur due to the inability of their models to handle Black Swans: rare, unpredictable events that dominate common, predictable ones. Taleb would have us protect ourselves from negative Black Swan events, while maximising our exposure to positive ones.
What causes the increasing importance of Black Swan's in our world? The "winner take all" effect is partly to blame. The smaller and more connected the world becomes, the more dominant is this effect. Mediocre writers and singers do not have a captive audience any more, the "flat" world increasingly craves only Pavarotti and Rowling. Wait, she's kind of mediocre, but you get the drift.
Taleb strongly criticizes the use of the Gaussian (or "bell") curve to model extremistani processes, going so far as to call it the "great intellectual fraud". He sees this at the root of most of the ills of the modern financial system - for example, why investment firms with Nobel laureates supplying the brains fail. He would rather have us use self similar fractal models of probablity, which are supposedly much friendlier towards Black Swans. He justifies this by showing how a self-similar model matches the distribution of wealth better than a Gaussian one.
Taleb does deal a strong blow to the generally held opinion about the ubiquity of Gaussian processes even outside the natural world.
One is stuck by the vehemence and depth of negative feeling Taleb has for the Gaussian. He has enough and more venom to spare for the "philosophers" of the modern world - especially financial "experts" who drive others to loss using their Gaussian models of reality, which sooner or later crash into a Black Swan.
In summary: we're not built to properly handle the world around us, we just cannot predict accurately enough. Our use of simplified, especially Gaussian, models of reality ensure our failure, even when we have Nobel laureates on our side.
There is much to agree with Taleb on. Seeing patterns in randomness, and over-induction is a sin one sees way too often, especially in management. One sees an abundance of absolute certainty born out of limited success. Models of limited value are dime-a-dozen, most created by herculean efforts at drawing conclusions from limited data and seeing patterns where there are none. The culture of fashion, fads and over-rationalization needs to be blamed for much of the inadequacies we see around us. A good dose of skepticism would do a lot of good for a lot of people. Unfortunately, skepticism breeds a healthy lack of absolute certainty, and thats exactly the reverse of what those with money to invest seem to want.
It is kind of a reviewer's cliche to blame authors for precisely the sins they rail against. Taleb anticipates this objection against his metaphor by acknowledging that he uses an excessively precise metaphorical treatment of Black Swans largely due to the need to communicate his ideas.
Taleb does seem to be a little too eager to generalize, and use anecdotal evidence (which should, by the force of his own argument, be useless) to bolster his claims. One can easily call for more skepticism about some of his theorizing around Black Swans, but that is a sin easily forgiven. The basic point to be made is the Black Swan and the inadequacies of our models.
The anti-Gaussian crusade is a bit over the top, often degenerating from savage but effective thrursts to tilting at windmills. Indeed, the Gaussian is over-used, but to call it the "great intellectual fraud" needs a lot more justification. The Gaussian serves us well in modelling physical processes, and manufacturing processes which are rooted in physical processes. - semiconductor manufacturing is a multi-billion dollar example. Mother Nature is still mediocristani at heart.
In spite of what a professional financial trader might think, speculative trading is an incidental part of the world. There are many possible economic futures in which trading of the extremistani kind is less important. The large financial exposures to Black Swans we see are an artifact of modern capitalism and its assignation of marginal risk and benefit to the capitalist. True, we do not have a viable alternative today, but which empirical skeptic would want to be that that's the case for ever? Many winner-take-all processes are an artifact of winner-take-all capitalism, and may not be permanent. The undeserved success of mediocre well-advertised popular musicians, for example, is unlikely to survive the advertising industry.
Treating the ills of choosing the wrong (Gaussian) model as a general sickness of statistics is an error on Taleb's part. Variance, standard deviation, and correlation, which Taleb dismisses as Gaussian artifacts make as much sense in a different statistical model of reality as well. In some power-law distributions, the standard deviation would be infinite (I need to check this), but in a truncated versions, which are closer to reality, it most certainly wouldn't. Correlation is a statistical term that is independent of the underlying model chosen. In fact, one is tempted to accuse Taleb of throwing out the statistical baby with the Gaussian bathwater. He anticipates a lesser variant of this criticism, but does not answer it satisfactorily.
In contrast with the intemperate anti-Gaussian tirades is Taleb's all-too-clear longing for his aristocratic past - his teenage rebellion notwithstanding. His heroes are all of an aristocratic temperament, if not extraction. Perhaps nothing critical needs to be said about that: the world would be a lot better if we had more of that sort of temperament around. His praise for wide erudition over the modern fascination for narrow specialization is one to be applauded. The anecdote towards the end, of missing trains being painful only if you run after them, is almost worth the price of the whole book.
Taleb's fury seems to go beyond the Gaussian, and all the way to the basis of the Enlightenment, which is difficult to justify. He seems to admire the philosopher Al Ghazli (Algazel), and sees echoes of his criticism of our confirmation biases in Al Ghazli's criticism of philosophers. Ibn Rashid (Averroes), who was Al Ghazli's most prominent opponent and a defender of the rational method, was one of those who inspired the medieval rationalism of Europe and eventually, the Enlightenment. The story of the Al Ghazli - Ibn Rashid debate is rather interesting, and should be a good topic for another post. Arabia and the Muslim world was more influenced by Al Ghazli, and turned their backs on reason (though Taleb sees that as a stretch of Al Ghazli's idea, one is not convinced). The reason of the Greeks and Arabs and Medieval Europe does indeed suffer from a lack of skepticism. The main contribution of the Enlightenment, which Taleb seems to hold in low esteem, is the addition of empiricism to the mix. Our models of the world, be they of mediocristani nature or extremistani markets, are inaccurate, but they allow us to make real progress. The spirit of the Enlightenment lies in empirical validation of those models, and the development of better ones, which then get further tweaked as our empirical abilities get better, and so on. Just as we have a much better theory of gravity today, we should have a much better theory of financial risk tomorrow, if the Enlightenment wins over dogma.
Taleb seems to be too sympathetic towards religion, something that ideally should not be even a point of any relevance to be discussed in a book such as this. Other than mentioning Al Ghazli, Taleb refers to skeptics historically being friends of Religion. Given that dogma is the basis of most if not all religion, that is a visibly self-contradictory position. As with Al Ghazli's followers, overreacting to imperfections in models leads us straight to dogma. Towards the end, Taleb expresses irritation towards those who "criticize the Bishop, but not the financial Analyst". Perhaps outside his bubble of financial markets, most people are affected much more by Bishops than by analysts? Contemptible as his Gaussian-wielding pretenders are, they do not seem to be spreading physical ruin, poverty or causing major wars. For a self-professed skeptic, Taleb could have done better than to repeat a rather puerile argument: religion has killed people, but so has Stalin, so religion isn't to be criticized. Even priests, he says, go to doctors rather than Bishops when they're ill. Priests may know their charade well, but lay people are increasingly fooled by faith-healing charlatans, who seem to be running around without being criticized by Bishops. A world without Bishops would be one much better off than one without financial analysts. If one must have one of the two, the choice is clear. How many Africans contract AIDS because financial analysts prohibit them from condom use? How many people have died because financial analysts flew planes into buildings? How many financial analysts push legislation to ban stem cell research? How many candidates for US President are indulging in chest-thumping proclamations of their faith in Gaussian models at this time?
Given his admiration for the aristocratic demeanour and expression, Taleb's greatest fault seems to be his being completely otherwise in this book. Even for one in strong agreement with him, the book is difficult to read without inducing strong negative reactions. Taleb seems to admire Umberto Eco, and perhaps the most damning criticism of his expression would be that favourite phrase of Eco's unforgettable Piedmontese character, Jacopo Belpo:
ma gavte la nata!
