The Signal and the Noise

I've just finished reading Nate Silver's The Signal and the Noise, with mixed impressions. On the one hand, it is overall a pleasant read and the first best-seller starring Bayes' theorem (I wrote about this point before). Any book that keeps professional statisticians busy guessing whether the author is a true Bayesian or a frequentist in disguise can't be all bad! On the other hand, there are some disappointing points (discussed below.) Concentrating on the negative side might seem unfair, but should also be more interesting, since the reviews so far have mostly been elogious.

1. The style is conversational and easy to follow, but a few details are annoying, such as Silver's love for adverbs:  "literally" (used on almost every page, sometimes more than once) "essentially" (for "almost") and "incredibly" (for "very"). Mixed metaphors and chained clichés are not uncommon. I would have expected more thorough editing for what was clearly a potential best-seller.
2. One gets the impression that Silver proposes a technical solution to a social (and political) problem. This is (very briefly) touched upon by a review in Science¹ and, more incisively, here. In the Introduction, he says:
   ... a single lax assumption in the credit ratings agencies' models played a huge role in bringing down the whole global financial system. 
   (namely, that mortgage defaults will be uncorrelated). Although he does nuance his position throughout the book, he does not consider the stark fact that the ratings agencies have a strong financial incentive to make the most favorable prediction, not the most accurate one. In fact, they are paid to certify a given version of the future, without having to bet on it. This problem cannot be solved by pushing them to adopt more elaborate (and thus even easier to fudge) prediction techniques, but rather by changing the reward mechanism.

Microscopy image

Cliodynamics

I've been reading Peter Turchin's War and Peace and War. He aims at nothing less than a science of historical evolution (cliodynamics), in particular as applied to the birth and decline of empires. To this end, he puts forward three main ideas. I find at least the first two unconvincing:

1. The concept of asabiya (borrowed from Ibn Khaldun), which he defines as "the capacity of a social group for concerted collective action". All other things being equal, the group with the highest asabiya wins. 

• The major weakness of this notion is its lack of a precise, independently verifiable definition. The winners obviously have better asabiya if we define the latter as the qualities required for winning, but this kind of circular reasoning would hardly be satisfying. Turchin is of course aware of the problem, and he states (in Chapter 12):

Suppose all I proposed in this book would be to say that different nations have different asabiya and therefore that is why some rise and others decline. Would that qualify as a scientific theory? Not at all, because asabiya would be just a different name for the fact that some empires grow and others fall. Just coining a term does not explain anything. If I can tell you why asabiya grows and why it declines, however, we have the beginnings of a theory.

after which he carries on, without giving more details. When does asabiya grow and decline, then? At the beginning of Chapter 11, Turchin states "Competition between societies leads to asabiya increase, whereas competition within a society causes its asabiya to decline." The second part is true, but trivially so: inner fighting is equivalent (by definition) to lower social cohesion. The first part seems plausible (several groups with diverging interests can unite against an external enemy), but is not generally true, since external pressure can also divide a society (examples range from the pro- and anti-Macedonian factions in Athens to the controversy on the Iraq intervention in our days).

[UPDATE 25/08/2013] One should also be wary of the confusion between cause and effect; a successful group is presumably more cohesive, for two main reasons:

• the members of the group are more likely to rally around a winning leader.
• the said leader has more resources by which to impose a social structure.

2. The frontier theory states that new empires form at the edge of older ones, where the asabiya is stronger.

• What about change that comes from within? The Soviet Union did not start at the periphery of the Russian Empire
• On the other hand, Spain was not a frontier region before building its empire –except in the trivial sense that every territory is on someone's frontier– and the territories it conquered were far away.

This theory may be useful as a rationalization of historical events after the fact, but it won't predict when an empire will fall and to what adversary.

3. The secular cycles, oscillations in the social structures (coupled of course with the evolution of asabiya). For now, I do not have much to say on this.

All in all, we are still far from Hari Seldon's psychohistory... 

Almost quasicrystals