The Memory Bank

From Bell-Curve to Power-Law

From bell-curve to power-law: distributional models between national and world society 

For the Social Analysis Forum on Reductionism

Statistical patterns can be found in nature and society. Their distribution may conform to mathematical models. Thus, if two unbiased dice are rolled a thousand times, the number seven will occur with six times the frequency of two or twelve. The resulting histogram will be symmetrical with one peak where the mean, median and mode coincide. Or take a large sample of adult human beings and measure their height. Most cases will fall between five and six feet with very few less than four or more than seven feet. Because this is a continuous variable, the results can be plotted on a graph to which a curve may be fitted. It too will have a single peak with fan tails on the high and low ends. We call this the ‘normal’ distribution or popularly the ‘bell-curve’. For more than a century statistical inference has largely been based on this curve with its parameters of mean and standard deviation

More recently, another statistical pattern has been making the headlines. If you count the number of hits on 7,000 websites in a given day and plot them according to frequency, the curve hugs the vertical and horizontal axes, indicating a few very large numbers (headed by Yahoo) and many small ones (yours and mine). If the same data are plotted on a log-log scale, the result is a straight line sloping down from left to right. This is a typical manifestation of something called a ‘power-law’ distribution. A similar formula describes the frequency of words used in natural language; and the distribution of molecular reactions in cells reveals a few hubs linked to most reactions and many weakly connected molecules. The ‘new science of networks’, basing its statistical approach on the physics of complexity, has been announced by authors such as Albert-Laszlo Barabasi and Duncan Watts. Just as, in the late nineteenth century, the normal distribution seemed to lend unity to statistical patterns emerging in a number of apparently unrelated fields, such as criminology, astronomy and plant genetics, now the power-law distribution appears in fields as disparate as the worldwide web, stock markets, air transport, Hollywood actors’ networks, electric power grids, urban hierarchies and molecular biology.

Empirical phenomena lend weight to the mathematical models used by statisticians, but their relative prominence in our collective imagination reflects how we experience society in history. Durkheim and Mauss proposed in Primitive Classification that the forms through which we perceive cultural order in the world are social in origin. They supported this claim with reference to Australian totemism, the classification of animals corresponding to clan organization, and to Chinese astrology which reflected the hierarchical organization of that society. Marx had already pointed out that Darwin’s evolutionary biology shared many features with the Victorian capitalism of his day — the individualism, the idea of progress, natural selection as market competition and so on.

Modern statistics took off a century and a half ago as a way of regulating people through enumerating them. Soon the normal distribution became the basis for the development of ‘parametric’ or inferential statistics. The very word normal says it all — conformity to a standard revealed by a central tendency, meaning that a population can be described in terms of an average type. The key assumption is randomness. This means that every member of a group has an equal chance of being selected. The democratic premise is obvious. This is an egalitarian as well as an atomistic model. Moreover, the quantities have to be measured on an interval scale, so that size is a continuous variable, not broken up into the separate classes of nominal or ordinal scales. Paradoxically, the bell-curve, in the hands of writers like Charles Murray, later became notorious for sustaining racist theories of intelligence and educational performance; but this should not divert our attention from the underlying properties of the model.

Does the recent rise to prominence of the power-law distribution, with its premise of extreme inequality, tell us something about our collective experience of society today? Power-laws have been known for some time. In the nineteenth century, when urban economy was relatively free of national controls, they were discerned in the dramatically uneven growth of cities. Later both Zipf and Pareto proposed something similar in the form of rank-order distributions, the one for word frequencies and the other for income distribution. Pareto is credited with discovering the 20/80 rule — the idea that 20% of the people own 80% of the wealth. But the premise of inequality contained in this rule was not adapted to the ideology of twentieth century society and it remained an anomaly.

The ‘normal’ image of the natural and social world gained credibility from reflecting the premises of the national societies formed to regulate industrial capitalism. In the last century, anthropologists transposed the idea of the nation-state as the typical form of society to ethnographic descriptions of so-called primitive societies, thereby demonstrating that the model of cultural homogeneity was universal. The power-law distribution is characterized by a few very large quantities and many small ones. The curve reflects an exponential rate of growth. In network science, it is commonly observed that networks consist of a few hubs with many links and a large number of weakly-connected nodes. The discovery of power-laws is related to the physics of complexity, the attempt to study interconnectivity in a non-reductionist way (as opposed to the isolated atoms of the random universe). This science is mainly concerned with the edge between order and chaos and with critical moments of transition, as when chaotic water molecules assume the rigid pattern of ice. It is now thought that self-organization, including life, flourishes in this interstitial zone.

Specialized study of networks in social science arose in the 1950s as a result of the development of graph theory in mathematics. This theory was based on a number of assumptions that are now seen to be unrealistic. It described an inventory of nodes whose number is fixed and remains unchanged throughout the life of the network. Second, all nodes are equivalent and are linked together randomly. These assumptions of randomness, stasis and equivalence were unquestioned for forty years. Territorial states lent some credibility to networks configured in their own image. Thus road maps do not diverge markedly from the model, each centre having roughly the same number of links as the others.

Stanley Milgram conducted an experiment in 1967 to see how many personal links would be needed to connect any two individuals in the United States. He found the median number of links was 5.5 and this gave rise to the popular idea of ‘six degrees of separation’, that all humanity is connected on average by six links. This ‘small world’ phenomenon does not sit well with the assumptions of a random universe. Then it was discovered that most Hollywood actors were linked by two or three degrees to Kevin Bacon through appearances in the same movies. Mark Granovetter established in 1973 that job-seeking networks formed clusters with weak links conducting information between them. And the typical clustering of networks was modeled by Watts and Strogatz in 1998. But until now the basic assumptions of original graph theory still held. The key shift emerged with the recognition that some network nodes are hubs and some persons are ‘connectors’ (Malcolm Gladwell The Tipping Point). People vary enormously in their ability to make social connection and in this they resemble the air traffic grid of the United States, with a few O’Hares and many small airports. By now networks were coming to be seen as intrinsically unequal in the size distribution of nodes and dependent on a few highly connected individuals. But what produces this effect?

Barabasi established the fit between the pattern of website links and the power-law distribution. Such networks are ‘scale-free’ and they lack the parameters of the normal distribution. There is no characteristic node in the continuous hierarchy described by the power-law. The exponential character of the curve reflects the fact that networks grow over time and the skewed distribution of links may be accounted for by ‘preferential attachment’. Growth with preferences both accounts for the hub phenomenon (early comers tend to attract more links) and requires us to abandon graph theory’s key assumptions of randomness, stasis and equivalence. This is consistent with the market principle that ‘the rich get richer’. Indeed in the network economy, as the Microsoft case confirms, it can even be summarized as ‘winner takes all’. The winner is often unpredictable until one node crosses a threshold and takes off. The trick is to find the threshold. When hubs are undermined, the network as a whole may be visited by ‘cascading failure’.

The convergence of world markets and the internet has multiplied opportunities for scale-free networks. If corporate hierarchy was well-suited to the era of mass production for national markets, the rise of a web or network model of economy involves a shift from vertical integration to flat, virtual integration, as Castells has long insisted. I have shown in a recent book that when the money circuit is detached from real production and trade, the market is revealed as a weighted and directed network, with the mass of ordinary stocks following a few market leaders. Already the power-law distribution has been harnessed to predictive models based on analysis of the movement of the eight or so main stocks in a given sector.

“Nature normally hates power-laws”, says Barabasi. Hitherto physicists have found them near the critical point of phase transitions, as when a metal is magnetized. Even if it can be shown to be regular, exponential growth is unpredictable. Statisticians can only say that sometimes a variable crosses a threshold and then it takes off. We have been led to believe for more than a century that the bell-curve is preponderant in the physical world and this has helped to make prevailing social ideologies ‘natural’. European societies still largely hold to these ideologies. But the Americans have long held that income inequality is inevitable and today even the radical democratic wing of internet society, the bloggers and the peer-to-peer activists, tend to accept the fact of power-law distributions, claiming that as long as choices can be made freely (equal opportunity), this inequality is acceptable, one might say ‘natural’ or even ‘normal’.

This whole paradigm shift in scientific and statistical models coincides with the breakdown of the nation-state’s monopoly of society and with it the corporatist premises of twentieth century economy, such as jobs for life and social planning. For a quarter-century now neo-conservative liberals have subordinated national economy to global markets; and the digital revolution has given us a new emergent model of society in the internet. The norm of this new world market is stark inequality. The egalitarian premises of nation-states, seeking to curb capitalism’s polarizing tendencies, have given way to a world society in which the winner takes all. We may or may not be on the way to a world order capable of curbing the excesses of capitalism. But for now the power-law is king. It’s a different model of statistics, for sure. Perhaps it captures society poised between national and world forms. Or maybe we have reverted to the imbalance between market and state typical of the nineteenth century, before national regulation aspired to curb domestic capitalism. The pressing political question for humanity is whether new forms of association will enable us to harness the polarities of the network economy for common ends.

There is an objective basis of sorts for these statistical models in nature and society. But the one that attracts most attention in a given period is likely to reflect underlying tendencies in social experience. Having been raised in the heyday of British social democracy, only to face the new liberalism now, I feel like I have had to internalize a radical paradigm shift at several levels. The shift from bell-curve to power-law is one way of talking about this momentous transition.

When I carried out fieldwork in Ghana during the 1960s, I was amazed by how migrants found their relatives, after traveling 500 miles to an unknown city of a million people. They had no addresses or phone numbers written down. When they arrived in the central lorry park, they would look for someone wearing Northern dress and ask him where they could find people like themselves. Directed to a particular district, they would seek out a leading figure in the ethnic community. They might then be directed to someone else from their home village. By all means, within an hour or two, they would be sitting with their relative. These African migrants knew that we live in small worlds connected by fewer links than most of us imagine. They used contingent human encounters and network hubs like local big men, not street maps. Their method was news to me then, but it shouldn’t be now.

Comments |1|