Summary: Human progress requires a culture of openness to change and innovation, which historically has been rare and resisted by established elites. Periods of remarkable achievement, like that seen in Enlightenment Europe, occurred when societies embraced new ideas and allowed for intellectual and economic freedom. The key to sustained progress lies in maintaining a culture of optimism and a politico-economic system that encourages innovation rather than suppressing it.
To make progress, we must do something differently from what we did yesterday, and we must do it faster, better, or with less effort. To accomplish that, we innovate, and we imitate. That takes a certain openness to surprises, and that openness is rare. It is difficult to come up with something that never existed. It’s also dangerous, since most innovations fail.
If you live close to subsistence level, you don’t have a margin for error. So, if someone wants to hunt in a new way or experiment with a new crop, it is not necessarily popular. There is a reason why most historical societies that came up with a way of sustaining themselves tried to stick to that recipe and considered innovators troublemakers.
That means that innovation depended on stumbling on a new way of doing things. Someone came up with a new and better tool or method by accident or by imitating nature or another tribe. But when populations were small, few people accidentally came across a great new way of doing things, and there were few people to imitate. In other words, there is a limit to what can be done in small, isolated societies.
It took greater population density and links to other groups to get the process of innovation and specialization going. Cultures at the crossroads between different civilizations and traditions were exposed to other ways of life as merchants, migrants, and military moved around. By combining different ideas, they set the process of innovation in motion. Ideas started having sex with each other, in the British writer Matt Ridley’s memorable phrase.
Such openness gave rise to extraordinary periods of achievement in cultures like ancient Greece and Rome, Abbasid Baghdad, and Song China. They were, as the American economist Jack Goldstone calls them, “efflorescences”—sharp and unexpected upturns that did not become self-sustaining and accelerating. They did not last.
The American economic historian Joel Mokyr talks about that as Cardwell’s Law—named after the technology historian D. S. L. Cardwell, who observed that most societies remained creative only for a short period. Often, they were ruined by external enemies, since poorer states and roving bandits are attracted by the former’s wealth.
But there are also enemies within. Every act of major technological innovation is “an act of rebellion against conventional wisdom and vested interests,” explains Mokyr. And conventional wisdom and vested interest have a way of fighting back.
Economic, intellectual, and political elites in every society have built their power on specific methods of production and a certain set of mythologies and ideas. The vested interests have an incentive to stop or at least control innovations that risk upsetting the status quo. They try to reimpose orthodoxies and reduce the potential for surprises, and sooner or later they win, the efflorescence is stamped out, and society reverts to the long stagnation.
An escape from stagnation requires a culture of optimism and progress to justify and encourage innovation, and it takes a particular politico-economic system to give people the freedom to engage in the continuous creation of novelty.
Enlightenment and Classical Liberalism
Luckily, this culture emerged forcefully in western Europe in the 17th and 18th centuries, in the form of the Enlightenment, which replaced superstition and authority with the ideals of reason, science, and humanism, as the Canadian psychologist Steven Pinker summarizes it, and classical liberalism, which removed political barriers to thought, debate, innovation, and trade.
It was the combined forces of the Enlightenment and classical liberalism that reduced intellectual and economic elites’ power to stamp out progress. Cardwell’s Law started to break down, and the road opened for individualists, innovators, and industrialists to change our world forever.
Why did this happen in Europe, and why then? There are two traditionally competing narratives, one associated with the right and one with the left, and they are equally wrong. According to the first, it was because Europeans were better than others (perhaps because of natural superiority, the legacy of the ancients, or Christianity). According to the second, it was because Europeans were worse than others (perhaps because of slavery, colonialism, and imperialism).
The problem with the first explanation is that experimentation in science, technology, and capitalism had been present in previous pagan, Muslim, Confucian, and other cultures. In fact, Europe imported and improved upon many non-European advances. The problem with the second explanation is that all previous civilizations also engaged in slavery, colonialism, and imperialism when they had a chance. Yet, they remained poor. So, what made Europe more successful must have been something else.
As noted, elites everywhere reacted to surprising innovations by trying to enforce political authority and intellectual orthodoxy. What made Europe different was that the elites failed. Unlike the Chinese or Ottoman empires, Europe was blessed with political and jurisdictional fragmentation, which has been emphasized by scholars like the British-Australian economic historian Eric Jones and the English historian Stephen Davies.
European rulers had the same ambitions to conquer and control, but on a peninsula of peninsulas, they were halted at mountain ranges, bodies of waters, riverine marshes, and forested landscape. Therefore, Europe was split into a mindboggling array of polities, independent cities, autonomous universities, and different religious denominations.
Hundreds of different sovereigns could not coordinate repression and impose one orthodoxy on all. That always left room for thinkers, entrepreneurs, and banned books to migrate to the jurisdiction most hospitable to their particular heresy. The Protestant Reformation was a further blow to ambitions for universal authority. How can you revert to a trusted authority when you don’t know which authority to trust? Nullius in verba (take nobody’s word for it), was not just the motto of the Royal Society, founded in London in 1660, but the spirit of the whole Enlightenment project.
European princes discovered that rivals who welcomed more migrant scientists, entrepreneurs, and technologies also acquired more wealth and thereby more war-making capacity. Disruptive innovations still threatened the elite power base in the long term, but a lack of innovation might threaten their lives instantly—via a superior invading army. In a fragmented Europe, sovereigns faced the opposite incentive of rulers of vast empires, who feared domestic discord more than they feared foreign conquest.
Fear of change therefore began to give way to a fear of stagnation. “And thus, it is,” wrote the German philosopher Immanuel Kant in 1784, that the Enlightenment gradually arises “from the selfish purposes of aggrandizement on the part of its rulers, if they understand what is for their own advantage.”
Scientific and Industrial Advances
The associated classical liberal transformation, pioneered by the Dutch Republic, and then taken further by Great Britain and the United States, simultaneously widened the freedom for new experiments and enterprises through greater equality under the law, more secure property rights, and freer domestic economy and expanding markets.
That created a virtuous circle, since the scientific endeavor, businesses forced to compete, and an open society are by their natures works in progress, subject to constant challenge and improvement. They allow more people to experiment with new ideas and methods and combine them in unexpected ways.
As the American economic historian Deirdre McCloskey has shown, these processes went hand in hand with a profound reevaluation of urban and bourgeois life. Whereas commerce and innovation used to be seen at best as necessary evils to fund a hierarchical and aristocratic society, they now started to be seen as desirable, even honorable.
This relative freedom for inquisitiveness and irreverence unleashed first a scientific revolution and then an industrial one. The cumulative nature of knowledge instilled a powerful sense of optimism. When telescopes, microscopes, and the English scientist Isaac Newton unlocked nature’s mysteries, the whole world soon learned about it and started thinking about how natural regularities could be exploited for practical purposes.
Through migrations, correspondence, the printing press, coffee shops, and learned societies, scientists and entrepreneurs systematized knowledge in mechanics, metallurgy, geology, chemistry, soil science, and materials science. That made it possible to consciously manipulate, debug, and adapt methods, materials, and machines to changing needs. New knowledge pointed to new experiments that could be used to interrogate nature further, and the results of those interrogations pointed to new technologies that could be used to grow more food, prevent or cure disease, shape materials, and exploit energy sources.
The modern corporation and financial markets emerged as vehicles for systematically transforming capital and knowledge into goods and services that improved people’s lives. No longer did mankind have to wait for someone, somewhere to stumble on a breakthrough at widely dispersed intervals. An economic and intellectual system devoted to the systematic pursuit of discoveries and innovations had been created. From Manchester and Menlo Park to Silicon Valley, pioneers methodically pushed the technological frontiers further into the unknown, and free competition and international trade made such wonders widely accessible at everyday low prices.
Therefore, for the first time in history, progress did not come to a sudden halt. It continued and accelerated. More people than ever looked at the world’s problems and were free to come up with their own suggested solutions. Finally, mankind reached escape velocity, and progress was no longer a bump on a flat line of human development but a hockey stick, pointing sharply upward.
“It may be that the Enlightenment has ‘tried’ to happen countless times,” writes the British physicist David Deutsch in The Beginning of Infinity. And therefore, it puts our own lucky escape into stark perspective: All previous efforts were cut short, “always snuffed out, usually without a trace. Except this once.”
It should make us deeply grateful that we are among the few who happen to be born in the only era of self-sustained, global progress. But it should also make us focused and combative. History teaches us that progress is not automatic. It only happened because people fought hard for it and for the system of liberty that made it possible.
If we want to remain the one great exception to history’s rule of oppression and stagnation, every new generation must find it within itself the desire to make the world safe for progress.