Last Saturday, a uniquely-trusted source sent me this e-mail: I am reading Tim Palmer’s new book “The Primacy of Doubt” … if, by chance, you have not yet read this one, I think you would love it.
Advice from a trusted source? The name Tim Palmer, a leading light, not merely in a single field, but two – meteorology and basic physics? And the subtitle: From Quantum Physics to Climate Change, How the Science of Uncertainty Can Help Us Understand Our Chaotic World?
How could a meteorologist remain unmoved? Of course I bought it! The book has already been out for six months (eight billion people are busily accomplishing a lot while our backs are turned), so my reading was belated. Fortunately, in this age of instant gratification, the Kindle edition was in my hands within minutes. Had finished my remedial reading[1] by midday Tuesday.
Bottom line? If you’re a regular reader of LOTRW, chances are good you’ll want to buy this book and master it. Unique in its accessible presentation of chaos theory and its application across basic physics, meteorology and climatology, economics/financial crises, pandemics, war, science of the brain; and even our free will, consciousness, and God. No chaotic system left unturned.
Those interested can find any number of more comprehensive reviews of The Primacy of Doubt online. These are uniformly positive. Instead of merely piling on, I’ll focus on a few quick initial impressions and personal takeaways.
Inter alia, the book makes clear:
- Doubt is truly primary. Back in 2007, NASEM’s Board on Atmospheric Sciences and Climate published a landmark report entitled Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts. They captured the rationale this way “Uncertainty is a fundamental characteristic of weather, seasonal climate, and hydrological prediction, and no forecast is complete without a description of its uncertainty.” Palmer is suggesting rather the opposite: characterizing the uncertainty is not some mere add-on, but rather a foundation of the weather forecast process. And furthermore, he argues it would make for a similarly useful starting mindset in the initial approach to economics[2], public health, and all the other applications he treats.
- Ensemble forecasting is essential – not merely useful. Palmer reminds us that when confronted with a chaotic system such as (but not limited to) the Earth’s atmosphere, the means to improved predictions do not lie so much through more detailed, accurate measurements of the initial state and greater model resolution, but rather through multiple less-detailed forecast runs designed to reveal the sensitivity of the forecast to slightly different initial conditions.
- Limits to the predictability of chaotic systems are intrinsic, not simply technological.
- Predictions per se differ from projections (the latter are conditionally dependent upon possible system responses to the predictions – e.g., human responses to environmental and economic forecasts).
- Uncertainty and doubt shape the implications of climate science for policy.
(Could add much more, but this gives a flavor…)
In closing, The Primacy of Doubt provided me a small personal gift – two (quite tenuous) touchpoints to my father Robert Hooke and his work, which I’ve mentioned from time to time in LOTRW. First, in discussing the geometry of chaos, Palmer dips into fractals, noting in passing that the arithmetic of fractals can be accomplished using p-adic numbers. I’d heard this latter term, but only because my dad had earned his Princeton Ph.D. with a thesis entitled Linear p-adic groups and their Lie algebras (published a year later in Annals of Mathematics, in 1942). This was an occasional dinner-table topic when my younger brother and I were growing up. At the time my puny adolescent brain absorbed little more than the thesis title and the idea (promulgated over my entire childhood by our mother, who was my dad’s biggest fan) that dad had mathematical superpowers.
Second, in The Primacy of Doubt chapter on weather, Palmer discusses a so-called annealing algorithm that uses noise to avoid being trapped by local peaks in searches for true optima.
This also struck me as vaguely familiar. In mid-career my dad had left pure mathematics to go into statistics and operational research, doing a postdoc under John Tukey at Princeton. Years later, while at Westinghouse, dad and a colleague, Terry Jeeves, published “Direct Search” Solution of Numerical and Statistical Problems in the Journal of the Association for Computing Machinery (JACM)., 8 (2) (1961), pp. 212-229. The paper described the mathematics behind an analog device they’d patented for identifying optima in the absence of guiding mathematical formulas (a paper that’s since been cited nearly 6000 times in this A.I. world). My (still) puny but now “mature” brain wondered if there might be an association between annealing algorithms and this work. Willy-nilly, I googled the phrase “simulated annealing Hooke Jeeves” – and was rewarded with a sizeable number of hits.
Isaac Newton once said “If I have seen further, it is by standing on the shoulders of giants.” Tim Palmer’s wonderful book drives that point home, putting into context the work of Einstein, Mandelbrot, Lorenz, Penrose (and even Palmer himself). But at the same time we’re reminded that we stand equally on the shoulders of the ordinary but far more numerous unknown scientists who made quotidian advances into the endless frontier.
Thanks, Tim! (Thanks, Dad!)
[1] (Full disclosure), the word “read” in this context requires some explanation. The Primacy of Doubt is a serious work. Though intended to be accessible to the general public, in places it can be heavy going. Instead, I read this book the same way I read Tolstoy’s War and Peace – in a single day, years ago. I was home sick from high school; my 1950’s vintage world didn’t include today’s rich abundance of internet entertainment options. Needing something to do, I started reading before breakfast and confronted immediately a thicket of Russian characters, each with several patronymics. Impossible to sort out who was who! But I decided to forge on, willy-nilly, trusting that the people who mattered would be familiar to me by book’s end. Sure enough, it worked (though the material that “sticks” today deals more with the Napoleonic wars, Russia’s scorched earth policy, and conditions in tsarist Russia than any of its literary chops).
Applied the same strategy here. Figured that even if I didn’t get a concept that mattered the first time around, the topic would be revisited, applied to another context, expanded, and clarified later on. This worked to some extent, though not so well for the quantum-mechanical bits! Spent some time the past couple of days rereading those, with good results, though if Tim Palmer were to show up at my door brandishing a pop quiz, I’d flunk.
No worries! The Primacy of Doubt is now a member of that small handful of books worth reading again and again. I’ll master it eventually (though, by then, Palmer will likely have turned his quantum mechanical conjectures into groundbreaking theoretical advances… and further papers and books).
[2] Palmer cites Richard Bookstaber’s The End of Theory: Financial Crises, the Failure of Economics, and the Sweep of Human Interaction here. LOTRW provided a review when the book was first published.