A stranger walks into a room and sits down behind a table. He picks up a piece of paper and read aloud a generic-sounding weather report. He completes his “report” in about 90 seconds and walks out of the room.

Next, you’re asked to guess his IQ.

You’re part of a psychological experiment, and you object to the absurdity of the request. I don’t know anything about that guy. He just came into a room and read a report. It wasn’t even his report- you gave it to him to read! How am I supposed to know his IQ?

Reluctantly, you make a wild guess. Separately, Fake Weatherman is asked to guess his own IQ. Who made a better guess?

Amazingly, you did, even though you know nothing about Fake Weatherman. Two (German) psychologists … conducted this experiment, and they found that the strangers’ IQ predictions were better than the predictions of those whose IQ was being predicted- about 66 percent more accurate.

To be clear, it’s not so much that you’re a brilliant predictor; it’s that he’s a lousy self-evaluator. We’re all lousy self-evaluators. College students do a superior job predicting the longevity of their roommates’ romantic relationships than their own.

Savor, for a moment, the preposterousness of these findings. Fake Weatherman has all the information, and you’ve got none. He’s got decades of data- year’s worth of grades, college entrance exams cores, job evaluations, and more. Fake Weatherman should be the worlds foremost expert on Fake Weatherman!

Chip & Dan Heath, Switch

March 22, 2023

The Secret of Success

March 22, 2023/ Stephen Goforth

Survivorship bias pulls you toward bestselling diet gurus, celebrity CEOs, and superstar athletes. You look to the successful for clues about the hidden, about how to better live your life, about how you too can survive similar forces against which you too struggle. Colleges and conferences prefer speakers who shine as examples of making it through adversity, of struggling against the odds and winning.

The problem here is that you rarely take away from these inspirational figures advice on what not to do, on what you should avoid, and that’s because they don’t know. Information like that is lost along with the people who don’t make it out of bad situations or who don’t make it on the cover of business magazines – people who don’t get invited to speak at graduations and commencements and inaugurations.

The actors who traveled from Louisiana to Los Angeles only to return to Louisiana after a few years don’t get to sit next to James Lipton and watch clips of their Oscar-winning performances as students eagerly gobble up their crumbs of wisdom. In short, the advice business is a monopoly run by survivors. As the psychologist Daniel Kahneman writes in his book Thinking Fast and Slow, “A stupid decision that works out well becomes a brilliant decision in hindsight.”

Before you emulate the history of a famous company, Kahneman says, you should imagine going back in time when that company was just getting by and ask yourself if the outcome of its decisions were in any way predictable. If not, you are probably seeing patterns in hindsight where there was only chaos in the moment. He sums it up like so, “If you group successes together and look for what makes them similar, the only real answer will be luck.”

Entrepreneur Jason Cohen, in writing about survivorship bias, points out that since we can’t go back in time and start 20 identical Starbucks across the planet, we can never know if that business model is the source of the chain’s immense popularity or if something completely random and out of the control of the decision makers led to a Starbucks on just about every street corner in North America. That means you should be skeptical of any book promising you the secrets of winning at the game of life through following any particular example.

David McRaney Read more here

June 22, 2022

Media Growth Predictions

June 22, 2022/ Stephen Goforth

Here are some takeaways from the annual PricewaterhouseCoopers (PwC) Global Entertainment & Media report:

U.S. digital newspaper ad revenue expected to surpass print by 2026.
Online TV’s ad growth (10%) will come at the expense of terrestrial TV’s ad growth, which will decrease from 66.6% in 2021 to 63.1% in 2026.
Print still dominates the book market, accounting for 77.4% of total revenue in 2021, with electronic books contributing 22.6%.
Virtual reality continues to be the fastest-growing segment of media, albeit from a relatively small base.
Global internet advertising revenue will expand at an impressive 9.1% CAGR in the next five years to reach $723.6 billion in 2026, at which point 74% of internet ad revenue will be mobile.
Teenagers are now spending more time in immersive virtual worlds like Roblox and Fortnite than they are on TikTok.

A Common Prediction Mistake

April 05, 2022/ Stephen Goforth

Suppose you’re told that a man named John is extremely well-educated, smokes a pipe, and wears tweed jackets with patches on the sleeve—is he more likely to be a particle physicist or a janitor? A physicist, you immediately think. But you’d likely be wrong, because janitors are common and particle physicists rare. The chances that you’d happen upon a very well-educated, tweed wearing, pipe-smoking janitor are higher than those that you’d meet a physicist who meets the same profile.

Laurie Abraham writing in Slate

October 25, 2021

Who is best at predicting the future

October 25, 2021/ Stephen Goforth

(In a contest involving hundreds of geopolitical questions) a small number of forecasters began to pull clear of the pack: the titular “superforecasters”. Their performance was consistently impressive. With nothing more than an internet connection and their own brains, they consistently beat everything from financial markets to trained intelligence analysts with access to top-secret information.

They were an eclectic bunch: housewives, unemployed factory workers and professors of mathematics. But Philip Tetlock (who teaches at the Wharton School of Business) and his collaborators were able to extract some common personality traits. Superforecasters are clever, on average, but by no means geniuses. More important than sheer intelligence was mental attitude. Borrowing from Sir Isaiah Berlin, a Latvian-born British philosopher, Mr Tetlock divides people into two categories: hedgehogs, whose understanding of the world depends on one or two big ideas, and foxes, who think the world is too complicated to boil down into a single slogan. Superforecasters are drawn exclusively from the ranks of the foxes.

Humility in the face of a complex world makes superforecasters subtle thinkers. They tend to be comfortable with numbers and statistical concepts such as “regression to the mean” (which essentially says that most of the time things are pretty normal, so any large deviation is likely to be followed by a shift back towards normality). But they are not statisticians: unlike celebrity pollsters such as Nate Silver, they tend not to build explicit mathematical models.

But superforecasters do have a healthy appetite for information, a willingness to revisit their predictions in light of new data, and the ability to synthesise material from sources with very different outlooks on the world. They think in fine gradations.

Most important is what Mr Tetlock calls a “growth mindset”: a mix of determination, self-reflection and willingness to learn from one’s mistakes. The best forecasters were less interested in whether they were right or wrong than in why they were right or wrong. They were always looking for ways to improve their performance. In other words, prediction is not only possible, it is teachable.

Prediction, like medicine in the early 20th century, is still mostly based on eminence rather than evidence. The most famous forecasters in the world are newspaper columnists and television pundits. Superforecasters make for bad media stars. Caution, nuance and healthy scepticism are less telegenic than big hair, a dazzling smile and simplistic, confident pronouncements.

From a review in The Economist of the book Superforecasting: The Art and Science of Prediction by Philip Tetlock and Dan Gardner

October 20, 2021

Goodhart’s law

October 20, 2021/ Stephen Goforth

Once a useful number becomes a measure of success, it ceases to be a useful number. This is known as Goodhart’s law, and it reminds us that the human world can move once you start to measure it. Deborah Stone writes about Soviet factories and farms that were given production quotas, on which jobs and livelihoods depended.

Numbers can be at their most dangerous when they are used to control things rather than to understand them. Yet Goodhart’s law is really just hinting at a much more basic limitation of a data- driven view of the world … there’s a critical gap between even the best proxies and the real thing— between what we’re able to measure and what we actually care about.

Hannah Fry writing in The New Yorker

July 27, 2021

Faith in Numbers

July 27, 2021/ Stephen Goforth

When polls have faltered in predicting the outcome of elections, we hear calls for more and better data. But, if more data isn’t always the answer, maybe we need instead to reassess our relationship with predictions—to accept that there are inevitable limits on what numbers can offer, and to stop expecting mathematical models on their own to carry us through times of uncertainty.

To recognize the limitations of a data-driven view of reality is not to downplay its might. It’s possible for two things to be true: for numbers to come up short before the nuances of reality, while also being the most powerful instrument we have when it comes to understanding that reality.

Hannah Fry writing in The New Yorker

May 26, 2021

Predicting the future

May 26, 2021/ Stephen Goforth

How much reliance can we place on regression to the mean in judging what the future will bring? What are we to make of a concept that has great power under some conditions but leads to disaster under others? Keynes admitted that “as living and moving beings, we are forced to act … (even when) our existing knowledge does not provide a sufficient basis for a calculated mathematical expectation.”

With rules of thumb, experience, instinct and conventions – in other words, gut - we manage to stumble from the present into the future … The trick is to be flexible enough to recognize that regression to the mean is only a tool; it is not a religion with immutable dogma and ceremonies. Used to make mechanical extrapolations of the past … regression to the mean is little more than mumbo-jumbo. Never depend upon it to come into play without constantly questioning the relevance of the assumptions that support the procedure. Francis Galton spoke wisely when he urged us to “revel in more comprehensive views” that the average.

Peter Bernstein, Against the Gods

August 28, 2020

Superforecasters

August 28, 2020/ Stephen Goforth

They are called “superforecasters” and they make surprisingly accurate predictions about world events. Tara Law writes about these semi-professional forecasters in TIME magazine:

Superforecasters tend to share certain personality traits, including humility, reflectiveness and comfort with numbers. These characteristics might mean that they’re better at putting their ego aside, and are willing to change their minds when challenged with new data or ideas…they may also be more flexible than traditional scientists, because they’re not bound to a particular discipline or approach. Their predictions incorporate research and hard data, but also news reports and gut feelings. They tend to be actively open-minded and curious. They’re in “perpetual beta” mode—always striving to update their beliefs and improve themselves. A willingness to change your mind when presented with new information, contend with your biases, challenge one another’s ideas, and break down problems into specific questions are all desirable qualities in people who make big, important decisions.

August 18, 2019

Pascal’s Wager

August 18, 2019/ Stephen Goforth

Pascal’s argument (written in the 1600’s) went like this: Suppose you concede that you don’t know whether or not God exists and therefore assign a 50 percent chance to either proposition How should you weight these odds when decided whether to lead a pious life? If you act piously and God exists, Pascal argued, your gain – eternal happiness - is infinite. If, on the other hand, God does not exist, your loss, or negative return, is small – the sacrifices of piety. To weigh these possible gains and losses, Pascal proposed, you multiply the probability of each possible outcomes by its payoff and add them all up, forming a kind of average or expected payoff.

In other words, the mathematical expectation of your return on piety is one-half infinity (your gain if God exists) minus one-half a small number (your loss if he does not exist). Pascal knew enough about infinity to know that the answer to this calculation is infinite, and thus the expected return on piety is infinitely positive. Every reasonable person, Pascal concluded, should therefore follow the laws of God. Today this argument is know as Pascal’s wager.

Pascal’s wager is often considered the founding of the mathematical discipline of game theory, the quantitative study of optimal decision strategies in games.

Leonard Mlodinow, The Drunkard's Walk: How Randomness Rules Our Lives

February 08, 2019

Assume They’re Wrong

February 08, 2019/ Stephen Goforth

From military predictions to technological predictions to sports predictions, when experts foretell the future, it’s always safest to assume they’re wrong.

Because they are deeply knowledgeable in a particular field, experts are more prone than others to view the world through a too-narrow lens, assuming that the current trends they understand so well are indicators of what is to come. Their expertise reinforces their confidence in their own analysis, blinding them to contrary data or disconfirming evidence.

As you listen to their smart, persuasive, credible prophecies, just remember: Most of them, most of the time, will be wrong. (You can take my word for it. After all, I’m an expert.)

Jeff Jacoby is a columnist for The Boston Globe

Becoming

ideas that challenge / comfort / inspire

22 recent articles about where AI will go in 2024

We’re All Lousy Self-Evaluators