Jason Feifer: This is Built for Tomorrow, a podcast about the smartest solutions to our most misunderstood problems. I’m Jason Feifer, and in each episode, I take something that seems concerning or confusing today, and figure out where it came from, what important things we’re missing and how we can create more opportunity tomorrow.

Jason Feifer: Don’t you wish you could predict the future? And I don’t mean in a street corner psychic kind of way. No, I mean, it in a more personal, meaningful way to know what is coming, to look at a situation and say, this is the right decision to make for me. This is what I should choose. This is where I should go next because I am pretty sure I know what’ll happen. There would be no greater, more powerful gift in the world than that because everything we do in some way or another is about placing a bet on the future. We are all gamblers at the casino of time. The unknowableness of the future is our greatest equalizer because nobody is at an advantage over anyone else, or are they?

Jason Feifer: Here is a curious piece of information. There is a group of people out in this world who carry the title of superforecaster. Why? Because they are better than the average person at anticipating what is going to come next. And in some cases they’re even better than government spies who have classified information on that thing. The superforecasters were in fact, born out of a government project meant to identify average people with this exact super power, which I know sounds like the setup of a comic book movie, but I am not making this up. It is absolutely true. And when I heard about this, of course, I had to wonder what makes them so special, and is there something the rest of us can learn from them? I am happy to report I got in touch with the leader of this… Actually it went more like this.

Warren Hatch: I do have a hard stop at one minute before the hour.

Jason Feifer: Because the leader of the superforecasters is of course, always aware of what’s coming next. Anyway, the conversation was totally fascinating and I came away with, well, honestly, I came away with an extremely embarrassing recording of myself, which I promise to play for year later, as well as a lot of amazing advice that you can use right now. But before we get to any of that, I think we need to back up a little, not just to who these superforecasters are, but really to why most of us are not anywhere near superforecaster level despite how much we try and why we are willing to listen to other people who are equally bad, because let us be clear, give someone a microphone or access to a digital publishing platform, and they will tell you exactly what they think will happen in the future, exactly. They will do it with the confidence of someone who just earned their superforecaster status. Like Steve Ballmer, who is Microsoft CEO when it’s rival, Apple released a silly little thing called the iPhone.

Voice Clip (Steve Ballmer): $500 fully subsidized with a plan. I said, that is the most expensive phone in the world. And it doesn’t appeal to business customers because it doesn’t have a keyboard, which makes it not a very good email machine.

Jason Feifer: Turns out people don’t really care about the keyboard. And here is another strong prediction. This one from 2012, when Rob Parker went on ESPN to say this.

Voice Clip (Rob Parker): Tom Brady’s on the downside of his career. And if you ask me if he’ll ever get to another Super Bowl, I have one answer, no way, no how.

Jason Feifer: Tom Brady of course, went on to win four more Super Bowls. This sounds fun, right? Just say whatever comes to mind, doesn’t matter. And I got to wondering, do these people learn anything from this? So I track down someone who has some experience in the matter. His name is Mike Masnick and he’s the founder and editor of a fantastic tech site called Techdirt. You have written how many things for Techdirt?

Mike Masnick: Over 50,000 articles.

Jason Feifer: I imagine that of those 50,000, not all of them stood up to the test of time.

Mike Masnick: 99.9% are perfect.

Jason Feifer: And here’s from the 0.1% of articles that were not perfect. It comes from March 17th, 2011, Headline. It took the New York Times 14 months and 40 million to build the world’s stupidest paywall? And the article is just Mike unloading on what was then the new paywall for the New York Times, which he predicted would be a massive failure. At the end of his article, he even seemed to imagine a future in which someone like me comes along, and questions his prediction, but then he doubles down. He writes “Perhaps I’ll be proven wrong, but I can’t see how something like this succeeds. It’s like a giant experiment in wrongness.” So who actually performed a giant experiment in wrongness?

Mike Masnick: Obviously I was wrong, right? I think New York Times paywall has been one of the most successful if not the most successful news paywall that has existed.

Jason Feifer: More than seven million people currently pay for times digital products.

Mike Masnick: And so what I’m looking at these things I’m always looking at which thing did I get wrong and how did I get it wrong? Were my original assumptions wrong? Did something change? What did I miss in here?

Jason Feifer: I love the questions that Mike is asking, and he has really thoughtful answers, answers that can also help us get to the core of how we make better predictions and therefore better decisions for our lives and our careers. So here’s what we’re going to do on this episode of Build for Tomorrow. First, we are going to take a step back even further, and we’re going to ask why has humanity always been obsessed with seeing into the future? And has anyone across time actually been good at it? Like Nostradamus, what’s up with that guy? Then we will meet the man at the top of the superforecasters, the superforecaster, superforecaster, the man who has a hard stop at one minute before the hour. And he will offer many words of wisdom and finally we’ll come back to Mike and see what he learned. And what’s still left to learn from all the things he’s gotten wrong. It’s all coming up after the break. All right, we’re back. So before we talk about how to see the future, let’s talk about the past. How far can we go back where we find people looking forward?

Laura Ackerman Smoller: Gosh, almost from the earliest recorded history we have, we have records of people trying to figure out the future.

Jason Feifer: This is Laura.

Laura Ackerman Smoller: Okay. I’m Laura Ackerman Smoller.

Jason Feifer: She’s a professor of history at the University of Rochester and is currently serving as the department chair there. And what do we see from these earliest records of recorded history? The Babylonians for example, were developing astrology and horoscopes. Other people were reading nature for signs, getting news of the future from the path of stars or the flight of birds.

Laura Ackerman Smoller: Or the famous Roman augers who would cut open and inspect the entrails and organs of sacrificial animals to try and read the future.

Jason Feifer: By medieval Europe, which is Laura’s area of expertise, elites and common folk were both prognosticating in pretty much every way they could. They’d be possessed by spirit it or read phases of the moon or get signs from thunder. By the later middle ages and into the Renaissance, astrology became a high art involving complicated mathematics and science and Latin textbooks. In the way that today we might go see a doctor and then get a second opinion from another doctor, people would do the same with astrologists.

Jason Feifer: And then the printing press came a along in the middle of the 1400, people were now able to write their predictions down and distribute them, which gave birth to a new kind of fortune teller, the celebrity fortune teller.

Voice Clip (Miss Cleo): Call me now for your free Tarot reading.

Jason Feifer: Well, I mean, it would take a few centuries to get to Miss Cleo.

Laura Ackerman Smoller: So the first really big splash was an astrologer, well, he’s a bit of a plagiarist. Well, he is a plagiarist also, but an astrologer named Johannes Lichtenberger. And he published a prognostication that just kind of went global viral.

Jason Feifer: Lichtenberger that damn plagiarist, he ripped off the more sober work of another astrologer named Paul of Middleburg. Then he married it to a bunch of crazy nonsense that was trending at the time.

Laura Ackerman Smoller: It’s a whole class of prophecies in the ancient and medieval and Renaissance world that are attributed to women seers known as the Sibyls. And just as people would write new texts and attach a name like Ptolemy to them, people would invent new prophecies and say, “Oh yes, this was uttered by the whichever Sibyl, the Cumayan Sybil, for example.

Jason Feifer: So this is starting to sound kind of familiar, isn’t it? You legitimize yourself with the base of authoritative information, then layer on a bunch of crazy, make up and attribution for it. And oh, on top of that, add some self-declared experts.

Laura Ackerman Smoller: He also pulls in material from people who said that they had received divine inspiration.

Jason Feifer: And at a splash of marketing.

Laura Ackerman Smoller: He puts all of this together with amazingly enigmatic and intriguing wood cut illustrations.

Jason Feifer: And voila, the medieval internet is set on fire.

Laura Ackerman Smoller: It goes absolutely viral, it goes through a whole bunch of printings and it inspires graphs and rafts of other such prophetic writings.

Jason Feifer: And of course, that’s basically today’s recipe for millions of podcast downloads and YouTube views. And maybe even your own elected office or cable news show. Now Johan Lichtenberger died in 1503 and his fame clearly didn’t last. I had never heard of him and I bet you haven’t either. But the same year that he died, another man was born who would follow a slightly more refined playbook and have very powerful results. He was a Frenchman named Michel de Notredame and he was for most of his life accomplished, but obscure.

Jason Feifer: He knew several languages, was well traveled and studied medicine. He even worked as a doctor during a plague outbreak. His wife and first two children died when he was in his 30s. And a few decades later, he married a rich widow and started a second family. He became proficient in astrology, which earned him a role as an advisor to France’s queen mother. And at some point he started going by the name Nostradamus. In 1555, he published the first edition of a book called The Prophecies, a collection of 942 four line poems, which described death and destruction. Here for example, is one of those poems.

Voice Clip (Nostradamus): The city is besieged and assaulted by night. You have escaped a battle not far from the sea, a woman faints with joy at the return of her son poisoned in the folds of the hidden letters.

Jason Feifer: It is all like that. And that all would’ve sounded familiar back then. The 1500s was dominated by religious wars where cities were regularly sieged and mothers were regularly wondering if their son survived. So if he really could see the future, it would’ve been pretty helpful if he told everyone what city he was writing about or when this assault would take place. But of course, he did not.

Laura Ackerman Smoller: He was really quite skilled at self-promotion and created a mystique about himself also by giving vague enough predictions that they always seem that they can be coming true.

Jason Feifer: Nostradamus died of gout at the age of 62 in the year 1566, but his work was just vague and captivating enough to live on, getting a renewed boost during moments of major western upheaval when people felt like maybe, just maybe he was talking about their time. His work became very popular during the French revolution, for example. And then again, during World War II and for obvious reasons, it’s having a moment right now, like…

Voice Clip (Youtube Prediction): Want to know what kind of potential calamities could be looming for next year? One famous astrologer has you covered.

Jason Feifer: That’s a YouTube video with 300,000 views called Nostradamus’ Predictions For 2022 Sound Pretty Bleak. And what did this man who died of gout 500 years ago have to say about 2022 specifically? Well, among the predictions, apparently one of Nostradamus’ poems reference famine. So the video says maybe we’ll run out of food in 2022. And in all likelihood of course, we will not all run out of food in 2022, but maybe someone will run out of food in 2022. Certainly food has become more expensive in 2022. And so if you squint enough, you might just be able to find some kind of confirmation in this, which Laura says is the reason these kinds of ideas retain their power. It’s never really about the individual prediction. It’s about the faith in the system that the prediction comes from.

Laura Ackerman Smoller: The fact that you might get it wrong never invalidates the science, just as we don’t discount modern medicine, because every medicine our doctor gives us doesn’t work 100% effectively all the time. Once you’re inside and you accept the premises, you tend to say, “Oh, okay, well maybe there was a miscalculation here. There are other mitigating factors. There’s always human free will.”

Jason Feifer: I look at all this and want to step back and ask a more fundamental question, why do we so deeply believe that the future is knowable at all? Laura’s hypothesis is that it comes from the root of so much of our theology. That if there is a God that knows the past, present and future, then that full story is already written, which means that all we need to do is figure out…

Laura Ackerman Smoller: So where am I? Where are we now in that story? And whether one is in good times or bad times, sometimes it’s really helpful to be able to position one’s self there.

Jason Feifer: And surely the answer has to be somewhere, right? So, okay, fast forward a few centuries and our world today obviously does not operate the way that the medieval world does. We don’t see it as subject to supernatural forces, and we organize ourselves by laws and incentives and cause and effect, but we retain that belief in some way that there are ways to see a few steps ahead, that the story we live in is a logical one, and that there’s some way to orient ourselves inside of it, or at least to take a really educated guess.

Jason Feifer: But how? Well, look, there is a lot of scholarship on this and I could bore you with a 57 hour podcast about predictive modeling and the difference between supervised models and unsupervised models, but, eh, let’s just make a phone call.

Warren Hatch: Yeah. Hello.

Jason Feifer: Hello. Can you hear me? There you are.

Warren Hatch: Yeah. All connected.

Jason Feifer: Fantastic. This is Warren Hatch, who I told you about at the beginning of the show, the leader of the superforecasters.

Warren Hatch: And I’m still a card carrying member. Although I do find the day job interferes with doing as much forecasting as I would like.

Jason Feifer: And what’s the day job? Well, okay. Let’s just back up a moment and explain. The story really starts with 9/11, which set off a crisis in the American Intelligence Community. How could they not have anticipated and stopped this terrorist attack and what can they do of course, to make sure it never happens again? In 2006, the government created a group called the Intelligence Advanced Research Projects Activity, which looks for innovative ways to improve American intelligence. It tried a whole bunch of stuff off, including in 2011, creating a forecasting tournament that was open to the public.

Jason Feifer: The idea was to see if the wisdom of the crowd could beat the best the government had to offer. Five teams joined this competition and through them, thousands of ordinary people tried to predict global events. Two years later, four of those five teams were eliminated because they weren’t accurate enough. This meant only one team remained. It was called Good Judgment. And what was Good Judgment secret? Well, it was led by Philip Tetlock and Barbara Mellers, both professors at the University of Pennsylvania’s Wharton School who have studied the science of forecasting and developed a systematic approach to making predictions as well as to identifying people who are good at predictions.

Jason Feifer: They had recruited 3200 volunteers and they ran then through a series of tests to see who was genuinely skilled at predicting the future, which sure not to be about 2% of those who applied. And one of those people was…

Warren Hatch: I came up through the ranks as a test subject.

Jason Feifer: Warren Hatch, who started out on Wall Street, left Morgan Stanley to set up his own investment firm and was trading on a forecasting platform on the side. He joined the competition and passed, and these people like Warren were given the title of superforecasters.

Jason Feifer: This team was 50% more accurate than the control him set up by the government. And in some cases were even better than government analysts that had access to classified information. As the tournament came to a close, the people running Good Judgment realized, hey, we can do more than just win a competition. We could start a company where clients will ask us to forecast the questions that matter to them. What’s going to happen in a certain industry with certain prices and certain markets? That kind of thing.

Jason Feifer: So that is what Good Judgment became. And Warren eventually became its CEO. They are still driven by the wisdom of these superforecasters that people around the world who are uniquely excellent at looking at data and anticipating what will come next. And Good Judgment is always looking to recruit more.

Warren Hatch: They’re from all over the world. They speak many languages. They are experts. Typically in at least one domain area, about a third have at least one PhD. I have one only, but what they all have in common is they are very good, they’re experts at forecasting.

Jason Feifer: So, okay. Now you can understand my curiosity. What makes these people so good at this? Warren says that through all the research Good Judgment has done, they have identified some core characteristics of a superforecaster.

Warren Hatch: Being good at pattern recognition is a big part of it. Filling in a mosaic faster than everybody else, being open minded. So many people, if they have a view of the world, they’ll defend it to the death. They don’t tend to make good forecasters, although they may be people who pose good questions. Being open-minded is another indicator then, and also being cognitive or reflective. When you are presented with a new problem, you want to pause and ask yourself is the most obvious answer the right answer? So be cognitive reflective. And those are the things you can test for.

Jason Feifer: And also superforecasters are not over confident, which is a pretty rare skill. If everyone listening to this podcast was in a room together right now, and I asked you all to raise your hand if you are over confident, very few people would raise their hands. Of course, nobody thinks they’re over confident.

Warren Hatch: But if you go through and test yourself, most people actually are overconfident.

Jason Feifer: To which of course, I then wanted to know how exactly do you test for overconfidence? Warren says that there are a lot of tests for this and they can be quite detailed. Then he offered an example.

Warren Hatch: So what year was Gandhi born then? And I want you to give me two numbers representing your 10% confidence and your 90% confidence.

Jason Feifer: Oh my God.

Warren Hatch: And we’ll do that across 10 questions and you should get nine of them inside the range and one of them outside the range. So in the case of Gandhi, just so you know, do a guess, what’s the earliest you think he was born and what’s the latest you think he was born representing your 90% and 10% confidence?

Jason Feifer: Now. Okay. Do you have an answer to this? Put aside the 90% and 10% confidence stuff, which I didn’t entirely know how to wrap my head around and instead, just focus on the simpler part of this quiz. What is the earliest year that Gandhi was born, and what was the latest year? Maybe you know the actual answer, but guys, I did not. I want to state right now for the record, the next part of this audio is deeply embarrassing. I am ashamed, ashamed at the answer that you are about to hear me speak, but I am going to play it for you anyway, because well, Warren’s response is just so powerful that I will sacrifice my dignity for it. Okay. Here we go. So I guess I’m going to say the earliest he could have been born was like 1940. And what was the other part of that?

Warren Hatch: The latest he would’ve been born.

Jason Feifer: The latest [crosstalk 00:22:10] 1955.

Warren Hatch: Yeah. So it turns out he was born in 1869 [inaudible 00:22:19].

Jason Feifer: Oh, I don’t know anything about Gandhi.

Warren Hatch: But here’s the thing, that that’s okay. You don’t need to say, I don’t know, so I’m going to pass on it. What you do is go, “Okay, I tend to have my bands too narrow. I mean, that was 15 years. It was not my band, but I don’t know anything about Gandhi.” That would be kind of a… So you should want to have wider bands.

Jason Feifer: Oh, this is a very good point. Right, that’s really interesting. Look what I just did. I didn’t know the first thing about Gandhi, but I just really narrowed it in. I was afraid to be so damn broad. What I should have done is I was like, I don’t know anything about Gandhi. The earliest that he was born was 1600, and the latest that he was born was 1980, right? That’s what I should have done. Why did I not do that?

Warren Hatch: Because you were over confidant, but now you’ve learned. And the next time you’re confronted with something, you’ll be cognitive reflective and you go, “Oh, you know what? I don’t know that much, let me widen my bands.”

Jason Feifer: And what’s the value of widening my bands? This sounds like a stupid question, but why am I doing that?

Warren Hatch: What we’re really getting at is how sure are you about what you know? So it’s not important that when Gandhi was born, what’s important is to know how confident you should be in the knowledge you think you have, because if you are over confident in what you think you know, you are going to be making decisions informed by probabilities that are not going to align with reality and they compound. So if you make a decision being, well, I know this, and then you make another decision based on that, because I know that, and so forth, it’s a geometric equation there and you can find yourself really far out on a statistical limb needlessly so. So the value is to slow yourself down so you can get to a better outcome.

Jason Feifer: Now, let me tell you something, I have done this Gandhi quiz with dozens of people since Warren ran me through it. I also speak to companies and at conferences about how people can become more adaptable. And I do the Gandhi quiz with them too. And the crazy thing is almost everyone does some version of what I did. Sure their dates are usually a little more accurate than mine, but they all guess a narrow band of years, usually 15 to 20 years in which Gandhi they believe could have been born, even though they have no idea what the actual answer is.

Jason Feifer: This is what we do. Even when we do not have enough information, we limit ourselves to the knowledge we think we have. And Warren says, if you want to be better at forecasting the future, or just better at making any kind of difficult decision, then you must be very aware of what you know and what you simply do not know. So what happens after we have accepted our ignorance? How do we decide what to do next? That’s coming up after the break.

Jason Feifer: All right, we’re back. So now that we’ve heard Warren talk about widening your bands, I want to take us back for a moment to Mike, the editor of Techdirt, and author of 50,000 articles of which…

Mike Masnick: 99.9% are perfect.

Jason Feifer: And remember, Mike and I were talking about how he’d gotten a very specific prediction wrong. He predicted in a series of articles that the New York Times paywall will fail. But in fact, the New York Times paywall became a massive success. So I wanted to know what he thinks went wrong there. And he said that there were a number of factors that he missed, one of the biggest was the question of why people would want to pay for something like the New York Times to begin with? Back then he was thinking that people just won’t pay for news.

Mike Masnick: My argument, if you look back at some of them, I talked about, there were ways that I thought that the New York Times could do a better paywall. And it’s actually, if you look at it, the thing that I was saying was that they should be building the paywall as a community and adding different value for that community.

Jason Feifer: Community, that is what people will pay for, he thought. They want to join a kind of club and they are willing to pay admission to that club. But the times was just charged for news.

Mike Masnick: Where it really took off was after the election of Donald Trump and suddenly lots and lots of people felt that they had to support the institutes or institutions that would hold him to account.

Jason Feifer: In this way, Mike was sort of right all along, people did want to pay for community, but he did not anticipate an event that for millions of people would turn the subscription of a news product into a sense of community. And as Mike said this, I had a thought about how we try to game out what a future looks like. So I said, you have to take what is happening now, and then advance it into the future and then grapple with what about the future is fixed and what has changed. And that’s very hard to do because it’s easy to do one and one, right? Like the product remains fixed, but the world in which it exists will change, or the product will change, but the world in which it exists is fixed. And just like I know it now, and that’s kind of abstract.

Jason Feifer: So let me give you an example, for almost all of human history, there was basically only one way to send a message across a long distance. You needed to write it down, then have someone carry that message on a horse and then hand the message over to a person who might learn something terrible that is now too late to fix. That in fact, is exactly what happened to a painter named Samuel Morse in 1825. He had traveled to Washington DC to do some work and got a letter by horseback saying that his wife was very ill. But this news of course was days old by the time it reached him.

Jason Feifer: The next day, another a letter arrived by horseback saying his wife had died. He rushed back home to Connecticut, and then learned that she had already been buried. This sent him on a quest to develop a technology that could move information faster. And by the mid-1800s, he had created the first commercially sustainable Telegraph technology. Then he showed up in Congress asking for it to be funded.

Tom Standage: He sets this whole thing up on a desk, and he’s clicking a button on one end and there’s an electromagnetic clicking on the other end. And he says, “Look at this, isn’t it amazing?” And there could be miles and miles between this switch over here and this electromagnetic over here. And honestly, people just thought he was deranged.

Jason Feifer: That is Tom Standage, he’s an editor at the Economist and author of a book about the Telegraph’s history called the Victorian Internet. We actually spoke couple years ago for an older episode of this podcast about the Telegraph. Samuel Morse, eventually of course, did get the funding that he needed. And he introduced the Telegraph to the world, and many people were very excited and understandably so, but many others were not. They thought it was a burden or dangerous, or just kind of useless. My favorite quote from this time comes from Henry David Thoreau, 1854 classic Walden, where he writes…

Voice Clip (Henry David Thoreau): We are in great haze to construct a magnetic Telegraph from Maine to Texas, but Maine and Texas, it may be have nothing important to communicate.

Jason Feifer: And sure, you could just dismiss Thoreau as lacking vision, but indulge me for a moment and go back to that idea I floated to Mike about predictions. I said people need to figure out what is fixed and what is not fixed. Like Thoreau was writing in a world in which Maine and Texas really may not have had that much to communicate. He imagined that although the Telegraph was new and therefore an unfixed factor, the physical distance between these two states would remain a fixed barrier. If people are too far away to know each other or to do commerce with each other, then what would they have to talk about? Nothing. Fixed.

Jason Feifer: But what he could not have imagined is that absolutely nothing about his world would remain fixed. Wider communication would lead to wider commerce and new transportation methods would allow people to move around and ultimately live further from where they were born and Maine in Texas would in fact have a lot to talk about, not just because of the Telegraph, of course, but because of everything that changed because of, or at least alongside the Telegraph. So that is my theory.

Jason Feifer: We run into problems when we try to imagine what is fixed and what is not, when in fact pretty much nothing is fixed. This is then what I floated to Mike, and he said, “Yeah, that’s interesting, but…”

Mike Masnick: I frame it a little bit differently, which is that I think of it in terms of models.

Jason Feifer: It’s like he built a prediction model in his head. And the hardest part is figuring out which variables to include in the model and which to skip.

Mike Masnick: And maybe this is like me getting old, but the more and more I look at the world and trying to predict things, the more and more I realize there are always so many more variables than anyone takes into account. And everybody assumes if this, then that. And they don’t realize that the actual equation probably has like 50 more important variables. It’s not just one or two.

Jason Feifer: I asked Mike if getting things wrong ever gives him pause. And he said, “No, the point isn’t to get things 100% correct all the time. That’s just not possible. The point is to get better at getting things right.”

Mike Masnick: The only thing in my mind that it does is it reinforces the idea of trying to think through all of the possible variables that you want to put to any model.

Jason Feifer: Which reminds me a lot of what Warren from Good Judgment said.

Warren Hatch: The next time you’re confronted with something, you’ll be cognitive reflective and you go, “Oh, you know what? I don’t know that much. Let me widen my bands.”

Jason Feifer: The way to get things right in other words is to learn from getting them wrong. And Warren also advises to think in terms of percentage. Like instead of thinking of every prediction as binary, where you are either right or wrong, you create percentages, you are 80% confident that this will happen, or it is 65% probable that this will happen. That might sound academic. But Warren says there’s value there. First, by putting a percentage to something, you’re admitting upfront, that there’s a chance wrong, which creates a window for you to learn more. Could you do something that might bump you up to 90% confidence? And, okay, that’s interesting in a way, but I don’t know. It’s also not very satisfying if you’re trying to make a decision yourself and trying to game out what will happen in your own life.

Jason Feifer: We are all throughout our days and weeks, and months and years facing big decisions and trying to get them right. And it’s a pretty cold comfort to say, “Well, I don’t know, I’m 63% sure about leaving my job or moving to another city or breaking up with this person. And if I get it wrong, then I’ll just learn to get something else right later.” I mean, nobody wants that. There’s got to be something better, right? So I said to Warren, what can we learn from the superforecasters and how do they make predictions in a way that can help us improve our own decisions? And Warren said that the most important thing you can do aside from widening your bands is to learn how to filter out noise.

Warren Hatch: There’s a lot of opportunity out in the world. There’s a lot of information out in the world. There’s no shortage of opinions out in the world. How do you filter those out and zero in on the diagnostically useful, both for the questions you need to ask, and the forecast that you will focus on.

Jason Feifer: And this noise comes externally, but also internally. External is in some ways clearer. There are a lot of people out there making a lot of noise. Do you need all that noise? No. Are there people or sources of information that you should just stop listening to? Yes, but what about internally? You’ve got a lot of ideas in your head. A lot of things you think are important, but some of them are just not as important. And it is hard to figure out which part of your own thoughts are noise, to which Warren said, if you want to do that, especially if you’re trying to figure out a personal decision, you should start with this.

Warren Hatch: Working on almost a pros and cons of these things that you’re thinking about doing.

Jason Feifer: A pro/con list. Not a new idea. And to be honest, not an that I have ever found all that useful or appealing, but also I had never thought about doing it this way.

Warren Hatch: Let it sit, come back in a week and do another list of pros and cons without looking at the old one. And now look at it and see how many of them overlapped. Did some of the pros show up regularly, or maybe some of them crossed over? That’s one thing that you can do, and then discuss it out.

Jason Feifer: Because a pro/con list that we write is not an objective document. It’s just a reflection of whatever we happen to be valuing at the moment when we write it. But some of those things are going to change. So we want to test ourselves. What is so important that it remains consistent and what are we a lot more flexible on? And then Warren says you do a pre-mortem. And what’s that? Well, maybe you’re familiar with a postmortem, which people do all the time in business, usually after a big project wraps up.

Warren Hatch: You go back and you revisit your thinking, were you right for the right reason or did you miss something? And that’s the way you get feedback.

Jason Feifer: But what if you did a version of that before you make the decision? You pre-mort.

Warren Hatch: Ask yourself, challenge yourself. Say, if I make this decision and six months from now, I’m regretting it, why might it be? What is the thing I’ll look back on and go, “Gee, I wish I had not missed that.” And that can be a great way to help give definition to your thought process, as well as uncover perhaps some hidden assumptions that are going on and that you can bring to lighting and think through more carefully.

Jason Feifer: Warren says it’s also important to pool information, which is to say, to share information with others and gain access to their information too. We don’t do this very often though. It’s something I also heard from another expert I spoke to whose name is Katy Milkman. She’s a professor at the Wharton School of the University of Pennsylvania who also researches how to change behaviors in positive ways. And she wrote a book called How to Change.

Katy Milkman: My collaborator, Angela Duckworth and I have studied this. And one of the things that always is boggles or minus when students come to us struggling in a class, asking for our guidance on how to do better. And we ask them, have you asked your friends who are doing well in the class what’s working for them? And we get a lot of blank stares.

Jason Feifer: And why is that?

Katy Milkman: There’s really great research showing that in general, we think other people have more similar cognitions and knowledge to us than they actually do. So if you, like me live in a city, and think living in a city is the best, you probably think everyone you see must agree with that. So it’s the false consensus effect is a name for it. And because we assume other people know the same stuff we do and think the same things we do, we may underestimate the value of having a very frank conversation where we can learn from them.

Jason Feifer: Warren gave me similar advice, but said he often sees a problem when people do this. They will build teams to help them forecast something, but will assemble a bunch of people who think just like them.

Warren Hatch: And so really you have one point of you being cloned multiple times.

Jason Feifer: The important thing is to get access to different points of view. And Katy says she sees the result of this in her research.

Katy Milkman: If you are randomly assigned a college roommate who earns higher grades, your grades improve. So it’s happening somehow, but it’s normally not a super deliberate process. And what we found in our research is that if you just tell people, go deliberately copy and paste a strategy that’s working for someone else who’s pursuing the same goal, that improves outcomes, rather than just telling people to make a simple plan for instance.

Jason Feifer: I asked Katy what someone could do now, like right now. If they’re facing a decision and trying to figure out what the right pathway is, they want to gaze into the future. They want to figure out what the correct course of action for themselves is. What can they do?

Katy Milkman: This is going to sound like the weirdest piece of advice. But the answer that I would give, especially if you really cannot figure out, like it’s not obvious what the next step is, one of the things we do too little of when we’re trying to figure out what’s the right kind of change or the right next step is experiment.

Jason Feifer: Just try something. We tend to treat all our decisions as permanent, all our choices as full commitments, but they don’t actually have to be.

Katy Milkman: We need to be more comfortable actually telling ourselves, labeling it, this is an experiment and it’s not the end. It’s not my end goal. I’m exploring. And so you ideally make a few pivots, you try a few different things and treat those as data that you’re gathering. It’s a data gathering exercise.

Jason Feifer: Because what’s the harm in gathering data? Of trying something out even if it doesn’t work, there’s no harm really. Now you know more than you did before. You got to glimpse one possible future, which reminds me in a weird way of the final question that I had from Mike, the editor of Techdirt. Have you ever experienced a consequence of being wrong? Does everybody reach out to you and say, “Hey, idiot, this thing you wrote in 2011 was wrong,” does it actually matter?

Mike Masnick: That’s a good question. Not so much in that way. I mean, certainly some people, people who always disliked me will point to something that I’m wrong. But the funny thing is they never really get the ones that I was actually wrong on. This is kind of amazing to me. I mean, the things that people accuse me of and argue over are things that they misread, misrepresent, and so [crosstalk 00:39:33].

Jason Feifer: So they were wrong?

Mike Masnick: Yeah. Which is a little weird because there are real things that I got wrong that you could call me to task over, but that’s not the ones that people pick up on. And I don’t know why that is, and now I’m suddenly interested in that.

Jason Feifer: And this makes me realize something that I hadn’t actually considered when I began this research. I started out wondering why people get things wrong and whether there’s a way to get things right. And now I wonder if I was putting too much emphasis on right and wrong. I mean, obviously there is a difference between right and wrong actions. And sometimes things turn out very right or very wrong. Lives and careers can be defined by a single moment in the decision that was made. But I suspect that most of the time a decision or prediction that turns out right or wrong is just a moment in time, but not the moment in time. Maybe I make a decision that was wrong in the short term, but it puts me in a position for something better in the long term. Maybe someone predicts something and gets it super wrong, and that’s embarrassing in a way, but the wrongness informs them in a way that’s much more powerful later.

Jason Feifer: Should we want to get it right as often as possible? Yeah, of course, we should filter out the noise, we should widen our bands. But Mike and Warren and Katy all seem to be driving towards something that is more liberating than just that mode of thinking, that unlike the way Nostradamus viewed the world, there is not just one story to tell and not just one path we must go on. The paths are infinite. The best thing that we can do is not just stand frozen at an intersection, but instead to pick something and to learn from it and to feel a little more assured the next time. The future is not knowable in this way, but that’s fine, the future is creatable, and that’s our episode.

Jason Feifer: But hey, I have one more prediction for you. It is a question of whether a medieval French cardinal predicted one of Europe’s most pivotal events. I will tell you about it in a minute. But first, as I said at the beginning, this episode is drawn from my book also called Build for Tomorrow, which is an action plan for how to embrace change, adapt fast, and future proof your career. It comes out in September, but you can pre-order your copy now, and let me know if you do so that I can thank you personally.

Jason Feifer: You can find it on Amazon or wherever you get books, or by going to jasonfeifer.com/book. And if you want even more advice and encouragement on how to adapt fast, then sign up for my newsletter. You can find that by going to jasonfeifer.bulletin.com, jasonfeifer.bulletin.com. You can also get in touch with me directly at my website. You know what it is, jasonfeifer.com or you can follow me on Twitter or Instagram. I am @heyfeifer.

Jason Feifer: This episode was reported and written by me, Jason Feifer with additional help from Britta Lockton, sound editing by Alec Bayless. Our theme music is by Casper Babypants. Learn more at babypantsmusic.com. Thanks to you Adam Soccolich for production help. Some of the background on Good Judgment came from my colleague, Liz Brody’s reporting in Entrepreneur Magazine. The Nostradamus reading you heard came from a YouTube video read by someone crediting themselves as Tom B and the voice of Henry David Thoreau might have been familiar. It was Dallas Taylor, host of the great podcast, Twenty Thousand Hertz, who voiced a bunch of quotes in my original episode about the telegram. You can go back and find that. And finally, thank you to Taylor Barkley, whose longtime support of this show has helped it grow into what it is and who is now moving on to an exciting new adventure.

Jason Feifer: Taylor, thank you for everything. This show is supported in part by the Stand Together trust. The Stand Together trust believes that advances in technology have transformed society for the better and is looking to support scholars, policy experts, and other projects and creators who focus on embracing innovation, creating a society that fosters innovation and encouraging people to engineer the next great idea. If that’s you, then get in touch with them. Proposals for projects in law, economics, history, political science, and philosophy are encouraged.

Jason Feifer: To learn more about their partnership criteria, you can find them on Twitter @together_trust. All right now, as promised, let’s talk hot predictions. Back when we were talking to History Professor, Laura Ackerman Smoller, we got a little into the idea of how the rightness or wrongness of an individual prediction is never quite as important as a faith in the system of prediction itself, which reminded her of a French cardinal she had researched who was alive in the 1400s while the church was being torn apart by not one, not two but three competing popes. And this cardinal starts to think he is watching the rise of the antichrist.

Laura Ackerman Smoller: So my French cardinal cared that he tried to use astrology to figure out, okay, when is antichrist going to arrive? And what I absolutely love is the answer he came up with is 1789. And he’s a Frenchman, and that’s the year that the French revolution begins. So later authors in the 19th and early 20th centuries, often some very pious Catholic writers writing about this French cardinal who was very important in the history of the church get a little freaked out that he also, in their mind predicted the French revolution.

Jason Feifer: I mean, you want to be remembered forever? Here it is, three simple steps. Number one, pick a year in the future. Number two, say something bad will happen in that year. And then number three, let them read it however they want. Thanks for listening. I’m Jason Feifer and let’s keep building for tomorrow.