AUG 11, 2022
A former Google data scientist says you can use big data to find happiness and success
JUL 16, 2022
“You can make better life decisions. Big Data can help you.” So begins Don’t Trust Your Gut, a new book by Seth Stephens-Davidowitz. Seth, a former Google data scientist, has mined massive data sets in order to answer some of life’s most vexing questions: “What predicts a happy marriage?” and “How do you get rich?” and “What really makes us happy?” The answers may surprise you.
Listen to Seth’s appearance on the Next Big Idea podcast below, or read a few key highlights. And follow host Rufus Griscom on LinkedIn for behind-the-scenes looks into the show.
Rufus: You say one of the reasons that Big Data is so useful in making decisions is that the basic facts about the world are hidden from us. How are the basic facts of the world hidden?
Seth: Well, some of it’s just lying. My first book was everybody lies, and that was all about the secrets that can be uncovered with Big Data about sexuality and racism and child abuse. But even from a self-help perspective there are many things hidden for us. I have a big section on who’s secretly rich in the United States. That’s complicated because so many people are playing up or down their wealth—we don’t totally know where people are really making their money.
Another thing is the media is very misleading in that they’re always telling us these crazy stories that we’re all drawn to, and it just gives us a very misleading view of how the world works. I talk [in the book] about the age of successful entrepreneurs. If you look in magazines, successful entrepreneurs are in their twenties because that just makes a better story. I think the average age of entrepreneurs featured in business magazines is about 27. If you look at the entire universe of entrepreneurs, the average age of a successful entrepreneur is 42. So we’re getting lied to in many ways about what success typically looks like.
Rufus: As a 54-year-old entrepreneur, I was underlining that section with great enthusiasm. And I think there’s also a data point that the probability of success increases until the age of 60.
Seth: Yeah, which is pretty shocking to me. Nobody thinks of a 60 year old entrepreneur.
Rufus: Music to my ears.
Seth: There were many motivations for this book. One of them was just following the data. But another motivation for the book is that I’m an enormous baseball fan, and any baseball fan has noticed that the game has radically changed driven by data analytics. This has proven so successful in baseball. It’s proven so successful in finance, in Silicon Valley, in the business world. But it is striking that [when it comes to] these major life decisions, the vast majority of us fly blind.
Rufus: Let’s get into the nuts and bolts here. You use data science in this book, Seth, to answer, by my count, four key questions. How should you approach dating and relationships? What makes a good parent? How do we achieve success? And what makes people happy? Why don’t we start with dating. We’ve got these new large data sets from OkCupid and other dating sites. What have you learned about how to be effective in dating?
Seth: There are two questions. There’s how to be effective and get more dates, and there’s who should you try to pick?
For effectiveness, there are these surprising counterintuitive results. Christian Rudder, analyzing OkCupid data, has found that one of the ways you can increase your odds of being successful in dating is being polarizing. So there are all these people on dating sites with what some might consider wacky looks. And they do really well in online dating. The reason for that is because some people are really into them. And that’s what you want in dating. You want some people really, really into you rather than everyone thinking you’re kind of OK.
That’s actually something I used in my own life. I don’t think it’s gonna surprise anybody that I’m considered extremely nerdy. My friends, in my dark, long decades of single life, were giving me advice. They were saying, “Be less nerdy!” “Be less weird!” “Learn how to be more normal!” And I think the data suggests the opposite. Nerd it up! Yes, a lot of people are not gonna be into you. But some people will be really, really, really into you, including my girlfriend, who, it turns out, had a thing for nerds. So I’m glad that I played up who I was.
Rufus: Moving on to what makes for a successful marriage or a successful long term coupling, which is, ostensibly, what people are looking for when they’re out there on dating sites, you talk about this really interesting study by Samantha Joel who gathered this huge dataset—I think over 11,000 couples—to find out what predicts successful relationships. What did she find?
Seth: It was Joel and 85 other scientists. There were more than 11,000 couples, more than a hundred variables on every couple. And the biggest lesson in the data is that it’s surprisingly hard to predict romantic happiness. Predicting whether two people are going to be happy together is not like predicting the weather tomorrow: it’s like predicting the weather four weeks down the road. I talked to one of the authors on the paper, and he said he’s moving to an idea that maybe relationships are, similar to the weather, chaotic systems where slight changes in initial conditions can take things in horrible directions or spectacular directions.
That was the number one lesson, but within that, the biggest predictor, by far of increasing your odds of being happy in your romantic relationship is being happy outside your romantic relationship. So if you generally like life, you’re gonna be happy in your relationship as well. There’s a lot of truth to the idea that you should work on yourself first before you look for someone else to make you happy.
And then, about the other person, if there’s anything that increases the odds of being happy with that person, it’s psychological traits, things like having a secure attachment style, growth mindset, conscientiousness satisfaction with life—all these stupid psychological quizzes that I hate taking seem to be the only things that predict romantic happiness in a partner.
Rufus: This study that Raj Chetty did of siblings who moved was totally fascinating and a great example of how to use Big Data creatively to gain insight into human behavior. Could you tell us about that study?
Seth: I basically had finished the chapter [on parenting], and I’m like, “OK, all I need to say is that everything you do as a parent doesn’t really matter that much.” You might have 8,000 decisions as a parent, and if the best evidence is that a great parent is going to improve a child’s outcome by 20%, divide 20 by 8,000 and each decision is just minuscule in how much it’s going to affect things. So just chill out. Do what you think is about right. And thenI remembered this research by a former professor of mine, Raj Chetty, and some other researchers where they [analyzed] the entire universe of taxpayers—tens of millions of people—to measure how much the neighborhood a child grows up in influences how they turn out. They did this very clever thing: comparing families that moved at a certain age. So one kid, let’s say, had 10 years in one town and another kid had no years in that town. What happens? How do kids turn out? And by comparing kids from the same family, you control for genetics and parenting and other factors.
They found living in certain neighborhoods consistently gives kids an edge on anything they could measure in tax data—income, going to college, marrying at a certain age. It doesn’t measure everything. We’d love it if they could measure character and other factors that matter. But I was thinking about this, and I’m like, “Well, what that means is that that decision seems to be the really important one.” The neighborhood you raise your kids in seems to matter a lot, even if everything else matters little.
Rufus: Let’s talk about how to be successful. It turns out there are a bunch of stories we tell ourselves about what success looks like. And very often those stories are not accurate, right? I’m a serial entrepreneur. The Next Big Idea Club is my fourth company. So this section on the characteristics of successful entrepreneurs I found totally riveting. You say there are three myths debunked by the data: the advantages of youth, the outsider’s edge, and the power of the marginal. What is this myth of the successful entrepreneur and what’s the reality?
Seth: These myths about entrepreneurial success that a lot of people believe, when you think about it, don’t make a whole lot of sense, and the data helps us correct these myths.
So the myth of youth. Totally not true in the data. The odds of success increase to the age of 60.
Being an outsider. David Epstein wrote an excellent book, Range, but in one chapter he talks about the outsider’s edge. If you’re too close to a field, you’re going to be stuck in the old ways, and you’re not going to be able to think of a creative solution to a problem. Totally not true in the data. The closer inside you are to a field, the higher your odds of success. A soap manufacturer’s going to have a much higher chance of creating a successful soap manufacturing business than a shampoo manufacturer. You want to be as close as possible to the field.
And then the power of the marginal. This is an essay by Paul Graham, whom I generally love and find very provocative. [He says] that it’s an advantage to be unsuccessful so you’re not weighed down by eminence when you’re creating a business. Totally untrue in the data. The most successful entrepreneurs are in the 99.9th percentile of income before they started their business.
There’s a formula for entrepreneurial success. You spend many years mastering a very narrow field. Prove your worth in that field. And then start your business, when you’re ready, in that field where you have all this expertise.
Rufus: Let’s turn to a topic of universal interests: human happiness. I was interested to read in your book that GDP has doubled in the last 50 years, we have all these wonderful technologies that now make it easy to learn and communicate, but we’re no happier. In 1972, 30% of Americans described themselves as very happy, and today, 50 years later, it’s about the same number. But maybe Big Data can help make us happier. The mappiness study is just so interesting. You wanna tell us about that?
Seth: This is two British economists, and they had this great insight that because of smartphones, we could do these really cool studies on happiness where you could just ping people and ask them a bunch of questions: “What are you doing?” “Who are you with?” “How happy are you?” They built this data set of more than 60,000 people, more than 3 million happiness points, and they could do all these amazing studies that I thought were really convincing on what tends to make people happy. I’ve made some big changes in my life based on what that project found.
Rufus: How have you changed your own behavior based on the mappiness data?
Seth: One of the big lessons for me was the value of activity. If you look at the leisure activities that score really high, they tend to require some energy. Sex, going to a museum, going to a show, exercise, hunting and fishing, taking a hike, taking a walk. These are things that require some startup energy. Then you look at the leisure activities that score very low—browsing the internet, watching TV, reading, iPhone games—they’re the types of things that just feel easy. Someone read my book and said, “There’s a difference between a comfortable activity and an enjoyable activity.” I think a lot of times we’re tricked by comfort and ease. Everybody knows the feeling. Your friends invited you to go out, and it’s 8:00 p.m., and you’re feeling tired, and there’s something you want to watch on TV. Should I just cancel? Tell them I have COVID? And I think the mappiness data tells us that’s a trap we’re all falling into—the trap of doing something that’s easy, that doesn’t require a lot of energy.
Natural Language Processing