How the Best Investors Approach Their Mistakes

Jonathan Berk: Well, welcome back, everybody. Jules, we’re a little late today, aren’t we?

Jules van Binsbergen: Yeah, this episode is coming out a little irregularly, but it is the summer. So during the summer season, the episodes will not come out on the strict biweekly schedule. Once we’re back to the fall, we’re back to the regular schedule that we’ve been doing before.

Jonathan Berk: Today, we’re going to talk about mistakes. And we’ve been talking all along about the importance of admitting our mistakes.

Jules van Binsbergen: But what we haven’t spoken much about is to know when you actually have made a mistake, or let’s even take a step back further, which is, how do you define a mistake? What is a mistake really?

Jonathan Berk: Yeah, some people might think it’s obvious, but I don’t think it’s quite that obvious. I always use as an example, back in the day at Stanford, we used to teach a course called Critical Analytical Thinking. And the course was designed to make better business decisions. And so I would start the course out by asking students what a mistake was. And I would say to students, okay, if, let’s say, you make a decision and things don’t turn out well – they turn out badly, does that automatically mean you’ve made a mistake? And conversely, if you make a decision and things turn out well, does that mean you haven’t made a mistake?

And I was always surprised that even at Stanford, the majority of students would agree with that statement. They would say, if the decision turned out well, then that means it was not a mistake. And if it turned out badly, it was a mistake.

Jules van Binsbergen: Yeah, I know. So the way that I in class try to really make the difference between good decisions and good outcomes salient is through an example from this TV show that, unfortunately, isn’t on TV anymore. It was called Deal or No Deal. And so the basic premise of the show was that there are a bunch of suitcases that have monetary amounts in them, and there’s a fixed number of amounts. You pick one of those suitcases but, of course, you don’t know what amount is in your suitcase. And then gradually, the suitcases are opened, and then it becomes clearer and clearer what amount might be in your suitcase.

And then I say to the students, suppose that we’ve ended up at the end of the show, it’s one cent or a million dollars. So you have a 50-50 bet between getting a million dollars or getting nothing. And then one feature of the show is that the bank can make an offer for the suitcase that you have. And let’s just make it really salient and say the bank offers 999,000 dollars for the suitcase that you have there.

Jonathan Berk: So Jules, let me just get this straight. The guy’s been offered the following: He has a 50-50 probability of either having a million dollars or one cent, and they’re offering him 99 hundred thousand dollars for sure.

Jules van Binsbergen; Nine hundred ninety-nine thousand dollars for sure. You would think that that’s an obvious decision. And so what this person does is they say, you can keep your 999,000 dollars. I’m going to go with whatever is in my suitcase because, you know, I am very confident that I am going to be right. And so what happens is the suitcase is opened, and a million dollars is in the suitcase. And then Howie Mandel says this one sentence that really annoys me a bit every time, and that is – he says to the contestant, “You have made the right decision.”

Now assuming that this person really was facing a 50-50 bet between one cent and a million dollars and didn’t have inside information, can we agree that this person just made the stupidest decision in their lifetime but managed to get away with it?

Jonathan Berk: Absolutely. That person made a mistake, even though the outcome was good.

Jules van Binsbergen: Exactly. So now you’ve really defined differently what is a good decision, which was to take the offer from the bank, versus what is a good outcome, which was he turned down the offer from the bank and managed to get away with it because there was a million dollars in the suitcase. And the other way around, that example also holds. Suppose that the bank had offered him only a thousand dollars for the suitcase for this 50-50 bet. Can we agree that the right decision is to turn down the offer? And then it is possible that there’s going to be one cent in the suitcase. But even if that’s the case, we call that bad luck. We don’t call that a bad decision.

Jonathan Berk: Exactly. So this always reminds me of one time I did a case on a very large portfolio manager called T. Rowe Price. And when I went and visited T. Rowe Price and I spoke to the employees there, I had a very interesting conversation with the most successful portfolio manager in the organization. And he said to me, “Jonathan, I can turn any smart person into a top portfolio manager. They just have to have one characteristic.” And I said, “Really?” He said, “Yes. The one characteristic they need to have is they need to admit their mistakes.” I looked at him, and he said, “You see, Jonathan, this is an industry where you’re a top performer if you are right 55 percent of the time, which means the best portfolio managers are wrong 45 percent of the time. And if you as an employee in that 45 percent always say, ‘Oh, it was just a bad outcome; I made the right decision – it was always a bad outcome,’ I can’t teach you.” So the key part of it is for that person to understand when they made a mistake.

Jules van Binsbergen: Now what I also really like about that example is how difficult it is. I mean, if you only get to make one decision once with the odds you just mentioned, it is nearly impossible without additional information to really say whether you made a mistake or not, if it’s a 55-45 percent bet. And so when we have a lot of repeats of decisions, it becomes a little easier because, if we see a hundred of these decisions, then gradually the 55-45 odds will start to work out, and we start to see that the person is a good decision maker because he gets things right 55 percent of the time.

Jonathan Berk: Yeah. So the question of how many repeats are required depends on the applications. And sometimes, the ratio of noise to skill is low, and you don’t need many repeats of the outcome, and you can tell it’s a mistake. But very often, the ratio of noise to skill is very high. And so the outcome gives you very, very little information.

Jules van Binsbergen: Yeah, and so I think that it really depends on the context. And let’s just give a couple of examples of professions where we go from a low noise-to-skill ratio to a high one. Think about a math professor whose job it is to prove a particular theorem. Now clearly, there’s very little randomness to that. And so that person either did that right or not. And if the proof is wrong, then they made a mistake. And if the proof is right, they didn’t make a mistake. It’s really that simple. When it comes to musicians, they have a very low error rate. If a top musician would play ten clearly wrong notes in a concert, everybody would really think it was a bad performance. So most of the time, they are right on. And so again, the ratio of noise to skill is very low.

But as we go through other professions like surgeons or lawyers or the example that you gave, when we get to the point of portfolio managers, it is in fact the name of the game that you are betting on very noisy things. That is the point of it. And so if you can get it right just a little over 50 percent, then you’re a very skilled portfolio manager.

Jonathan Berk: Yeah. So let’s take the collapse of the Florida condo. I mean, almost certainly, that was a mistake because we expect condos to not collapse, right? I mean, obviously, if there’s a major earthquake and a condo collapses, [one] could argue was that a mistake or was that – that you can’t design a building that will always survive every earthquake. But in that case, there’s nothing different. It just collapsed by itself. So almost certainly, that’s a case of where it’s a mistake.

Jules van Binsbergen: Yeah. So let’s make it a little more difficult. What about the Challenger Space Shuttle that exploded? Was that clearly a mistake? I think a lot of people would say it was clearly a mistake. But there was a particular confluence of circumstances that seemed to be pretty unlikely. The fact that the rings that were not able to withstand the very low temperatures that were going on there, is that something that you should have been able to foresee? And you can see that there it’s already a little trickier. When it comes to selecting stocks or selecting investments, then the name of the game, as we said earlier, is that you take risks.

I mean, so it really depends on the objective. I think that if you run a business, I think there’s no way around the issue that you can not only invest in risk-free things where you don’t take a chance. And so if you want to innovate and if you want to invest in risky new things that potentially have upside, then you’re going to be wrong quite a lot of the time. And you cannot say every time that things didn’t work out well that that was a mistake because, as we just illustrated, there’s a clear difference between good decisions and good outcomes. And so if you want to become a better decision maker, it is important to not every time conclude when something doesn’t work out well that you made a mistake, or, if it did work out well that you think that you’re a fantastic decision maker. It’s possible that there’s a disconnect between those two things.

Jonathan Berk: So Jules, I would say though that the typical business decision is much more like a portfolio manager than a musician. In other words, it’s very difficult in a business organization to ascribe responsibility to any one worker. There’s so much noise, it is extremely difficult based on outcomes to decide if a person made a good decision. And especially since often these decisions take a very long time to – the outcome takes a very long time to come. You can make a decision today and it may be years before the outcome is obvious. And the person may not even be in the same job. So I would say in a business, using outcomes as the sole discretion for whether somebody makes a good decision is a business mistake.

You know, going back to T. Rowe Price, one of the things that really surprised me about that organization is how much effort went into determining compensation of the senior executives. They spent something like half their time for three months just on the question of compensation for their portfolio managers and analysts. And what’s interesting about that exercise is how little the outcomes played in the decision-making process. They didn’t even use one-year returns. They only used three-year returns and five-year returns. But they used a wealth of other information.

They used when an analyst recommended a stock, how convincing they were. They looked at not only stocks that the portfolio manager held, they looked at stocks they didn’t hold and see how they did. So they used a wealth of information to determine compensation.

Jules van Binsbergen: Yeah. So what I really like particularly about the fact that they both look at the investments you did make as the ones that you didn’t is that there are two types of errors that people make, and one of them I think gets way more attention than the other. The first one is the error of doing the wrong thing. That’s such a salient thing that everybody pays attention to it. And when something goes south, everybody’s debating it and discussing it. But there’s another error, and that is the error of failing to do the right thing. And so I think that when we want to [assess] decision-making more broadly, it is important to look at both these mistakes and properly trade them off.

Jonathan Berk: Yes. I think this is a good time to introduce our guest, Hadley Mullin, who is Senior Managing Director of TSG Consumer, a 20-billion-dollar private equity fund. And what TSG Consumer does is invest in companies. And they have to make decisions about companies and whether or not they are going to be good investments. And so Hadley, welcome to the show.

Hadley Mullin: Thank you. I’m thrilled to be here with you all today.

Jules van Binsbergen: Thank you. We are very happy to have you with us. So just to set the stage a bit, Hadley, so there are a lot of opportunities of what you could invest in, what fraction of opportunities that you get to see do you eventually invest in?

Hadley Mullin: Yeah, I will share with you that the percentage of deals that come in the door or that we bring in the door and that we actually close an investment – in which we actually invest is very, very small – I’d say less than one percent [laughs] – by a meaningful margin, less than one percent. We are very selective in what we’ll even take a look at. As Jonathan described at the top of the call, all we do at TSG Consumer is focus on investing in consumer-facing businesses. They can be selling products, or they can be selling services.

Jules van Binsbergen: So as you know, the topic of our episode is to try to tell apart the difference between good decisions and good outcomes. And so there are two types of decisions that you may regret that you might view as a mistake. And so when you perceive that you’ve made a mistake, how often is it an investment that you did not make versus an investment that you ended up making and that didn’t perform the way you thought it would?

Hadley Mullin: I would say with respect to decisions, we think about it very similarly to what you’ve just described, which is there’s kind of the error of omission where we choose not to move forward with an investment because we have come across some risk or set of risks that we just can’t underwrite or get comfortable with. That is far and away our biggest – the most frequent form of a mistake, if you will, that we make. And we can very much measure these mistakes [laughs] because one of our brethren will make this investment, and then we watch as the business over-executes. The management team over-executes on their plan, and maybe there are some exogenous factors that give a further tailwind, and it ends up being a home run.

And we lick our wounds over those ones. I would say, in general, we tend to have a lower loss ratio. And certainly, a number of us grew up in Bain & Company’s private equity practice where we were hired to conduct due diligence and find kind of everything that could possibly on a commercial basis be wrong with a business. And sometimes I think that bias causes us to pass on opportunities or over-assume there’s a risk there that may be easier to mitigate than we realized. But we’d rather stand on the side of preserving capital.

There’s the other mistake is that you conduct your due diligence. The business looks good. You’ve identified risks, yes, but you’ve gotten yourself comfortable with those risks. And for a variety of reasons, both specific to the company or specific to the market, or even some exogenous effects or issues, COVID of course being one of them, that was unforeseen, the business doesn’t go well, and you lose your money. Or maybe you get your money back, but that’s about it. You don’t cover the cost of capital for your investors. Those are really bad outcomes, and we do postmortems on those. We take that very seriously so that we can learn from our mistakes.

Jonathan Berk: So Hadley, do you perceive the cost of those two different mistakes differently?

Hadley Mullin: Yeah. I mean, one’s an opportunity cost, and one’s an actual realized loss. So the latter is – we perceive as far worse than the former. Of course, we missed out with the former, where we should have done a deal but we didn’t, and it ends up being a great outcome for whichever one of our peers did make the investment in said company. Um, that’s a huge opportunity cost to us because we could have made good money for our investors. But again, far worse is when we actually lose our investors’ money.

Jonathan Berk: Am I right in saying that the second mistake, when you make an investment that doesn’t go well, there’s much more learning that you get from that because you have much more information about what actually happened? When you watch somebody else do really well, where you don’t have the information to do a postmortem.

Hadley Mullin: That is true at a high level. Because we focus exclusively on consumer, we do often, when we fail to make an investment in a company and one of our peers does it and it does really well, chances are, when that other investor brings – three to five years later will bring the company to market, and we will get a detailed offering memorandum as part of their sale process, that other PE firm sale process, that will show us exactly what the management team has done and the board has advised over the last three to five years.

So, for instance, we may have passed on a company because it has a very narrow product range that sells in just one geography, and we weren’t sure that it had universal appeal sufficient to scale and sell for a price that would deliver our minimum returns. We may, in fact, then, when the business comes back to market several years later, we may in fact realize that, huh, this is what they did. They were, in fact, able to extend into adjacent geographies and product categories, et cetera. How did they do that? So there is some learning to be had, but it’s much delayed because, to your point, we don’t – we’re not under the covers on that particular investment once we pass on it, and we have to wait until it’s brought to market again.

Jonathan Berk: Let’s say you’re doing postmortems, both good or bad – how to you deal with the incentive and behavioral issues in the room? Obviously, people are going to be defensive. It’s hard for people to admit that they made a mistake. So how does the organization deal with that stuff?

Hadley Mullin: Right. We have a very different culture, I believe, in the private equity arena. And that is that we have a very we, not me, culture. We in fact kind of strike – I mean, if somebody refers to a deal as Steven’s deal or Tom’s deal, that is really discouraged. We – all we do is consumer, and we have over ten partners focused exclusively on consumer. We’re going to have three to four partners on every deal.

Jules van Binsbergen: So does that mean that, for that reason, you think you can sort of avoid these standard problems where it’s hard for people to admit that they made a mistake? Or is it still the case that a group of people can still fall for the same sort of behavioral decision-making fallacies of being defensive about it, because a postmortem – in one of the previous episodes, we had Ruth Porat from Google, and she said, “What we are really trying to implement is blameless postmortem,” which implies that it’s encouraged to actively admit to the mistakes because everybody makes mistakes; there’s just no way around that, and so, therefore, it is an incentive to actually admit things that went wrong, just to make sure the next time that wouldn’t happen?

Hadley Mullin: I couldn’t agree more that it’s really important to set up a culture where there is just radical truth and acceptance, and it’s not a blame game. The best investors make mistakes; but the very best investors learn from those mistakes. It would be far worse not to do a detailed assessment of what went wrong such that we – in that situation, you’re not able to incorporate those learnings and actually continue to sort of sharpen the saw, if you will. Because of this team-based approach, we really do – it’s depersonalized.

And what we reward, culturally, is that as we’re evaluating a particular investment, a really kind of sober assessment, somebody always – if somebody’s particularly passionate about a deal, of course they’re going to advocate for that deal. But what we really look for is we focus on balance. We want that particular partner who may, for whatever reason, be championing a deal, we want to see him or her have a very thorough assessment of the negatives. And we also are staffing multiple partners and team members to a particular due diligence process because we don’t like deal fever, which can happen when maybe there’s one – some other firms, they may have a generalist firm, and they have one consumer partner.

And that one consumer partner is going to do one or two deals per fund, right? And if you’re running up towards the end of the fund and you haven’t necessarily done a deal yet, what does that mean? You may have a confirmation bias in your assessment. By having multiple partners and really moving away from any sort of attribution credit – and this is a really important point – by again not focusing on whose deal this is, but it is collectively our deal, and we are going to make the best darned decision we can with the collective capabilities around – on this – around this table, it’s that setup of when we make the investment decision that I think it makes us better positioned to do a sort of unbiased postmortem where we’re not pointing fingers.

I mean, if anything, we’re all just looking in the mirror and saying we all joined hands, we locked hands and we made this decision together. So I didn’t hear – if you didn’t speak up and vote no in an investment committee, this is as much your deal as it is everyone who worked on the deal specifically. That’s very core to our culture.

Jules van Binsbergen: It sounds like you’ve built a truly team oriented culture at your company, Hadley, you must be very proud of that culture.

Jonathan Berk: Thank you Hadley, for spending the time with us today.

Hadley Mullin: ‘Bye-bye.

Jules van Binsbergen: ‘Bye.

Jonathan Berk: Thank you. ‘Bye.

Jonathan Berk: Thank you for joining us today.

Jules van Binsbergen: You’ve been listening to All Else Equal, please rate and review us on Apple podcasts. We love to hear from our listeners, be sure to catch our future episodes wherever you get your episodes. For more information, visit allelseequalpodcast.com, or follow us on LinkedIn. The All Else Equal podcast: Making Better Decisions, is a joint venture from Stanford Graduate School of Business and the Wharton School at the University of Pennsylvania and is produced by Podium Podcast Co.