243. A Quantitative Approach to PMF, Tribe’s 8 Ball for Objective Evaluation, and Approaches to Reduce Talent Biases (Jonathan Hsu)

A Quantitative Approach to PMF, Tribe's Magic 8 Ball for Objective Evaluation, and Approaches to Reduce Talent Biases
Nick Moran Angel List

Jonathan Hsu of Tribe Capital joins Nick to discuss A Quantitative Approach to PMF, Tribe’s Magic 8 Ball for Objective Evaluation, and Approaches to Reduce Talent Biases. In this episode, we cover:

  • The thesis at Tribe Capital?
  • How do you define PMF?
  • Could you explain what the 8 Ball diligence framework is?
  • When is a deal too early — how much data (or over what time continuum do you need data) in order for the model to assess appropriately?
  • Just for evaluation or also sourcing?
  • Can you apply this tool to a range of business types (ie. SaaS vs. Maretkplaces vs. User-Growth, etc.)?
  • How to avoid false positives? Data looks great for early phase — early market… How do you know that it isn’t luck and the company didn’t stumble onto something with early signals of PMF but they don’t have the insight or flexibility to evolve the business through growth and scale phases?
  • What aspect of early stage investing don’t you use data for? Why is this not appropriate to measure with data?
  • Loss ratio goes down — do you think it increases potential outcome size?
  • On the evaluating side of things, it seems like Tribe has a big emphasis on using data to understand early product market fit. What is your definition of product market fit?
  • Does Tribe conduct portfolio support in a similar way?
  • What happens when data and intuition clash?
  • Do you always lean one way or another? Examples of either?
  • How do you account for exogenous factors?
  • Ever been a scenario w/ a company misrepresenting data (fake data)?
  • How long did it take to build this out — what is the current team structure of investors, developers, and data scientists?
  • Where do you and the team at Tribe need to improve most?

Guest Links:

Transcribed with AI

Intro 0:02
welcome to the podcast about venture capital, where investors and founders alike can learn how VCs make decisions and reach conviction. Your host is Nick Moran. And this is the full ratchet.

Nick Moran 0:19
Jonathan Hsu joins us today from San Francisco. He is co founder and general partner at tribe capital, an early stage venture fund recognized for its emphasis on early product market fit and data driven investment. Prior to Dr. Capital, he was a partner at social capital, led the creation of analytics and data science at Facebook, and was a physicist earning his PhD and masters at Stanford. Jonathan, is Protagoras your real middle name?

Jonathan Hsu 0:46
That is actually my middle name. With a very bad sense of humor. That’s,

Nick Moran 0:52
that’s hilarious. Well, welcome to the show. It’s a pleasure to get a chance to to connect. Glad to be here. I’ve chatted with your partner or June before but not you, can you give us a sense for your background and your path to venture?

Jonathan Hsu 1:04
Yeah, you know, I think you you kind of covered it. But you know, I started out as a physicist, I was at Stanford studying black holes and string theory, in the early 2000s, which was, you know, it’s, it’s really kind of interesting, but sort of truly, deeply useless. I did the form of physics that involves like, no computers, this was in the early 2000s. This is like, you know, chalkboard physics. So towards the end of my PhD Career, I didn’t want to be an academic. So I ended up leaving and going, I joined Microsoft to be a product manager, I was only there for a little while. And the Facebook platform opened up. And me and a couple of friends built one of those very early social networking applications was called Super poke way back in the day, I don’t know if you remember that one. And yeah, you could throw sheep on the internet. So awful in 2007. And we ended up selling it to a company called slide, which was Mac’s legends company, time. And so I went there, and I ran data for max for a couple of years, that slide. And then I joined Facebook in 2009. When I joined Facebook, there were about five or six of us kind of doing data science and analytics across the company is not a lot of us. And then by the time I left, you know, we had this big sort of 150 person org, I really helped, you know, sort of put that together. And you know, now it’s like, 1000s of people doing this stuff. And so by 2014, I’ve been there for five years, and, you know, is interested in doing something else. And you know, had a couple of friends in social capital. So join social capital in 2014. And yeah, and that’s that was sort of my first step into venture, I guess. Awesome.

Nick Moran 2:32
And then when did the transition occur from social to founding tribe?

Jonathan Hsu 2:37
Yeah, so as a social capital for four years, from 2014, to 2018. And in 2018, you know, Arjun, Ted and myself, we sort of decided to spin out to create to create tribe. So it’s been two years now, almost to the day. Got

Nick Moran 2:52
it. Well, congrats on that. Tribe is a really interesting firm with a really interesting lens on things. I’d like, maybe we could just start out, maybe you could describe the thesis, and then we could unpack some sort of your differentiators? Yeah,

Jonathan Hsu 3:06
yeah, a try, we really are really focused on recognizing and amplifying early stage product market fit. Okay, so. So what does that mean to us, that means to us really utilizing a bunch of these data science and analytics techniques to really help us understand the early pattern of you know, how customers interact with a business with a product, and then use that quantitatively based understanding, obviously, to help us make investment decisions, but maybe more importantly, help founders, you know, I think this really stems from, you know, sort of our history and social gaming or history in the social web. Part of what the social web taught us in the sort of mid 2000s Is that you can use data to help yourself grow, you know, you can use it to help make good strategic decisions executed the next level. And so, you know, we spent all this time doing that as operators from the mid 2000s on, and then, you know, part of what we, what we like to do here is to use that wisdom with company, you know, it doesn’t mean always grow at all costs, but it means use data to just help you make the best decisions possible. And we really aim to use data that way. And then as a side side effect, you know, it helps us make good investment decisions as well.

Nick Moran 4:12
And, you know, are you investing across stages? Are there certain sectors or areas that you focus on? Yeah,

Jonathan Hsu 4:18
we focus primarily seed through Series B, although we’ve definitely invested all the way up. You know, we call it this last round of Carta, for instance, which was, I think, a Series F. So we, you know, we’ve definitely gone up and down stage wise, I would say that the idea of using data sort of at the series A is kind of a special bit. I usually say that when a company is in the seed stage, you’re really investing in a team right now that that is a full team that it they kind of have a product, they don’t really have a business. That’s been a Series C company or a D. You know, that’s a business at that point. It has income statements, and you can treat it like a business. In the series A and a B. It’s kind of not yet a business, but it’s more than a team. There’s more than it’s Team, there’s more than just people in there that you’re investing in. And so it’s sort of the most interesting part. And we believe that that’s actually the place where, where, you know, what we do is sort of most differentiated by really recognizing product market fit. You know, when we say product market fit, that’s actually in some sense, that’s the definition of the series A right like companies go through their seed stage. They they’re dramatically changing the products. And then they come to the series investor, and they say, I have found product market fit, here’s a line that goes up, maybe the line is revenue, and maybe it’s user or something. And they say, if you pour money on me, this line goes up forever. We know that’s right, like series A investments about you know, half of them just go to zero. So clearly, there’s something sort of amiss there. And so we really view it as our sort of thing that we focus on our expertise is really recognizing that early pattern, and being able to understand what’s going to happen, if you put money on this, if you start putting fuel into this thing, you know, will it have legs to get to the next level? You know, will it have a really a shot at becoming something really big? So

Nick Moran 5:54
I hate to even ask this question, because it’s almost become so cliche, but how do you define product market fit?

Speaker 2 6:01
Yeah, no, it’s, it’s a good question. So I’m gonna answer that and kind of a roundabout way. So I have this saying that, I like to say that accountants were the first data scientists. Okay, so what does that mean? So, accountants, you know, they take a big pile of raw data, you know, the ledger, and they turn it into something useful, like an income statement, or a balance sheet. That’s actually all that data scientists write, they take a pile of raw data, and they turn it into something useful. Maybe they use a bunch of fancy math, fancy statistics. But at the end of the day, if you’re a strategic decision maker, you don’t care about that. You want it to be useful, right? You want it to be something you can trust. So in the accounting world, sort of the analogy here is how do they define the concept of profitable? Well, you know, that profitable actually doesn’t have a single definition, right? Profitable is actually a family of concepts, right? If you’re like a private equity guy, maybe you want EBIT, da, you know, maybe you’re interested in Unit margin, maybe you’re interested in contribution, profit, maybe you’re interested in net profit, you know, these are all different concepts. And so when you say something is profitable, what do you mean, depending on where you come from, you might mean something else. But that said, it is still a family of concepts that are measurable. And so for us product market fit is a family of concepts that is measurable, we have a sort of underlying framework that looks kind of like accounting, and we call that the eight ball, sort of the underlying framework, and then that thing outputs a bunch of different types of numbers. And depending on how you put them together, you may have different forms of product market fit the same way that you know, accountant would put together different forms of profitability.

Nick Moran 7:25
Got it? So let’s get let’s go a bit deeper there. Right, you know, what was the the architecture of that? And you know, what are the results look like?

Speaker 2 7:33
Yeah, really, we view it as a framework. It’s an intellectual framework, like accounting, right? So if you think about accounting, accounting is amazing, right? What it does is, so, accounting is amazing, because it applies to every business out there, which is kind of remarkable, right? The fact that you have one intellectual measurable framework that you can use on every single business is pretty remarkable. So it’s super broad. It’s not that deep, right. So if you dig into account, if you let’s say, you are an expert at insurance companies, you know, top level accounting is kind of going to be somewhat useful, but then there’s going to be a bunch of very specific insurance accounting, that’s sort of really specific to your domain. And so, you know, accounting is a sort of a field has this interesting aspect that it’s super widely applicable. And it sort of has, you know, sort of ramifications and derivations that are more specific within within certain sub segments. And, and so product market fit for us, the eight ball for us is similar, it’s really around understanding the pattern of user interaction and customer interaction within the context of some business. You know, if you think about accounting, really what accounting is, is it’s an incredibly detailed way to understand costs. If you ever look at an income statement, basically, revenue is like one line. And then there’s 20 lines of costs, because it’s like, freeware, where the dollars went. Now, the thing is that in the world of software, right, costs are really simple. You know, building software is super cheap, right? So having a lot of knowledge about the cost structure doesn’t really give you a lot of insight into what’s going on. As such, you know, really, the thing that matters to us is really above that it’s really revenue in the structure of revenue, but even as a leading indicator to revenue, what is the product market fit that leads revenue? Right? What are the signs of engagements? And how can you pick that apart? And once again, a standardized way across businesses, right, that helps give you insight into what’s going on, on the demand side, pushing off the cost question until later.

Nick Moran 9:22
So let’s talk about a simple example. Right? Let’s take like software as a service. Are these blocks of data? Is it you know, cohort analysis is it quick ratios, MRR hours, obviously, you’re looking at growth, you’re looking at engagement, you’re looking at churn numbers, I mean, what are the major if you were to simplify this down? Because I’m sure you’re acquiring and analyzing lots and lots of data, but if you were to simplify it, what are some of the those major blocks of data?

Speaker 2 9:47
Yeah, the three I guess, so the underlying block of data, just like for an accountant, the underlying block is the ledger. So for us, the underlying block is usually something like a transaction log of some sort. That’s the underlying data. Now, you know, For an accounting, they’ll take that block or that underlying data and turn it into basically sort of three standardized statements, right? There’s an income statement, a balance sheet and sort of a statement of cash flows. Roughly for us, we think about three sort of families of analytical techniques, right? There’s growth accounting, which is really okay, you’re growing, what are the contributors to grow? If you like, how do we break down growth? Which pieces? Where’s growth coming from? How much of it is from expansion? How much is it from new customers? What is churn look like? What is contraction look like? So there’s that that’s growth accounting. The second area is cohorts, which as you describe cohorts, okay, cohorts have an LTV component, they have a retention component. And then the third sort of area that we look at is concentration. So you know, this concept of 8020, right, where like 20% of the customers generate 80% of the value on the internet, that’s actually not the case, usually, things are 6020. We only know this because we’ve measured this hundreds and hundreds of times. And so you know, understanding sort of the distribution of product market fit within within the system. Now, so those are the three families. Now, the thing that’s interesting is that you can use those three families of analytical techniques on many different things to see. Whereas accounting really only looks at dollars, you know, we use these three sort of analytical approaches to engagement logs, we use it to revenue, SAS, we can also use it to pay out so let’s say you have a bunch of drivers, your Uber and you have drivers who you’re paying, well, you can measure the payout structure, this way, you can measure the rides, this way, you can measure time spent on the application, this way, you can measure commits to an open source repository, this way, you can measure the product market fit of a single open source repository with some audience of software engineers, this way you can measure just about any interaction of some product with some set of users with these abstract concepts. It doesn’t have to be dollars is

Nick Moran 11:44
working capital cash flows float, is that a part of it? Does that matter to you?

Speaker 2 11:49
Um, you know, the analogy between accounting and sort of our eight ball sort of I don’t I don’t know if it goes that deep, right? I think it’s, it’s more like this concept of there’s an underlying dataset, and there are things you can compute from it. And then, you know, those can be useful to you, I think, the concept of working capital cash flow for a company’s finances, you know, is there a good analog of that in the eight ball side? I’m not off the top of my head. But I think it’s also because the specific concepts don’t translate necessarily mean, the we take is really more that there’s an underlying log of something going on. Now, what are the standard things you measure from the log? If the log happens to be costs and revenue, then the standard thing you do is you make an income statement? If the log happens to be users logging in? Well, maybe you do something else, right? In

Nick Moran 12:33
talking to Marcelino, and in our June, your colleagues about this, it sounds like it’s something that you can assess very quickly for startups. So give me a sense for what is the input data? What is the source data? Is it? Do you go all the way back to things like QuickBooks? Or what are you plugging into to be able to acquire this data, munge it and then get a lens on where the company is at from a product market fit standpoint?

Speaker 2 12:58
Yeah, our preference is usually anonymized raw data, that’s kind of our preferred mode, you know, where they just give us literally raw data, somewhat, you know, anonymized obviously, to protect, to protect anything sensitive. And then we use that to sort of build it up. Now, that said, a lot of these sort of analytical outputs, founders can do them on their own as part of why we’ve we’ve written big, you know, these giant blogs about it, explaining how to do them. And then we can always take the outputs of the of the analysis rather than the inputs were really good either way. But really, the goal is to develop that viewpoint, right to be able to get to that viewpoint of ground truth. What is Product Market Fit doing? What is the pattern of growth right now? What is the pattern of engagements? And can we use that to build a very quick bottoms up understanding of how this company is doing before doing all of the others traditional venture work?

Nick Moran 13:43
Jonathan, when is a deal too early? Right? You talked earlier about, you know, Series A, you can get a sense for where things are trending and what the data looks like. But I’m curious how much data do you need? Or Or maybe over what time continuum? Do you need data in order for the model to assess the opportunity appropriately?

Speaker 2 14:05
So going back to the accounting analogy, right, you can make income statements for a company arbitrarily early, the question then becomes more how much income statements you need to feel somewhat confident about the next whatever N months to next n years, right? So in a similar way, we can apply these frameworks to a company arbitrarily early, right, they have one customer, we can do it. But it tends not to be super useful. So for us, it tends not to be useful, sort of in the same way that an income statement for a company that’s extremely early as not useful. So for us, we tend to focus on companies once they have at least three months of data. Typically, like when you talk about b2b SaaS, the most common thing that we see is sort of, you know, anywhere from five to 20 b2b customers, and those customers will have underneath them, each one will have five to 10 users touching the software, right? So on the underlying, you know, the underlying piece may have actually hundreds of users touching the software. And so that’s a pretty common thing that we’ll see where we will then analyze the revenue structure and In relation and the sort of what the revenue streams look like, that looks like pretty conventional SAS, and then we will also analyze the underlying usage, the engagement piece, and then between the two of those, we’ll be able to get a very good sense of what’s going on, before it starts going deeper.

Nick Moran 15:14
Got it? And is this just a tool that you you all use for evaluation? Or is it also, is there a way that you can use it for sourcing as well?

Speaker 2 15:23
Well, not not so much for sourcing, I would say there really the the value of it comes more from portfolio right, for companies that are in the portfolio, right? You know, we really developed all this stuff at social capital, initially, because, you know, we were interested in helping portfolio companies get to the next level, right? The logic was, you know, 2014, the logic was something like, Well, you know, Facebook, the social web companies had done so well leveraging data to just grow and execute, and out execute everybody else, can we take some of that philosophy and give it to our portfolio companies? Right. And so that was the goal, the goal was to use these frameworks in those companies. And it was, when we did that, that we realized, oh, there’s actually a bunch of commonalities. You know, when you apply the framework to Slack, when you apply it to this education company over here, you apply to this FinTech company, the SAS company, oh, wait, there’s a bunch of similarities in language, then you can actually abstract away an analytical framework, and then we abstracted it away, then that’s sort of the result. And even now, you know, when we do this analytical work, that’s why I said, you know, the, the main thing that we’re interested in is helping entrepreneurs give the value back to the entrepreneur, you know, yeah, it helps us make a better investment decision. But I think if you’re an entrepreneur, the goal should be more okay, chat with tribe, because they’ll give you an a differentiated viewpoint on your company. And you will likely find it useful and helpful, because no one else is going to give you a viewpoint like this, maybe they’ll invest, maybe they won’t, but even if they don’t, at least you get something valuable and interesting. Got

Nick Moran 16:44
it. I know that there’s a number of firms out there that are monitoring a variety of data exhaust online, and then using that, you know, for sourcing techniques, I didn’t know if there was a instantiation of this that did that as

Speaker 2 16:56
well, it goes back to the the notion of like how you believe value is created adventure, right? I think that we have a belief that value is created by expanding Product Market Fit by helping things grow. That’s what we believe matters most. And so our goal is to is to help entrepreneurs do that. And then yeah, we obviously hope to make investment returns. But number one has to be helped companies do that. It’s not really so clear to me that if I build it, this giant machine learning thing to help me, that doesn’t really help companies grow, you know what I mean? And it’s not, it’s not, you’re not adding value to the system. When you do that. That’s part of why we, you know, we do some of that, to be honest, but it’s not a big part of what we do, because we don’t think it’s the most important value to add to the system.

Nick Moran 17:34
Is there a cadence by which you run the analysis for the existing portfolio companies and provide that?

Speaker 2 17:39
Yeah, so with our existing, you know, when we invest in a company, usually it starts getting integrated into how they do their own board reporting, it’s how they it gets integrated into how they think about their company, that’s the goal, right? The goal is that hopefully, we can help you think about your own company, you know, a little bit differently. In which case, we ended up seeing the results, you know, on a regular basis, typically, we will do, you know, sort of a very complete and thorough rundown on the data, when they are preparing to do their next round of financing, right, it helps us sort of understand what our reinvestment is going to be. And it helps them you know, sort of clarify their story. And, you know, oftentimes entrepreneurs will actually literally just take the work that we do for them, you know, our portfolio company, and use it in their fundraising. And it’s like, here’s what the tribe guys did. It’s kinda like an audit, right? Here’s what they found. Do with it? What you may? And

Nick Moran 18:30
does this work for a range of business types? We talked about SAS before but you know, marketplaces, various ecommerce businesses, user growth businesses, you know, what is it optimized for?

Speaker 2 18:41
All of those? Yes, it works in consumer, it works in consumer, we can think of a consumer transactional consumer subscription, a SAS top down b2b marketplace, every form of marketplace, we’ve measured, really, really the only things it doesn’t it really works anywhere, where there’s a customer interacting with a product, which sounds really broad, but it’s not everything, right? If you look at like really hard tech build outs, where they raise a Series A to just build the thing, because it’s so hard, it doesn’t really help you there, right, because there’s no customer interacting with the product. So we don’t do a lot of like, really intense robotics are like, you know, really, really hardcore technology tends to be more situations where there is some form of customer using some form of product.

Nick Moran 19:20
And how do you avoid false positives? Right? Let’s say there’s a business that has early traction, beachhead market, you know, bunch of innovators, early, early adopters. But the dynamics of the business do not work, either through the growth and scale phase. Or maybe the team got lucky and stumbled on something early. But maybe the team is not versatile and flexible enough to evolve with the business over time and take it where it needs to go, you know, how do you how do you avoid just looking at the data snapshot in the early years and making sure that it’s applicable and it’s going to continue in the right direction?

Speaker 2 19:55
Yeah, it’s a good question. Really, the question you’re asking is how do you use the data? To inform an investment decision. And you know, I should be very clear here, even though we do all this data work by no means is it the sole thing that goes into an investment? It’s just like with accounting, right? How do you avoid, you know, false positives when you look at income statements? Well, you know, you would never invest based only on an income statement, that would be the same way here, you would never invest based only on this eight ball stuff, that would be foolish, right. But that said, if you’re Warren Buffett, you certainly look at the income statement, you know, you should look at it. And sort of that’s our goal, we should look at it. And we should, you know, try to understand it, and hope that we are on the same page with management about what it says, before going into the rest of, you know, sort of the the venture process, it

Nick Moran 20:39
seems very reasonable to me that your loss ratios are going to decline with something like this. Do you also believe that upside and potential outcomes increase as well.

Speaker 2 20:49
So our loss ratios historically have been lower, when we use this process, both social capital and a try, but it’s not honestly, it’s not what we’re aiming for, you know, really, what we’re aiming for is finding companies that are going to be huge, right. But the problem with with with just looking at that is that you sort of miss all the execution, you miss the fact that in order to get huge, you need to be able to buy yourself time, you need to be able to like, get there, right and and that’s really what the series A and B is it’s about bridging that gap from, you know, you’re a team with a prototype to a business. And so the question is, are we gonna be able to get you to a business? Can we even get you, you know, how, how confident can we be that you can get to a solid business? And then step two is, can you become huge, right. And so even though we spend a lot of we have historically spent a lot of time doing this data work, and we obviously spend a lot of time with founders on this, it’s honestly, you know, it’s something that we are just, it’s just a part of our language. Now, you know, most of our time is actually sort of hard thinking time is spent more on the, if this thing becomes if this thing can get to that stage of a reasonable business, does it have a chance of getting huge, because we think that we really understand that early stage, we really understand the next roughly two years of evolution, if we can understand that really deeply a D risks a big part of the investment, and then we can focus on this question of will it become huge, the problem I think that a lot of venture investors have is they only worry about the will this thing get huge thing, but most of their companies then die much sooner than that. They never, ever get a shot to get huge, because they don’t even make it right. So they don’t even make it to reasonable, much less huge.

Nick Moran 22:17
What percentage of deals pass the analytical data base test, but do not pass the qualitative assessment that you guys bring in after the analysis is complete? Yeah,

Speaker 2 22:32
I mean, the you know, once again, going back to the accounting example, right, it’s kind of like this, what what we do is we, you know, we do this data work, we actually do it hundreds of times a year, because we have built up the capability to do it a lot. But then, you know, basically, if a company is looking like, it’s sort of top quintile top decile in multiple sort of categories, then that warrants further inspection, right. And so in a given year, typically, we will have on the order of about 50, companies that are sort of like, this looks pretty good, you know, this, this is going to make it to the next level. But at which point, you start having this question, okay, now, can it get really big, right, it goes back to the other thing, you said, can it get really big, can we can we help them get to that next, you know, really the next next level, not just not just the next level of scale, but the next two orders of magnitude, right? And you know, of the 50, that sort of look reasonable, I will end up investing in like, maybe five to 10. So that’s kind of what it looks like, you know, we ended up doing, we really tried to do the data work hundreds of times a year, because we really view it as a way to give value to the entrepreneurs. And that’s really the goal, we give the value back. And entrepreneurs pretty much, you know, obviously, most the time we pass, but entrepreneurs say to something like, Hey, you know, a bunch of passing, but, you know, this is like 15 pages of intense analytical work that nobody else has done for me, You’ve helped me look at my business in a different way. Thanks for that, at least, you know, even though you’ve passed. So you know, the whole point here is that we’re adding value most of the time when we’re passing. And then yeah, some of the time when when we have to go super deep, hopefully, we can get down to the, you know, pick the right, whatever five to 10 that we really want to get behind. But it becomes much less data driven, sort of when you get down deep into the funnel, it becomes much more market, you know, the team stuff really starts to make a difference and sort of our viewpoints on on a bunch of bigger questions, but we really don’t, we really only focus that energy. You know, once we already know, there’s sort of a base of product market fit from which to work.

Nick Moran 24:18
That’s great, though you provide these, this feedback to the founders and the teams that you don’t invest in, I assume that you also indicate the areas that require improvement?

Speaker 2 24:28
Absolutely. You know, very often, it’ll be like, Hey, we’re passing, here’s your 15 page report. Right? That was really good. You know, actually, from our point of view, that’s like a median number. This number here you thought was, you know, sort of median, that’s actually pretty good, you know, and sort of give them sort of that benchmark, you know, without without obviously we don’t tell them about your competitor over there as well. We don’t tell them that we just say hey, this is this is better, this is worse and sort of give them that and sort of talk through that. And usually that also involves something like, oh, here are a couple of ways that we measure things that you’ve probably never even thought of, you know, we met During this specific aspect of your product this way, this is why, and this is good, not good. And this is why, and then the entrepreneur is like, Hmm, I didn’t know, I could even measure that that way. That’s useful. And then it gives him something to think about.

Nick Moran 25:13
Love that. How do you think, you know, any analytics tool or any database assessment, you know, is, you know, whether it’s venture or hedge funds, or what have you is sometimes at risk of the things that cannot be measured, right? Like exogenous factors, we’ve had one this year, pretty significant. So give us a sense for how has that kind of impacted all the numbers? And has it reframed any anything? Is it adjusted assumptions? Or what are you doing now that you’ve seen the the massive exogenous factor affect the portfolio, and you can look at all the data?

Speaker 2 25:49
Yeah, I mean, it really hasn’t broken assumptions in the same way that like, the pandemic doesn’t make accounting irrelevant, right? One still does accounting, right? You know, we still do the data work, it’s just give you if anything, it’s great, because it’s giving us a lens, to see how these frameworks express what’s going on in a measurable way, you see, like, you can think about the pandemic happening, it’s kind of this big shock, when you look at a company, if you’re an investor, right, depending on the type of investor you are, you kind of have some mental model of what a company is, right? Like, if you, let’s say, you are sort of a high frequency trader, if you’re a high frequency trader, then a company is a symbol, and a stock price and a line. And so you know, here’s how much like the, you know, here’s the price of the asset, it’s moving up and down. That’s it, that’s all a company has, right? If you are a seed investor, a company has like a talent, these are the founders, these are their, their lives and their pains, and all of this sort of soft stuff. That’s what a company is. See, if you are you know, most investors sort of in this private equity middle fundamentals zone, they tend to look at companies a big piece of it is financials, right, that’s sort of the abstraction that you use to understand the company. And then there’s obviously a bunch of other stuff, strategic stuff. But you know, when a shock, like this happens, you can see Oh, through my lens of accounting, how does how has the shock affected everything? Oh, look, you can see, you know, like, you’re gaining very important experience. And for us, because we have this specific way of looking at the world, we can literally see what’s going on, right through a lens that is somewhat orthogonal to the traditional financial lens, it’s orthogonal to the team lens, you know, I’m sure if you’re a team oriented investor, you’d say, oh, you know, this, this pandemic has stressed teams, and all these different ways, here are the ways in which the psyches of the founders have moved around. And that would be, you know, definitely insightful. And we do some level of that. But you know, that for us, you know, we have a special lens, it’s this lens of product market fit, and we can see how it’s playing out. You no wonder what one of the wonderful things about this product market fit stuff is that there are various datasets out there that you can get your hands on. And you can see into the histories of these companies, you can actually see what various other shocks have done to other companies in the past. So one example here, through credit card panel data, you can, for instance, go back and see what happened to Chipotle during the E. coli scare of I think it was four years ago or five years ago. And you can literally see how these things unfold. Historically, and you know, doing that exercise of doing this historical learning is just part of developing sort of this overall discipline of being able to treat product market fit as a quantifiable sort of thing. Have

Nick Moran 28:13
you ever been in a situation a scenario where a company without naming names, of course, but a company misrepresented data?

Speaker 2 28:21
You know, not really, and the thing about it is, it’s very hard to represent misrepresent this data, if you think about it, right? If you’re an investor and an entrepreneur hands you an income statement, it’s literally like one page with a bunch of numbers on it, it’s very easy to manipulate that right? You could just change the numbers, right? If somebody asks you, okay, can you send me every login event over the last four months? That’s gonna be like millions of rows? That’s pretty hard to fabricate. It’s not saying you can’t do it, you can certainly do it. But it’s much harder. And so that’s the reality is that it’s very hard to fake this stuff. And you know, we have not really run into that problem yet. Now, I’m sure at some point, it will. I mean, of course, if you look at very early stage companies, there are ways for this to be fabricated at the very, very early stage. But you know, like I said, this is just one lens for us, by no means is that the only lens that we use to invest, right, I think we’re differentiated and that we have a lens that very few other people really use. We try to use that to get value back. But from our investment decision making point of view is just one lens of money.

Nick Moran 29:18
How long did it take to build out the eight ball? Yeah,

Speaker 2 29:23
well, I mean, like, kind of, like I mentioned, it’s really based off of a bunch of frameworks that people developed in the social web. You see, like, if you look historically, in the 2000s, what happened is, everybody all of a sudden had this great awakening of what data could do. What happened was that all of a sudden, in the mid 2000s, it became cheap to store tons of data. Okay, and, and the compute computational technology got to the point where you could count to a billion, you may not I don’t know if you may remember this. But in 2000, it was really hard to count a billion unique objects on a machine because like, a computer couldn’t handle right. And so people developed like Hadoop, all this distributed computing stuff to do it. And so in the mid 2000s, wasn’t all of a sudden these companies had a ton of data, they could count. And they were like, wait, I should use this to make better decisions. Facebook came to this conclusion, Twitter, Google, all these companies roughly at the same time, you know, Zynga came to the conclusion of sort of like, what the value of this data could be for their strategic execution. And you know, after sort of going through that myself in Facebook, and social gaming, and watching a bunch of other people do it, really all we did was sort of abstract the thing away from it, every one of those companies thought that their framework was theirs, and that it was unique to their snowflake company. But the actuality was that they were all roughly getting at the same thing. And that’s really what we did was we sort of like tied together, okay, this is the abstract thing that all of us were doing, that we were doing in all of our past lives. So in some sense, it took a long time, but also some sense, it took no time. Because, you know, we’d already done all the hard work along with everybody else in sort of the analytics community. What

Nick Moran 30:48
is the team makeup a tribe? Like? What’s the balance of investors versus data scientists, developers?

Speaker 2 30:55
Yeah. So right now, there are three founding partners, you know, three of us sort of our GPS, we sit on sit on boards, lead investments, but we’re all, you know, all three of us sort of have our own historical expertise, and, you know, some some amount of capability and competence at other areas, you know, like, Arjun has been, you know, serial founders. So he’s gone through that, that journey much more than the rest of us. Ted has been a career venture capitalist, you know, he’s been at AOL ventures, usbp, co founder of social capital, he’s been through a lot, right. And, you know, obviously, my history is in is in science, quantification, and data science. So that’s the three of us. And then, you know, beyond that, we have a team of three data scientists, who combined sort of HBs Bridgewater, Facebook, data science, Slack, data, science, all of these sort of crazy skills. I think across the whole team, we now have three PhDs in physics. And, you know, all of us are capable of being technical at some level, we also have a product whose background is our partner whose background is more in product management, and sort of product definition, product marketing and product design, which is really helpful. You know, and that’s, that’s kind of the core of the of the investment team is really that those folks, you know, I think the the data scientists that we have, tend to be much more like hybrid MBA data scientist, quant types, rather than, you know, sort of the data scientist, you might find building ml models at Uber, they’re not like that.

Nick Moran 32:20
Do you guys invest outside of the states? Yeah,

Speaker 2 32:23
a lot. Actually, we have, I think three companies in Canada company in India, when the UK, you know, we’re sort of, I think a two thirds of our portfolio is in the US. And of the two thirds, about half of that is west coast.

Nick Moran 32:39
I guess accounting is non discriminating across geographies and, and borders. But do you see different revenue demand engagement patterns when you go in different countries, broadly speaking,

Speaker 2 32:53
you we do at some level, but you know, the great thing about it is that because we have a mental model, a quantifiable mental model, that is, you know, sort of inherently pretty objective, we’re able to sort of calibrate that appropriately, you know, on the demand side, and this is maybe one of the wonderful things about software, right software, you know, internet delivered software is kind of the same, no matter where you are, their you know, their phone is better, your phone is worse, maybe your product context is different. So at some level, you know, our analytical methods, they work really well sort of no matter where you are, but of course, you have to layer on a totally different context, after you look at the data, right? The context of the local market, the context of the local competitors, whatever. And that piece, obviously becomes really different. Well,

Nick Moran 33:36
I would imagine, outside of the data and the analysis, you have to think a lot about moats and defensibility, on top of everything you’ve just measured, right? Because you don’t you don’t want to build something wonderful that that has none.

Speaker 2 33:48
That’s absolutely right. You know, and I think one of the one of the reasons why one develops models like this is so that you can understand something like Moats. Okay, what does a moat mean? A moat means like, you have a competitor, presumably, the competitor is more than an income statement, the competitor has their own pattern of product market fit. Can you understand that pattern in some non trivial way? Right. And, you know, we spent a lot of time doing that. Obviously, we’ve obviously we know, the patterns of Facebook and all the social web stuff really well, because we live through it, right? So we know what it means to try to go up against them, not just sort of a surface level of like, Oh, they’re big, but more at the more detailed level of what does it mean to win one minute of a user’s attention? It’s not trivial. And like, what does it take? And how to, how am I convinced if you show me some social networking product? What will you have to show me to convince me that you’re actually winning those minutes away? You know, we’re calibrated for that kind of stuff.

Nick Moran 34:38
I mean, it must be fascinating when you see businesses that have similar profiles, either cross sector or maybe even, you know, head to head competitors. When you’re looking at this data side by side and you you see the differences in unit economics, you see the differences in engagement rates and growth. I mean, that’s, that’s got to be a compelling exercise. Yeah,

Speaker 2 34:58
I think that’s right. And in fact that That’s sort of points to sort of a large, you know, investment theme for us. You know, this spans back from social capital is this notion of consumerization of b2b Right? Like consumerization of b2b, what does that mean? Really, it’s about the fact that like, you know, people in a b2b context who are making a b2b purchasing decision, they themselves have lives as consumers, they themselves have been using, you know, Snapchat, or Facebook, or Instagram or whatever. So they have expectations of what a product is. So if you can appeal to that, you know, they’ll admit they may make a purchasing decision in your favor. And that’s social capital, right? We were early investors in Slack. And a lot of that was based on this recognition that like, oh, there are certain patterns of consumer oriented product market fit, that have not really exposed themselves in b2b. And once you started seeing them, while some interesting things can happen, and that’s kind of what slack was a lot of what we’re, you know, part of what we’re focusing on these days is pushing this notion even further, what does it mean to consumer as the SMBs life right, a consumerized technology world, for, you know, an auto dealer, that’s actually one of our portfolio companies? What does it mean for like fintech? What does it mean for the controller of your, you know, financial department to have a consumerized experience? How do their purchasing decisions work? And so when we think about sort of, like b2b software, this is sort of the macro wave that we’re thinking about sort of consumerization oftentimes, it’s a FinTech angle, and like, what are those buyers doing? And how can we make their lives better? Sort of wanting to time? Yeah, recall,

Nick Moran 36:21
talking about slack with Moon probably five years ago, I can’t remember how fresh the investment was, but really an amazing ride there. What do you think? In what area does tribe need to get better? You know, where, where’s the area where you guys are focused? Most or or think you can improve most?

Speaker 2 36:41
Well, I think all of them, you know, the reality of venture is it’s sort of an unsolvable, extremely hard game, right? Like, there’s a sourcing thing, which is, which is hard? I don’t think anybody you know, it’s not possible to master and solve sourcing, right? I mean, you talk to the guys at Sequoia they’re still busting ass out there trying to meet people, and we have to do the same. Everybody has to do that, you know, you could build some computer to help you do that. But at the end of the day, like, it’s just a hard game, right? And, you know, evaluating is hard. I told you, we have another mental model, but that doesn’t make it easier. It just gives us another another lens on which to look at a situation, right. And so that’s always hard. And then obviously, helping portfolios. I mean, they’re starting companies, that’s hard. So you know, we’re always looking to improve across all of you know, I think that, you know, one thing that we found is sort of just doing something a little differently give value back to everybody, even if we’re not investing in them. So, you know, we can always improve in that. But I would say that that’s a little bit more to zero to one, right? Like, we do that in a way that a lot of venture firms just don’t even do that. Right. And in a systematic way, we do it in a very systematic way. So you know, we think about sort of getting to the next level, we’re thinking about all of these, how do we give more value to founders, even when we don’t invest in them? How do we use that to help us see more deals? How do we get better evaluation? How do we be better partners with founders that we’ve invested in with CO investors? Really all of the above? We are always trying to get better all of them?

Nick Moran 37:59
Is there a frontier sort of internal tech that is going to attempt to do I don’t know the next level thing with data that you guys are working on? And if so, is this something you can talk about? Um,

Speaker 2 38:13
yeah, you know, we’re working on stuff, you know, I guess maybe here’s how one can think about it. So if you look at how, like hedge funds work, like a quantitative hedge fund, I don’t know, maybe like Bridgewater de showing these guys, the way those guys are organized, they have a team of people whose job it is to acquire data, they go out there, they buy all this data. And then they have a team of engineers that do nothing but sort of integrate the data, okay, I got this nasty, gross dataset, how do I stick it into the system and make it usable? Okay, then there’s sort of a bunch of software engineers who basically build abstractions on the raw data, okay? You need that layer. Because if you have that layer, then you can hire PhD economists who sit on top and manipulate abstractions, they don’t have to deal with like, the gross pneus of software, and the closeness of infrastructure, they can deal in abstractions. And then they sit on top, and they do a bunch of research, and they do a bunch of, you know, they find a bunch of interesting insights on the world. So that’s roughly the stack for a quantitative hedge fund, right. It’s a, you know, acquire data, build abstractions, and then do research on top of it. Our viewpoint is basically to do something like that. And that’s kind of what we’re doing, right? You know, we see a lot of data, we get all sorts of data from different places, we’re building a bunch of technology to help to help give us abstractions that we can manipulate on top of it, so that we can get to insights faster about the world, both about the world in general and about individual opportunities, as we see them, you know, hopefully, over time, that whole system compounds, you know, to our advantage, and to the advantage of, you know, the ecosystem at large as we as we give that value back. Jonathan, how

Nick Moran 39:37
do you attempt to measure or assess talent?

Speaker 2 39:42
Well, it’s a big, big topic. So we don’t actually do much of this at at try, but it’s something we’ve thought a lot about, you know, when I think about using data, you know, you’ve heard me sort of walk through all the ways we use it. At some meta level. The point of using the data is to really remove bias and incomplete Isn’t biased and income families just an income statement? It’s how you interpret the income statement that can be really biased, right? And so that’s kind of the point of data does is it gives you some input, hopefully, it’ll reduce the odds that you are biased, but you may still interpret in a biased way. Now, when you apply this to talent, right? There are actually examples where this works pretty well. Right? I would say, you know, this might be somewhat controversial. But I would say that the best example of this is actually college admissions as a system over the last 50 years now, people is probably controversial, because I think most people would say, college admissions is clearly obviously horribly biased. And they would give all sorts of reasons and they’re hard to argue with. But if you look at college admissions as a system, overall, not only one college overall, now versus 100 years ago, or 50 years ago, you know, the college admissions, now they’ve taken all these data points, right? As at whatever like, grade, they look at your grades, maybe they didn’t even do that 50 years ago, they may look at you know, they tried to have multiple reviewers review it and do it independently. So they can sort of D bias each other. They’re all trying to developing, they’re trying really hard to develop ways to D bias their systems, and D biasing their systems involve some form of structured data, right? It doesn’t mean numbers everywhere. But it may mean like, you know, reviewing an application blindly by different people. Sure. So it’s a process development, process development, you know, being systematic and creating checklists. And part of that is, you know, quantifying some aspects. And then hopefully, overall, you end up with a system that is less biased. And so when I think about like, what would it mean to use data and talent? That’s what I think about I think about what would it mean to do something less bias, right? You know, there are examples out there, like I said, like college admissions, other examples, if you wish, you know, it’d be like the software engineering hiring process at Google and Facebook. If you look at how they interview and hire engineers, it’s really different from how like legacy companies hire, they do this whole system of blind interviews, they aren’t even allowed to like a Facebook, apparently, they’re not even allowed to use gender and pronouns and their interview feedback, specifically, troll that bias. So, so they’re making all of these moves one at a time to D bias, how they assess talent, right? And they’ve developed these big systems to do that. Now, if you look at venture and how they assess talent, none of this happens. Like the way that venture firms assess talent is kind of like how companies were assessing talent 30 years ago. And so if you were to ask me, you know, even though I don’t, we’re not doing a great job on this right now. But you know, if I were to prognosticate on what the world looks like, in 1020 years, I think that we’ll see more of that, right, we’ll see more systems where venture investors are assessing talent, with systems in place to try to control bias, right. And the systems in place are sort of making data but they’re, but really, they’re trying to get to the same end goal, which is d biasing as much as possible.

Nick Moran 42:45
For sure. We tend to hire a lot of interns here, we’ve been through almost 30. And we’re doing a big cohort, this this fall. And, uniquely, I think we don’t look at resumes until the very end. So we run them through this whole process, we don’t have to, because the other portions of the process will tell us who’s special and not the resume is, is almost a formality.

Speaker 2 43:06
I really think that all of these things, like I don’t think that there’s like, you know, specific tactics that are better or worse. But I think that matters is like that people view it as like something that they should work on and just continually improve. You know, when I think about college admissions, that’s how I view it. It’s like, maybe it’s, it’s biased now. But the thing is, they’re clearly making efforts as a system, you know, like, clearly the system has been making efforts for decades now. You know, and then that’s kind of how I do that.

Nick Moran 43:30
Is there a thought that you’re going to bring some sort of talent lens to, you know, the startup founders that you’re vetting?

Speaker 2 43:36
Yeah, I mean, we, you know, you know, like I said, we do have a talent lens, it’s just not super systematized? Right, right. You know, what would it mean to systematize? It? I have a good like sense of it, but it’s not, I haven’t yet fully worked it out. And we have not even attempted to implement this.

Nick Moran 43:51
Jonathan, if we could cover any topic here on the show, what topics do you think should be addressed? And who would you like to hear speak about it?

Speaker 2 43:57
I’m not sure. You know, I’m a big fan of talking to people who are working in just really different industries and doing it at the top of their game, you know, like, you know, like, I spend most of my I don’t really read Twitter, I don’t really read most of what other venture investors are writing, because it’s like, I spend all my time and venture right. I’m more interested in what people who are far away are thinking, what are the what are the fixed income investors thinking? What are the guys who run insurance companies, what are they thinking? And it’s not really relevant, per se, but it’s giving you really orthogonal views of the world. And, you know, it’s when you take all those really orthogonal views that that you sometimes you know, get lucky and come up with a view that’s actually possibly novel.

Nick Moran 44:36
Like that. And then finally, what’s the best way for listeners to connect with you?

Unknown Speaker 44:42
Twitter, I guess. Yes.

Nick Moran 44:43
As you say, you’re not on Twitter,

Speaker 2 44:45
whatever, but I do get the thing. messages me. Forget

Nick Moran 44:49
Good. Well, Jonathan, this was a huge pleasure. I mean, we’ve we’ve heard a lot about tribe, a lot of great things about tribe of I’ve really enjoyed my interactions with members of the firm and Really pleased that you came on and talk to us about the approach.

Unknown Speaker 45:02
Yeah, thank you so much for having me had a great time.

Speaker 3 45:10
That will wrap up today’s episode. Thanks for joining us here on the show. And if you’d like to get involved further, you can join our investment group for free on AngelList. Head over to angel.co and search for new stack ventures. There you can back the syndicate to see our deal flow. See how we choose startups to invest in and read our thesis on investment in each startup we choose. As always show notes and links for the interview are at full ratchet.net And until next time, remember to over prepare, choose carefully and invest confidently thanks for joining us