28. The Investment Readiness Level (Steve Blank)

 

Nick:      The one and only, Steve Blank, joins us from Pescadero, California. For a man that truly needs no introduction, he’s a serial entrepreneur turned educator, author, investor, and is credited with launching the Lean Startup movement. I am so fortunate to have him join me today. Steve, welcome to the program and thank you for joining us.

Steve:   Thanks for having me, Nick.

Nick:      So before we launch into the topic – the investment readiness level – can you tell us a little bit about your background and what led you into the startup and venture space?

Steve:   Well you know, I’ve been doing entrepreneurship since I came out to Silicon Valley in the late 1970’s and, well, it was already a technology center. It wasn’t quite understood as the innovation capital of the world then as of yet. And I spent 21 years as a serial entrepreneur, did eight startups, and retired in the last bubble and was lucky enough to be able to do that, and then spent the last thirteen or fourteen years now as an educator, coming up with a theory of entrepreneurship and then a practice on how to both teach it and implement it, called the Lean Startup. And that’s what I’ve been doing.

Nick:      I know that you spend your time teaching in a variety of different forms – accelerators, universities. Can you tell us about some of those places that you teach?

Steve:   Sure. I teach at Stanford in the Engineering School. I teach at Berkeley in the Haas business school. I also teach in the business school at Columbia in New York and I teach at NYU. And my class at Stanford became the basis of how the National Science Foundation or Nations Research Organization commercializes science now. The class which is in university is called the Lean LaunchPad. I adapted it for the NSF – National Science Foundations. It became what’s known as the I-Corps or Innovation Corps, and we’ve put an aggregate of probably over 700 teams of this country’s best scientists and engineers through the process. And literally as we speak, we’re now running the same program as pilot in their National Institutes of Health, and the Department of Energy has announced they have adopted the same methodology and process as well.

Nick:      Well, unbeknownst to you, you have been my professor for a long time, although the classroom has been your blog. So, appreciate that and everything you’ve done.

Steve:   Great, thank you.

Nick:      Topic for today is the investment readiness level. Steve, can you talk about how incubators and accelerator investors traditionally evaluated fledgling startups and what’s changed that provides an opportunity for a much better approach?

Steve:   Well, you know, it’s kind-of hard to generalize, and I can’t speak for every VC or every investor, but you know, historically I can speak as an entrepreneur. You essentially wrote up a plan and then you summarized your plan in nine or ten slides and got an introduction to an investor and you had… you know, if you got lucky you might have had an intro and a preliminary meeting with kind-of a gatekeeper and analyst or junior VC, and then you got to pitch a partner or partners about your idea, your team, and the size and scale and scope of the opportunity. But you essentially got one shot, and typically, not always, it was based on vision and maybe some technology you would build or an opportunity you’ve seen. You know, when we started doing things like Y Combinator and other accelerators, we kind-of honed that into an art of, you know, you have a couple of minutes essentially, to pitch your product and give a demo. And basically, part of the things that you get taught is how to pitch. I just happened to believe that all those are exactly the wrong pieces of data to make investment decisions on. And given that we were focused, at least in our case, on some of the top scientists in the country, I had a very different model. I think looking at one data point of an entrepreneur and their team is kind-of ludicrous. You know, it’s like saying the half-time score is three. You know, three to what? What I really thought would be probably more efficient, if you had the time and, more importantly, built it into the process, is to turn this into evidence-based entrepreneurship. That is, not only describe the opportunity to me, but show me some evidence that the risky parts of your business about who the customers are and pricing and competitors and channels and how would you create demand, can you find out some of that stuff before you even show up in front of me? That is, could we put some discipline into the process? And more importantly, can I actually see your progress in doing that? Because I also want to measure your velocity and your trajectory, not just a point in time. And that’s a big idea because almost every startup is simply wrong on day one, and what they do is, they learn very quickly or they go out of business – by running out of cash.

Nick:      Sure.

Steve:   And so we put together a process that allowed entrepreneurs over a period of eight to ten weeks, to talk to over 100 customers and partners, etc. And then when they do what we call a lessons learned presentation, it’s not, “Here’s my great font, and you’ve told me to use lots of adjectives and project.” We actually teach them to tell us the story. What did you think on day one? Tell me your hypothesis. What was your initial belief about your vision and your customers, etc.? That’s great. What happened next? “Oh, we got out of the building and we spoke to thirty people and we found out we were wrong.” So what did you do? “Well, we did a major pivot. That is, we made a sensitive change to one or more parts of our business model.” Than what happened? “Well, then we talked to more people.” Well then, what did you find? Now, this could be could be done on the same literal credit card as the startup who’s giving you an eight minute pitch about how smart they are. This is a pitch about how much we’ve learned and we’ve done it on almost no dollars. The difference between the two is just pretty dramatic. And the velocity and trajectory of those teams as they come out of this program is also equally dramatic. One is, we’ve taught them something. In the other case, I’ll contend you’ve just taught them how to pitch to VCs, but didn’t make them particularly smarter about their company.

Nick:      So you talk about how Demo Day has now become Lessons Learned Day.

Steve:   Right.

Nick:      So why is it more valuable to understand the journey and what’s changed as opposed to looking at the progress and a snapshot in time?

Steve:   One of the interesting things about great entrepreneurs is, you tend to reinvent the past. Wherever you are, you tend to say, well that’s where I started.

Nick:      Yeah.

Steve:   Yeah, right? Well, I was a genius from day one. And it’s the nature of founders. It’s not bad or evil, it just is. What I remind my students is, I actually have your original slides. So you can’t do that. You can’t claim you knew it all the time and you have no idea what you were doing in the class. I actually can show you what you thought on day one and where you actually got this idea from and what you learned. And I’ve seen this happen because I always… my teaching team is always made up of venture capitalists as well as experienced entrepreneurs sitting in the back of the room, watching these teams. I’ve seen VCs who kind-of go, “Steve, why are we having them presenting weekly? We get it the first time.” And of course VCs see these same teams come in on week two, three, you know, and n, watching their teams change dramatically. You could see the venture capitalists doing a literal slap on the forehead, realizing the teams they would have turned down on day one turned out to be very different and sometimes spectacular teams given what they’ve learned on week eight.

Nick:      Interesting.

Steve:   Or vice versa. Teams you thought gave a great day one pitch actually end up learning nothing and not progressing very much. And you think, “Well, there goes half a million dollars in seed money I would have spent, because they look great, but they didn’t know how to make any progress.” And so being able to sample over time with the same urgency as you would have in a startup, almost in a classroom/accelerator environment, I think is an enrichment process for both teams and for investors being able to make a higher quality decision. Because, as I said, you’re now doing it on more evidence other than… Some VCs are spectacular at pattern recognition of 20 years. I could tell by the team or the technology or the things they say. But for the middle of the VC pack, this allows them to add more data to have better judgment criteria. And the investment readiness level kind-of fell out of that.

Nick:      There’s a lot of elements to the investment readiness level that I’d like to get into. But I’m curios. Do you find that flexibility and adapting and learning, are those some of the most critical elements that you look for in startup people, startup founders?

Steve:   Well again, you know, I have different criteria than investors who do this for a living.

Nick:      Sure.

Steve:   You know, I’m trying to teach a methodology that these teams will take with them for life. And there are some individuals or teams where this process just won’t work, meaning… You know, this is voluntary. It’s not a required class. You know, it’s interesting to talk about the ones that don’t fit this process or those who are in day one, are convinced that they don’t just have a vision, but they have a vision from God, and nothing you could tell them or they’re going to learn is going to change it. And they might be right, but it’s the wrong class or process for them. Does that make sense?

Nick:      Yeah, sure. Yeah.

Steve:   But almost everybody else, and I at first, by the way, thought that this only would work with students who are 22 and hoodies and flip-flops, but wouldn’t work with experienced principle investigators and tenured faculty or 50-years-old and they wouldn’t be able to do this because they’d be so set in their ways. And I’ve now seen the biggest surprise, is that if you are willing to actually understand that all you have on day one is a series of untested hypotheses, and while you might believe you’re a visionary, the data that we now know says the odds are that you’re actually hallucinating. We could help you turn what’s essentially a faith-based company – because all you have is faith in your hypothesis – into a fact-based company very quickly. And while it doesn’t guarantee success, it does guarantee we will put you on a more efficient path on how to spend time and money. And I think, if you think about it, in a startup those are the two most valuable things you have. That’s just your idea or your customers [00:10:16.03] but on day you’re burning cash. And if you’re the CEO you’re losing sleep, thinking about three numbers 24/7, is, what’s your burn rate, how much cash you have left in the bank, and what date do you run out of it?

Nick:      Yup.

Steve:   And what this process does is allow you to take a lot more shots on the goal with the same amount of cash and time. It just points you in the place we know you should be spending your time. It doesn’t guarantee your success, but it gives you more evidence and data about where it should be. And you know, now that the Lean Startup models become more popular it’s… people who don’t quite understand it but understand the phrase, tend to think it’s for… well, it works for existing products or existing markets but doesn’t work for moonshots. You know, the favorite critic is Peter Thiel. As smart as he is, he clearly doesn’t read other people’s books because he believes Lean doesn’t work for, you know, new ventures or disruptive stuff. And it couldn’t be further from the truth. In fact, my whole career, types of startups I did were exactly in that space. And the mythology is Steve Jobs sat in a lead lined room and is in lotus position with lightning bolts coming out of his head and that’s where all his ideas came from. And that’s just simply not true. It’s great mythology, but Jobs was probably the best out-of-the-building guy in terms of observation about science and art and was probably one of the best pattern recognizers of what form those should take in terms of products for a set of users who didn’t even know what they wanted. And so the mythology of you can’t get out of the building and ask customers about products they don’t know yet exist is simply wrong. You can get out of the building and understand what a day in the life of a customer looks like today, and then get enough data, and integrate it with trying to understand what’s the day in a life of a customer going to look like when you change the world. I’ll contend, there’s no possible way you can do that up in your building thinking smart thoughts. Possible but probably improbable. And so the whole Lean and startup investment development process works for both existing products and new disruptive ones as well.

Nick:      Yeah, it’s an interesting point about Steve Jobs. One of the things we’ve talked about on the show is, you know, as an investor, you’re often not the target market. In a lot of cases, even the founders aren’t the primary core target market. I kind-of wonder myself, was Steve Jobs the epitome of his target when he was developing that product? Or did he just have the best most intimate understanding of his customers’ needs and the most appropriate solutions for those needs?

Steve:   Yeah, I think Jobs had a vision of the world as he wanted to see it, and I think the apocryphal story of him coming into Xerox Park and finally seeing something that closely matched what he kind-of… in the back of his head, that could computing could be. The irony is that no one tells the other half of the story, which is… and ignored the other like 80% of spectacular inventions that took Apple another twenty years to figure out. You know, Steve Jobs never believed in networking. He famously used to run around holding a floppy and saying, “This is the only network you need.” He was just fabulously wrong. You know, didn’t understand distributed computing. It was all about personal computing again. He did not understand the power of the network. But the part he did get right, which he focused insanely on the user interface and user experience, and that was the part he took out of Park and built it spectacularly well. So I think he had a vision but then actually saw an implementation, and said, “Well, I want one of those.” So, you know, that’s to me the most famous part of, Steve Jobs did get out of the building and did well. He copied the part that he understood and just relentlessly focused on delivering that experience in a NAS market way that Xerox couldn’t because they had no… their R&D group was disconnected from any sales function and therefore had no feedback from sales and no direct customer experience.

Nick:      Right. So before we jump into the specific elements of the investment readiness level, you’ve cited Moneyball and even NASA as inspiration for it. Can you tell us more about the origin of the IRL?

Steve:   Yeah, so the investment readiness level is just a simple technique. You think of it as a cheap hack, and it’s developed two things. One is a common language for people looking at commercial startups to think about, are these guys or women ready for more money or not? And it came from my experience with the hundreds of startups we’ve been working with in the US government. At the end of the class – you can think of it, at the end of the accelerator – the question is, is this a go or no-go for additional funding?

Nick:      Sure.

Steve:   And that’s a great question. I mean, given it’s government going to put some more bucks in it, and what should we tell the teams? And for the first couple of years, the answer was go or no-go. And I thought, well, that’s kind of a waste. We’ve just learned a ton about these teams and they’ve learned a ton as well. We’ve learned, did they understand product market fit? Have they achieved it? Did they have early customer validation? Did they have some users or payers or did they understand the channel? Did they understand how to create demand? Did they understand what activities, resources, partners they need, for example? Did they understand regulation and, if they’re a medical device, have they applied for a PMA or 510(k)? Did they understand the process to do that? Have they picked a clinical trials partner? Did they even understand what the right endpoints are for a clinical trials if they’re in therapeutics or diagnostics or devices? Or if they’re in a hardware company, did they understand whether they’re licensing the technology or OEMing it or setting up direct sales channels? That is, after ten weeks, they kind-of know the answers to good numbers of these questions, but we were just saying go or no go. And it dawned on me that we maybe could use more granularity in those discussions. So I went back and looked at how do venture capitalists make decisions in partner meetings, and it turns out that after fifty years, it’s still kind-of like a custom craft. There are no standards.

Nick:      Right.

Steve:   Every VC will tell you pattern recognition about teams and about processes and about technologies, so it’s not like VCs don’t have them, but there are no standard set of heuristics. And well, one could argue, it will always be a craft business. I kind-of believe, yes it will, but we could raise the bar about what are the things we want to give the teams feedback on? That is still one of the other worst parts about which every entrepreneur could tell you. You come out of a VC meeting, and you’re not sure whether you just had sex or not. I mean it’s, you know… It’s kind-of a bizarre feeling until you either get a term sheet or they stop returning your phone calls.

Nick:      Yeah.

Steve:   And so actually it’s a unidirectional meeting. You never learned anything out of the meeting about what you should do better. And again, as an educator, I thought that was another missing piece of what we weren’t providing. So that’s a little background of the problem I wanted to solve. And it dawned on me, I had run into – at NASA – something called the technology readiness level. And it’s worth about a minute to explain it, is, by the 1980’s, NASA kind-of figured out how to launch rockets to space without most of them blowing up, which was a big accomplishment after since 1958 it was kind-of a good thing to do. They could get most of the rockets up into space reliably. But then they had another problem – the payloads that is the satellites. Not the ones with the men or women in them, but the robotic satellites still failed at a disconcerting rate. And what NASA realized when they did a survey of why that was, was that the satellites were being built by a wide variety of vendors, everything from the government themselves and NASA Labs to commercial vendors – Boeing and Lockheed and Port Aerospace and [00:18:02.06] and others. And what was clear was, in the mid 80’s there was no standard for what was flight readiness. There wasn’t even a common set of metrics between all these vendors of satellites of, is this thing spaceflight ready? And so NASA came up with what’s called the technology readiness level, which was a… think of it as a church bake sale thermometer visually, you know, a thermometer with gradations of zero through nine. And level zero was, it’s a breadboard in the lab. Level one was, it’s in [00:18:34.25], you know. Level five or six might be, we’ve tested in the vibration chamber or we kind-of have it in the vacuum chamber. And level nine finally is, it meets all our criteria for flight readiness. And NASA published this and then established a common language – a common set of criteria – and surprisingly, for the government, it wasn’t a 700 page document. It literally is about three pages, but gave everybody a language to kind-of understand, “Oh, if we’re level six, everybody understood a) we weren’t ready for primetime, and b) here are the other activities we needed to achieve.” And this was not only adopted by NASA. Soon the European Space Agency looked at it and said, “Good enough for us.” And then the FAA looked at it, “Good enough for us.” And then the Department of Defense adopted it. So in the technology environment in the government, there’s kind-of a set of standards of, if you say you’re at technology level seven, people will understand where you are kind-of in the get ready to either go into space or be deployed. But I realized, there was no common thermometer of what’s commercialization readiness or investor readiness. Everything was kind-of an individual craft conversation going on in every venture capital boardroom with zero feedback after, to teams. And what I realized is, because we’re now collecting enough evidence, and having these teams go through a canonical that is a standard process of talking to ten to fifteen customers a week, we can actually give them very specific feedback of, you haven’t achieved product market fit, or you really don’t understand demand creation, or you still haven’t figure out your cost, or whatever combination it was, and actually have the teams kind-of say, “Yeah, we agree. Oh, good. Thank you for telling us. Here’s the next big thing we could work on.” Now in the background, you still might say, “I don’t care if they ever do that, it’s a weak team.” And whether we ever have a, you know, team competence level that we publish or not, but every VC is kind-of running that as a background process as well. But at minimum, we could give them some feedback that’s very specifically targeted to the things we know that they’re going to have to accomplish to have a viable business. So that was the investment readiness level, and it allows us to play essentially money-ball with startups, meaning it’s not sufficient data, but it is more data than we had before and allows us to kind-of use that data to maybe start investing or at least using it as part of an investment methodology, that even when we talk to co-investors who don’t use my firm’s investment method, we could kind-of say, “Well, here are the things they were missing,” and all start using a common language. So that was the rationale. Just as an aside, I’ve been stunned in the last literally week and a half. I’ve heard from – and I won’t name the companies – but a company in a Nordic country that makes large games versus a genetic engineering company in Silicon Valley who both have adopted the investment readiness level for their internal no-go/go decision process.

Steve:   Is that right?

Nick:      And both of them, independently… I don’t know what it was about the timing – maybe it’s taken six months for kind-of this to catch on – and sent me photos of the teams with the scores right behind them on the same damn thermometer. It’s like…

Nick:      You should get a royalty on each one.

Steve:   Well, no. You know, my email was like, “Why did you guys do this?” And the answer was essentially the conversation or the soliloquy I just gave you, which was, “Listen. It’s not perfect, and it’s probably not even, you know, perfectly right, but it’s better than what we were doing, which was all individual opinions with no set criteria.” And my whole point about the investment readiness level is, you don’t use Steve’s metrics. Steve’s metrics for each level are kind-of placeholders for your industry and your company. And so, for example, even in life sciences we have a different investment readiness level set of metrics for diagnostics than we do for med devices than we do for therapeutics than we do for hardware than we do for web mobile. But it’s the same common idea with kind-of the details, just like we’re different depending on your domain.

Nick:      Yeah.

Steve:   And this is what I saw in the photo. And I was both amused and pleased at the same time.

Nick:      Yeah, I bet. So it sounds like some of these inputs change depending on vertical industry, etc.

Steve:   Yeah, sure.

Nick:      But I’m curious, you know, to learn a little bit more about how this works and the main elements. So I came from corporate M&A where we had an evaluation process ourselves, right? And we tried to quantify decision making as best we could. I’m curious, are the elements of the investment readiness level more phases and gates and sort-of binary questions and answers or requirements, or is it, are these factors that they each have a spectrum or a scale that you can sit on and it allows sort-of the startup to parade, “Oh, here’s the biggest sources of not being ready that we either need to go address or maybe they’re not even addressable, like wrong market”?

Steve:   Yeah, and so the answer to the question there… Let’s start out with at least my version of the investment readiness level is biased to what I teach, which is this Lean Startup methodology, which says, “You’re testing nine different hypotheses, all focused around Alexander Osterwalder’s business model canvas, and you’re gathering evidence whether you have validated or invalidated those hypotheses.” It turns out, in some industries – let’s just say social media – product market fit, the fit between value proposition and customer segments is probably number one and two. So if you haven’t validated that, you haven’t gotten anywhere in social media. But I’ll give you a different example, you’re in medical devices. Well, product market fit is nice, but unless you validated reimbursement, regulation, IP, and clinical trials, you don’t have a business in that business.

Nick:      Yup.

Steve:   But those are irrelevant in social media.

Nick:      Sure.

Steve:   But they all focus around those hypotheses which are framed by the canvas. And they’re framed by… we taught you that these are important and we’ve asked you to focus on them, because we know they’re important in this specific business. How well did you do? And by the way, you could have done great, but the data could have come back – people aren’t interested or bigger obstacles. So what are you going to do about that? And so the IRL a) is dependent on industry, focused on hypothesis testing, driven by customer development, but on those hypotheses we have articulated that are important. You might be a venture capitalist and say, “That’s great, but it’s all about the team.” Okay, well, you could come up with a different IRL which… maybe a third one, which I really do believe people are running in the back of their head but don’t want to share with the teams, are, “Is this a team that’s going to be successful or not, or strong or weak?”

Nick:      Sure.

Steve:   So there are some investors, just invest on the basis of the team. Some invest on the basis of the team and the market. Some of them have different criteria. I’m not going to suggest that the whole world is going to go to just using IRLs. I think that’s ludicrous. I think what it does is just simply raise the bar to have a common language for a set of things that we should be looking at anyway.

Nick:      Yeah.

Steve:   So much like a Lean Startup. I don’t think is a replacement for all the other kind-of tangibles about a startup. I think it just simplified what were the things we were supposed to be being tangible about. But it doesn’t guarantee success or failure. I think of the IRL the same way, is it in fact, does it replace great judgment and pattern recognition of an investor, or change your firm’s or individual’s investment thesis. But it now allows us to just kind-of take that to the next level. And it’s a tool, and you can decide to use it or not. As I said, I’ve been surprised how quickly it’s been adopted in places I never would have expected.

Nick:      Yeah. So to park on customers a little bit, you write a lot about customer development, product market fit, but in the product adoption lifecycle or technology adoption lifecycle within innovators, early adopters, majority, etc., how do you think about, are the early users – the innovators – are they representative of sort-of the later majority of the market or not?

Steve:   Well, you know, one of the first books that I thought was useful for Silicon Valley entrepreneurs, since everything else wrote on entrepreneurship in the 20th century was mostly focused on large corporate entrepreneurship, was Jeff Moore’s Crossing the Chasm, which he took the work of Everett Rogers who had come up with this notion of technology lifecycle adoption. And I think Jeff did a great job of popularizing this idea that there is a gap between these early adopters and mainstream customers. With the web and mobile apps, some of that disappears in some markets, but it is still true in others. The good news about customer development is that… and customer discovery, is that if you want to stay in business, you constantly need to do this.

Nick:      My own curiosity is, you know, I’ll see startups, and they’re… maybe they’ve got some product market fit and they’ve got some customer traction, and yeah, how do you cross that chasm? You know, is the timing right? Are those customers going to reflect sort-of the general public or are they too early?

Steve:   Yeah, so one of the things I always used to talk about, even before I came up with the customer development process, was always worrying about are you optimizing a local maximum or a global maximum. And the bad news is, those customers don’t come with little memos that says, “I am a local maximum.” And so what you need to understand is, are these customers going to help you get global scale or are they an intermediate point that you’ll use to actually have to figure out how to get that larger market? I just want to just tell you a short aside, is that when I used to run marketing, one of the companies – just to make the point about customer discovery goes on forever – to attend my marketing staff meeting, every member of marketing including the admin had to call two customers a week and summarize what they found with a standard questionnaire I made up in the staff meeting before we even could discuss any of the staff meeting issues. And given, by this time the program grew to almost forty people. We were talking to eighty customers a week. That’s 4000 a year.

Nick:      Wow.

Steve:   We probably had more insight, you know, than anybody on the planet about changing customer needs, and more importantly, I kind-of aligned marketing to customers without having to beat them with a stick. It was just a criteria of like, you don’t get to stay employed unless you’re doing this. And what was great is then, I then had marketers arguing with each other about, well my customer said X. Well, mine said Y. Well, I was, “Great! What a great thing to be arguing about!”

Nick:      Yeah.

Steve:   And so it made us incredibly dangerous in terms of as a competitor, because while other people were guessing, we were operating with real data and a continuous stream of data. So it kind-of gave us x-ray vision in their market. We went from 11% to 68% market share in two and a half years.

Nick:      Wow.

Steve:   After I joined. And one of the first times I got to experiment with some of these, what would become, customer discovery techniques, but this one was incredibly effective. And so when companies start growing, people start thinking, “Well, I found the market. I don’t need to do this discovery stuff anymore.” Oh man, let me tell you. That’s when it becomes the most fun. Because it’s no longer survival. It’s in fact, how do you look like a blur to competitors? And we always looked like a blur. How did they know that? How were they able to move over here? The market changed and these guys were right ahead of it. And it wasn’t like we were smart. Actually I don’t think we were very smart. All we did was, we listened really well and we understood what the battlefield looked like continually. It was like having a permanent drone over the battlefield.

Nick:      That’s so cool. I love the practitioners that are customer-obsessed, and sounds like you figured out a good way to sort-of build in a culture of customer obsession into the organization which is just awesome. But anyway, we’ll have to get you back another time also to talk about sort-of how to translate customer needs and problems and learnings into solutions, because that’s always a bit tricky. But anyway, circling back and to round out the topic here, can you just walk us through an example of how an investor would use the data from the IRL?

Steve:   Sure, and we’re actually doing it… you know, there’s a new firm started called M34 Capital that the founder, Errol Arkilic, had been the guy who ran the US government’s commercialization efforts at the National Science Foundation, and he decided after 15 years for the government, that he wanted to do this in private capital. And so he’s now funding kind-of the best science that comes out of government research. And to make a long story short, the way he’s looking at those deals, not the only way, but we actually use both the technology readiness level and the investment readiness level as two thermometers on every deal we look at in the portfolio review. How mature is their technology? How mature is their understanding of the market? And it’s not… obviously the conversations always also get into teams and market opportunities and vision and trajectory and velocity, but we now have a common set of conversations about technology readiness and investment readiness, [00:04:10.03] the business model canvas and how much discovery have they done. And it’s not the… you know, you don’t add up the numbers and do the investment or cancel the investment, but it now makes that committee kind-of like, “Okay, we know what we’re looking for.” And by the way, we’ve tweaked what it is we look for in those metrics, but as I said, it makes the conversation a lot more efficient, because then we could have the rest of the conversation. “Yeah, but I still don’t like those guys,” or, “Yeah, forget the numbers. These guys will go through walls,” or you know, “Yeah, but you don’t understand they like… one of them probably will win the Nobel Prize and that counts for three more points.” Does that help?

Nick:      Yeah, sounds like your process is a lot more thoughtful than mine, but even my own is a scorecard. And I spend my time with the startups that are scoring, you know, over an 80 on my scorecard, and that’s where I get really deep into questions and evaluation whereas the ones that are scoring 40s and 50s, I’ll tell them the parado bars that are highest, here are the biggest challenges for me, and tell them to come talk to me again.

Steve:   Alright, but this way we can actually tell the entrepreneur… and by the way, when we do this in a class, in our accelerator, we don’t give them the score first. Guess what we do. We actually make them, as part of their final lessons learned pitch, their last slide is, “Here’s our self-assessment of our score.” And then we tell them what we think their score is. So now here’s their number, here’s our number, and we explain why.

Nick:      Interesting. Do you find that the ones that are scoring higher than you or lower than you are often the ones that become investable?

Steve:   It turns out that so far, out of maybe – I think we’ve done this now with fifty, maybe seventy teams – I can’t think of more than one or two that their number and our number have been more than one apart. Pretty impressive.

Nick:      Wow.

Steve:   Yeah. Only because we’ve had them for n number of weeks, and they know what we’re looking for. So it’s not like they’re like making it up. You know, every once in a while you get some team that, “Oh, we’re a nine.” No, you’re a three. I mean, the couple that have been far apart are the ones where they’ve kind-of been enmeshed in their own reality distortion field, and while they think they’re visionaries, we know they’re hallucinating.

Nick:      Yeah. So like employer reviews those stubborn employees that don’t do anything but think they’re a ten.

Steve:   Those guys either are Steve Jobs and Elon Musk, or they’re truly going to be large creators. But you’d be surprised. If you teach them the methodology how thoughtful they are, because it’s not like I’m going to fake the score, because it’s an evidence-based score. It’s not some… I get to pick some number. It’s, “Well, yeah. I really didn’t find product market fit. I’m still looking, but no, we didn’t find.” Or, “No, we really don’t understand customer acquisition cost because right now it’s ridiculously high. So agree, we still need to be working on that.” Man, if you get the teams to be able to say that, holy cow! Because now you’ve really enriched the process on everybody’s side, and you still might decide to invest going, “I know they’re going to figure that out.”

Nick:      If I could have the startups that come to me evaluate themselves first on something like this, that would really change the game.

Steve:   That’s exactly the goal of this, right? We can’t pick or make winners and losers, but we could raise the bar for everybody. We could raise the bar on what teams look like by the time they come in to early stage investors, and we could allow early stage investors to finally give some feedback that’s productive rather than being, you know, “I pass,” or soft circle, or whatever, because every investor is afraid to tell entrepreneurs the truth, because maybe their next deal will be Facebook, but you pissed them off. They’ll never come back to you, right? That’s the fear of giving honest feedback to entrepreneurs.

Nick:      Right.

Steve:   I think we might have a way to do that in a way that, “Oh, well that was very helpful and constructive. Boy, I hope I get to work with that investor again.”

Nick:      I’m taking notes, Steve. So to close things out here, Steve, what are you currently most focused on?

Steve:   You know, my current focus as an educator is helping the US government figure out how to commercialize their science. We invest, as a country, about $125 billion a year in basic and applied research. That’s a huge number. Even if you discount that half of it might be Department of Defense for applied weapon systems, that’s still $60 billion a year. $30 billion of it goes to the National Institutes of Health, and the rest goes to the National Science Foundation, Department of Energy, etc. There’s a mandate to spend 3% of that number on commercialization – that is, turning these ideas into real companies. And there was never a process. There was a funding mechanism called the SPIR program and STTR, but there was never a… how do we teach scientists who want to do this how to do this? And basically, there was a smart guy named Errol Arkilic at the National Science Foundation who looked at my class at Stanford and said, “My God. You invented a scientific method for entrepreneurship. This is what the country’s been looking for, for thirty years.” And so we’ve now been running this program and I’d say we probably have maybe even more graduates than Y Combinator to the extent that we’ve put over 400 teams through this in the national program and these universities – twenty of them – who are teaching the same methodology, have put maybe another 300 through the original program. It’s 700 teams that have been exposed to Lean Startup methodologies in how to take your science and get out the building and figure out whether it is a business. We never were able to close that gap. And what it does is allow scientists to finally, when they are through the I-Corps or walk into a venture capital office, not just wave hands about the paper that they have in Science or Nature or Cell, but actually talk about its applicability to some commercial activity, and just bridging that gap as an educational, forget about as an accelerator or incubator, but just to bridge that gap as an educational component. I think we’ve made a major impact on science in the United States, because a good number of those scientists are still running their labs, their universities, now training generations or grad students how to actually think about commercial opportunities for science. We’ve made a major dent about the effectiveness of our science in this country. You ain’t seen anything yet. I think in the next five years, this impact is going to be evident.

Nick:      That’s incredible. If you ever decide to launch some satellites in other cities, that’s something that would be really interesting for a lot of people. But Steve, if we could cover any topic in venture investing, what topic do you think should be addressed, and who would you like to hear speak on it?

Steve:   Well, you know, Mark Suster has been reinventing LA singlehandedly if for, as a venture hub. It’d be great to hear Mark speak, though I would not let him pitch now. I would let him tell you the story of what he saw, why he wanted to change it, and where he thinks he’s going. I’d be really interested in that. And the other is, I think one of the most underrated stories about venture is how Bloomberg singlehandedly turned New York into a serious player for entrepreneurship. You know, I think Fred Wilson and others are emblematic in Union Square of the transformation of New York into a entrepreneurial hub, but I have yet to hear the story being told about how one guy singlehandedly engineered New York in an incredibly short period of time by throwing money at different accelerators, incubators, hubs, startups, etc., and turning New York from what we used to think of as just a financial and media capital, into… now you would say New York is finance, media, and entrepreneurship. I think that’s a story that’s yet to be told.

Nick:      Yeah, I was actually talking with Fred last week, and it turns out that Joanne Wilson is going to come on the program.

Steve:   Well, I think both Joanne and Fred are indicators of the change in New York City, and I would just go ask her, “What was it like pre-Bloomberg, and what is it like now?” And what does she think the reasons were this happened. But I truly would understand that there was somebody with a vision of actually engineering the cluster, rather than just participating in the cluster. And I don’t think anybody has asked him, “What did you do to do this? And did you actually know it would happen?”

Nick:      I hope we see the same thing happen here in Chicago. So Steve, what’s the best way for listeners to connect with you?

Steve:   Well, I have a website called steveblank.com. You know, there’s tabs on the top, one called Startup Tools. If there’s anything missing, just put a comment at the end. If some of your listeners are interested in how Silicon Valley got started, there’s a tab called the Secret History of Silicon Valley. If you’re going to visit the Valley, there’s kind-of a tab on what to see and what to do, and then just read the post themselves. Feel free to subscribe and I’m happy to get some feedback.

Nick:      All of Steve’s information and links will be in the show-notes. Steve, you’re a huge inspiration for me. I really appreciate you spending the time today, and thanks for all you do for entrepreneurs.

Steve:   Great. Take care, Nick.