Founders as experiment designers
A few days ago, I posted about what I think is one of the most dangerous myths in the startup world: that being successful is all about perseverance, grinding, and grit.
Now, of course, there are many moments in the life of a founder when grinding is exactly what it takes: getting a challenging fundraise over the finish line, chasing a dream hire for months on end, sending the 10th follow-up to finally break into that F500 account.
But, if you convince yourself that all it takes to win is grinding, because “that’s what successful founders do,” that can lead to wasting years of your life on something that likely won’t work.
Instead, it’s my experience that the best founders aren’t necessarily those who grind the longest, but those who are the most “greedy” with their time. Said another way, I’m constantly inspired by founders who, with limited information, quickly make the call on whether something is working or not, and then either double-down or move on.
Making quick decisions given limited information is not easy. It requires thinking in bets, which is unnatural and uncomfortable. What often happens, instead, is that founders fall in love with the initial idea, and won’t make a significant enough change even when, after years, there’s little evidence that it’s working. (Even worse is when investors, so tied to their original hypothesis on the investment, bully founders into staying the course, even when the founder’s intuition tells them to try something new.)
This is what I’m most worried about, and the idea that I want to explore.
So, for the founders out there, how do you know when to make the call on whether something is working or not?
My advice: run everything as an experiment.
Putting your product in front of a customer for the first time? Targeting a new customer segment? Launching a new feature? Testing out new positioning? No matter what you’re doing…think of it as an experiment.
What does that mean?
First, decide on a metric that matters. Early on, when you barely have a product, the metric will undoubtedly have to be a proxy for value (e.g. the number of organic shares of a social post about the problem you’re trying to solve, or the conversion rate from a cold email to a call), and you’ll have to use a lot of your own intuition to decide whether or not the result is meaningful. As time goes on, this will be less of a problem.
Second, timebox it. Most likely, this is not a “real” experiment, so there’s no need to run it until you hit statistical significance. Instead, set a time limit up front and stick to it. Keep up the momentum.
Third, make big (enough) changes. As I detailed in an old post about building a growth engineering team, the reality is that, for most startups, you have so few users that you’ll never hit statistical significance on any experiment. One way to counter that is to test out changes that you think will have a huge effect. As an example, instead of changing the color of the sign up button, change the entire landing page.
Fourth, and finally, record the results. As Richard Feynman said, “the first principle is that you must not fool yourself and you are the easiest person to fool.” It’s easy to convince yourself that something worked after the fact. Keep yourself honest. Write down what success would look like, record the results and compare. If you do this, you’ll get better at estimating the results over time as well.
I’ve come to agree with Elad Gil that things that work, tend to work quickly. That doesn’t mean you should give up on your idea. The core hypothesis may still be correct. But it could mean that a pivot is in order to figure out the path from here to there. If a pivot is needed, then do it fast.
Remember, at its core, a startup is perhaps best defined as a small group of people dedicated to testing a novel hypothesis for creating outsized value. If you’ve just raised a round of capital, you have between 12–24 months to run a series of experiments to put that hypothesis to the test. In other words, clock is ticking. So run as many experiments as you can!