It’s easy to say your enterprise is data-driven. Or claim it nurtures a culture where people are empowered to be change agents and mad scientists.
“Move fast and break things,” as Mark Zuckerberg advised.
But how do you make it a reality? How can you splice the growth hacking mindset into the DNA of your organization?
In my experience, one of the most powerful ways is by embracing the “1 Percent Experiment” philosophy. I’ve been at a lot of startups that took a data-driven approach — including my current one — and saw it at close range.
A data-driven approach has been central to Twitter’s success and you’ll find a similar approach at other agile, highly innovative enterprises.
Getting to the Heart of Product Innovation
At Twitter, the 1 percent experiment gives product developers, engineers, marketers or others the permission to test new ideas on 1 percent of user sessions, pretty much at will — so long as the experiment is designed for careful measurement of results.
Done right, it can be a primary growth driver, just like its close cousin high velocity testing. It’s how a huge range of ideas that improved the user experience at Twitter arose from the ground up versus being dictated from the top down.
Traditional testing tends to reinforce traditional silos and bureaucracy. Taking the 1 percent route is about democratizing experimentation and devolving authority from centralized decision-makers out to the trenches.
That encourages people to seize the initiative, work together and break down barriers. Which results in cross-functional thinking, and fosters the kind of “synaptic” workplace where individual ingenuity and collective collaboration coincide in the best possible way.
It promotes a feeling of unity and joint responsibility, and makes people proud to work in a place that puts innovation and proactivity on a pedestal.
It’s what you see at a company like Netflix, which promotes a culture of “freedom and responsibility” meant to attract self-motivating and exceptional workers who are expected to continually test new approaches to optimize consumer value.
"Testing our product ideas frees us to make big bets, to try radical or unpopular ideas,” according to John Ciancutti, Netflix’s former VP of Personalization Technology. “It allows the best product thinkers to build a track record based on real customer value. It allows us to build consensus out of debate and to build on our best ideas. It helps us avoid the tyranny of ‘or,’ because we can test many approaches to solving the hardest challenges we face."
The 1 Percent Process
A strict process is at work here, and if it looks familiar, that’s because it is.
It’s the classic scientific method, but now you’re letting almost everybody into the laboratory to play with the test tubes.
To start, build a clear hypothesis: What’s the specific theory or idea being tested? What would you consider to be a successful outcome? As you’re building a hypothesis, make certain you’re being rigorous in keeping it user-focused.
That empathy keeps you from getting sidetracked by tasty-looking ideas that ultimately don’t benefit users or your business.
Define the success metrics for the idea. What are the very specific, most relevant metrics to use, and what outcomes would prove its feasibility or value?
Learning Opportunities
Webinar
Apr
5
Adopt a Modern Digital-First Marketing Strategy
Learn how to use a composable DXP to your advantage
Webinar
Apr
13
Composable Commerce The Future of Retail
The Adobe Commerce approach to composable and why it's best in class
Webinar
Apr
13
Accelerating Content Velocity Across the Digital Supply Chain
Learn to integrate DAM with upstream tools to improve workflow efficiency.
Webinar
Apr
18
Drive Business Growth Through Customer Service and Support
Defining the future of technology. What is AI’s role in the future of customer service?
Be explicit: At Twitter, forcing experimenters to lock down these metrics prevented “cherry-picking” and “HARKing,” where they’re tempted to pick out only metrics that support their hypothesis, or even alter a hypothesis to fit results.
Test the hypothesis by building both a working implementation to demo it and the test framework for measuring results. Ideally, apply the same framework across your whole organization, ensuring both cost effectiveness and consistency whenever different teams test their own ideas.
You can create your own customized test framework, but there are great existing platforms like Optimizely available, too.
Learn and iterate: Brace yourself for the very likely chance your experiment results in no meaningful change in metrics. That’s perfectly cool: most experiments get exactly those results, but that’s a key learning in itself.
So be ready to iterate your experiment many times over. You might find variations on your initial idea that work better, or different levers that change your metrics. It’s all good, because it’s all data.
And remember: keep testing even after you’ve moved the metrics needle. Double down on what works and push the envelope to discover the limits or constraints of your idea.
Ship: If you get meaningful positive results, sell your innovation to the organization, then (hopefully) watch it get released to big wide world. Congratulations!
Repeat the process: Create even more ideas, and use what you’ve learned from the last round of experimentation to build new hypotheses.
Experimentation Examples
One feature that saw adoption at Twitter thanks to the 1 percent experiment philosophy was the “quote tweet.” After the initial hypothesis was created, a whole series of variations were tested prior to settling on the one that was the most successful — the feature users employ today.
Other recent changes at Twitter began life as 1 percent experiments before the rollout to all users. Like the shift from “favorites” (represented by stars) to “likes” (using hearts), and the introduction of bigger innovations like Moments.
Tests will nearly always draw ire or confusion from some users or the media, but that’s a ground rule of innovation: somebody is going to gripe about it. That kind of negative feedback is okay: it helps you understand the limits of your change.
Instilling a 1 Percent Mindset
So what are some keys to nurturing a 1 percent experiment mentality in your organization?
- Commitment: If you’re in a software or service business, innovation is a survival tool, and your team is your best innovation asset. So empower them with a 1 percent experiment philosophy as a way to survive — and thrive
- Set expectations: Make sure everybody knows taking a hand in experimentation is part of their job description
- Delegation: Push the ownership of innovation down to your team
- Education: Make sure they’re trained in the right methodologies
- Tools: Give them the resources and equipment they need
- Recognition: Reward people for experimentation (even if it’s not successful) — and for evangelizing that culture to others.
Learn how you can join our contributor community.
