Why Statistics Kill Innovation

By: Stephen Shapiro

Which will help your business to be more successful: statistics or probability? Underwriters at insurance companies use statistics to assess future risks. This is based on years of collected data. Probability is what card counters in Vegas use to increase their odds of success. This is based on real-time, real-life experience. If you want to play it safe, use statistics. If you want to win big, use probability.

Statistics and incremental innovation

Businesses are increasingly using statistics to manage decision making, as evidenced by popular books like Tom Davenport’s Competing on Analytics and the boom in Customer Relationship Management (CRM) system usage.

The belief is that if we gather more data we can make better decisions. But this may not be true when it comes to innovation.

If you are crunching numbers, you are probably gathering information from existing customers. This will give you insight into their buying habits, usability behaviors, and other patterns. But most likely you are only gathering data on YOUR customers. This represents the middle of the bell curve or the norm. This information may be useful in “incremental” improvement, but it will rarely lead to significant innovations.

When you move beyond the norm to the far ends of the bell curve, you will find the real interesting ideas.

The value of non-customers

On the far right-hand side of the curve are the market leaders; the advanced users. They may not be your customers because you can’t meet their high-end needs. Or maybe they were once your customers and they left. When someone is not a customer it is difficult to gain insights into their wants and needs. If you could somehow understand their perspectives, you may find opportunities for “advanced” innovation and insights on where the industry may be going in the near future. These innovations would be more radical, yet continuous in nature. Think of this as the Blu-ray improvement on the standard DVD.

On the far left-hand side of the curve are the laggards; the less sophisticated users. Your products/services may be too advanced, too complicated, or too expensive for their needs. Again, you are probably not gathering statistics on these individuals or organizations. But here lies the greatest opportunity for discontinuous innovation. If you can find a way of “dumbing down” your offerings, you might find new and untapped sources of revenue. Quite often these products become the de facto standard, much like when PCs replaced the more sophisticated mainframes and mini-computers.

We can use numbers to justify anything we want. But quite often they justify the wrong actions.

The problem is, it is very difficult to get data about the ends of the bell curve. Focus groups, surveys, and other traditional data gathering techniques are useless. Scott Cook from Intuit once said, “For every one of our failures, we had spreadsheets that looked awesome.” We can use numbers to justify anything we want. But quite often they justify the wrong actions.

Probability and innovation

If a statistics-driven innovation model does not work, what would a probability-based model look like? Probability tells me that if everything is equal, the more bets I have, the more likely one will be successful. The odds of 1 success out of 200 are greater than 1 success out of 20.

But how can you have more bets without diluting your efforts and potential returns? The key is to learn as you go. This is what card counters to. Let’s contrast a statistics-driven model with a probability-based model. To do so, we will use two exceedingly simplistic examples. With innovation model #1, you make a few “big bets” based on analytics you gathered from your customers (a statistics-driven model). Innovation model #2 is a more experiential “learn as you go” model (a probability-based model).

In both examples, let’s assume you have $100 million to invest in innovation.

Innovation model #1: Big bets

This is the most common approach and is highly driven by statistics. You identify a number of large innovations you want to invest in. For this example we’ll use 20 projects @ $5 million each. No matter how much data you have, most innovations will fail. We know that to be true. And of the successes, most will not achieve the predicted ROI. In the end, if you are lucky, you’ll have a few wins out of 20. This feels like putting all of your money on 35 black on the roulette table and crossing your fingers. Your wins/successes had better pay out big to cover your losses.

Innovation model #2: Learn as you go

Let’s look at a different model. Instead of 20 large projects, you start with 200 smaller projects. Again, you know that most of these will fail – but you don’t yet know which ones. You initially invest a small amount ($10M or $50K per project) to test the ideas as low-risk, low-cost experiments.

Based on this experience, you decide that 40% (80) of the ideas still show some promise. But you are not yet ready to bet the house. This time you allocate an additional $20M ($250K each) to do further testing. You now eliminate 70% of the projects, keeping 24 alive. You now invest another $20M (nearly $1M per project). Of these 24, you decide that 5 are real winners.

At this point you have only spent half of your money and yet you were able to eliminate 195 ideas. That’s incredibly valuable information learned by doing rather than by analyzing. You now invest the remaining $50M on those 5 ($10M each).

Why doing is better than guessing

With innovation model #1 you must “guess” which ideas will be successful up front. A major high-tech company recently announced it is moving to this model by consolidating 150 ventures into only 20 big bets. This feels like a bad idea. What if the 20 they choose all turn out to be duds? What if the real winners are in the 130 they eliminated?

With innovation model #2, you make lots of small bets to eliminate the bad investments. Then, as you see the odds of success improving, you increase your investments.

This is what blackjack card counters do. At the beginning of a deck, they place small bets. Then, as “real time” information is gathered, they can determine when the odds have shifted in their favor. That is when they increase their wager. The big bets are only placed when there is a high certainty of success.

Innovation and the real world

Of course the real world is a lot more complex than these simple models suggest. The ultimate solution is a combination of big-bets and experimentation; a mix of data analysis and learning as you go.

For the learn-as-you-go model to work, you must take into consideration your organization’s culture:

• Tolerance of failure: Your organization must be able to embrace an overall 2.5% (5 out of 200) success rate. For some companies – and employees – this high failure rate may be disheartening.

• Fast decision making: Although the learn-as-you go approach can move quickly by avoiding analysis paralysis, if go/no-go decisions get bogged down in bureaucracy, this iterative approach can grind to a halt.

• Talent: When you have a large number of projects, you run the risk of diluting your most talented resources. What can you do to get the right people with the right skills on the right projects? The learn-as-you-go method works exceptionally well in blackjack. There is a reason why casinos don’t like card counters. Card counting shifts the advantage to the player. Anyone who read Bringing Down the House: The Inside Story of Six M.I.T. Students Who Took Vegas for Millions knows that this approach works well in gambling.

But does this approach work in business? Can an experimentation mentality shift the innovation advantage to your organization? Absolutely. If you read Charles Koch’s The Science of Success, you will discover that this experimentation mindset has been fundamental to the success of Koch Industries, the largest privately held company in the world with $90B in revenues and a growth rate seven times faster than the S&P 500 for the past 40 years.

The Koch model is based on a free market philosophy. Realizing that central control is “fatal conceit” for his company, Charles eliminated the old command-and-control style and instead asked employees to run their businesses as if they owned them. Koch Industries is now a network of employee-entrepreneurs who have decision-making powers. When someone is brought in to do a job, they are immediately given the authority to spend money and to move people when and where they choose. The result could be described as Darwinian. All good ideas get seed money, while only the best survive. The less successful ideas are changed, cannibalized, or jettisoned altogether. This approach helped the company grow 200-fold over three decades, and helped it expand into new business areas previously not considered.

These learn-as-you-go approaches are powerful. They provide insights unattainable through traditional market research, they increase agility and adaptability, and they can give your organization an innovation advantage.

And that you can take to the bank.

By Stephen Shapiro

Photo credit: Messy Man from Shutterstock.com