Nearly every company’s strategy these days is to grow through innovation, yet many fall short. We all know the standard reasons: innovation is hard, innovation is uncertain or innovation grinds against the gears of the operating organization. They are all more or less true, but they are also simplistic, not really guiding executives on how to actually get more innovation.

I’ve come across surprising and actionable reasons why companies don’t have enough innovation. Here are my top four, from least important to most important.

4) Not Enough Ideas
3) Innovation is a Different Process
2) Not Enough Opportunity
1) Incremental Crowds Out Innovative

Reason #4: Not Enough Ideas

“Not Enough Ideas” is the most common self-diagnosis and nearly every innovation officer’s starting point: let’s get more or better ideas, and let’s engage and activate the innovation community in our organization. A cottage industry of consultants, books, methods and software is built up around this diagnosis.

Sometimes this reason is correct, or at least correct enough that a major initiative around idea generation is warranted. But often these efforts run into frustration because a lack of ideas isn’t really the issue.

Think Big

Most organizations I have come across have unintentionally suppressed their big ideas. Many projects have large upsides that are not in evidence or really being pursued, even though there are capable members of the organization dreaming about these big ideas.

The usual culprit is a culture of a reliable promise, in which people promise what they can deliver and then exceed expectations. While this is a superb approach to project management and accountability, it often has the unintended side effect of causing people to lower their aspirations. They make a small promise and propose a small step against a big idea. Rarely can the small step actually deliver substantive results, leading to a sort of spiral of mediocrity. In bad cases, people can actually be afraid to even mention their big dream.

No idea generation system can overcome this cultural issue. Either the proposed ideas will be small, or big ones will get crushed as they go through the grinder of the stage gate, annual operating plan, and other routine business processes. The almost inevitable result is a wave of cautious enthusiasm for innovation, followed by increased cynicism and frustration that makes it harder to drive growth.

To find the hidden upside in innovation opportunities, you need to incorporate uncertainty directly into your evaluations and treat innovation with a real option structure. Assume the innovation works and explore upside scenarios, expressed financially, and then explore the downside scenarios. The difference will tell you which uncertainties really drive the size of the innovation: focus on those. A tool called the tornado diagram is very helpful here. Then analyze what proof points you need to demonstrate to make the case that you are on the path to the upside (or at least a good scenario). Create a learning plan around these proof points.

Reason #3: Innovation is a Different Process

The approach for evaluation sketched above for evaluating innovation as an option and putting uncertainty at the center of your evaluation process is not common. It is one of many ways things need to be done differently to support innovation. The basic problem is that traditional project management, stage gate, and accountability processes all tend to kill innovation. And the culture that builds up around these processes also kills innovation.

Most companies know that innovation needs different ground rules. If you are putting your ideas and innovations into the standard processes and culture, it won’t work very well. Your company is unlikely to meet its growth goals.

Happily, in the last decade or so, there has been a tremendous explosion of helpful methods and processes to help innovation: Agile, Lean, Discovery-Driven, etc. They all have a lot of overlap, and each has its own special strengths. So, if you haven’t made a good space for innovation, get some of the books on these techniques, read them and follow them. Go to conferences and share experiences.

It is no longer a mystery how to set up an environment for innovation to work. It is not easy, but it is reasonably clear what needs to be done.

Don’t Just Say, Do

The biggest issue I see is that people adopt the new language but not really the spirit of the new methods. For example, if you want to look agile and look like you are complying to the approach of hypothesis testing, the easiest thing to do is to reframe whatever you would have done under your old approach in hypothesis language and do the same thing.

Let’s say your organization is comfortable with a process that goes more or less gets customer requirements and then seeks to engineer a solution to those requirements by building prototypes. A way of preserving the old approach (to avoid change) is to relabel it by saying you want test the hypothesis that a customer wants something by creating a minimum viable product. Customer requirement maps onto hypothesis and prototype maps onto minimum viable product.

In other words, people don’t want to change. Or maybe they just don’t want to do (or aren’t skilled at) doing the critical thinking about the new opportunities required by some of the new approaches. They’d rather just do the next step in a familiar process.

One high leverage thing you can do is insist on a Formulation Step. Require the team to develop several strategies for developing their innovation. Require the team to really understand the uncertainties that drive the upside. Then step back to create proof points. Don’t just let the team run off with the first thing that comes to mind as the “hypothesis”.

Reason #2: Not Enough Opportunity

So you’ve got some ideas, and you are well underway with the new processes, and you still aren’t getting the growth you need from innovation. Many companies abandon their efforts in frustration, while others double down – generating even more ideas or trying even harder to make better processes. Often these efforts are just a way of avoiding a frightening fact:

All businesses die.

Even worse, every business is always dying. Every business is slowly sliding into lower value, towards commoditization or obsolescence. Every one, all the time.

But not every organization dies. Organizations survive through renewal, the nurturing and generation of new businesses, even as the old business dies.

The Business Lifecycle

So where is your business on its lifecycle? Trying to convert a late stage business into a growth business is a recipe for frustration. Yet many executives pursue this approach, hoping “innovation” will somehow restore them to their former glory.

To prevent this issue, you’ve got to get crystal clear about the gap between the top-down aspiration and bottom-up reality. What are your actual projects and proposals? What are they worth? What is the upside? How difficult are they? How big could they be?

Then make sure your business strategy and your innovation portfolio are aligned. Too often they are not, with strategic demands not being realistic given the bottom up reality.

Here is a typical strategic taxonomy of innovation opportunity.

  • Cambrian Explosion. Opportunity is everywhere. The basic forms and parameters of the new types of businesses are being created, tested and transformed. Successful companies are regularly transforming themselves. As examples, think of Amazon or Netflix. A challenge in this area is that opportunities are so abundant, you need to be thoughtful about which ones to pursue. It is easy to go too many directions or get lost. But those who invent the new forms get tremendous competitive advantage and create tremendous value.
  • Growth. While the basic forms of business life are established, lots of speciation is happening. There are ample adjacent opportunities, and even good transformational ones. Good ideas in this phase are harder to come by, but the can be developed and exploited. The value created remains high. As an example, consider DuPont in the 1940s with Nylon, pursuing many novel applications of a new material. Many executives want to be or believe they have growth opportunity of this character; not all are right.
  • Exploitation. Companies become defined more or less by their current capabilities and their current customer classes. Innovation can become more about optimizing manufacturing, keeping high volumes in the line, or adapting their technology to specific customer requirements. Companies stand out through differentiation, being a little better, faster or cheaper; or more responsive or trustworthy than their competitors. Many executives implicitly pursue an Exploitation strategy, even while aspiring to Growth. This creates a huge disconnect, and innovation will pretty much fail. If your business is truly in the exploit stage, run it that way. Focus your innovation on creating businesses outside the core.
  • Commodity. In this world, whoever can do a fairly standardized thing the best wins. Innovation shifts almost exclusively to process and cost improvements. Trying to grow a commodity business through new innovation is nearly impossible.

At each level, the innovation opportunity gets smaller. Don’t fall into the trap of driving your organization to a higher level when it is in fact at a lower level. You cannot move backwards: you have to reinvent and renew, creating new businesses within the old organization and letting the current businesses age gracefully.

Human nature leads us to denial. If we imagine our company as a growth company, we will set “strategic objectives” and require that the organization creates a growth plan. This top-down thinking is insufficient. Unless you have a rigorous bottom-up view of the merits of the project in your portfolio and a clear line of sight to the gap between the bottom-up reality and the top-down aspirations, you will probably fall into the trap.

Reason #1: Incremental Crowds Out Innovative

For many companies, innovation is a like a pet. We all like innovation and want it. We want to point to innovative things our company is doing. It is a source of pride and a symbol of freshness.

But innovation as a pet eats the table scraps of the core businesses. As long as innovation is cute and doesn’t eat too much, we keep it.

The problem occurs when innovative projects need real resources or focused attention. When the projects challenge the status quo or otherwise compete for importance with the core business, the core business fights back. When the pet jumps up on the table and starts eating the main course, we smack it down.

I’ve never met an executive against innovation. They just don’t want to cut into the core business to support it.

This concern is exaggerated by the way we tend to evaluate innovation. It is “strategic,” based on a hand-wavy argument about “different rules”. So an executive is put in the position of choosing between a relatively certain and familiar investment in a core opportunity and an uncertain investment in an unfamiliar innovation opportunity.

Analyze Innovation Rigorously

To address the concern and remove its power, you have to make innovation stand up rigorously in financial terms so that it can actually compete with core investments. The executive needs to be in a position of having to make a choice between two investment grade opportunities.

Most financial analysis completely botches the value of an innovation investment. To avoid this error, you need to use a rigorous financial approach that allows innovation to express its real value. This requires putting uncertainty at the center of your evaluation system, not forcing innovators into making reliable promises. If you can do this, you will find that innovation can actually compete.

For example, the Innovation Screen above shows the probability of success of an opportunity versus the size if successful. Projects in the upper left, easy but small, are Bread & Butters, typical core investments. Projects in the lower right, difficult but big, are Oysters. Most of them won’t in fact work. But an Oyster farm can fairly reliably produce Pearls – the big and very valuable businesses that produce reliable returns – even if we cannot tell in advance which Oyster will turn into a Pearl.

The core business faces a relentless pressure to deliver. Bread & Butter projects abound, creating a kind of clutter. This clutter is the greatest threat to innovation.

Cut Out Clutter

The irony is that the clutter is rarely actually a good investment. It just gets a pass because it is more familiar. Risky, innovative projects are often much better than the small and difficult projects that they often compete with.

So show the value of your innovation, head to head with bread and butter investments. Make sure the executives are really clear about their choices in financial terms, between clutter or innovation.

When you reach this level of clarity and transparency, people choose innovation regularly – not only the executives, but people all through the organization. Strategic Portfolio Management unleashes this innovation by finding the true value of innovation opportunities; applying the right process to innovation management; aligning the innovation portfolio with the business strategy; and dealing with the biggest, most prevalent and most important source of failure in innovation: failing to say “no” to the clutter.

By David Matheson, President & CEO, SmartOrg, Inc.

About the author

David has helped senior management teams from some of the world’s largest corporations to improve their results from strategic portfolio management, product development, innovation, and R & D.  He is an expert at measuring value and managing uncertainty.  David is co-author of The Smart Organization: Creating Value through Strategic R & D. He is a Stanford University Ph. D. and a Fellow with the Society of Decision Professionals.