How do you Measure Innovation Results and Outcomes?

By: Chuck Frey

The sixth major issue to be tackled by our distinguished panel of innovation practitioners is the all-import issue of metrics: How do you measure innovation results and outcomes and motivate the organization to deliver across all stages of the process, and what are the best metrics for measuring innovation performance?

1. How do you measure innovation results and outcomes and motivate the organization to deliver across all stages of the process?

Innovation is a process that is best managed with a long term perspective, not necessarily measured in long time increments (e.g., months, years) but rather in completion of targeted goals. This requires separating the innovation process into three implementable stages: 1) identification of goals and exploration activities, 2) short term deliverables and 3) near term development.

The first stage, identification of goals and exploration activities, defines the course of action and establishes the motivational inspiration for the entire innovation process.  Setting forth a vision for the innovation goal and providing opportunities to explore various solutions enables innovator buy-in to the goal.   Once the goal has been identified, the steps that need to be accomplished for success can be prioritized, assigned to stage 2 or stage 3, and executed accordingly.  It is important to realize that stage 2 and stage 3 are not static, and should be routinely reviewed and updated.  As goals in stage 2 are completed, some of those in stage 3 move into stage 2 to provide the basis for a new set of measurable results and outcomes. It is management’s responsibility to assess performance to goals in each stage and to determine when a goal has been completed or moved into a different stage.

By splitting the execution phase into 2 stages, the innovation process is positioned to yield a continuous flow of near term successes, which maintains innovator motivation.  Furthermore, if corrections to the initial strategy need to be implemented, they can be done in a timely fashion and at relatively low cost.

— Marc Chason, Motorola Labs

It is important to understand that a clear definition of what constitutes innovation is critical to  the success of measurement. If we define innovation as “people creating new value and capturing value in a new way ,” there are basically three focal points to measure it:

• Past / current innovation performance
• The demonstrated ability to create and capture sustainable and profitable value from innovation
• Future/expected innovation potential
• Effective/efficient innovation capacity
• The activated capacity to realize the firm’s full growth and innovation potential

Critical also to have a balanced group of metrics around all innovation management dimensions — innovation strategy , innovation partnerships, innovation platforms, innovation portfolios, innovation process and systems, innovation and entrepreneurship culture.

— Victor Fernandes, Natura

There are many key performance indicators (KPIs) that are talked about for measuring innovation performance. One that is used by many companies is the “Innovation Sales Rate” (ISR). The ISR can be variously defined, but usually is a measure of the percentage of sales that is sales of “new” products. No doubt, this leaves room for a variety of interpretations, but still is a good measure.

Often, the average margin rates of new products are measured as a key innovation metric. The challenge in large diversified companies comes from having an enterprise-wide description that is consistent and fair. Some business units within a large corporation may be in a mature phase, while others may be like startups. Different expectations have to be applied to these businesses. For motivation, the incentives have to be skewed towards new product and innovative product sales and margin performance. I advocate well designed, persistent incentives, such as based on sales results that are measured over a period of time, like moving averages, so that employees reap long term benefits for longer term planning and performance. This also deflects focus from the next quarter and rewards intrapreneurs for the longevity of their contributions.

— Dr. Makarand “Chips” Chipalkotti, Osram Sylvania

This is a particularly challenging question. Inherent in innovation is exploring the unknown and that brings with it a higher rate of failure than many are accustomed to. Accordingly, it’s important to measure things as a whole, with a portfolio mentality.  Each individual effort cannot and should not be measured at the innovation state. To do so will stifle innovation. Portfolio thinking comes in two flavors: across many projects during a single period, and over time.  So the performance of an individual or group can be measured, but only by looking at their portfolio. If you’re evaluating a manager with many projects, that’s straightforward.  For individual contributors working on one project at a time, you need to look at their efforts over a period of time across many projects.

— David Silverstein, Breakthrough Management Group

a. How to measure innovation?
In Japan, innovation and creativity are not exclusive to a select group of design engineers. Innovation and creativity can come from any employee. At Toyota during the 1980’s, the average employee gave 70 implemented ideas and just a year ago Subaru was getting 108 written implemented ideas per employee and saving over $4,000 per year per employee. Sure, most of the ideas were very small, not like the new spectacular new iPhone, but it was the accumulation of these small ideas from all employees that represented the real success of Japanese companies. And one of these small ideas might become the next “Post-it-Notes,” or “Q-Tip.”

So, one important key measure of innovation and creativity is how many ideas per month are your getting from all of your employees. According to a recent article in the New Yorker magazine the average Japanese company receives 100 times more written ideas then the average American company.

b. How to motivate the organization to deliver across all stages of the process?
Simply evaluate every supervisor and manager on the number of written implemented ideas that they are receiving from their employees. You reverse the process. When a worker comes up with an idea, it is the job of the manager to listen and help the worker implement it. Case in point: one worker in one of my recent classes said, “When I move the windows along the factory floor, I have to go over bumps on the floor and sometimes the windows crack. I go to my supervisor and tell him her about the problem and he only tells me to be more careful.” The supervisor won’t tell his manager about the problem because his manager will only tell him to tell the worker to be more careful.

Now, if you recognize that the worker has the ability to solve the problem on his or her own, when the worker comes to you with a problem, you say to the worker, “What can you do to solve the problem?” The worker says, “We need to re-cement the floor. I don’t know how to do it.” The boss, then says, “Learn how to do it. Go ask Mike to teach you how to do it.” You simply reverse the process.  It is called “bottom up” management. You ask the worker. You continually ask the worker; you don’t tell them how to do it. You just ask. Then watch the innovation and creativity work. Look, if the Japanese can do it so can we.

— Norman Bodek, PCS Inc.

In my experience, the most important thing is to keep the measures simple and focused on what is important to measure — not what is easy to measure. We did a major survey with Rice and Stanford several years ago, and the major finding was that companies were measuring what was easy to measure instead of what was important, and most were measuring far too many things.

The next most important thing is to link these measures and metrics with reward and recognition systems — both are equally important. In that spirit I would recommend that the following three types of measures be incorporated into a balanced innovation scorecard and linked to performance evaluation and reward and recognition systems.

a. Leading Innovation Measures:

• Richness and robustness of growth and innovation platforms and clusters of ideas or opportunities selected and developed
• Strength of strategic and leadership commitment to growth through innovation as expressed in strategic initiatives, targets and leadership metrics

b. In-process Innovation Measures:

• The risk-adjusted net present value of the innovation pipeline and the return on investment in that pipeline
  Innovation capacity and capability building (including partnerships and networks) relative to targets and competition

c. Lagging Innovation Measures:

• Amount of  earnings or revenue growth achieved through innovation relative to targets and industry competitors and overall competitive position
• Success of individual innovation projects (from concept to customer) and overall platform or new business development programs

I also believe it is important to track a company’s progress in capturing key innovation outcomes or premiums relative to the rest of their industry:

• Shareholder value premiums in terms of superior shareholder returns, company value and price-to-earning ratios that reflect growth prospects
• Customer and market premiums in terms of market share, brand equity and customer loyalty
• Value chain premiums in terms of partnership preferences, networks, and positioning
• Workplace premiums in terms of employee retention, attraction and motivation.

— Ron Jonash, Monitor Group

The “Type A” answer is that the best measure of innovation results is ultimate financial success in the marketplace. While that is definitely a reasonable expectation, I’m hopeful that results are also measured by the learning gained throughout the discovery and commercialization process of innovation. Financial success is ultimately imperative to feed the innovation engine, but hopefully, there is enough patience and “lifeline” allowed for products and/or projects to fail along the way. It may sound trite, but you’ll learn more through the difficult times than you will through smooth-sailing success. In fact, there’s a high probability that the successes are built on the backs of many failures. So I’d measure the results of innovation in terms of learning gained, patience developed and wisdom refined as much as eventual financial success. If you keep these measures at the forefront of your innovation practice, you’ll have no difficulty managing the motivation level of the organization throughout the process.

— Troy Geesaman, brandimage

Our Analysis Innovation results are difficult to measure because they include results from (a) completion of ideas and projects in the innovation pipeline and (b) the expansion of innovation capacity at the firm.  Dr. Chips Chipalkatti leans towards the latter by using the Innovation Sales Rate (percentage of sales of new products) as a measure, while Normon Bodek focuses on capacity expansion by using the measure of number of ideas per person.  Troy Geesaman and others would like to include both including measurements for learning through failures. David Silverstein reminds us that innovation projects across a portfolio and innovation projects executed in series by any individual will inherently have higher than average failure rate than other types of projects.  Marc Chason provides further insight by writing that innovation should be managed with long-term perspectives with short-term deliverables and goals.  The long-term perspective allows for experimentation and learning while the short-term deliverables communicate quick wins and momentum to carry through the long term. Victor Fernandes has a more holistic view of innovation results and provides a broader list of elements to measure.  Ronald Jonash organizes this list and links it to a reward-recognition system through an “Innovation Balanced Scorecard” (I-BSC).  The I-BSC organizes the key measures into leading, in-process and lagging indicators.  It measures the (a) risk adjusted value and size, shape and speed of innovation projects pipeline, (b) investment in new platforms, partners, and competences, (c) new earnings and revenue contributed by the pipeline. It is clear that innovation results are not easy to measure.  Far too many companies measure what is easy to measure but not what is important to measure and get it wrong.  Innovation management is a system and the innovation breakthroughs require the whole organization to work as a team.  Innovation measures should be similarly encompass the organization.  Senior leaders should be rewarded for harvesting a pipeline (real sales) and for building a pipeline (future sales).  Their scorecards should be designed to milking the pipeline and also replenishing it, even during a financial crisis or a focus on Six Sigma. — Chuck Frey/Hitendra Patel

2. What are the best metrics for measuring innovation performance?

There is no best metric, since single measurement processes can sometimes negatively impact the innovation processes they are attempting to measure. Preferably, a suite of metrics should be used to measure the innovation process. In order to mitigate this negative impact and increase the value of the innovation measurement process, management should use these reviews as “teachable moments” to reward, correct or guide innovator performance appropriately. To this end, there are objective and subjective metrics.

Objective metrics might include:

• Deliverables to goals (e.g., preapproved innovator performance targets, meeting corporate initiatives, etc.)
• Completing activities that enhance the brand image (e.g., publications, conference presentations, interviews, etc.)
• Production of intellectual property (e.g., patents, trade secrets, etc.)

Subjective metrics include attaining reach-out goals and roadmap targets. For example, a goal such as “Develop two new processes that increase office productivity” enables the innovator(s) to identify bottom-up opportunities with significant opportunities for self-motivation.

— Marc Chason, Motorola Labs

Most companies produce products or services; if they want to compete, they need new products or services. Since many companies I have dealt with tend to “metric” themselves into a paralysis, there are two measurements that I think are simple.

1. Speed to market
2. Number of new product (services) launches.

Speed to market is valuable because it ties in all of a company’s operations.  Everyone understands the need to get something out before the competitor launches a competing product.

I like the “number of new products” metric as because it leaves open the possibility that some of the new products may fail. That is what innovation is about.  Sometimes you flop. The fast food industry is great with this metric. This industry has new offerings all the time, as well as some offerings that fade away.  Wendy’s had a buffet style salad bar (gone), but it just launched a whole line of new Frosty products. McDonald’s had the McLean (gone) but also launched the snack wraps. Pizza Hut reinvents pizza every few months.

The other reason I like these metrics that it takes away the “find a way to do it cheaper” mantra. Some believe that finding a way to do something for lower cost is innovative and needs to be captured on an innovation scorecard.

Frequently, though, the reason for doing something cheaper is to free up resources to develop new products and services or bring them to market faster. Therefore, metrics indicating lowering operation costs are subordinate to speed to market and new products.

— Jonathan Rowe, Gene Express

In the end, one must examine the theory of why innovation is so important. Competitive differentiation, market leadership and higher profitability and sales are the key drivers. When looking at metrics, there are two groups to consider: The innovation creators and managers, and the executors. The interface between these two groups is typically a troubled one. The success of the enterprise relies heavily on a successful hand-off at the interface. So while metrics for ideation and idea management are important for one group, and sales and margin performance for the latter, I believe the most critical metric is one that measures the hand-off between the two groups.

— Dr. Makarand “Chips” Chipalkotti, Osram Sylvania

There are a lot of new metrics being used today. Most are garbage. Things that need to be looked at include overall portfolio performance and the “funnel.” The top of the funnel — new ideas — should be getting bigger. The quality of what comes out of the funnel should be improving. And most importantly, companies need to get much better at learning to kill projects when it’s clear they’re not going to deliver value. All too often we only measure the final result. In innovation, the intermediate steps must be measured, too.

— David Silverstein, Breakthrough Management Group

At the front door of Stu Leonard’s, one the of the largest grocery stores in America is a large rock with the words:

“Our Policy:

Rule 1 – The customer is always right!
Rule 2  – If the customer is ever wrong reread rule 1.”

The best metrics is to reread my first few paragraphs and get everyone implementing their small ideas. And I recommend you just get up from your desk and learn about Quick and Easy Kaizen and do it. Yes, be brave and do it! Stop looking for excuses not to do. “Just do it!”

— Norman Bodek, PCS Inc.

Metrics: a necessary evil? Well, maybe not quite evil, but metrics can seem like a sterile, incompetent way to measure some of the beneficial, intangible measures of innovation. Yes, financial, volume and time metrics are critical to the discipline of innovation.  If you’re part of a successful enterprise, they will naturally be embedded into the process. What’s often missing, however, is a means of capturing, measuring and highly valuing the intangibles of learning gained, patience developed and wisdom refined. How to measure and document these intangibles should be totally contextual to the corporate environment and existing reporting systems. What should be absolute is the inclusion of these measures in personal and project performance, and not just measure innovation performance on financial, volume and time metrics alone.

— Troy Geesaman, brandimage

Our Analysis Our experts are quick to point out that there is no one best metric for measuring innovation.  However, they all had their own favorite metric that they felt should be included.  We have compiled this list below: Increase in value of ideas at top of funnel Number of new ideas implemented Risk adjusted net present value of pipeline Number of projects killed Number of successful handoffs Speed to market Number of new offerings launched Lessons learned from failures (and successes) David Silverstein believes a focus on bigger ideas should deliver bigger innovation results.  Norm Bodek advocates that ultimately it is the implementation of new ideas that drives overall innovation. Ronald Jonash is an advocate on the Risk Adjusted Net Present Value (RANPV) of the pipeline and using this process to provide guidance for adding new projects, slowing or speeding projects or killing bad projects.  David’s focus on killing bad projects supports that since it removes negative value projects out of the portfolio and increases the RANPV of the pipeline.  Similarly, Jonathan Rowe’s emphasis on speed makes sense as it increases the net present value of the project and creates a competitive advantage in the market. Jonathan explains in detail about the benefit of having a metric that measures number of new offerings versus successful offerings.  He embeds the notion of failures and learning in this metric and its importance for future innovations.  Troy Geesaman requires that organizations should require formal mechanisms for capturing these learnings. The above list could be a good starting point for many companies wanting to start their innovation journey and metrics.  Innovation is a system and requires more than one metric.   Measure what is important and not what is easy. — Chuck Frey/Hitendra Patel

In the next interview:

How do you find and build the people, capabilities and competencies required to be successful in Innovation?

Feedback?

Please share your comments below, where we hope to create a lively discussion about the ideas and strategies outlined in this “virtual roundtable” interview.

Published by Chuck Frey, InnovationTools, and Hitendra Patel, The Center for Innovation Excellence & Leadership, Hult International School of Business

Photo by eskay lim on Unsplash