Framing the Critical Question: Insights from Survey Research

By: Doug Collins

Campaign teams cover a lot of ground as they work with the sponsor of a collaborative innovation challenge. In this article, innovation architect Doug Collins makes the case that campaign teams should focus their energies on helping the sponsor develop the critical question that serves as the basis for convening the community. Forming the powerful question—the question that accurately reflects the sponsor’s intent and that resonates with the community—yields the greatest return on time spent in developing the campaign, relative to its ultimate success.

Get the question right

Developing a challenge for a collaborative innovation campaign can take months. The campaign team spends time during the period seeking commitment from people to serve in roles such as sponsor, moderator, and coach. They spend time developing the campaign mechanics, including when and how to engage the community. They cover a lot of ground with a lot of people.

In my experience the most important job that the campaign team performs during these months—the activity that most affects the ultimate success of the campaign—is helping the sponsor and their core group of advisors formulate the critical question upon which they want to invite the community to discuss and contribute their insights.

Get this part right and they achieve results, even if they permit other elements of the campaign to slip.

Get this part right and they achieve results, even if they permit other elements of the campaign to slip. Get this part wrong and they find themselves with an overabundance of invalid insights and ideas, the surplus driven by their flawless execution of the plan to engage the community.

Lamentably, too, I find that campaign teams do not spend enough time perfecting this critical element of the campaign. They hesitate to request the needed level of commitment, or engagement, from the sponsor. They lack the guidelines to help them develop the question properly. Yet, the essence of the practice of collaborative innovation lies in the discipline of framing and reframing the critical question, then engaging the community as they respond.

In this article I address the latter challenge: question formation. The campaign team, once they have a vision for what makes for a quality question, feels more confident in seeking the sponsor’s time to develop it. To this end, the article may help the campaign team make headway in resolving the former challenge, too.

Drawing lessons from the discipline of survey research

A number of valuable resources on the subject of challenge question formation exist, including most notably Getting to Innovation by Arthur Van Gundy. For this article, I draw primarily from Jack Fowler and his book, Designing Survey Questions. Fowler approaches the subject of question design from the more mechanical, empirical perspective of designing effective survey instruments for field research. The field of survey research has for years grappled with the challenge of proper question formation, recognizing the important role that question design plays in yielding valid results. Fowler, specifically, lays out a set of principles to guide one’s thinking on what makes for proper question design. I will relate them as follows to the design of challenge questions, as his guidance and insight maps well to the practice of collaborative innovation.

Principle: ask people about their firsthand experience

In survey research, asking people to speculate yields unreliable responses. People know if they have been a victim of crime. They have no reliable perspective on crime county-wide, for example. Relative to the practice of collaborative innovation, I always recommend grounding the enquiry to the community members in the reality of their own experience. Specifically, regardless of whether the challenge question is aspirational or practical in nature, we solicit their observations: What caused you, the contributor to have this idea?

If the nature of the challenge question is around improving customer experience, then we encourage them to reflect first on their experiences from having served customers. The core idea may prove to be highly speculative, which is welcome. However, the idea becomes compelling in this context when it ties to real-world experience (e.g., “I notice that customers must wait 30 minutes before they can receive a response. We can cut the wait time in half by doing X.”).

Principle: ask one question at a time

Van Gundy likewise offers the same “keep it simple” advice, with the additional guidance not to embed potential solutions in the enquiry. This principle falls under the umbrella of common sense. Yet, the campaign team, fearing the undesirable response, will often go to extreme lengths to preclude surprises, leading to constructions such as, “How might we reduce inventory carrying costs by 10%… in a way that involves hiring left-handed tuba players who spent their formative years in Hamburg?”

The remedy for this scenario is for the campaign team to ask themselves if they are, in fact, leading the witness—and then be merciless in cutting the lead. Likewise, be merciless in breaking apart compound questions.

Principle: word questions such that each respondent is answering the same enquiry

Clarity differs from comprehension. The following challenge question is straightforward: “How might we enter the education market?” However, community members are left to wonder whether the sponsor has the local preschool, vocational program, or university campus in mind.

Provide context ahead of the challenge question—not in terms of the optimal solution but rather the optimal target.

To this end, campaign teams are wise to provide context ahead of the challenge question—not in terms of the optimal solution but rather the optimal target (e.g., “please focus your enquiries on vocational programs, in particular”). The context can come in the form of a letter from the sponsor which serves double duty as an instrument of greater comprehension and reassurance that the organization condones their spending time to contribute to the campaign. Likewise, testing your question with a handful of colleagues who have not participated in its development helps you see likely points of ambiguity.

Organizations that have achieved a level of maturity in developing campaigns often invite the community to form the critical question together through approaches such as The World Café or Open Space. In doing so, the community provides its own context and meaning.

Principle: clearly communicate expectations

Clearly communicate to all community members the kind of response that constitutes an adequate answer to a question. Adhering to this principle confounds both market researchers and campaign teams. The team devotes the necessary time to develop a clear, contextualized enquiry to the community. A member, pressed for time, short hands their response, providing only the germ of what could be a compelling idea, leaving the campaign team wondering what to make of the contribution. Here, I strongly recommend using the observation, implication, application approach developed by consultancy Innovation Focus and described in a previous article.

In brief, seek their observations. What did they experience that caused them to have this idea? Ask them to draw implications. What does the idea mean to the organization? To the intended recipients or beneficiaries? Then, ask them for their perspective on next steps. What should the organization do next—tomorrow—to explore and realize the benefit from the idea?

Campaign teams hesitate to burden the innovator for fear of losing their contribution, altogether. My experience suggests otherwise. What I find is that people, inspired to have and to share an idea, will willingly and joyfully make the effort to fully convey their insight. It’s a commitment they make as an expression of the personal mastery they exhibit with respect to the topic at hand. Campaign teams have the opportunity to engage in powerful ways with the innovator when they respect their commitment. Doing so puts them in the best position to fully explore the potential that the idea represents.

In closing, people who practice collaborative innovation can enhance their own mastery by reflecting on the insights and experiences gained from related, enquiry-led disciplines such as survey research. These practitioners, too, struggle with framing the critical question, providing relevant context, and deriving essential meaning from the responses they elicit. They have in turn developed empirically sound guidelines for improving question design.

By Doug Collins