Managing Fundraising Innovation: the Dangers of the ‘Echo-Chamber’

By: Paul Vanags

This case study explores the results of an innovation research process undertaken by Oxfam, which compared internal feedback vs. general public feedback to identical sets of ideas. In comparing responses between these two audiences, Oxfam discovered an immediate and obvious divide between their staff’s opinions about which fundraising ideas would perform the best, versus what the general public preferred – an important lesson about avoiding the bubble of the echo chamber.

The algorithm. Fast becoming one of the most powerful forces shaping our society today. “People who like that also like this” Amazon tells us. “People you might know” suggests Facebook, at the same time as suggesting products we might like based on its analysis of our posts and news feed. (Incidentally, Facebook seems to think I need razors and gay dating – who am I to argue?) Some of it is useful, some of it’s downright scary.

The problems of our own personal ‘echo-chambers’ have been well documented and are attested to by the shock many people felt from Trump and Brexit.

But back home on the ranch what can this tell us about how we judge new ideas? Are our own organizations effectively ‘echo-chambers’ of a different sort, that give us a false view of the world inhabited by our supporters?

One of my favorite facts about the Oxfam supporter database (it’s a long list!), is the following. Most Oxfam supporters vote Conservative and read the Daily Mail. When I tell people this fact (note: fact = a thing that is known or proved to be true, just in case anyone is unsure), they are usually very surprised. Why? Because it doesn’t fit with people’s thin-sliced intuition about Oxfam of course. We’re sort of leftie, liberal, save the world types, padding round in second-hand jumpers and moccasins and eating lentils right? Wrong.

And thin-slicing is a brilliant way of quickly assessing a situation and tapping into those ‘intuitions’ about whether something ‘feels’ right. Malcolm Gladwell got a whole book out of it. But it’s important to remember that we all have biases and to get the full picture you need to talk to the people you’re trying to persuade.

This case study from Oxfam uses scores of a range of identical innovation concepts and compares how they fared in two pieces of research – one internal and one with the general public. The results make fascinating reading and we believe carry an important and useful lesson for all innovators.

A few years back Oxfam embarked on a new innovation process, setting out to generate radical new ideas to find significant new income streams. In this particular process was that it utilized internal “crowd-sourcing” to build/test the ideas and then compared these results to an external concept screener.

Broadly the process we followed was as follows:

We used an online innovation platform to test the concepts internally. A broad selection of Oxfam personnel were shown concepts and asked to vote whether or not they liked the idea with a “thumbs up” or a “thumbs down”. They were also able to comment or leave additional ideas or questions on each idea.

We received over 1,000 votes and over 150 comments on the concepts, a huge amount which if nothing else demonstrates how interested and engaged people are in innovation at the ideas stage.

We used this feedback to filter out a few concepts, although we kept in a number of concepts that we felt strongly about which had not scored well internally. None of the concepts which were present in both studies were changed in any way.

We then put the remaining concepts into a proprietary concept screening research, Brainjuicer’s “Predictive Markets” tool. This tool asks the question “If you owned shares in this idea would you probably buy or probably sell them?”

These topline results are give the following output, the key measure being what percentage of people said they would “probably buy” shares in the idea:

This gives a comparable output to the like/dislike question(1) with the only difference being in audience – people who work at Oxfam vs. a nationally representative sample of the general public.

Now for the big reveal! Here are those two data sets of scores for exactly the same idea presented alongside one another.

Three things are immediately obvious from the comparison;

1. In general, people who work at Oxfam are more positive about new fundraising ideas than the general public
2. There are significant differences between Oxfam and the general public
3. Those differences have no particular pattern

In short – people who work at Oxfam are in general a very poor guide to what supporters are going to think. The bubble of the echo-chamber at work.

In this specific example, the idea that tested the best (by some margin) with consumers was poorly rated internally and attracted a great deal of criticism.

This has serious implications for innovators in charities – if this study is representative (and we believe it will be) your colleagues are going to be very bad at judging what the public will think.

Just think of what that means for an innovator trying to get their new idea through their organization and into the market. Without target audience research to help steer the prioritization of new ideas, the ones picked out by internal influence is no better than a lottery. You might get lucky, but most probably at the earliest stage in the process you will be discarding great ideas and pursuing hopeless ones. The rest of all your innovation effort depends on choosing the best ideas. Poor decisions now affect the whole process, you are squandering precious time and resources on what is effectively a random selection of ideas.

Even with some good research with your audiences to help you prioritize, you may still face stiff opposition in getting your ideas accepted by key decision makers. Even those with knowledge and experience of the challenges of innovation will find it difficult to drop their personal preferences and prejudices (we’re only human after all).

Of course, there are many reasons why an organization may choose to reject an idea, even if the consumer likes it. It could be expensive to get to market, it could carry a significant reputational risk, it may not show a return quickly enough or might not be on strategy.

This is all good and right – and any decent innovation process will force out these debates and discard ideas along the way for all of those reasons. But the important point here is that it is wrong to discard ideas too early. To throw new ideas away without properly exploring them is like going to the trouble of baking a cake and then throwing it away before you’ve tasted it.

So what to do about it?

We have found a huge benefit internally in exposing our ideas to people – it makes the innovation process more transparent and less ‘black box’, it is engaging and helps give your innovation programme awareness and momentum within the organization. It gets people talking.

However, the inputs you get from it need to be treated with caution. At this stage of the process, the watchwords for the innovator are inquisitiveness and flexibility. Do not be put off by negative comments, interrogate them, understand them, but be happy to discard them if they’re not useful or they take you off to somewhere you don’t want to go.

The people who work for your organization, particularly in the Third Sector, are a wonderful, passionate bunch of people. But they’re self-selected advocates of the cause, and unless your innovations are all aimed at people who work in your building, they’re not going to help you judge which of your ideas will succeed in the world outside the front door. For that, you need to ask the target audience.

Prototyping is also a great mitigator of risk – getting some kind of emulation of your idea physically in the hands of some real people can be very enlightening in all sorts of unpredictable and useful ways.

My top tips:

• Beware the echo-chamber – it is NOT your friend
• Use research (but not like a drunk uses a lamp-post)
• Remain inquisitive and flexible
• Remember doing innovation is more important than talking about innovation

Good luck!

By Paul Vanags

(1) The methodologies are different as are the actual questions and so we are open here to the criticism that the two metrics are not directly comparable. I understand that but “like/dislike” vs “buy/sell” are similar enough in my view to warrant comparison and remember we are looking at the pattern of responses, not the absolute and the former are mathematically unambiguous.