Medicine can Teach Business and Government a Powerful Innovation Lesson

By: Paul Sloane

Many companies develop their new products in secret and behind their own closed doors. They then launch their radical new approach with a fanfare of marketing expenditure. They are often disappointed. Paul Sloane looks at how organizations can harness the power of randomized control trials to drive successful innovation.

For centuries doctors tried different remedies for treating illnesses and diligently observed what happened.  But they were unable to be certain what caused the outcomes that occurred.  When the patient recovered that seemed to vindicate whatever treatment had been used – even it was bloodletting or the application of leeches.  The big innovation which transformed medical research and saved countless lives is the Randomised Controlled Trial (RCT).  The idea is simple. A new treatment is given to a number of patients chosen at random, while others receive the current standard care.  This gives a rigorous way to determine whether a cause and effect relationship exists.

In the 1920s the eminent English statistician Ronald Fisher studied the effects of fertilisers on potatoes.  He divided fields at Rothampstead into small plots and randomly chose which plots would have fertiliser. He was able to rigorously analyse the yields and the effects of the fertilisers.  His methods were taken up by statisticians in armaments factories during WW2 and then by the medical establishment after the war.

According to the British Medical Journal  the key features of a RCT are:

• Random allocation to intervention groups.
• Patients and trialists should remain unaware of which treatment was given until the study is completed.
• All intervention groups are treated identically except for the experimental treatment.
• Patients are analysed within the group to which they were allocated.
• The analysis is focused on estimating the difference in outcomes between intervention groups.

RCTs are now widely used in social sciences; for example in the fight against poverty.  In India researchers tested the hypothesis that giving assets directly to poor people would have long term benefits.  They gave some people a goat, a cow or some chickens together with instruction on husbandry. Five years later these people were significantly better off than the control group.  The RCT showed that it was the aid which made the difference and not some other factor.

In many areas RCTs can show what works and also (just as usefully) show what does not. In cities in the USA the Restoring Inner City Peace (RIP) programme was designed to reduce violence by teenagers by shocking them with visits to prisons and mortuaries. The idea appealed to policy makers but a randomised control trial found that participants were normalised to the violence and thus became actually more likely to engage in criminal actions.  An RCT in New York showed that paying all teachers in a school more when their pupils did better at exams did not improve educational standards.

Governments in the West undertake major changes in healthcare, education or social security plans without thorough random trials.

Aid donors like to ensure that their largesse is being used effectively so there are many RCTs underway in developing countries. However, there are relatively few in the developed world.  Governments in the West undertake major changes in healthcare, education or social security plans without thorough random trials.  Obamacare in the USA is a prime example.  Why the reluctance to use the proven power of RCTs?  According to the Economist, the electoral cycle is the main reason why politicians are reluctant to run proper trials which often take years.  Politicians want to be seen to be taking actions now – and with results before the next election.  Do heavy prison sentences deter crime?  Do high taxes raise significant extra revenues?  A good way to test these hypotheses would be with RCTs.  Yet many politicians and commentators seem to prefer their own strong opinions to hard evidence.

And what of business?  How can organizations harness the power of RCTs to drive successful innovation?  Marketing departments have long used trials to test offers.  Direct mail campaigns were prime examples where the same product would be offered to some people on the list at $49 and to others at $59 or $69 to see which price point produced the best overall returns. Of course if you do this randomly you will get two people in the same department getting different offers and this can result in complaints.  But the temptation to select who gets what offer must be resisted; only the random trial can overcome hidden biases.

Every proposed new product, service or method is based on a hypothesis – usually one that assumes customers will prefer the new approach to the one they currently use.  Many companies develop their new products in secret and behind their own closed doors.  They then launch their radical new approach with a fanfare of marketing expenditure.  They are often disappointed.  The Segway was an arch example of this approach – and it flopped for reasons that could probably have been anticipated.

Other companies use focus groups of existing customers and people with the characteristics of the target market.  They test their new ideas with panels, feedback sheets and discussion groups.  When the chosen product is launched it often fails to meet expectations.  New Coca Cola was such a product.  It succeeded with focus groups but failed in the real market.

True innovators do not trust their instincts, or those of the R&D department, the marketing department or the customer focus group.  They trust experiments. In particular they trust randomised trials.  They move quickly and put a minimum viable version of their product idea in front of a random selection of prospective customers.  They ask questions like, ‘Would you buy this?’  They compare the actions of the trial group with the actions of a control group.  The decision on whether to develop the product is not based on the opinions and biases of executives; it is based on the results of the trial.

The randomised controlled trial is the great lesson that government and business can learn from medicine.

By Paul Sloane