The combination of Big Data and ethnographics can be a potent toolset for uncovering innovation opportunities, as a growing number researchers are discovering.

For many years, marketing has been in love with Big Data as a means of discerning what customers want and delivering products and services to them. In this role, it has been a lagging indicator, telling us what consumers are doing and have done, but it has been of limited utility by itself as a driver of innovation.

At the same time, ethnography has been growing in importance as a tool to “go deep” and understand what challenges customers are having by observing them in their natural environment. Often, this illuminates jobs to be done and pain points that cannot be discovered in any other way. This type of input tends to be more reliable than focus groups, where customers are brought into a sanitized environment and asked a series of questions. Taking them out of the context in which they use a product tends to skew their answers, and once again is backward looking. Consumers can easily tell us what they are using and doing today, but are notoriously bad at articulating what their unmet or future needs may be. For example, a focus group could have never envisioned the Apple iPhone.

Recently, companies like Xerox are finding that the combination of the two disciplines – ethnography and the analysis of large data sets (often referred to as “Big Data”) can be a powerful combination. In a recent interview on, several Xerox executives shared their observations on how these quantitative and qualitative disciplines have come together to generate some powerful new solutions for government agencies.

Ken Mihalyov, Xerox’s Chief Innovation Officer for Transportation Central and Local Government, explains how ethnography helps his team to ensure that they are focusing on the right problems, and that the conclusions they’re drawing from their analysis of Big Data are validated in the real world:

“There are things that we can certainly accomplish with our algorithms and Big Data alone. We can look at the data and see trends that we would not otherwise see. Ethnography is a strong counterpart to looking at the data a certain way and drawing conclusions from it. We can confirm that we’re working on the right problem, that we haven’t missed something and that our interpretations are correct.  Ethnography helps us confirm those factors and that we’re seeing the bigger picture that includes human interaction.”

A related article on the Ethnography Matters Blog highlights some of the intriguing possibilities that are emerging from this unique combination of disciplines:

One study developed a database of consumer usage of computing devices, which was interpreted in the form of color-coded charts that showed the intensity of device use at different times of day. This behaviorial data was then shown to survey participants, who helped researchers to interpret the patterns the data showed and to explain more about the reasoning behind their device usage habits.

The availability of massive databases of web search data, available from services like Google Correlate and Hadoop, can be combined with ethnographic studies to open up new consumer research insights.

Another study analyzed demographic data in several south African countries on cell phone usage, which showed a strong gender demographic skew toward male ownership and use of cell phones. Researchers then combined that data with a deep understanding of marital and family dynamics, gathered via ethnography. This combination of research methods explained why far fewer women used cell phones than men in those countries. It also helped researchers discern that cell phone usage by women was directly related to education and income; affluent, well-educated women in those countries are more likely to own and use cell phones.

Challenges with combining ethnography and Big Data

Combining ethnographic observations and data isn’t necessarily easy, of course. One problem with Big Data is its accuracy – or in many cases, the lack thereof. “Dirty” data entered inconsistently or missing segments of it can easily skew our interpretations of it.

Also, people who are able to interpret and draw meaning from massive quantities of clickstream and other data are often hard to come by. It’s distressingly easy to draw haphazardly and selectively from data, which can give us a distorted picture. In other cases, researchers may inadvertently use data to reinforce or justify our premeditated conclusions about customer behavior and their needs.

Another shortcoming that the Ethnographers Complete Guide to Big Data blog post series points out is that big data tends to under-represent poorer segments of the population. Their behaviors are likely to not show up in traditional sources of Big Data, such as web site traffic and conversion reports.

In addition, the author points out that combining the two disciplines sometimes makes it even harder to figure out how much is enough. How much data do I really need to collect? How much more time should I spend in the field observing customers? What don’t I know?

As researchers continue to explore the intersection of big data and ethnography, I predict that many more valuable models will emerge that can be employed and adapted by innovation practitioners. Together, these disciplines can provide us with deeper, more nuanced pictures of customer behaviors and the meanings that drive them. Ultimately, that will help us to develop new products, services and business models that better predict and anticipate their needs.

By Chuck Frey

About the author

Chuck Frey Senior Editor, founded and served as its publisher from its launch in 2002 until the partnership with Innovation Management in 2012. He is the publisher of The Mind Mapping Software Blog, the definitive souce for news, trends, tips and best practices for visual mapping tools. A journalist by trade, Chuck has over 14 years of experience in online marketing, and over 10 years experience in business-to-business public relations. His interests include creative problem solving, visual thinking, photography, business strategy and technology. His unique combination of experience and influences enables him to envision new possibilities and opportunities.