The case for treating audience research as investigative journalism

This is an excerpt from my monthly newsletter, Re:filtered. You can subscribe here.

My main project this summer is conducting research on the information needs of a group of people I can't meet.

These people live in another country. Their lived experiences are vastly different from mine. While we speak the same language, we do so in very different ways.

I've asked some media practitioners and scholars for guidance and some told me that it couldn't be done. Confronted with that skepticism, I worry they're right.

The last time I felt this way was ten years ago, when an editor at the South China Morning Post asked me to write a feature with a China angle to mark the centennial of the outbreak of World War I.

I set out to find descendants of the hundreds of thousands of Chinese laborers who supported the war effort in Europe and the Middle East, a contribution that still isn't widely recognized. There was once a massive war mural in Paris that showed France surrounded by her Allies. To make room for the Americans when the United States entered the war, Chinese laborers were painted over.

I couldn't go to mainland China for this story. I had no idea how to find people two to four generations apart. It took many weeks of dread and dead ends to find someone. This photo of a graveyard visit in France saved the day:

Kneeling in the middle is Cheng Lin. She had posted this photo on Weibo, mentioning the name of the cemetery in the post. It is the grave of her grandfather, Bi Cuide.

A century earlier, the native of Shandong province had joined hundreds of thousands of Chinese men, mostly from the countryside, to help Britain, France and the other members of the Entente topple the empires of Austria-Hungary, the Ottomans and Germany.

Like so often in reporting, this was a series of rare strokes of luck: Cheng found the grave thanks to a British commemorative medal bestowed to Bi after the war and which miraculously reached and stayed in the family.

Cheng first discovered the curious disc when she visited her ancestral home in Laiwu in the 1970s. Then a teenager, she remembered noticing the number etched along the rim: 97237. Half a century later, the number, her grandfather's military ID, led her to a database which then led her to the grave.

I am now experiencing a similar dread with this audience research. But it's a good sense of dread that is so familiar from investigative reporting and that pushes you on. It's the addictive thrill of a reporter's looking for information, fueled by the knowledge that something is out there. It just needs to be found in a similar mix of perseverance, learning and luck.

This is so different from what I originally thought audience research would be.

When I first encountered its practices in newsrooms, it was all about data wrangling to identify trends in past consumption, and draw conclusions about the future.

I wish someone had told me back then that that's only a tactical part of a broader effort that's much more similar to investigative journalism.

Ideally, audience research is reporting, before reporting.

In "Just Enough Research," Erika Hall has this helpful distinction: 

1. Generative research: "What is a good problem to solve?"  
2. Explanatory research: "What is the best way to solve the problem I've identified?"
3. Evaluative research: "Are we getting close to solving this problem?
4. Causal research: "Why is this happening?"

In media organizations, so much energy, resources and technical development go into #3 and #4, and so little into #1 and #2.

The problem here is that evaluative and causal research can only optimize a status-quo. It can't identify needs and opportunities. These feedback loops have supplanted (instead of building on) the more exploratory strategic research that often just doesn't happen.

That's especially tricky because consumption data is so compellingly precise while generative and explorative research often lack the such appearance of scientific precision.

For civic media, evaluative research can't lead to product/market fit and will not lead to any tangible social change – unless you're really, really lucky and talented, which no one ever is consistently over time.

This is a massive problem in public media or donor-funded media organizations that can't impose discipline through monetization. I've worked in some newsrooms where audience analysts, journalists and marketers have framed evaluative research as generative as a way to impose their vision of what the media organization should be doing. Perhaps I have too.

This is also a key issue for the major ongoing donor initiatives that have invested a lot of energy into data dashboards. These have their place, esp. for benchmarking, but they're secondary to strategy. The methodologies of research to get to strategy are fundamentally different.

The "Jobs to Be Done Playbook" by Jim Kalbach is helpful in how it structures  generative research:

1. Discover value: find the right problem to solve for the people you serve.
2. Define value: set the direction for addressing the problem you've identified.
3. Design value: Create solutions that are desirable, viable, and useful.
4. Deliver value: Present the solution.

One methodology he mentions are goal-based personas, based on the work of Alan Cooper in "About Face 2.0. The essentials of interaction design:"

Step 1 – Interview people: "You can interview for jobs and personas in the same session. Rather than asking users about their preferences and desires, focus on their intent, as well as what frustrates them and what success looks like."

Step 2 – Map interviews to variables: Think of variables as needs with two endpoints that create multiple ranges (e.g., functional: factual vs. interpretive, specialized vs. general; social: community vs. universal, consumption vs. participation; psychological: reassurance vs. challenge, inspiration vs. utility, education vs. entertainment).

Step 3 – Identify patterns in goals: Look for clusters.

Step 4 – Describe the resulting personas: For each cluster, create a persona with shared circumstances when reaching the goal, workarounds and frustrations. Avoid adding irrelevant detail.

Result: Fictional name; behaviors and actions; demographic and psychographic details; needs and pain points. 

In my own past work, I have perhaps overly focused on demographics rather than for pain points for sizing these groups. I'm going to give this another try.

This is an excerpt from my monthly newsletter, Re:filtered. You can subscribe here.