Obsessing Over Bias

Recognizing and understanding research bias is crucial for determining the utility of study results and an essential aspect of decision making in marketing. Research plans that lack clear mechanisms to minimize bias are unlikely to be viewed favorably and the end results dismissed. But what are the rules for qualitative research studies? Whenever I am reviewing a plan or a proposal, whether quantitative or qualitative, and I come across attempts to manage “bias,” it always gives me pause. Why? Because I have concerns that the growing tendency of qualitative researchers trying to manage “bias” in their work is due to the increasing pressure to demonstrate research outputs lead to quantifiable impact. I get the desire for ROI, KPIs, etc., but there are times when the fixation on quantifiability is grossly misguided.

Instead, qualitative researchers generally agree that considering concepts such as rigor and trustworthiness are more pertinent to the reflexive, subjective nature of qualitative research. A host of strategies for upholding these concepts during the have been developed and written about extensively, and engaging with this literature is a rite of passage for most experienced and novice researchers who are new to qualitative methodology.

But the issue of bias is raising its head with increasing regularity. Stories of clients or departments rejecting proposed qualitative methods due to subjectivity and bias are common. One of the most frequent questions I get asked when pitching qualitative research is whether directed or probing questions from an interviewer is evidence of bias, that is, that they are mining for data that will affirm their own preconceptions. I understand their confusion. But this begs the question: how much of a researcher’s own values and opinions need to be reflected in qualitative study questions, data collection methods, or findings for it to constitute bias? The answer, of course, is that the question is fallacious. Those carrying out qualitative research are an integral part of the process and final product, and separation from this is neither possible nor desirable. The concern instead should be whether the researcher has been transparent and reflexive about the processes by which data have been collected, analyzed, and presented. 

Numerous people have written far more eloquently than I on the challenges and complexities of the evidence-based movement for understanding the potential contributions of qualitative research. And they offer some sage advice that can help us identify a way forward. Principally, that the challenge is not to try and convince that qualitative work reflects objective, opinion-free neutrality. Rather, it is to better articulate the unique value that qualitatively derived knowledge can play within a system that measures impact through an evidence-based lens. Although it may be more difficult to quantify the impact of qualitative research, we should resist the temptation to reach for a positivist tape measure to solve the problem. To do so will lead us to become apologists for the subjectivity that is the very strength of interpretive work.

Published by gavinjohnston67

Take an ex-chef who’s now a full-fledge anthropologist and set him free to conduct qualitative research, ethnography, brand positioning, strategy and sociolinguistics studies and you have Gavin. He is committed to understand design and business problems by looking at them through an anthropological lens. He believes deeply in turning research findings into actionable results that provide solid business strategies and design ideas. It's not an insight until you do something with it. With over 18 years of experience in strategy, research, and communications, he has done research worldwide for a diverse set of clients within retail, legal, banking, automotive, telecommunications, health care and consumer products industries.

Leave a comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: