Sampling: Why Individuals Don’t Matter

We spend a lot of time talking about samples when talking with our clients.  Samples are constructed differently in ethnography than for focus groups or surveys. Ethnographers sample settings and interactions as much as individual people. The individual is rarely the unit of analysis.  Sample is defined in the social interaction and the contexts in which activities occur. Asking a person if, say, they have specific impressions of a brand of beer will doubt yield information.  Unfortunately, it’s simply nothing more than a first-hand account of what you get from a survey.  Interacting with a group of people as they move from bar, to dinner, to party will yield significantly more information about brands and the contexts of selection and use.

All too frequently, sample devolves into a discussion of validity, with a portion of the clients pointing fingers and declaring the shortcomings of the methods in question because, quite frankly, they in fact know precious little about statistics and the epistemological constructs around them.  But reliability and validity are by no means symmetrical.  It is possible to obtain perfect reliability with no validity, but perfect validity would assure perfect reliability because every observation would yield the complete and exact truth.  This notion is what leads to an obsession with “typing” individuals and limiting our ability to uncover new, meaningful insights.

Loosely speaking, “reliability” is the extent to which a measurement procedure yields the same answer however and whenever it is carried out; “validity” is the extent to which it gives the correct answer.  As an example, imagine 100 people participate in a survey on grocery shopping to determine the optimal placement of goods on the shelves, thereby increasing how quickly people get in and out of a store.  Regardless of the specific questions, the point is that the survey will produce statistically reliable data about individual units of analysis (or what I like to call “people”).  The questions it does not address are what people really do when they shop and why do they do it.  Understanding THAT requires thinking about the sample in terms of context, not individuals.

The number of individual participants involved depends on the relevant diversity of the target population. A skilled ethnographer may use multiple methods in the recruiting process and not rely only on professional recruiters. This different approach to sampling also means that sampling is often built as part of fieldwork, and refined once a team is on the ground and collecting data. This can, and often does, scare a client absolutely shitless.  After all, they are working from psychographic models and segmentation schemes, all the while worrying about which of the other stakeholders will call the work out as a way of currying favor or establishing greater power in the boardroom.  But, the fact is that while statistical work is valid in many respects, it is only one way of envisioning the world.  When faced with the complexities of human interaction, these schemas break down.  In practice, the power of an ethnographic process lies in uncovering unexpected patterns, not in reifying the segmentation work that has already been done. While an ethnographer will no doubt have specific sampling parameters from a client, they should also be able to articulate why sampling may change once the research begins. If they can’t, then you’ve wasted your money.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s