Finding Balance: Data, FIeldwork, and Creativity

There is perhaps nothing new about the ongoing battle between data and qualitative work, and the influence they have on creativity and design. Data is everything, creativity is dead vs. the argument that creativity is paramount and data is a distraction. Neither position is true, though there is some truth in each argument. The goal is to deliver insight that inspires creativity, regardless of the methods by which we gain those insights. The central need is to determine how data and inspiration work together to drive change.

As advertising, marketing, and design come to rely more on technology, we are forced to reconsider what constitutes creative quality. It also means being honest with ourselves and recognizing that data is not a panacea. It, like qualitative work, is part of a thinking process that helps identify the underlying story we need to divine and craft tools that inspire action. At times that can be found in the data alone, but more often it’s found among outliers. Without the two sides working hand in hand, we get half truths.

For marketers, nothing could better define both the essence and preeminence of creativity than empathy. We all recognize the pace of technological change and changing customer behaviors. And we all recognize there is tremendous opportunity in being able to derive greater targeting from the data we collect. But behavioral measurement shouldn’t lull us away from using the creative process to intuit what customers will experience, whether we’re trying to convince them to take an action or building a tool to meet a need. Data underpins everything, but meaningful success will come to those who can augment data with a deeper understanding of the audience. What role does symbolism play? What metaphors connect? How does the object we create make sense in their lives? These are the sorts of things we come to understand through deep immersion.

As an example, some years ago I did work on a medication used in treating schizophrenia. Based on the success rates and data collected about patient behavior, it should have been an easy product to market. However, the sales were flat. It wasn’t until we began examining the process of schizophrenia that we were able to tease out where the problems were. Access to transportation, difficulties with case management, distrust of the psychiatric community, and the role of friends and family all had a significant impact on how the medication was understood. This wasn’t the sort of thing you could get at via data analysis. And yet, using the two methods together allowed the team to develop creative work that resonated deeply and was targeted at the right place and the right time.

What we need to be doing is rebooting brand planning as a qualitative and a quantitative art. What designer, strategist, etc. tasked with building a tool or developing an engaging brand experience wouldn’t want to know a bit about how the audience for their art behaves? How they engage with content? How they engage with a device? But the trick is not getting caught up in the numbers at the expense of the human being behind them.

Advertisements

From Personas to Stories: Creating Better Tools for Design and Marketing

Design ethnography takes the position than human behavior and the ways in which people construct meaning of their lives are contextually mitigated, highly variable and culturally specific. on the central premise of ethnography is that it assumes that we must first discover what people actually do and why they do it before we can assign to their actions and behaviors to design changes or innovation. The ultimate goal is to uncover pertinent insights about a population’s experience and translate their actions, goals, worldview and perspectives as they directly relate to a brand, object or activity, and the role that these pieces play with regards to interactions with their environment. Often, the information results in a large-scale, broad document, but it also often results in the development of personas.

The idea is that personas bring customer research to life and make it actionable, ensuring the right decisions are made by a design or marketing team based on the right information. The approach to persona development typically draws from both quantitative and qualitative tools and methodologies, but because of the very personal nature of ethnography, the methodology often leads the charge. The use of ethnographic research helps the creation of a number of archetype (fictions, in the most positive sense) that can be used to develop products that deliver positive user experiences. They personalize the information and allow designers and marketers to think about creating around specific individuals.

But there are problems with personas. Don’t get me wrong, I believe personas can be useful and help design teams. But I also believe they can reduce the human condition to a series of attributes and lose the spirit of what personas are designed to do. First, in terms of scientific logic, because personas are fictional, they have no clear relationship to real customers and therefore cannot be considered scientific. So much for the science.

For practical implementation, personas often distance a team from engagement with real users and their needs by reducing them to a series of parts. The personas, then, do the opposite of what they are intended to, forcing design teams down a path that gives the illusion of user-centricity while actually reflecting the interpretations or the individual designers. Creating hypothetical users with real names, stories and personalities may seem unserious and whimsical to some teams within an organization and be, consequently, dismissed as so much fluff. But by far, the biggest problem, at least to my way of seeing things, is that while we want to use personas to humanize potential customers and users, we in fact reduce them to objects and a laundry list of actions, personality quirks and minimalist descriptions.

I’m not advocating the dismissal of personas, but I am suggesting that perhaps there are alternatives. One place to start is to admit we are writing fiction when we construct these tools and expand upon that notion. We should be adding to the mix humanistic narratives. Customer novellas, so to speak. It requires more time and effort, both on the part of the person/people creating them as well as those using them, but it also gives greater depth and insight into the needs, beliefs and practices of the people for whom we design and to whom we market. Rather than relying exclusively on a dry report or a poster with a list of attributes.  In this model, the idea is to create a short story in which actors (the eventual personas) engage with each other, a wider range of people, and a range of contexts. Doing so allows us to see interactions and situations that lead to greater insights. It allows us to look at symbolic and functional relationships and tease out elements that get at the heart of the fictional characters we create.

Why is that important? Because it does precisely what personas are meant to do but typically fail at – provide depth and characterization, establish a sense of personal connection between designers and users and provide breakthrough insight and inspiration. Anyone who has read history vs. historical novels is familiar with the idea. It is easy to reduce Julius Caesar to a series of exploits and personality traits, but in doing so we lose the feel for who the man was. A historical novel, in contrast, adds flavor by injecting conversation, feelings, motivations and interactions. We walk away with a feeling for who he was and what affect he had on others, good and bad.

Imagine developing a persona for Frodo from The Lord of the Rings. We could say the following and attribute it to all Hobbits: Frodo is enamored by adventure but frightened by it. He loves mushrooms, has no wife, is extremely loyal to his friends and will work at any task he is given until it is done, regardless of the difficulty or potential for personal harm. He disdains shoes and has a love of waist coats.

There’s nothing wrong with this description, but for anyone who had read the trilogy or even seen the movies, the shortcomings are obvious. We miss the bulk of Frodo’s personality. In exploring the novel, we come to develop a rich understanding of Frodo, a deep understanding of his motivations and personality and his relationship with other members of the party, including the Ring.

For the literalists out there, I am not suggesting we create anything as vast as a novel, particularly one as expansive as The Lord of the Rings, but I am suggesting that we move beyond attributes and create stories that more fully develop the people behind the personas. Several pages of engaging writing is sufficient. Not only does it provide deeper insights, but it engages the reader more fully, inspiring them to go beyond the “data” and explore a wider array of design, brand and marketing options. Again, it isn’t meant to replace personas (or the research report), but to add to it. It requires more effort and time on the part of the person creating it as well as the person consuming it, something people are often disinclined to do, but the end result is better design, greater innovation and a more complete vision of what could be.

Why Training Matters for Good Ethnography

Ethnography is a powerful tool, but it’s being so watered down as to become nearly meaningless in many cases. What ethnographers do, or should do, is uncover meaning and complexity. There is, frankly, a lot of crap being produced by so-called ethnographers. Being able to conduct a good interview does not make a person an ethnographer anymore than being able to balance a checkbook makes someone a mathematician. It comes down to being able to talk about depth of knowledge and make connections that others overlook.  Not everyone is a painter and we accept that. Not everyone is an ethnographer. While it may come across as arrogant, that is not the intention. The point is to say that what we learn from training and experience has value and while the goal in the current economic climate is to be good, fast and cheap (something that can, in fact, be attained), it is ultimately none of these things if the end work is not grounded in solid methodology or training. This isn’t to say one needs a PhD in anthropology from the Harvard or the University of Chicago, but it is to say that simply calling yourself an ethnographer doesn’t make it so.

And to be fair, there are times that it is possible to be, or at least appear, too academic. It is a criticism well deserved. Don’t get me wrong, I admire the output and thinking depth of academics, but in a business context it’s difficult make the transition. They are not trained to think in business terms — they simply don’t speak the native tongue. Some are tossing that perspective out the window as much out of necessity as anything else. Some anthropologists, both in and out of academia, I think, are afraid of losing their “anthropologist” identity. That can be a tremendously threatening thing. Anthropologists started as rogue methodologists in many ways, developing theories and barrowing methods in order to get to a deeper truth. They no doubt need to return to that in all areas of anthropology, but especially on the applied side. People like Boas were looking for understanding the human condition in the broadest sense. By 1960 it was about defining the discipline.

But returning to the original point, a solid academic grounding in behavioral and cultural theory is imperative to doing the job well, whether it’s in helping create a marketing plan or designing a new product. Simply taking into account what people tell you in an interview is misleading and often dangerous. For example, if participants tell you that they make a point of eating dinner every night as a family, it would be easy to take that information and build a marketing plan or product around that statement. The catch is it doesn’t address the unsaid. How much clutter is on the dining room table? What discarded boxes are in the garbage? What is the weekly schedule of activities? Are the kids there when the fieldwork takes place at 6:00 p.m.?

At a deeper level, the underpinnings of meaning are lost. What are the various meanings of “family” in a given context. How is dinner time used to establish or co-create meaning? What is the symbolic role of food? How does ritual factor into purchase and preparation choices? How does that carry over in the store?  These are the host of observational data points that are frequently overlooked by researchers who lack a theoretical grounding. Now imagine what it means to lose that depth of understanding when designing something as complex and expensive as a new type of car. If you get it wrong, you may well waste millions going down a rabbit hole. Regardless of the product, service or message you are designing it makes a great deal of sense to have a research team that can get at these issues and translate them into meaningful insights. Business anthropology represents the synthesis of academic anthropology with the professional practice of marketing and design. It seeks to understand what it means to be human, the diversity of human practices and values, and then turn these practices and values into tangible experiences. Getting it right means getting the right people.

Getting Past the Hawthorn Effect

In 1924, the National Research Council sent two engineers to supervise a series of industrial experiments at a large telephone-parts factory called the Hawthorne Plant near Chicago. The idea was that they would learn how shop-floor lighting affected workers’ productivity. Instead, the studies ended up giving their name to the “Hawthorne effect”, the notion that that the act of being observed or experimented upon changes a subject’s behavior.

The theory arose because of the unexpected behavior of the women who assembled relays and wound coils of wire in the plant. The data collected during the study demonstrated that their hourly output rose when lighting was increased, but also when it was dimmed. Simply, as long as something was changed, productivity rose. Out of this arose the notion that as long as the women knew they were being observed, there would be a behavioral change.

But Steven Levitt and John List, two economists at the University of Chicago, decided to analyze the data, which was still available, and see what they found. Contrary to the descriptions in the literature, they found no systematic evidence that levels of productivity in the factory rose whenever changes in lighting were implemented. Now that was unexpected.

It turns out that idiosyncrasies in the way the experiments were conducted may have led to misleading interpretations of what happened. For example, lighting was always changed on a Sunday, when the plant was closed. When it reopened on Monday, output duly rose compared with Saturday, the last working day before the change, and continued to rise for the next couple of days. But a comparison with data for weeks when there was no experimentation showed that output always went up on Mondays. Another of the original observations was that output fell when the trials ceased, suggesting that the act of experimentation caused increased productivity. But the experiment stopped in the summer, and when examining records after the experiment stopped it turns out that output tended to fall in the summer anyway.

It’s all very interesting, yes, but why does it matter?  It matters particularly to ethnographers because one of the central criticisms of the methodology is that our presence negates any of the findings on the basis that we alter the behavior of our participants.  As it turns out, the problem may not be as notable as the critics claim.

I will be the first to admit that our presence does shape the interactions and behavior of the participants, but only in a limited way, and those ethnographers worth their weight in salt are able to establish rapport in such a way that changes are minimal. Time is, of course, the driving factor in this. Participant observation, the foundation of ethnography, refers to a methodology in which the researcher takes on a role in the social situation under observation. The social researcher immerses herself in the social setting under study, getting to know key actors in that location in a role which is either covert or overt, although in practice, the researcher will often move between these two roles. The aim is to experience events in the manner in which the subjects under study also experience these events. Success is defined, in many respects, by the nature of the relationship that develops. As such, a good ethnographer becomes another actor rather than simply an observer, thus largely negating or minimizing the changes subjects display.

What this means for the researcher is that conducting ethnographic work means doing more than interviewing. It means learning to conduct research that involves a range of anthropologically-informed tools. For the buyer of researcher, it means questioning your vendor, thinking through what they propose and be willing to do research in a way that may make you initially uncomfortable – digging through the dirt with an HVAC installer or bar hopping with a twenty-something through NY may seem a little daunting at first, but these are the things that make for good research and, more importantly, good insights.

 

Video as a Replacement to the Ethnographer

Video is one of the most important and effective ways of communicating research findings. As such, video is often used to convey participant stories and communicate ethnographic findings. Increasingly, video has become a substitute for note taking and in some case, it has essentially been billed as a cheap, quick alternative to fieldwork.

But it isn’t a replacement for fieldwork and the trained ethnographer, regardless of what some might say. Claiming that it can do what fieldwork can do is akin to saying that hotplates can replace all other modes of cooking – in some instances that’s true, but not when you’re talking about cooking a meal for multiple people on a daily basis. Of course the analogy isn’t perfect here, but it hopefully conveys the point that while video ethnography is part of the tool kit in qualitative research, claiming it can replace ethnographic fieldwork is misleading and, well, often flat wrong.  Video is a tool. As with any tool, knowing when and how to use it is pivotal to its success. And while anyone can use a hammer, in the hands of a professional carpenter, the results will probably be superior to those of the average person.

So what do I mean when talking about video ethnography. Video ethnography is the recording of the stream of activity of subjects in their natural setting, in order to experience, interpret, and represent culture and society. At least, that is what it has meant.  Unlike ethnographic film, it cannot be used independently of other ethnographic methods, but rather as part of the process of creation and representation of societal, cultural, and individual knowledge.  Uses of video in ethnography include the recording of certain processes and activities, visual note-taking, and ethnographic diary-keeping. Video is not a replacement for fieldwork or the fieldworker. There are a couple of reasons for this. First, assuming that putting a camcorder in the hands of a participant and thinking they will capture everything needed for analysis assumes that the participant isn’t self-selecting. People record what they want, not what you need – context is often overlooked, unpleasant or uncomfortable situations are omitted, and the subjects of the video are driven by the participant’s biases. Second, using participant video as a substitute for the ethnographer on the ground means that the right questions to ask rarely emerge. It is like the story of the three blind men and the elephant. We end up with only the tail and base or analysis and recommendation on a small portion of the observed rather than the whole.  So, without accompanying fieldwork the video is of limited value and may yield conclusion that are misleading of flat out dangerous.

Video ethnography involves:

  • Observation, including extensive filming of practitioners
  • Allowing practitioners to view the video recorded material and reflexively discuss their practice
  • Building the capacity for the ongoing and critical appraisal of practice

Video-ethnographic methods seek to foreground practitioner knowledge, expertise, and insight into the dynamics of their own work processes. This is achieved by first talking with practitioners about their beliefs, structures, work and organizational processes, and by seeking an articulation of the social, professional, environmental, and organizational contingencies that both enable and constrain their practice. By allowing practitioners to discuss their practices in response to video footage researchers gain insight into areas of practice that may be benefit from redesign. Video ethnography is contingent on the researcher gaining the trust of practitioners, on becoming familiar with the site and on being trusted to be present at time and in places where critical conducts are undertaken. And that combined, collaborative structure of the research design is what produces real insight.

Despite the new rhetoric of empathy and inclusiveness, of involving the user and understanding people’s needs, the person pointing the camera still occupies a position of authority in relation to the subject. This is no less real just because it is concealed beneath a soft blanket of warm feeling. Whether the camera is held by the practitioner or the subject/researcher, the fact remains that even in an increasingly video-centric world, the camera is still an intrusion, altering the situation.  This is why we occasionally turn the camera off – seeing the changes that emerge when recording is off is as important as what we capture on film. So eliminating the researcher from the field equation means relying on a medium that is fraught with unresolved issues as subjects of the video negotiate power and meaning. In other words, if the camera is all you have to go on, especially if there isn’t even an ethnographer using it, there will people an enormous number of misleading statements and representations.

So what am I suggesting? It’s rather simple. Anyone saying they can produce ethnographic research and analysis without the use of an ethnographer in the field is selling a bill of goods.  It is cheap and fast, but yields information that is decidedly limited. As a tool in the larger project it has become indispensable, but as a replacement it is lacking.  In an era of budget cuts and the ever-present need to get insights quickly, it is tempting to look at something like video ethnography as it is often being billed (i.e. putting cameras in the hands of participants and leaving it largely at that) as a viable alternative to more complete research. But sometimes cheap and fast simply don’t make the grade.  For a marketer or designer, the question becomes, are the upfront savings worth getting your product or message wrong?

Writing Your RFP Response

Landing a new client is, as everyone knows, a difficult process. For every ten RFP responses your write, only a handful will result in actual work. And when you consider the effort that goes into building an RFP response, that means many hours of work that result in very little payoff. But it’s important to remember that the RFP is not about the researcher, it’s about the client. Writing a proposal, then, is more than a matter of research plan design, it is the first step in a courtship.

The proposal is, in effect, an intellectual (not legal) contract between you and the person or people asking for work to be done. It specifies what you will do, how you will do it, how you will interpret the results, and how you will make them useful for the client. In specifying what will be done it also gives criteria for determining whether it is an actuality done. In approving the proposal and awarding you the bid, the client gives their best judgment that the approach to the research is reasonable and likely to yield the anticipated results.  And those results are about more than the research – they are about making the client look good in the eyes of their peers.  The research may in fact demonstrate the smartest, most productive, most innovative work imaginable, but if it doesn’t address the explicit and implicit needs of the client, it won’t get off the ground.

The objective in writing a proposal is to describe what you will do, why it should be done, how you will do it and how it will benefit the client. A vague, weak or fuzzy proposal can lead to a long, painful, and often unsuccessful bid A clean, well thought-out, proposal not only secures a job, it forms the backbone for the long-term relationship with the client.

Proposals help you estimate the size of a project. Don’t make the project too big or too small – make it fit the needs of the client. This isn’t to say that the quality of the work should be compromised or that there’s no room for adding imaginative elements to the work, but it is to say that it needs to fit the scope of what a potential client has asked for.  If they need a needs assessment and only have $50K to spend, then writing a proposal based on a lengthy, expensive process will be fruitless. If you’re unwilling to do the smaller job, walk away. The client will remember that frankness when they do have the time and money to conduct a project that fits your recommendation. The key point is simply this; write the proposal that fits what they’ve asked for first.

Proof read your proposal before it is sent. It is a simple enough task that is too often overlooked.  Many proposals are sent out with idiotic mistakes, omissions, and errors of all sorts. Having been on both sides of the vendor fence, it is amazing how quickly a simple mistake can destroy all credibility. Clients have seen proposals come in with research schedules pasted directly from other proposals unchanged, with dates, prices and methods that are clearly irrelevant research tasks. Proposals have been submitted to the wrong person at the company. Proposals have been submitted with misspellings in the title. These proposals were, of course, not successful. Stupid things like this kill a proposal. It is easy to catch them with a simple, but careful, proof reading. Don’t spend six or eight weeks writing a proposal just to kill it with stupid mistakes that are easily prevented.

Finally, no matter how much experience, training, and expertise you have, everyone retains a bit of skepticism. “Does this guy really knew what he was talking about?,” is the question on the minds of those involved in the selection process. Remember that knowing your stuff isn’t enough. You will need to answer questions, make changes and compromise.