By Kevin Gray
Some populations are hard to survey because they are hard to find, others because they are hard to sample and still others because they are hard to persuade to participate in surveys. Some are simply hard to interview. Examples of hard-to-survey populations include migrants, immigrants, homeless people, persons with intellectual difficulties, the visually impaired, drug users, political extremists, sex workers and people in difficult settings such as war zones or homeless shelters.
These sorts of populations usually do not figure in marketing research. However, some of us conduct research for national or local governments or NGOs that involves hard-to-survey respondents. Other examples are medical and pharmaceutical projects, which may include interviews with people suffering from conditions that make it difficult for them to be interviewed. Business leaders and key decision makers also can be challenging cases and young single males are notoriously hard to reach. Though GPS, satellite imagery and the diffusion of mobile devices are making it easier to conduct research in remote areas, it remains challenging.
I should make clear that I’m using “survey” to include qualitative research and ethnography, not only large-scale quantitative studies. Though I cannot claim to be an authority on this subject, I have had quite a lot of experience over the years with some kinds of hard-to-survey populations in connection with B2B and medical/pharma studies. I’ve also worked on many projects in developing nations and on projects in which fieldwork has been disrupted because of civil unrest or natural disasters.
Based on the expertise that I do have I can recommend Hard-to-Survey Populations (Tourangeau et al.) to those of you who also are involved in projects such as the ones I have mentioned. This is a hefty, wide-ranging volume with more than 70 contributors from a number of countries, and the only book I know that addresses this topic comprehensively and in such detail. Some who do, or will be doing, cross-cultural research may also benefit. While the book is not specifically about that topic, hard-to-interview populations are by definition culturally “different” and in the book we are frequently reminded of the need to be sensitive to cultural and linguistic subtleties.
Rating scale usage styles vary considerably across the globe, to give one example. I still hear surprise that NPS or Employee Satisfaction scores differ so much from country-to-country. Another example would be mixed-gender group discussions or focus groups with both older and younger participants taking part in the same sessions, which many Westerners assume are acceptable anywhere in the 21st century. Moreover, concepts that are entirely clear to someone from one background can be ambiguous or meaningless to people from different cultures. Some ideas cannot be translated, except very approximately.
Reading the book I was reminded of the implicit assumptions marketing researchers often make when conducting studies in countries other than their own. One should not assume that because someone speaks your native language fluently and attended graduate school in your country that he or she is “just like us.” Many marketing researchers appear to be unaware of these important considerations. Working globally as I have for most of my career, I tend to be sensitive to them but, conversely, can fall into the trap of taking for granted that something that is plain as day to me will also be obvious to other Westerners!
Total survey error arises from various sources, such as coverage, sampling, non-response and measurement errors. It affects any kind of survey research but, as one would expect, is more of a concern with rare populations. True probability sampling is hard to achieve in run-of-the-mill consumer research but even more challenging with hard-to-survey populations. Sampling frames usually do not exist and little data are available about the population, this being one reason the research is being conducted in the first place. Hard-to-Survey Populations covers sampling in some detail. Disproportionate stratification, location sampling, adaptive cluster sampling, capture-recapture methods and snowball sampling are some of the many approaches covered. Obviously, care needs to be exercised and none of these methods is universally suitable. Ask a mobster, for instance, to introduce you to other members of his network and he will probably be very disappointed.
Potential respondents may be traumatized by hardship and unwilling or unable to cooperate or respond to questions accurately. Others, such as gays, illegal immigrants or drug users, understandably, may not wish to be identified. Randomized response is one method useful for sensitive topics. Some respondents may be suspicious of the purpose of the research or, conversely, try to game the interviewer, thinking they or their family will receive assistance from the government or an NGO if they answer in a certain way. On the other hand, some may be concerned about losing benefits if they participate in the research out of fear that the authorities will assume they’re alright and not in need of help. Monetary incentives may not always be appropriate and can also place interviewers at risk in certain situations.
Establishing a rapport with the respondent is especially important and social exchange theory even more pertinent than in typical interview situations. It is sometimes desirable to recruit and train members of the rare population to become peer interviewers. Special attention must be paid to questionnaire design as well as issues of custom, language and dialect. Illiteracy rates, something marketing researchers would not normally think about, are high in certain populations. People with learning difficulties are another example of a hard-to-survey population, and technologies such as audio computer assisted self-interviews can come in very handy for respondents with special disabilities. Multi-mode/method approaches are common when surveying atypical populations, but there is no one-size-fits-all strategy and the research must be tailored to the specific group or groups targeted by the research.
Though not a light read I did not find the book at all boring, and it would be a good resource to have on hand if you ever work with hard-to-survey populations. At the very least, it’s a strong reminder not to lose sight of some important things that should be obvious to researchers but are all too easy to neglect in today’s frenetic workplace. Perhaps the most important of these is to try to put yourself in your respondents’ shoes.
Kevin Gray is president of Cannon Gray, a marketing science and analytics consultancy.
By Marina Cozzika
On my way to ESOMAR’ sensory forum perched on high heels, trying to avoid the pitfalls of Paris’ pavements, holding a hot cup of tea with one hand while texting with the other, smelling with envy hot croissants and bread just baked, annoyed by the sound of cars passing by and enjoying the wonderful view of the Eiffel Tower against a clear blue sky, I thought to myself that Paris was the perfect city to enjoy a multi-sensory experience. My senses were stimulated, working in synaesthesia, an experience that lasted all day at the ESOMAR sensory forum. The day promised “an open dialogue for insight specialists and sensory scientists to connect and exchange innovations across methodologies and marketing, all geared to demonstrate the business impact of uniting multi-sensory expertise”. Did it live up to its promise?
Methodologies: Where we are now, challenges and opportunities
We started the day with Herb Meiselman who gave us a good view on how sensory and marketing have evolved throughout the years from where we are now, the challenges and opportunities we have as an industry. He stressed that we have to keep a balance between laboratory research and real-life research. Clearly it’s a difficult path to walk in sensory as both controlled studies and real life testing bring complimentary diagnosis. So is one more right than the other? Furthermore, the industry has changed a lot with the internet and the growth of home-use testing. He also pointed out that we need to develop methods that work globally and in the process identify local consumer issues, whilst uncovering global solutions, not an easy task.
Another key issue in sensory highlighted by Herb is the use of trained panels vs consumer panels. Whilst trained panels will contribute with clarity thanks to their rich semantic stock of sensory descriptors, they shall not be the reference for hedonistic scores. This is when consumer panels are needed. The situation is now in a state of flux as new groups of statistically based methods are making consumers panels more applicable. Companies are now testing consumer panels for sensory descriptions and trained panels are being asked to predict the liking scores of consumer panels. Herb Meiselman does not know how it will all turn out. What’s clear however is that now broader measures are being used like “satisfaction”. An encouraging shift as the industry is getting deeper into emotion measurement.
After Herb’s fundamental presentation, followed a flourish of complementary papers and discussions. Diana Derval of DervalResearch regaled us with a 360° presentation that covered – in a most entertaining way – all senses. Specific sense experts followed, from sound with Laurent Cochini of Sixième Son, to scent with Arnaud Montet of IFF, to Massimo Cealti the voice of the consumer or as he described himself, the dark side of the force, culture, and an original and innovative approach to sensory matters through a semiotic perspective by Fiona McNae of Space Doctors who uses culture to make sense of the world around us.
Connecting culture and sensations
Fiona McNae’s presentation brought us back to the importance of popular culture, an importance currently seized by two fascinating exhibitions currently taking place in London, the Tate Sensorium that draws together experts in sound, taste, scent and touch to bring the visitor a multi-sensory experience linked to four paintings, and the Alcoholic Architecture where you can walk through a cloud of gin tonic (note to myself, plan a trip to London!). Our industry saw a shift from the corporeal, the physicality of sensation, towards the abstract, but now it’s shifting back to the corporeal. A brand only exists in the heart and mind of the consumer through the senses: how it looks, how it smells, how it talks… Lush is a good example of a brand cohesive from the way it was designed, the way you are welcomed into the store, the sound you hear, the service you get…and on the receiving end of the brand experience. Why is it a success? The more sensory you put in your brand, the more likely you are to connect with your consumer. You will build stronger memories, rituals, recognition and in the end, preference.
We were further enlightened through concrete case studies that respectively encompassed all senses.
We learnt that taste is an important sense as the taste buds on our tongue determine the love/hate relationship we have with food and beverages. Some of us have 10 taste buds, some have 1000, they determine the intensity of taste that we experience and divide us into three profiles, the super taster, the medium taster and the non tester. Diana Derval used a Red Bull case study to make her point. The brand first targeted night-clubbers but soon found out that 70% of their cans were sold at extreme sports events. Research showed that most participants had a strong immune system (so they are “extreme sports” ready), which makes them non tasters and appreciate Red Bull. People with weak immune system have super taste buds and seem most likely not to like Red Bull.
The function of sound is important to create an emotional connection with a brand. Laurent Cochini of Sixième Son presented an interesting case study, the audio branding work they did for SNCF France’s national railway company. Hear it here. Four year after the launch, research shows an impressive brand attribution. 92% respondents recognize the brand after hearing the Audio logo, 88% recognize it after just two notes, 71% feel attracted/very attracted to the brand.
Our eyes have many receptors but all are not equal and depending on how much we have, we perceive color or contrast differently and with more or less subtlety. Research done by Derval for Ubisoft’ Assassin’s Creed to understand the profiles of gamers showed that gamers see very well contrast and the nuances of the same color. So Ubisoft found out that there was no need to add too much color to their videogames.
Arnaud Montet of IFF raised the question of how to bring rationality into the emotional world of fragrances with the Aqua di Gioia case study. The brief was to develop a feminine fragrance under the umbrella brand of Aqua di Gio. The concept was to convey joyfulness. The challenge was to maintain a level of freshness and create something engaging. IFF’s strategy was to connect with the consumer, to define it and translate it into an olfactive signature using qualitative research to define the emotional territories of freshness. The result: a success story.
We all have sensors located in the lines in our hands and in our body. This enables us to perceive touch, renders us sensible to cold or heat… we all produce vibrations that we perceive through our hands and process with our senses. There are three types of touch profiles: the super vibrator, the medium vibrator and non vibrator. If you are a super vibrator, chances are you need to touch to feel the product because of the sensitivity of your skin. A make-up brush by Sephora was first presented in a packaging that didn’t allow consumers to touch it. It did not sell! Research showed that consumers needed to touch the bristles to experiment the thickness of the brush. So packaging was changed.
All these examples and explanations have left my brain buzzing and my senses more alert to decode my environment. Sensory marketing and by extension sensory research clearly need to be championed by brands and manufacturers. They clearly leave a strong imprint.
Marina Cozzika, Added Value, France