Metric Monday



For the past year, Metric Monday has focused on metrics that matter for communication and PR professionals through specific measures of inputs, outputs and outcomes across a range of industries.

In this next series, our attention turns from “what” we measure to “how” we measure.

So if you’re back in the era of focus groups, content analysis and paper surveys, it’s time for a refresh and a look at the new and emerging research methods available to communicators.


New ways to measure:

Neuromarketing is taking consumer behaviour research to a whole new level by looking at brain responses to stimuli in real time – making it equally relevant to communicators interested in understanding consumer behaviour.

This new field uses medical technologies such as functional Magnetic Resonance Imaging (fMRI) to study the brain’s responses to marketing stimuli.

Researchers using neuromarketing aim to better measure and understand consumer preferences as the verbal response to the question “Do you like this product or organisation?” may not always be the true answer.

Technologies measure changes in activity in parts of the brain to learn why consumers make the decisions they do, and what part of the brain is telling them to do it – with the results being more revealing than self-reporting on surveys or via focus groups.

Neuromarketing is providing powerful insights into a range of consumer responses associated with eye gaze, packaging, colour, comparing campaigns, decision-making, customer satisfaction and much more.

Not surprisingly, this innovative form of research comes with its own challenges including cost and credibility along with issues of ethics and personal intrusion.



Metric Monday Archives


New ways to measure:
Research gamification

Here’s another tongue twister! Gaming techniques are being used to improve the quality of feedback from online research. Gamification involves using gaming techniques in a non-game setting like research.

Games and stories go hand in hand. Games are almost always driven by a back story (even a simplistic one). Games channel human behaviour into structured rules; the information we can collect from a game is not limited to just the outcome of who won or lost. In some cases, the act of taking part in a game creates more value for a research audience than observing a game.

Gamification has already entered many parts of our online lives. For example, social media networks reward us with social approval in the form of “likes,” “shares,” or “comments.” The psychology of gamification is undeniable; everyone likes to be rewarded.

By taking this approach to research and rewarding and challenging respondents, researchers may begin to see more thorough responses, leading to deeper insights within their target audience.

The bottom line is that the engaging nature of gamification research prompts respondents to spend more time on each question, and challenges them to dig deeper. The result is higher quality responses while engaging a higher quantity of respondents.



#MetricMonday New ways to measure - Mobile ethnography

New ways to measure:
Mobile ethnography

If you can wrap your mouth around this research method, you’ll be off to a great start!

Put simply, ethnography is the practice of spending a long period of time with a subject group in order to better understand their culture, interactions, behaviours, decisions, thoughts and opinions. Because this is an extremely time-intensive process, the modern practice of self-ethnography is becoming popular.

In this method, the target audience member becomes the researcher. Participants are asked to record their thoughts, feelings, and decisions on a regular basis through guided journaling, shooting video from their mobile, responding to prompts, and other research methods.

If the target audience member or stakeholder has an interaction with your organisation or another experience that you’d like documented, it’s going to be to your advantage to find out about it right away. The longer you wait, the more likely they are going to forget that experience.

Collecting data via a smartphone means that your subjects can give you information any time. They can record a video and tell you about a billboard they saw on their commute to work. Or they can tell you how they felt about a particular subject on your website. These things can be done milliseconds after the event or experience, before your subject forgets any details.

No matter how your participants like to communicate, you can find a way for them to engage in self-ethnography. With all of the options available, you’ll be sure to find something that works well for your sample. You could have your participants send their data via emails, blogs, social media, or even by leaving a voicemail. You could ask for reflections, send short surveys, have a standard set of multiple-choice questions, use numerical rating scales…the possibilities are almost endless.



#MetricMonday New ways to measure - Text Analysis – Webcam interviews

New ways to measure:
Webcam interviews

Webcam interviews are in-depth interviews conducted online and usually last 45 to 60 minutes.

The participant and moderator see each other and the stimuli.

We can also share the respondent’s screen and watch them navigate a topic/website of interest online.

The client team views remotely, and can suggest prompts or clarifications to the moderator via a chat box.

Ideal uses and benefits include:

  • In-depth interviews are ideal for getting instant feedback from consumers on concepts, taglines, package designs and other stimuli
  • Respondents can participate from the comfort of their own environment (eg. home, office)
  • Participants can communicate with the moderator or other viewers in different locations via text boxes (eg., probing questions) while the group is taking place
  • No travel needed for respondents, viewers or moderator
  • They work well when participants are scarce and/or geographically dispersed, eliminating time zones as a factor.



#MetricMonday New ways to measure - Text Analysis – Webcam focus groups

New ways to measure:
Webcam focus groups

Webcam focus groups are focus groups conducted online in real time for a set duration.

They are typically comprised of four to five respondents and last 60–90 minutes.

Webcam focus groups provide the personal interaction of in-person research with the benefits of online.

Group discussions via webcam are best when a group dynamic is preferred, immediate facial expression is critical to ascertain, and/or you need participants to show or do something in real time.

A shared screen feature allows stimulus (storyboard, video, etc) to be shown and marked up by both respondents and moderators, and a polling feature allows respondents to answer select questions privately before further discussion.

In the age of technology, it’s important to make sure all focus group participants undergo a “tech check” to reduce the chance of technical mishaps and to ensure their setting offers optimal lighting.

One common misconception about webcam focus groups is that they are less expensive than other methodologies.

While they do eliminate travel expenses and save time, successful (i.e., seamless) webcam focus groups require a trusted platform, skilled technicians, and an experienced webcam moderator.



#MetricMonday New ways to measure - Text Analysis – Sentiment analysis

New ways to measure:
Text analytics – Sentiment analysis

Analysing the opinion or tone of what people are saying about your company on social media or through your call centre can help you respond to issues faster, see how your product and service is performing, find out what customers are saying about competitors, and so on.

There are three popular ways of going about this kind of sentiment analysis:

1. Polarity analysis, where you simply identify if the tone of communications is positive or negative.

2. Categorisation, where tools get more fine-grained and identify if someone’s confused or angry, for example.

3. Putting a scale on emotion from ‘sad’ to ‘happy’ and from 0-10.

Affective Norms for English Words (ANEW) is a useful tool for emotional ratings and allows communication to be identified in more detail, such as mild concern or somewhat angry.

WordNet is another tool that relates words similar to each other, such as synonyms and antonyms, and allows users to build classification schemes using that semantic information to do semantic analysis.

A word of caution when it comes to sentiment analysis – picking up on sarcasm and irony remains a challenge especially with tweets and social media where people are ironic and sarcastic as a way of getting a message across.



New ways to measure:
Text analytics

In a profession focused on words, communicators should be totally au fait with methods for analysing the output of our work.

Text analytics is the process of deriving information from text sources. In fact, it is pretty much as old as the technology of language itself. Traditionally, people just called it “reading.”

The difference is that now, software can read text as easily as human beings, not to mention much more quickly. With text analytics, you can spot patterns in massive collections of textual data that an individual human mind could never detect. It’s used for several purposes:

  • summarising: trying to find the key content across a larger body of information or a single document
  • sentiment analysis: what is the nature of commentary on an issue
  • explicative: what is driving that commentary
  • investigative: what are the particular cases of a specific issue
  • classification: what subject or what key content pieces does the text talk about.

There are a variety of free tools to get started with including the Natural Language Toolkit and TextBlog in Python.



#MetricMonday New ways to measure - Facial Analysis

New ways to measure:
Facial analysis

Understanding emotions is powerful in research but extremely difficult to achieve. So it’s not surprising that facial and emotion recognition is an emerging area of technology development in communication research to test key messages and advertisements.

Brief flashes of emotion displayed on a respondent’s face – or “micro-expressions” – are understood to reveal a person’s beliefs and their propensity to act in a particular way. Emotion detection technology is being used to analyse people’s emotional reactions at the point of experience. This knowledge not only gives researchers a greater understanding of behaviour patterns but also helps predict likely future actions of those consumers.

The specific approaches vary but they all capture video and then analyse it for facial movements that correspond to emotions, typically based on the human-observed system called Facial Action Coding System (FACS).

The result is an unprecedented level of insight into what triggers customer emotions.

There are some issues to manage. First, people from different nationalities and cultures have different levels of emotional response and different facial structures, so benchmark data needs to take this into account.

A second issue is content delivery. While many people are now used to engaging with video content through a variety of media, facial recognition technology requires a two-way view. On a mobile device, chances are that a respondent moving around may hinder the technology’s ability to read critical facial expressions.

Finally, there’s the privacy issue which can be easily solved by asking respondents for permission to access their webcam and record their faces while they’re watching your content.



#MetricMonday New ways to measure - Eye tracking

New ways to measure:
Eye tracking

Eye tracking is a unique method to objectively measure consumers’ attention and spontaneous responses to communication and marketing messages. These insights help to effectively design communication to catch the consumer’s eye.

The trend to capture subconscious and unbiased data through implicit methods is growing and eye tracking is among the most effective of these techniques.

Eye tracking lets you see how consumers react to different marketing messages and understand their cognitive engagement, in real time. It minimises recall errors and the social desirability effect while revealing information conventional research methods normally miss.

Eye tracking can help academic and commercial researchers discover:

  • What communication elements capture the eye of the consumer?
  • Which parts of the communication consumers focus on and which parts are ignored?
  • What drives the decision-making process?

Implicit research offers the benefit of objective measurement. Real behaviours (such as actual purchases or movement through a physical or online store) are recorded, providing researchers with the data to substantiate more valuable insights into consumer behaviour. These methods are becoming standard for ad measurement and package design. Eye tracking fits into this overall trend as a powerful implicit method.

Advancements in wearable eye tracking allow for new types of studies that were not possible previously due to cumbersome technology. Now, researchers can equip subjects with eye tracking glasses and better understand how a person interacts with different messages and channels in any environment.

The use of eye tracking now opens up the possibility of measuring authentic behaviour from the point of view of the subject. This adds a new perspective and unique insights into how people perceive their world and behave within it.



#MetricMonday New ways to measure - Wearables

New ways to measure:

From Fitbits to the Apple Watch, Google glasses, smart clothes and jewellery, wearable devices are transforming the way research is carried out.

The most exciting development is that wearables are allowing communicators to join the customer research journey early in the process, giving time to focus on the whys behind the behaviours we are aiming to change instead of the whats.

For example, smart fitness devices can provide data on daily activities, heart rates, and sleep patterns. Smart watches can track media usage. They can also help to discover where someone was before and after he or she visited your bank, outlet or organisation, as well as how much time was spent in each place.

Beyond the wearables data, you can gain qualitative insights by asking relevant questions at the end of each day or week through any form of SMS, text, or email.

Collecting this data in this way offers advantages for respondents and researchers.

For respondents, the experience is seamless and unobtrusive – no long surveys to take, no fatigue, and no chance of having a fuzzy memory and incorrectly reporting their actions.

For researchers, wearables provide research without “doing” research saving time and money and providing a high level of accuracy.

As with all research, it’s important to consider the privacy concerns surrounding this type of passive data collection, but many consumers are growing more comfortable with the idea of granting access to some of their data in exchange for a benefit – perhaps a free wearable device.