This case study details an efficient blended approach, using surveys + interviews powered by our own solutions, to seamlessly combine regular, meaningful performance benchmarks with a deeper customer insight behind the numbers.
We followed up with select respondents who had completed a Voice of the Customer (VOC) tracking survey, with a focused one-on-one webcam interview. These interviews provided a more thorough picture of the customer experience and the sentiment behind their survey responses. Using our latest technology solutions, the joint product and marketing team delivered a quick and seamless research-to-insight lifecycle, compared to traditional uses of quantitative and qualitative research methods.
Surveys Alone Don’t Always Reveal the Full Picture
Like many companies, we at FocusVision regularly run VOC surveys to track customer perceptions of our brand and product offerings.
It is an always-on research approach that needs to be efficient and meaningful to remain part of the fabric of our business and product decisions. These surveys typically include customer loyalty metrics: for example, an NPS (Net Promoter Score) question or attribute ratings on key loyalty drivers(1). The surveys are short, in deference to a customer’s time. However, this means that a limited amount of information is collected and at times is not sufficient to make informed business decisions.
What is a Net Promoter Score?
A Net Promoter Score, or NPS, is used to measure customer experience, such as the loyalty of customer relationships, and helps predict business and revenue growth. Because of its success in the business world, the Net Promoter Score is considered the core measurement of customer experience management programs around the world.
The Net Promoter score is an index that ranges from -100 to 100 and it measures the willingness of customers to recommend the products or services of a company. The scores help identify the overall customer satisfaction and loyalty to the product/service and brand.
An example question when calculating Net Promoter Score might look something like this:
On a scale of 0 to 10, how likely are you to recommend this company’s product or service to a friend or a colleague?
Based on how they answered, the customers are then classified into three categories: detractors, passives, and promoters. The Net Promoter Score (NPS) is determined by subtracting the percentage of customers who are detractors from the percentage who are promoters. What is generated is a score between -100 and 100 called the Net Promoter Score.
For example, we ask respondents to rate their FocusVision Decipher survey software on “ease of use.” But what does it mean when the customer gives an “ease of use” rating of a 5, or any number for that matter? Which features allow them to work quicker? Is it easy to learn? Exactly where may they be having a problem? Do they have to troubleshoot error messages? Do they find the user interface confusing or simple?
To help flesh out the selection results, researchers often provide an open-ended question that allows the customer to provide more elaborate feedback. However, our team often finds that less than half of respondents actually write something. And when they do, the response often lacks the detail to provide a full picture. (Figure 1).
So, while the VOC survey provides a good aggregate benchmark of where the brand stands, it may not deliver a full picture of what is/is not connecting customers to the brand and where specific improvements can be made. In other words: How do they feel? Why do they feel that way? What are their individual stories and experiences?
Going Deeper with Webcam Interviews
Really getting to know your customer means you have to sit down and talk with them, ask about their experience, and listen. Webcam interviews are the perfect way to achieve this, and they help flesh out more details and insights related to survey results. Customers will provide more elaborate feedback during live 1:1 interviews compared to surveys, plus you can see their expressions, body language and hear their tone of voice — and sometimes these indicators are more insightful than the words they use. Using live video, interviewers can engage in a dialogue, flesh out the details of a customer’s story and experience, and probe further if body language indicates that there may be more to learn. And, because interview sessions are streamed over the internet, the need for travel is eliminated. All participants engage remotely, making the process more convenient and cost-effective for everyone.
Deeper Insight – A Turning Point
While running a previous FocusVision VOC tracking survey, we noticed that one metric for the Decipher survey software platform wasn’t performing as well as expected. We wanted to get a better understanding as to why, and what we could do to improve customer experience and perceptions. Ultimately we needed customers to provide examples on their computer screens of what they were experiencing, and what they were trying to accomplish at that time.
Shortly after fielding a specific wave of VOC surveys, we identified a list of customers to follow up with via live video. We specifically targeted respondents who gave an open end or a ratings response that we wanted to unpack in more detail. Special emphasis was placed on those giving low satisfaction scores for the specific areas we had concerns about.
How long? Within a short period of time, we scheduled fifteen webcam-based 1-on-1 interviews with the selected customers. The interviews were conducted via our InterVu video focus group solution, making it easy for both participant and moderator to join the call, and providing a recorded environment for simple review and analysis. Although an incentive was considered, it was not necessary to offer one – the customers were more than willing and excited to provide feedback.
Each interview lasted roughly 30 minutes and was conducted by a member of our product team. The conversation followed a discussion guide that was developed from the many questions the software designers had for customers. This included feedback on potential new features and likes/dislikes about the software. Customers were asked to explain why they rated the software the way they did in the survey. Questions were also used that would present a deeper picture of who the customer was and what motivated them. For example:
- What are your primary job responsibilities/roles?
- What are the job motivations and needs surrounding the use of FocusVision Decipher software?
- Describe a job situation in which FocusVision Decipher is being used; what is being delivered?
This line of questioning delivered detailed information about desired software improvements, who wanted them and why. Customer insights that are valuable to both product and marketing teams.
During the interview, participants were able to share their computer screens, to show or even demonstrate specifics areas of the FocusVision Decipher software they were referring to. Multiple internal team members watched privately in the background and messaged in comments to the moderator during the discussion. While a video recording was available for later playback, an appointed member of the team was used to be a “scribe”, and take copious notes of each interview. This was an important step that provided an internal document to share and discuss findings as a group immediately afterward.
The Outcome – Improved Team Discussions and Valuable User Personas Emerged
Our approach proved that 1-on-1 webcam interviews provided a fuller picture of the customer experience than an online survey alone. For example, in one interview, a single-sentence open end response from the online survey became a 30-minute discussion on that customer’s most frequently encountered difficulties, what context they were occurring in, what he was trying to accomplish at that time and the possible solutions that would be workable for him. This conversation, and others like it, yielded a wealth of information for the product and support teams. Distinctive patterns among those interviewed also emerged. There appeared to be unique classes of customers who shared profiles and faced similar situations and needs. This provided an unexpected benefit for us. Using these patterns, we were able to hypothesize a few key user personas. Marketers and designers have long understood the importance of defining user personas(2). They serve as archetypes, that stand in for a class of real users. This provides product designers with a clear vision of the final user(s) and enables them to build products according to actual user wants and needs. (Figure 2).
Meet Small Business Steve
Occupation: Small business owner
Firm: 3-5 employees
Industry: Custom luxury apparel
- Monitor how his customers feel about their purchase experiences
- Track customer complaints; make sure they get resolved
- Address customer issues in a quick and timely manner
- Most software is too expensive and loaded with features he doesn’t need
- Has trouble separating what needs his attention and what can be delegated to staff
- No system for prioritizing the severity of customer complaints
Small business Steve runs an online retail site. His products are customized, and as such, his clients are demanding and require high levels of service and attention. He used to handle his customer service solely via email and phone, but now his business has grown to the extent that his customer services processes requires a more streamlined system. Steve is often traveling or busy with client meetings so it’s important for him to delegate work to staff and step in only as needed.
Figure 2. User personas represent of class of users who share a common set of characteristics, goals and needs. User personas describe a type of customer and help designers visualize the individual(s) they are building a product for.
An added bonus to using webcam interviews was that recordings could also be used to make video clips — with full consent from participants on how they would be used. These video “testimonials” allowed stakeholders to more closely empathize, engage and connect with customer issues. Researchers know that it can often be difficult to socialize customer insights with numbers or charts on a page. But a video can tell a powerful story through the customer’s own voice, which will resonate more deeply with decision makers.
Combining interview-based research with always-on VOC surveys awarded us insights that would not have been possible if we had just used one method, such as the survey. The surveys themselves provide a scorecard of how a brand is performing, while the webcam interviews yield even deeper customer insights. Product or brand marketing managers naturally have many questions for their customers. Webcam interviews are a quick and convenient platform for asking these questions, connecting a company to its customers, hearing their stories, and identifying what they want, and the problems they face. The main objective is to have a design team regularly sit with customers, internalize their problems and situations, truly understand the customer’s experience from their point of view and build great products for them. With this valuable information at hand, further design discussions and appropriate business directions will inevitably take place while ensuring that the customer’s voice is kept in the equation.
- For more detail about the NPS question and how it is used in customer experience research, visit: https://hbr.org/2003/12/the-one-number-you-need-to-grow
- For a basic primer on user personas, visit: https://www.usability.gov/how-to-and-tools/methods/personas.html Usability.gov is a federal government sponsered website devoted to user experience best practices and guidelines.