Last week I was delighted to host Part Two of our “But What About The Sample” panel discussion with our FocusVision Decipher Sample Marketplace partners – Matt Gaffney, CEO & Founder at Branded, Lisa Wilding-Brown, Chief Research Officer at InnovateMR, Katy Mallios, Director of Partner Programs and Operation at Lucid, and Ted Pulsifer, Executive Vice President, Enterprise Solutions at Schlesinger Group (Market-Cube). These educational conversations covering the world of sample today have proved to be extremely popular. And for a good reason – there’s a lot to talk about.
As we concluded the discussion, three thoughts sat in my mind:
- Sample is a Complex, Technologically Sophisticated Field.
- Sample Quality Needs to be Addressed from Multiple Angles.
- We Need to Continue the Conversation.
Sample is a Complex, Technologically Sophisticated Field
As I outlined in my blog after the initial panel discussion, the world of internet sample has grown into a complex, technologically sophisticated field. By all accounts, Sample has leapfrogged many other areas in the insights industry over the past five years. Today conversations around APIs (Application Programming Interface), EPCs (Earnings Per Click), and many other acronyms are the norm. There are many approaches to recruiting, including panel pools, communities, and affiliate partners. In this regard, there are many similarities to the world of programmatic advertising.
For researchers, these changes bring new concerns about data quality, representativeness, incentives, and more. This is on top of many of the ‘old’ concerns, which haven’t been fully resolved.
Sample Quality Needs to be Addressed from Multiple Angles
So what can be done to help alleviate the concerns, particularly the most dominant question around quality? Well, throughout the discussions, it is clear that quality needs to be addressed from multiple angles.
The first is understanding how sample works today – if you don’t know the mechanics, it is difficult to make fully informed decisions about your participants and their data.
There are also steps to be taken when designing your questionnaire. Think carefully about how to reduce response burden by understanding the response process and the cognitive demands made on your participants. Then think about ways to make the questionnaire easy to complete and as engaging as possible. Mobile-first is essential.
Next, consider whether it makes sense to add specific quality questions within the questionnaire. These can be attention checks, red herrings, or duplicate questions. Note: quality questions play an important role in weeding out rogue participants, but they need to be employed thoughtfully to avoid offending genuine participants.
Lastly, employ various data checks to identify inattentive and bogus participants to remove them from your dataset. Three common assessments are straightlining, speeders, and gibberish responses.
For more on these approaches, see our White Paper: A 3 Step Guide for Better Research Data Quality.
We need to Continue the Conversation
Finally, I left with the certainty that this cannot be our last conversation about sample. There’s so much more to cover, and it is only through candid, transparent conversation that we’ll reach shared understandings and move forward together for the better of the industry.
I urge you to share your thoughts on LinkedIn and Twitter, to initiate conversations with your sample providers and FocusVision Decipher customer success managers. Let’s tackle this together.