At the heart of every successful online survey is good data collection. Recruiting and getting quality responses to your survey questions will yield invaluable customer insights for your business. However, not everyone you invite will choose to participate in your survey. While participation levels differ from study to study, the more difficult it is to recruit people to take a survey, the more time, energy and cost involved. Non-response error becomes an issue with diminishing response rates. This can bias the survey data and give the researcher an incomplete picture of the population. Similar to the silent majority – you may not hear from a segment of your customer base, but their opinions do matter.
You can improve survey participation by adhering to a few simple guidelines. These are tips we’ve discovered over the years analyzing the meta-data in surveys hosted with FocusVision Decipher. The figure below illustrates an example. The graph shows dropout rates and completion times for each page in a 19-page survey. The pages underlined in red are where dropout rate and completion times increase and this likely reflects a problem with the content or design of the page within the survey. Survey participants are spending extra time here, and more prone to abandon to the survey altogether. What’s going on? By examining the content on pages like these we’ve learned some valuable lessons on what trigger respondents to abandon a survey mid-way through. And these dropout triggers are ones you should take pains to avoid.
Question Designs with Higher Dropout Rates and Completion Times
Open-end questions require additional effort on behalf of the participant. Compared to a multiple-choice question, extra effort is required to formulate a point, articulate it, and type out a response, leading to increased dropouts. However, open ends are very popular in surveys, as they can be a rich source of unstructured feedback. Survey participants are able to elaborate on their feelings or narrate a key insight leading to new ideas and understanding of customer sentiment. Therefore, avoiding them entirely isn’t recommended. But do be mindful of the number of open-end question in your survey. Limit the number to a few, and don’t place open-ends near the start of your survey, since having them in the beginning further increases dropout. If you do need more, keep a close eye on participation rates while in field should you need to make adjustments.
Grids, especially repeated grids
A grid-formatted question is known in research to result in respondent fatigue and dropout. The participant must evaluate a list of items, rating each one on a scale. This format essentially condenses multiple questions into one grid, making the task a bit repetitive and laborious. A common survey method is to take a grid-formatted question and repeat it multiple times in a series. For each grid, the participant rates the same set of attributes, but for a different brand. This kind of grid repetition significantly increases dropout rates in a survey, and so it’s best to only include 2 or 3 brands in a rotation or avoid the repetition altogether.
Some interactive question types, such as the Card Sort can also be a suitable alternative to the grid-formatted question. While research has shown it achieves better data quality and participation rates, it’s always important to minimize the number of repetitive tasks within a survey.
Many Rows, Lots of Text
In question designs that include a lot of text or many answer options (rows) to read through, high dropout rates follow. This is simply cumbersome for the participant, so keep the number of rows to a minimum, and take pains to simplify your writing so that your survey is more user-friendly.
The 3D grid is a version of the grid-formatted question that attempts to condense even more information into a single question. In the example above, the participant is asked to indicate the gender, age, and education of each member of the household. 3D grids should be avoided entirely. Instead, split out the 3D question into multiple ones. The same information can be gathered in smaller bites over multiple pages.
Intro and First Question
In every survey we’ve analyzed, we’ve found that the page with the highest dropout rate is the first page in the survey. Participates click on your survey link to check it out. The first page gives them the first impression of your survey as they discover what it is all about and participants make the decision about whether to continue. Many do not. The second-highest number of abandons occur on the next page. Here, the participant is still in the evaluation stages, considering whether to invest time in your survey or abandon it.
Since participants are most vulnerable to drop out at the beginning of a survey, it’s important to pay particular care to engage respondents early by avoiding any survey dropout triggers. Questions that require a lot of reading or thinking should be saved for later. Instead, make early survey questions easy to read, fill out, mobile-friendly and visually appealing.
- Avoid open ends. Open numeric age question are common in screeners; best to move them further back
- Cut down lengthy introductions or questions
- Move simple questions (e.g. single select) upfront
Any part of a survey that is repetitive, laborious, and requires extra effort can encourage someone to abandon it. None of the recommendations provided here should be too surprising, and generally, a simple pre-test can indicate where problems might occur. Take your own survey, and if a question design feels cumbersome or tedious for you, it is highly likely that will also be the case for someone else. Also keep in mind improving survey engagement and participation is a multi-faceted endeavor. The design of the email invite, employing incentives, and using interactive question types can have an impact as well. You can find helpful suggestions on these other topics in our Definitive Guide to Effective Online Surveys. But follow the simple guidelines provided here and you’ll be well on your way to providing a more user-friendly survey taking experience for your participants.