Finding a better alternative to the grid question is perhaps a holy grail of survey question design. Grids are loved by many researchers as it is a seemingly efficient way to capture many data points with a single question. However, survey participants (and some researchers as well) find the question type tedious.
The typical grid formatted question employs a single rating scale (horizontally arranged) with a list of attributes (vertically arranged). The survey participant rates each item in the list. Despite evidence that this repetitive question format leads to respondent fatigue and survey dropout, it is still commonly used in online surveys (1).
Figure 1. Standard grid-formatted question
With the prevalence of mobile phones, the need for an alternative question type to the grid has become more imperative. Unless altered, traditional grid questions do not render properly on the smaller screens of mobile phones. Without any resizing / reformatting, the grid formatted question type doesn’t fit within the viewable screen of a mobile device. This is a problematic presentation and will lead to a data bias for participants to select the part of the scale that is visible on the screen. Some programming tools wrap attribute text to fit response options within the portrait screen, however, this display is often cluttered and results in up and down scrolling, often with the scale labels, presented above the first attribute, not being visible as the respondent scrolls down the screen resulting in potential quality issues.
Furthermore, this question design presents a grid of small radio buttons from which the user must make their choice. While the array of small selection areas may work fine as a point and click task using a desktop computer mouse, it is much trickier as a touch-tapping event on a mobile phone.
Figure 2. Standard grid question is not mobile-friendly
A Case for ‘Card Sort’
The ‘Card Sort’ question type is an effective replacement that addresses the issues of the grid format questions. Participants evaluate a set of cards by sorting them into categories or ‘buckets.’ It is an interactive question design where animated cards slide across the screen, one at a time, then respondents can drag and drop them into the desired category(s) or simply click the category to add the item to it. There are a number of aspects that make the Card Sort attractive and easy to use for survey participants:
- Respondents can carefully consider one item at a time.
- The item to be considered and the answer options are located in a single field of view, unlike a traditional grid.
- A traditional grid shows a matrix of input areas (radio or checkbox forms). With Card Sort, there are as few buttons as possible, simplifying the selection process.
- Animation helps create an interactive user experience, providing useful feedback in addition to more stimulating visual effects.
- Card Sort is mobile friendly. There is no horizontal scrolling and large ATM-style buttons are perfect for touch-tapping. The simplified view of the Card Sort question reduces the amount of content shown on the screen, allowing for enlarged content to help with readability and ease of use.
Figure 3. Card Sort is an improved, mobile-friendly alternative to the grid-formatted question.
Improved Data Quality and Survey Participation
In a recent study, Maritz CX, Dynata, and FocusVision, collected survey data from more than 2500 desktop, tablet, and mobile users to compare and test the standard grid format with the Card Sort (2). Questions of various scale and row lengths (5 and 10 point scales, 5, 13, and 24 rows) were examined.
In this investigation, Card Sort was found to:
- Significantly reduce straight-line ticking. In straight-lining, the respondent selects the same scale response for each of the items in the grid. This is one measure of data quality. As researchers, we hope participants vary their scale usage as different items are being rated. In most cases, the less straight-lining the better (though some questions may be quite legitimately answered by straight-lining).
- Improve survey participation. In a survey using Card Sort, mobile users were 13% more likely to complete the survey; desktop users were 6% more likely.
- Have strong cross-platform data consistency. One measure of question design performance is whether the data is consistent between desktop and mobile devices. Card Sort outperformed the standard grid on this measure.
- Improve survey enjoyment. A survey using Card Sort instead of standard grids resulted in directionally higher overall survey satisfaction among participants.
For your next online survey, consider replacing your grid questions with Card Sort. With improved survey participation and data quality, what’s not to love about it?
- Mick P. Couper, Roger Tourangeau, Frederick G. Conrad, and Chan Zhang. (2013). The Design of Grids in Web Surveys. Soc Sci Comput Rev, 31(3), 322–345
- Mobilize Me!! – Mobile Survey Design Enhancements – research presented at Insights Association NEXT, June 2019. [A webinar of the presentation can be viewed here.]