Horizontal vs. Vertical Survey Rating Scales: Does it Make a Data Difference?

I was recently asked whether or not there is a difference or change in response to a survey question if the answer options are arranged in vertical or horizontal format.

The question pertained to Decipher’s responsive survey design, where an 11 point horizontal scale renders in horizontal format on a PC device, but in vertical format on a smartphone device. This change to vertical format occurs when an 11 point horizontal scale is too wide to properly fit the narrow screen of a smartphone. There is an advantage in arranging it this way. Research has shown that mobile-friendly designs improve participation rates by as much as 50% for smartphone users.

Desktop view of an 11 point NPS question in a horizontal survey rating scale.
Figure 1. PC view – 11 point NPS question, horizontal scale

Impact on Data Quality in Survey Rating Scales

Mobile view of an 11 pt vertical survey rating scale.
Figure 2. A responsive design detects that an 11 pt horizontal scale is too wide for the narrow screen width of a smartphone. So it renders the scale in vertical format.

Back to the original question. Does the change in the scale arrangement affect the data? In other words: If you take the same rating scale question, but show it in horizontal alignment to a PC user and vertical alignment to a smartphone user – will that cause a difference in the data? Are the two comparable and can you aggregate data from PC users and smartphone users together?

Evidence says that yes, you can.

In a split cell test, we investigated  5, 7 and 11 point scales. We found no significant difference in responses between PC users rating horizontally arranged rating scales and smartphone users rating vertically arranged rating scales. We even looked at PC data from a 5-column x 9-row grid and compared that to breaking out the nine-row attributes into individual vertical scaled questions on the smartphone. We did not see a significant difference there, either.

The findings support that there shouldn’t be a concern when utilizing a responsive design in your survey.  It has helped improve participation rates for smartphone users, and we have yet to see evidence that it substantively alters respondent answers to our survey questions.

Frequency response distribution for desktop and mobile survey rating scales.
Figure 3. Frequency response distribution  – 11 point scale
Share this article:

Want to talk? Have a question? We’d love to hear from you. E-mail bloggers@focusvision.com

See for yourself: Begin your journey to better customer insights

Request a demo

Sign up to receive news from FocusVision