Horizontal vs. Vertical Survey Rating Scales: Does it Make a Data Difference?

I was recently asked whether or not there is a difference or change in response to a survey question if the answer options are arranged in vertical or horizontal format.

The question pertained to Decipher’s responsive survey design, where an 11 point horizontal scale renders in horizontal format on a PC device, but in vertical format on a smartphone device. This change to vertical format occurs when an 11 point horizontal scale is too wide to properly fit the narrow screen of a smartphone. There is an advantage in arranging it this way. Research has shown that mobile-friendly designs improve participation rates by as much as 50% for smartphone users.

Desktop view of an 11 point NPS question in a horizontal survey rating scale.
Figure 1. PC view – 11 point NPS question, horizontal scale

Impact on Data Quality in Survey Rating Scales

Mobile view of an 11 pt vertical survey rating scale.
Figure 2. A responsive design detects that an 11 pt horizontal scale is too wide for the narrow screen width of a smartphone. So it renders the scale in vertical format.

Back to the original question. Does the change in the scale arrangement affect the data? In other words: If you take the same rating scale question, but show it in horizontal alignment to a PC user and vertical alignment to a smartphone user – will that cause a difference in the data? Are the two comparable and can you aggregate data from PC users and smartphone users together?

Evidence says that yes, you can.

In a split cell test, we investigated  5, 7 and 11 point scales. We found no significant difference in responses between PC users rating horizontally arranged rating scales and smartphone users rating vertically arranged rating scales. We even looked at PC data from a 5-column x 9-row grid and compared that to breaking out the nine-row attributes into individual vertical scaled questions on the smartphone. We did not see a significant difference there, either.

The findings support that there shouldn’t be a concern when utilizing a responsive design in your survey.  It has helped improve participation rates for smartphone users, and we have yet to see evidence that it substantively alters respondent answers to our survey questions.

Frequency response distribution for desktop and mobile survey rating scales.
Figure 3. Frequency response distribution  – 11 point scale
Share this article:

Add comment

E-mail is already registered on the site. Please use the Login form or enter another.

You entered an incorrect username or password

Sorry, you must be logged in to post a comment.

2 comments

by Newest
by Best by Newest by Oldest
1

Do you have a published research paper with these findings? It would be useful.

2
Alisa Manjarrez

Hi, Larissa! We're talking about working on something for this. We'll email you when a paper comes out!

Begin your journey to better customer insights

Request a demo

Sign up to receive news from FocusVision

We collect this information to send you free content, offers, and product updates. Visit our recently updated privacy policy for details on how we protect and manage your submitted data.