Organizations that want to begin a new online market research panel initiative will need to answer a wide variety of related questions as part of the planning process. Not the least of which is how large the panel should be, as well as how large a panel might be achieved using existing lists and other resources that are available in-house.
The size of the panel is ultimately driven by its expected use. For example, for enterprises building Voice of the Customer (VoC) panels, Kinesis considers the following four factors in order to determine the optimal panel size:
- How many projects do you plan to field each month?
- Approximately how many completed interviews do you plan per project (on average)?
- What type of panel will this be?
- What type(s) of quotas will the projects have?
Working backwards from this information, the approximate size of the panel can be computed.
1. How many projects do you plan to field each month?
Except in unusual circumstances, most panel managers should plan on inviting panelists to at least two projects per month for which they qualify, or the panelists will not remain engaged or necessarily even remember that they are in the panel. In an inactive panel, there will be greater churn, because there is less opportunity to provide opinions as well as to earn rewards. Projects do not need to be surveys – consider communities and qualitative exercises as well – because the panel is a hub of all market research activity.
2. Approximately how many completed interviews do you plan per project?
Most projects range from an estimated number of completes from 500 to 10,000 per language. The project with the largest number of completed interviews should be kept in mind, but if there is only one or two projects per year with a large number of completes, it may be more cost-effect to recruit a smaller panel that serves the majority of needs, and augment sample from outside sources for the larger projects.
3. What type of panel is this?
The more specialized the panel, the smaller it needs to be. If everyone in the panel is a plumber, it will by necessity be relatively small, because the universe of plumbers (“the incidence rate”) is small. However, if it is a consumer panel, a broad range of types of individuals is likely to be required.
4. What type(s) of quotas will be used in your projects?
Projects with extensive quotas – for instance, by age, state or region, income, education, marital status, ethnicity, and other factors – necessarily dictate a larger panel. In addition, consider whether quotas will be nested or non-nested. A non-nested quota utilizes differing types of quotas that are independent of one another. For instance, a project may require 500 males and 500 females, and 500 individuals who have children, and 500 individuals who do not have children. A nested quota would require 500 males and 500 females, in within each of these categories, 250 would need to be filled by individuals with children, and 250 by individuals without children. The nested quota is always more difficult to obtain because as the project fills, the incidence rate of the targeted population decreases.
What Are Your Next Steps?
Once this information has been gathered, a more formulaic assessment can be made. Let’s assume that the largest project requires 5,000 completed interviews. The response rate is unknown for this panel and will itself depend upon several factors – the types and amounts of incentives used, the length, quality, and relevance of the projects to which panelists are invited, the frequency of contact, and of course, the underlying panel management rules and attention that panelists receive. However, a response rate of no greater than 20% should be assumed. Thus, a project requiring 5,000 completed interviews would require a panel with a minimum size of 25,000 members. This provides one metric for consideration. If it is highly unusual to have a project of this size, the option remains to obtain these individuals from another source – a separate list or an outside panel, for instance.
Next, consider the quotas. If quotas will be complex, these will increase the need for panelists. In addition, consider that some population segments – for example, high-income groups – are more reluctant to join panels than other segments. If these are likely to be important factors in your projects, consider adding an overage to the panel population to compensate.
Finally, let’s return to the number of projects and think about overall panel usage. If the number of projects is relatively small then the calculation of needed panelists based upon the paragraph above should be adequate. However, if a very large number of projects are anticipated, consider adding a percentage for overage, keeping in mind the invitation rules that you plan to implement. Invitation rules may be applied so that any given panelist is only sent a specific number of invitations in a given period. Keep in mind that the purpose of any panel is to invite respondents only to projects that are relevant to them – and not every project will necessarily be relevant to all panelists.
Remember also to track churn and plan for ongoing recruitment. All panels experience some dropout and bounceback on emails, and new recruits ensure that the panel’s voice continues to remain representative through time and does not rely upon a shrinking number of individuals who are more likely to respond than the average person.
Finally, remember that you can “recruit as you go.” Many VoC panels tend to recruit in a single wave, but recruiting to fill a panel on a quarterly basis for just what you need simplifies planning and helps reduce churn from non-activity. More frequent recruitment waves allow for more experimentation with recruitment approach, source, invitation wording, and other elements, to ensure a more balanced member base to your panel.