Get the HIGHEST VALUE from your Next Market Research Project!!

Carey Azzara(Fast Track Your Web Panel RFQ)

Introduction:

What do you want when you send out a request for quotes (RFQ) or request for proposals (RFP) for web-panel pricing to panel providers, besides a price?

In our rush-to-get-it-done world, we sometimes move too quickly and end up wasting time rather than using our time efficiently. Writing a proposal request or in the case of web panel providers a request for quote (RFQ) requires more than asking: “How much is it?”

The price and the cost may be two different dollar amounts. The price you pay an individual panel provider versus the cost of the decision to use one provider over another must factor in the value received. It’s your total cost of ownership (TCO) or perhaps in this case your total cost of sample development (TCSD) that is at stake. The quality of your research project is directly proportional to the quality of your sample. If you’ve ever heard me talk about research you’ve probably heard me say: “You can never fully recover from a poor sample – No amount of clever data processing, cleaning, or manipulation will save you”.  With that in mind, let’s work through how to write a web-panel sample RFQ for your next project.

Apples to Apples

To evaluate and understand the apples to apples value from each provider you’ve sent the RFQ to, you must set parameters in your request that are clear and precise.

For example, if you don’t specify the incidence rate expected or assumed you will very likely get back quotes (bids) with different rates and prices; some high priced with a low incidence assumption and some low priced with a high incidence assumption.

Which is correct? The answer is both. Which is better? The answer is neither.

However, the low incidence high price is probably more realistic – at least you are less likely to be handed a nasty surprise in the form of an unexpected invoice at the end of data collection.

What should you do?

First, you want to take control, which simply means you must be specific. You must learn to specify a number of key parameters. Doing so will reduce the time from sending the RFQ to receiving quotes you can compare. If you are not specific, you’ll be getting questions from panel providers individually and their questions will not necessarily overlap. Therefore, you will be updating each provider with the newest clarification you have based on the most recent question you have received from each provider – what a hassle!

Responding to their questions will require more work than creating a well thought out RFQ in the first place.

Where Should You Begin?

In most cases, it helps to state briefly, what the research plans to study. This can be as short; provide the working title of the study or a working report title. You can give more detail but the important thing is to provide a context to help the panel provider understand the study objectives.

Example:  Printer/Copier Selection Study.

Acme Printer Corporation is interested in conducting a U.S.focused research project targeting companies that will make printer and/or copier buying decisions during the next 12 to 24 months. Our focus is on the small and medium sized business (SMB) market.

The intent is obviously not to provide precise parameters. If the study is complex, perhaps including several country markets that information should be included, but the specific countries would not need to be listed; that information is specified later.

Example:  Multinational {product name} Mobile Device Use Study.

Bonanza Corporation is conducting a study on [name of product or function]. They need a total sample size N=650: US N=250, Germany and China N=200 each. The target audience is mobile device users in three company size categories. The objective is to explore the drivers and inhibitors of adoption for [name of product or function]. The time frame for the study is tight, we need a feasibility assessment immediately and price quotes soon after.

Basic Sampling Parameters

Sample Size and Geographic Scope

Start with the basics and there is nothing more basic than the size of the sample. However, the sample size is not simply one number. In addition to the overall sample size you may have specific quotas by geographic location (e.g., regions, countries or states). The study may also require that specific sample size quotas for different types of companies or households measured either by number of people or revenue and income, respectively.

Clearly, the sample size parameter is likely to be more than one number. Therefore, telling the panel provider you want the sample to be N=500 is not enough information. I know you knew that, but it’s an important point and for the sake of completeness it had to be said.

Sample size, geographic scope, and sample quotas all go hand-in-hand and it will be necessary to clearly state specifications for each.

Example: Total sample size N=650, US N=250,Germany and China N=200 each.

  • Company Size US: Maximum 100 per category, no minimum, must reach total of N=250
  • Company Size Germany: Maximum, 75 per category, no minimum, must reach total of N=200
  • Company Size China: Maximum 75 per category, no minimum, must reach total of N=200

Company Size Categories: 10 to 99, 100 to 499, 500+ employees

Respondents

Another basic parameter is to define the respondents you want to target. Start by identifying the basic class of respondents, that is, what type of study are you conducting?

  1. A consumer study (Specific ages, types of households)
  2. Business-to-Business
  3. Speciality audience, (Lawyers, medical professionals, technologists, oil and gas explorers)

You may want to talk with employees who are line managers in manufacturing companies, but only women; or you may want to talk to women who are the “heads of households” and work for manufacturing companies. While these two populations of women may overlap each is a different group and being precise will ensure you have the respondents you want in the sample.

Example: Respondent must be either:

Decision makers for [name of product or function]

Or must qualify as a [name of product or function] End User

US: Maximum 175 per category, no minimum, must reach total of N=250

Germany: Maximum 125 per category, no minimum, must reach total of N=200

China: Maximum 125 per category, no minimum, must reach total of N=200

Given these maximum N values no category in any country will have fewer than N=75. This approach safeguards the study against filling the total sample with too few respondents in one of the categories or worse yet filling the total sample with one type of respondent.

Incidence Rates

Incidence is the proportion of panelists who when invited qualify to participate in the study. The more restrictive you make the screening criteria the lower the incidence rate. Calculate the lowest projected incidence or qualifying rate based on prior studies with similar screening criteria. Unfortunately, this information is often not available. Researchers must estimate incidence based on experience and knowledge accumulated over the course of conducting many studies. To avoid surprises ask the panel provider for costs with incidence rates of 60% or less and in many cases it is wise to estimate incidence at 40% or less.

If the respondents are from a specialized professional group, be more conservative. For example, if you are estimating incidence among Computer Security Systems decision makers, incidence may be as low as 20%. If this is the case, ask for cost estimates for this rate of qualification. Ultimately, you want to achieve an apples-to-apples comparison.

Feasibility

Feasibility is the panel provider’s ability to fulfill all the sample requirements. When the pool of respondents is limited a panel provider will either decline to meet the sample quotas or no bid the project. For example, if the study requires a sample of respondents (N=200) with the title of CFO and the incidence rate for a CFO is very low (less than 10%) and the number of CFO panelists is limited, it may not be feasible to provide the required sample size.

To get the pricing right feasibility testing is a must-do task. Panel providers are typically willing to test feasibility because, like you, they do not like surprises. Testing also helps you better estimate incidence more accurately.

Time to Complete Questionnaire

Maximum Survey Length (minutes): 5, 10, 15, or 20+

Approximate the length of your research instrument (RI). This is a key parameter for the panel provider to have in order to give you an accurate quote. If you are unsure of the length ask for quotes for different length estimates.

A quick rule of thumb for estimating the length of a questionnaire is to divide the total number of questions by 2.5; the result is the length in minutes (Table 1). Obviously, this is only a rough rule of thumb and subject to error since it does not take into account the complexity of the questions. If the RI uses multipart questions and a number of attribute batteries, you should adjust the ratio to two (2) questions per minute. This will give you a more conservative ballpark estimate.

Use the Longest Path

Estimate the time in minutes for the longest path through the RI, that is, the path through the skip logic that will require the most time to complete. Have people unfamiliar with the RI read it as if she or he were a respondent. Use a stopwatch and start with the introduction. Ask each person to read each question and all the answer options, selecting items or ratings etc. as if they were completing the study online. Have two or three people do this exercise and check to see if times are roughly equal.

Table 1: Maximum Length

Estimated                          Number of Questions

Minutes                   Low Complexity      High Complexity

    10                                          25                            20

    15                                          37                            30

    20                                          50                           40

    22                                          55                            44

Complexity of the RI will affect the data collection on several levels. Complexity will have an impact on completion rates as well as the cost and time for programming. Complexity, is a major factor when soliciting bids and should be discussed if you believe it will directly or indirectly cause the cost of the sample to increase, that is, there is an unusual element in the RI (e.g., an atypical use of visuals) you what to draw the panel providers attention to.

Conducting the Study

Programming and Hosting

Let the panel provider know if you want them to program and host the study. Whether you intend to do these tasks or have another third party handle them it is best to make that clear from the outset of the project. If you want the panel provider to program and host the RI ask for separate pricing for these add-on services.

Screener questions

When possible supply the panel provider with the actual screening questions you intend to use. If you have not finalized the screening questions, provide the basic screening criteria detail. This is another key element for the estimation of sample costs.

Translations

Translation (or localization) of the RI is another element of the study we advise you to inform panel vendors about early in the process. If this service is required, some panel providers offer it or have partners they work with to create a turnkey solution. Once again, if you want the panel provider to do or manage translations for your RI ask for separate pricing.

Start and Stop Dates

The dates for starting and stopping the data collection phase of the project should be included in the RFQ. These dates frame the feasibility and allow the panel provider to schedule the work they need to do to meet your needs. Hence, it is in your best interest to give them as close an approximation on the start data as possible and to let them know if you have a hard out-of-field date too.

This RFQ approach provides you with the information you need to create a request that will yield apples-to-apples bids for your next project.

Carey AzzaraAuthor: Carey Azzara

Experienced business owner, with roles including President, CEO, and COO

Authored the book “Questionnaire Design for Business Research” released in 2010 by Tate Publishing, as well as numerous eBooks, articles, blogs, and other publications.

Website ; LinkedIn ; Twitter ; Book

Comments

  1. Great info Carey! You obviously know what you’re talking about.