Top 5 reasons Market Researchers DOUBT online data collection methodology!!

online panelsSocrates was executed primarily for bringing on the NEW IDEAS to a world living in shackles of the old beliefs and customs. Although in today’s modern society nobody meets such gruesome fate -nevertheless challenging old methods is no easy task. In comparison to other data collection methodologies – online is a new player. It is still in the experimental phase of acceptance by researchers. The article illustrates 5 reasons why the research community is still not able to deal more confidently with online data collection.


The above statement is partially true – ODC (i.e. online data collection) can cover only those target groups which are easily accessible via internet. Countries where internet penetration is limited ODC has comparatively lesser reach. It is quite evident in the fact that developed markets like USA & Europe have a better acceptance of ODC!

Now you must be thinking where the reason for doubt crops in is!!

Online Panel companies have LIMITATION for what they can serve but researchers are not clear on the same. It results in pressure to cover target groups which cannot be served by ODC. Some BRAVE online panel companies simply refuse to pick up such projects.

Alas! Due to financial constraints panel companies accept requests for TG’s where online response rate is poor – END RESULT = DISASTER!!

The onus is on Market Researchers to fully understand which target groups are available via ODC methodology and which are not. In some cases researchers are PUSHING panel partners to conduct ODC as their own end clients have said so. The resulting project failure then pushed on to the methodology itself.

Market Research companies should have a checklist created through historical information available – as to which regions + target groups + industry verticals can be covered by ODC.

ODC is a tool – now how you use this tool is up to you –the fault is not in the tool but the hands which hold them.

What if I give a HOCKEY STICK to the best batsman of the world –ASK YOURSELF!!


As with any industry with low entry barrier the problem of fake companies sprouting up is evident. Online Data Collection (ODC) industry is also facing similar problems. Although there are many genuine panel players in the market – research companies for reasons best known to them are experimenting with new unknown panel vendors.

2 BIG issues which I have personally failed to understand:

1- If there are small dubious panel vendors EXISTING in business – some clients are definitely giving them business. It really intrigues me as to when even a layman can gauge the authenticity of these fake panel vendors – how come these fake panel vendors are sustaining themselves. One probable reason may be that the price point they are offering is too lucrative for research companies to fall into their trap. As for any vendor if they don’t get business for 8-10 months they will close shops – so because some research clients are feeding these dubious companies – these are still existent in business. If I was a researcher looking for a panel source I would simply go to the BIG players – why take the risk. If I have USD 300 million for a Hollywood movie I would but naturally go to an A-STAR not some rank newcomer.

2- DATA is the lifeline of any market research project. You hire the best researcher in the world but if you take service of a dubious panel vendor the end result is always going to be a DISASTER. When data holds such key position in any MR project than why the selection of the panel vendor happens at the last moment and secondly why market researchers have to do all the cost cutting for their data collection agency??

ASK YOURSELF: would you be able to provide INSIGHTS using fudged data from a dubious panel vendor??


If any – let me REPEAT AGAIN– if any panel vendor tells you “as the TG is tough we would have to use MIXED MODE METHODOLOGY wherein some data would be collected online and some via field or CATI” – PLEASE DO NOT BUY INTO THIS STUPID ARGUMENT!!!!!!!!!!

There is always going to be some variation using different methodologies for data collection. If online data collection cannot give you the requisite number of complete – why even then go for it?? It is better that you use face to face or telephonic methodology for data collection.

Let me put it this way – normally how many completes does a general market research project requires – on an average I would say 300-500. Now if a panel vendor with “millions of double opt-in” respondent base cannot provide 300-500 completes – why even use them. Ask your panel vendor as to how from their millions they can’t even deliver few hundred completes – no matter what the excuse is.

Don’t mess up your project using mixed mode methodology – it is best to use a consistent technique for data collection. If the TG is tough via ODC – nobody is FORCING you to use it!!

As repeatedly mentioned above – the problem is not in the online data collection methodology – the problem is how ignorance is leading to project escalations.


The BIGGEST problem I had faced during my entire tenure as a Project Manager was my clients having NO CLUE as to the technical setup of an online data collection project.

It is strange that after multiple educational sessions for the same the clients were facing tech issues at the moment of project launch. Almost 60%-70% of project escalation that I had was because during final launch something went terribly wrong at the client side.

Online data collection methodology is here to stay and in coming future it is going to have lion’s share in this market. Not knowing the technicalities of ODC does not means that the problem is with the methodology.

After my discussions with many researchers I had found that the doubts they have for this methodology are mostly baseless. It is a new method for data collection – understanding it thoroughly is the best way to harness its immense potential.


I want to make this CRYSTAL CLEAR having worked in the panel industry for so long.

1- It is not CHEAP method of data collection.

2- It is not LIGHTENING FAST as some have wrongly construed.

It is COSTLY to recruit and maintain a healthy panel base – therefore if anyone is selling you CHEAP they are serving you FAKE data. ODC is comparatively faster than some other methodology but it does take time to setup an online project and also to get the requisite number of completes.

Like any other data collection method ODC has its limitation and strong points. If you understand this methodology you would be in a much better position to utilize it. The doubts related to ODC in most of the cases are uncalled for.

 Now you have a choice. You can comment, share, or implement. I prefer if you implement, but I’ll appreciate all three – Akshay Kanyal


  1. You have made some good valid points here.
    On your first point, some examples might help. Not everyone is connected to the internet equally, but more importantly, not all age-groups or respondent types are able to use a computer and internet and have little or no familiarity with these new fangled gadgets! And this applies not just to ‘online panel’ methods of self-completion, but even to interviewer-assisted, recruited to online facilities in a Central Location, or online CAPI. For instance, we have just completed a project here in China on Seniors aged 70-85 (which to their credit our panel provider admitted they could not sample) so very few of them were able to operate and enter their own responses, or follow the long (45 minute!) wordy questionnaire with the embedded MaxDiff section in the middle, let alone enable cookies and delete them at the end, and click on the right buttons and reach the (ambiguous, confusing) end succesfully and accurately. We had a hard job training our experienced interviewers in all this, so I can only wonder how any Seniors managed it alone, or even their 50 yr-old offspring who were the other segment under study.

    And that’s even if they could understand the questions and discriminate between subtly different responses, or remember the content they were being asked about, or any of the other sources of error which as you quite rightly say apply to ANY methodology which involved asking people questions and eliciting linguistic responses – oral or in writing.

    These are just 2 of my areas of agreement with you. More to follow once I have digested it further, or have more examples to elaborate with.

  2. Chris Robinson says:

    There are many valid reasons why clients have concerns about Online research. The biggest, which you seem to ignore, is the validity of the respondents. I have used online for years but hand on heart have some reservations about that high-incomed older aged respondent being really who he is. The second concern that is rarely reported on by online suppliers is the response patterns especially in image frames. I have looked at that a lot and get very concerned about straight-lining of answers and other patterns suggesting respondents are racing through the questionnaire. yes there are ways to deal with this, but panel providers won’t offer this as a validation exercise because they know this is a major problem. The other point I would disagree with strongly is the merging of online and offline data. We have seen data sets where the sparse data in certain cells was dealt with by online data and merged later by fusion methods. The comparisons between the offline response patterns and offline were close enough to make this call. In other words it is not a straight forward never do this consideration.

  3. Hi Thanks for Sharing this post.. I Looking for more post on online data collection. I also have site which provides online service for data collection from multiple sources, securely sharing and online collaboration. Ready online forms & excel like web database to update data plus email notification to act on data.