Is Mixed Mode Methodology of Online Data Collection a BIG FRAUD?

Let me share with you a naked truth – your online data collection vendor is fooling you big time and you have absolutely NO idea. “C’mmon Akshay Kanyal you are just making stuff up” I know that will be the reaction of many market researchers but stay on with me and I will tell you why is it so. If you have heard these words “this is hard to reach respondent group and therefore we will have to supplement our online samples with a mixed mode methodology” – you may be a victim of fraud. This article illustrates why market researchers are outright naive when they agree to a mixed mode methodology of data collection.
The moment you as a market researcher hear the words hard to reach respondents what do you do?

You innocently ACCEPT that the online data collection vendor is right and simply nod your head to what ever suggestion is put forward by the vendor.

Have you ever as a market researcher asked these 3 straight forward questions to your vendor?

Q1: How have you come to the conclusion that these are hard to reach respondents?

Q2: Is there any quantifiable metrics based on which it has been identified that these respondents are tough to target?

Q3: Is it just your experience that the respondent group is hard to reach or has it been accepted by the entire data collection industry?

I am sure if you ask these 3 questions to your online data collection vendor there are high chances that they don’t have a satisfactory reply for you.

What can be a hard to reach target group for one vendor may be an easy job for another vendor.

For example if you go to an online panel vendor who sells gen pop panels it will be tough for the vendor to target healthcare specialist but for a vendor like WorldOne who are healthcare panel providers it will be easily achievable.

But then you are a market researcher and a ‘very busy‘ person – you don’t have time for these trivial matters. If the online data collection vendor says it is a hard to reach respondent group then it must be true.

What really intrigues me is that the one profession which is completely based on quantifiable facts is of a market researcher but it is here that maximum amount of false assumptions are taken.

The first step of a FRAUD is when the person being duped ASSUMES that the other person is speaking the truth.

So when your online data collection vendor says that it is hard to reach respondent group and you readily accept it – that is exactly when the process of fraud starts.

I want to ask this question to the entire market research industry:

Where it has been written that if a study is not possible through online data collection methodology alone you MUST pursue it anyways?

If your online data collection vendor says “Sir it is hard to reach the respondent group via online methodology” why are you still hell bent on pursuing this methodology even if it means allowing the vendor to use mixed mode methodology?

The second step of fraud is when the person being duped due to pressure for example peer pressure becomes adamant on believing the other person to be true. In case of a market researcher it may be the lack of time or client sitting on their head – that researchers give a go ahead for the mixed mode methodology without verifying the vendor statement of hard to reach respondent group.

Right now many of you may be thinking “what is wrong with mixed mode methodology? For a moment if we keep aside whether the respondent is hard to reach or not why exactly mixed mode methodology should be avoided?”

Let me first give you an example:

Suppose you are trying to find whether increase in the price of table salt brand ‘A’ will have any impact on the consumer’s demand for it.

Now if you conduct this study via online data collection methodology the problem is that the internet panel members belong to a section of society which is brand aware, involved purchaser and forward thinking group – which probably won’t mind even a substantial increase in the price of table salt if it is accompanied with possible better health benefits. The outcome of such a study will probably give an indication that consumers are ok with a price change on exchange of increased value proposition.

As you can see the fundamental problem with this study is that the general populace probably won’t share the same thought process as an internet savvy conscious group of respondents.

In the same way a Doctor who is internet savvy will probably be tech friendly and someone more ready to a new state of the art surgical instrument than your typical Doctor who still prefers established norms.

As you will agree with me here – not in every circumstances the internet panels can be true representation of the national psyche.

Have you ever considered this angle before you start looking for an online data collection vendor?

I am sure many of you haven’t thought it this way and here is the root cause of all problems.

There are times where a face to face study is recommended and there are occasions where online data collection methodology is suitable.

The problem is that most market researchers decide on the methodology on 3 parameters:

1- what ever the client suggests – so if the client says I need online data collection methodology – so be it!

2- what ever the vendor suggests – so if the vendor says it can be easily done by online data collection methodology – so be it.

3- what ever come the cheapest – so if the online data collection methodology is the most cost effective one – so be it.

At no point of time it is analyzed whether the objective of the study can be attained by a particular methodology or not.

I like Chinese food as well as Italian cuisine but if you serve me noodles mixed with pasta – I am sure the dish will be a complete mess! and not to my liking.

Some of you I am sure will get my point.

Now let me share with you how exactly the fraud takes place.

In recent times many Indian data collection companies have realized that if they pitch that they possess online healthcare specialist panels there is a good business potential.

But the problem is that none of them possess such a healthcare specialist panel so the moment they receive a client’s RFP the first thing they write back is – “sir this is a hard to reach respondent group therefore we have to use mixed mode methodology wherein our online data collection methodology will be supplemented with face to face interviews wherein our representative will meet the doctor in person and get the questionnaire filled up. Later on the responses will be punched in the survey link”.

The primary reason for mixed mode methodology in case of healthcare specialists sited is that Doctors are busy people and therefore not so responsive via emails therefore there is a need to send representatives to their clinic/hospital to get the questionnaire filled.

The reason seems plausible and as you are a ‘very busy‘ market researcher you will readily agree to the vendor.

Now please don’t be offended by what I am going to say next:

“Are you f***ing idiot?”

Even a non-market researcher with simple common sense will understand that if ultimately you have to send a field executive to get the questionnaire filled from the Doctor why is there a need to use online data collection methodology? You can simply tell your data collection vendor to conduct face to face interviews.

But wait a minute I can understand your problem – it is called LOCS (lack of common sense).

If your data collection vendor suggests mixed mode methodology have you ever asked as to how many respondents will come via online panels and how many via face to face interviews?

I am sure it must be difficult to take some time out of your ‘very busy‘ schedule as a market researcher to ask such mundane questions.

Now that you have agreed to a mixed mode methodology and have not set as to how many minimum completes are required via online methodology your vendor is being given free hand as to which primary data collection methodology to use.

What if your vendor gets 99% completes via face to face interviews and just 1% via online panels? I am sure ethically the vendor is correct as he has in theory conducted mixed mode methodology!

Let me tell you a shocking truth: in case of face to face interviews you need to submit the hard copy of the interviews along with the visiting card of the Doctor attached – so that you can verify that an interview has actually taken place but in case of an online interview no such thing can happen as the responses are directly punched on to the survey link and in the name of ‘we don’t share the personal identifiable information of our respondents’ it is impossible to cross check whether an interview has taken place or not.

This BIG loop hole in the system has lead many dubious face to face data collection companies to claim that they possess healthcare specialists panels.

My Dad was a Doctor (trust me he had to really squeeze in time for us) and having personally conducted face to face interviews for healthcare specialists if some one in India says “oh we have a big healthcare online panels” I leave it to your common sense to decide the validity of their claims.

If at the end of the day your vendor suggests we have a big healthcare panel but for this hard to reach respondent group we will require mixed mode methodology – I leave it to your sensibilities to make the right judgment.

My grandmother always told me “son no one can make you an idiot – you yourself invite others to do the job”.

So don’t be an idiot – be sure about what methodology will best suit your research objectives and don’t I repeat don’t mix up things – if a study cannot be completely conducted via online data collection methodology – don’t fall into the trap of fancy words like ‘mixed mode methodology’.

Bonus point: how to catch if your vendor is using face to face interviews in the name of online data collection methodology. The answer is simple – in case of face to face interviews the number of responses are minimum in the first week and maximum in the last week – this is primarily because in case of face to face interviews the first week is all about setting up things (training interviewers, identifying target respondents, planning logistics etc.) – in case of online data collection methodology it is complete opposite – the number of responses are maximum in first week and it tends to slow down up to the last week – this is primarily because in online methodology we start getting responses as soon as the survey is made live and the survey invites are being send – the responses tend to slow down later on when quotas start getting filled up.

I will really appreciate your thoughts in the comments section below!

Akshay Kanyal

Akshay Kanyal

Akshay Kanyal writes survey research reviews on his popular blog Online MR. He’s an avid blogger, brand consultant and a content marketing expert, helping business owners to craft content that sells.

He provides content marketing advice to start-ups and innovation driven companies. He can be contacted at editor@onlinemr.com

You can also connect with him on LinkedIn ; Facebook ; Twitter

Trackbacks

  1. […] Source: EzineArticles.com So, what is niche market research? It is the process of doing research on specific markets in order…to find out what is it research, then you could end up putting a lot of time and effort into a niche […]