How the GRINCH stole MY DATA!

Online PanelsOnce upon a time in a far away land there was a BRAND that was lost in the jungle called ‘THE MARKET’ – it went to the wizard called the ‘RESEARCHER’ to find his way back home. The wizard opened the MAP of the city and showed the brand where is the right path but ALAS!!! The map was a FAKE and so the brand never reached back its home! I am not sure whether you have heard this sob story or not but unfortunately it is happening as you read this article. Data collection is an integral part of any research but with dubious quality data being served rampantly I have serious reservation as to what INFERENCE we can get from it. The article illustrates 5 sure shot ways to prevent your data being MANIPULATED!

RULE 1: BACK CHECKS, BACK CHECKS & BACK CHECKS!!

Back checks are the most potent weapon to ensure the data quality. I think this is the area where F2F & CATI score over online research – as in online data collection the vendor do not share the personally identifiable information of the respondent – you will not be able to do the back checks!!

Many a times “SMART” data collection agencies carefully select the participants for back checks (they are in fact there own employees!!) it therefore would defeat the purpose. A researcher should randomly select participants for back checks to avoid any kind of manipulations.

If there is even an IOTA of doubt as to the quality of the data you should investigate the whole matter thoroughly. I humbly feel that every party involved with the research process should understand that if we don’t provide true value to the brand – there would be no premium attained!! And we have to survive on the scraps.

RULE 2: PARTIAL DATA ANALYSIS!!

I guess most of the researchers follow the practice of partial data analysis to ensure that the data collection is happening as desired. The only glitch being that it happens mostly when 50% of the data has been collected.

The issue with it is that due to tight timelines even if there are small variations deducted in the data we are tempted to ignore it as there is no turning back. Ideally data collection agencies should implement tools & techniques so that researchers can analyze the LIVE DATA! It will ensure that if there is a problem it can be deducted and solved earlier on.

RULE 3: KNOW THY VENDOR!!

Not long back one of my clients told me that in Delhi itself there are more than 600 field agencies – I am not sure whether he exaggerated the numbers or not – but it cannot be ignored that data collection agencies have mushroomed all over the place. As the projects are limited the data collection agencies are bending their backs and compromising on the data quality to sustain them.

Here are few quick steps to avoid selecting a wrong data collection vendor:

1- Don’t change your current vendor frequently – if you are happy with the quality – DON’T EXPERIMENT.

2- Be careful with the NEW KIDS on the block – thoroughly understand their capabilities before sending your RFP request.

3- Create an “APPROVED” vendor list to be decided by senior management and implemented rigidly.

4- Don’t GO FOR MONEY GO FOR QUALITY – CHEAP BREEDS POOR QUALITY!!

Many new data collection are coming in the market with pompous claims of coverage – understand them! After all if something goes wrong you will have to face the client not your vendors.

RULE 4: SHARE THE BACKLIST!!

Almost all companies create their internal back lists of vendors who have shelved out dubious quality of data. The problem is that this list is INTERNAL to a company. Many fake agencies dupe a company, get black listed but then move on to the new gullible company to fraud them.

There are many industry verticals where they create a centralized vendor list which is given ratings as to how they have fared in an individual project. Such a centralized list will help companies to understand the background of the vendor before calling them on board.

RULE 5: TRAIN YOUR PARTNERS!!

Many a times it has been seen that poor data quality is not due to data fudging but rather as the vendors are not well trained in the research methodology. The companies should take time out to train their vendors and make them understand the importance of a good quality data. This will also help forge a strong bond with your vendor. After all you are as good as your vendor!!

We all (researchers & data collection agencies) are in the same boat and understanding each other will help us reach our destination safely!!

Now you have a choice. You can comment, share, or implement. I prefer if you implement, but I’ll appreciate all three – Akshay Kanyal