Podcast: Jason Buchanan talks fraud in quantitative research data

On this episode of Now that’s Significant, Horst Feldhaeuser, Group Services Director at Infotools was joined by Jason Buchanan, Managing Director at the Global Market Research Agency, Verve. Jason has a rich history in the insights sector, founding the Research Now APAC chapter and steering SSI’s APAC business to greener grass. Jason has a passion for exploring how the world works, the know-how for making it better, a deep understanding of consumers, and even has a degree in neuroscience.

Jason dove into world of online quantitative surveys and how businesses around the world would be shocked if they knew who was actually in their research sample data.  He wasn't talking about the claims that suppliers make about who is providing the data, but who the actual people were, if they actually are people at all, and how confident the suppliers are that the people answering surveys are who they say they are. 

This came to life for him when visiting family and chatting with some young nieces and nephews who actually told him (without knowing what he did for a living) that they were taking surveys to make spending money. They even explained to him how to trick the system so you could qualify for more surveys. Jason thought that the problem of fraud might be even bigger than he thought. To put this into persona talk, 'Fabricated Frank' sure does answer a lot of surveys.

He goes on to talk about the state of sample quality, and the focus that has been on stopping bots and click farms.  Quality has then become about how to identify fraudulent behavior at the end of the survey, once it is complete. He said this experience with his family brought the issue of identity verification to the forefront.  What are suppliers doing to verify the identity of people? He says "We have known as an industry that this has been an issue, and companies I speak with tell me that they are throwing away 30, 40, 50, some even said 60 percent of their sample, and the state of sample quality seems to be in perpetual decline." Jason is most interested in the reasons why, and he thinks there is perhaps too much focus on automation to be able to do things at scale, not enough being done to ensure that people are who they say they are.  

But do buyers actually care? Jason says "So I don’t like doing this, but I think it is accurate to say that the two types of people that emerge through this period will be those who actually care about their insights results and therefore ask more awkward questions of their suppliers, and those who are likely just to accept the status quo and not believe there is a problem.  And I am certainly not judging who fits into which of those, but I think everyone will have to make a decision about whether they actually care or not." It is big challenge to identify fraudulent behavior after the fact, and it messes with everything when you have to remove people from your sample. Unless a field supplier is focused on making sure that real and verified people are participating, it’s a problem and it will continue to be a problem.  

He also touches on respondent experience issues, something else that has been pervasive in the insights space. And now we have two major themes to address, who are the people and what is their experience? Jason says the two are joined at the hip and that it is up to us as an industry to be continually better in these areas. 

Listen in to hear the rest of their conversation, including Jason's involvement with The Juicy Effect, a Boutique Wellbeing and Healing Sanctuary in Bali, and how it has actually made him a better researcher.