Who do you think they are?

The recent town hall on fraud provided by the Insight Association provided considerable food for thought, although there’s nothing new about respondent misbehaviour in research. For example, there has been a significant level of personation within some elements of the qualitative research world for many years. At a relatively trivial level people have claimed to be a few years younger or older than they are, or to be earning rather more than they do, whilst others have falsely claimed to have the qualifications or experience required to attend a highly rewarded focus group. The occasional investigation of such behaviour always revealed a situation that a relatively small number of people were responsible for most of these occurrences. In a word – they were professional respondents, using the activity of attending focus groups to supplement their income. Little was ever publicised about the problem, which in truth was seen as an annoyance rather than an activity to be described as fraud!

Now, living in the ‘Meta world’ where people are positively encouraged to create avatars and alter-egos for gaming and other purposes, it is hardly surprising that such activity should be on the increase and spreading into many areas of online research, particularly into any research where a tangible reward is offered. Moreover, as the recent presentations on this subject had demonstrated, it is not a straightforward matter to detect fraudulent behaviour. Some of the methods used to identify fraud, such as questions that double check a previous answer can result in excluding bona fide respondents who lack sufficient education or simply make a mistake, indeed all the processes used for fraud detection have a level of uncertainty associated with them and, as with all fraud, it is difficult to detect the true professional.

However, despite the history, we are not aware of any instance where fraud has been identified as the cause of any significant error or bias in the results of the research. Is anyone aware of such as example?

Meanwhile, we review the potential effects of various levels of personation.

Firstly, respondents claiming to be older or younger than their actual age may not have serious implications for the outcome of the research, unless there is some consistent reason whereby older people claim to be significantly younger, because age does affect attitudes and behaviour. Similarly, pretending to be a different gender is likely to affect responses. Moving up the personation scale – claiming to earn more has definite lifestyle implications which would almost certainly affect the truth of the answers and, therefore the outcome of the research. Finally, and most seriously, there is the respondent who claims an entirely different job responsibility to earn a reward completing a serious B2B project, or equally important a consumer claiming ownership of a product that they don’t own. In each of these cases they really need to make it up as they go along.

We would suggest that if the research is truly important then it should be worth investing more to employ professional interviewers, who would be able to validate the respondents.

However, the important question that arises from this debate is “are enough of these fraudulent respondents behaving in a consistent manner, such as to actually bias the response”? We would hypothesise that this is not the case and that for the majority of research projects, they only create random ‘noise’ in the data, which does not affect the main findings.

Is there any evidence to contradict this opinion?

If you would like to find out more, please contact us by clicking here.