Home » Validity
Category Archives: Validity
I recently took part in the Crime Survey for England and Wales and, in the absence of something more interesting to talk about, I thought I would share with you how exchanging my interviewer hat for an interviewee one gave me cause to consider the potential impact that I could have on the data and the validity of the data itself. My reflections start with the ‘incentive’ used to encourage participation, which took the shape of a book of 6 first class stamps accompanying the initial selection letter. This is not uncommon and on the surface, is a fair way of encouraging or saying thank you to participants. Let’s face it, who doesn’t like a freebie especially a useful one such as stamps which are now stupidly expensive. The problem comes when you consider the implications of the gesture and the extent to which this really is a ‘freebie’, for instance in accepting the stamps was I then morally obliged to participate? There was nothing in the letter to suggest that if you didn’t want to take part you needed to return the stamps, so in theory at least I was under no obligation to participate when the researcher knocked on the door but in practice refusing to take part while accepting the stamps, would have made me feel uncomfortable. While the question of whether a book of first class stamps costing £3.90 (Royal Mail, 2018) truly equates to 50 minutes of my time is a moot point, the practice of offering incentives to participate in research raises a moral and/or ethical question of whether or not participation remains uncoerced and voluntary.
My next reflection is slightly more complex because it relates to the interconnected issues associated with the nature and construction of the questions themselves. Take for example the multitude of questions relating to sexual offending and the way in which similar questions are asked with the alteration of just one or two words such as ‘in the last 12 months’ or ‘in your lifetime’. If you were to not read the questions carefully, or felt uncomfortable answering such questions in the presence of a stranger and thus rushed them, you could easily provide an inaccurate answer. Furthermore, asking individuals if they have ‘ever’ experiences sexual offending (all types) raises questions for me as a researcher regarding the socially constructed nature of the topic. While the law around sexual offending is black and white and thus you either have or haven’t experienced what is defined by law as a sexual offence, such questions fail to acknowledge the social aspect of this offence and the way in which our own understanding, or acceptance of certain behaviours has changed over time. For instance, as an 18 year old I may not have considered certain behaviours within a club environment to be sexual assault in the same way that I might do now. With maturity, education and life experience our perception of behaviour changes as do our acceptance levels of them. In a similar vein, society’s perception of such actions has changed over time, shifting from something that ‘just happens’ to something that is unacceptable and inappropriate. I’m not saying that the action itself was right back then and is now wrong, but that quantitative data collected hold little value without a greater understanding of the narrative surrounding it. Such questions are only ever going to demonstrate (quantitatively) that sexual offending is problematic, increasing, and widely experienced. If we are honest, we have always known this, so the publication of quantitative figures does little to further our understanding of the problem beyond being able to say ‘x number of people have experienced sexual offending in their lifetime’. Furthermore, the clumping together of all, or certain sexual offences muddies the water further and fails to acknowledge the varying degree of severity and impact of offences on individuals and groups within society.
Interconnected with this issue of question relevance, is the issue of question construction. A number of questions ask you to reflect upon issues in your ‘local area’, with local being defined as being within a 10-15 minute walk of your home, which for me raised some challenges. Firstly, as I live in a village it was relatively easy for me to know where I could walk to in 10/15 minutes and thus the boundary associated with my responses but could the same be said for someone who 1) doesn’t walk anywhere or 2) lives in an urban environment? This issue is made more complex when it comes to knowing what crimes are happening in the ‘local’ area, firstly because not everyone is an active community member (as I am) therefore making any response speculative unless they have themselves been a victim of crime – which is not what these questions are asking. Secondly, most people spend a considerable amount of time away from home because of work, so can we really provide useful information on crime happening in an area that we spend little time in? In short, while the number of responses to these questions may alleviate some of these issues the credibility, and in turn usefulness of this data is questionable.
I encountered similar problems when asked about the presence and effectiveness of the local police. While I occasionally see a PCSO I have no real experience or accurate knowledge of their ‘local’ efficiency or effectiveness, not because they are not doing a good job but because I work away from home during the day, austerity measures impact on police performance and thus police visibility, and I have no reason to be actively aware of them. Once again, these questions will rely on speculative responses or those based on experiences of victimisation which is not what the question is actually asking. All in all, it is highly unlikely that the police will come out favourable to such questions because they are not constructed to elicit a positive response and give no room for explanation of your answer.
In starting this discussion, I realise that there is so much more I could say, but as I’ve already exceeded my word limit I’ll leave it here and conclude by commenting that although I was initially pleased to be part of something that we as Criminologist use in our working lives, I was left questioning its true purpose and whether my knowledge of the field actually allowed me to be an impartial participant.