Home » Quantitative
Category Archives: Quantitative
Reflecting on Research Access
I am currently undertaking a part-time PhD and, as part of my qualitative research project, I need to keep a reflexivity diary, reflecting on my own position in relation to the subjects of my research. My first entry reflects on the process of negotiating access, and I thought it might make an interesting subject for a criminology blog!
As a reflexive qualitative researcher it is important constantly to reflect on my own position in relation to my research and my participants. I am about to start collecting data but only after a very lengthy process of negotiating access (12 months). This process was one in which my own position and history had a significant influence.
For my first project, I am conducting detailed qualitative interviews with serving prisoners. This required approval from the HMPPS National Research Committee (NRC). As a former HMPPS employee, I thought that my inside knowledge would be helpful – indeed at a later stage, in negotiating access to specific prisons, I think it has been. However, at the stage of submitting my application for national approval my prior experience added emotional baggage.
I worked for the Prison Service for nearly 12 years. I really enjoyed working with prisoners and I also enjoyed the camaraderie of working within the tight community of a prison. I worked with some lovely individuals who were dedicated to helping people and who supported me personally in my career. However, the Prison Service is a large, unwieldy organisation and large unwieldy organisations do not always treat individuals well. There were several times in my career when I felt that the organisation had treated me badly: when a recruitment freeze was introduced just after I passed the assessment centre so that I was stuck in limbo without a permanent job; when the in-house MSc was abolished (with no replacement) just before I was due to start it; when I ended up taking on my boss’ responsibilities as well as my own with no promotion or increase in pay; when my request to work part-time after maternity leave was declined; when my post was put on the “surplus list” during a phase of job cuts. It was not all negative – as I say, I enjoyed the work very much, I was proud to be a prison psychologist and there were times when the organisation was good to me (I eventually had a different distance learning MSc funded by the Prison Service, and I was able to take a 12 month career break following my maternity leave) but those negative incidents felt like personal insults when I was working hard. The biggest kick in the teeth came more recently in 2020 when I applied to re-join HMPPS when my tenure on the Parole Board came to an end. I had an unremarkable remote interview and was then turned down. I knew that HMPPS advertised for qualified psychologists every month, so there were plenty of vacancies – that they didn’t want me when I knew I was a good psychologist and had already given them the best years of my career really hurt.
Since then, I have started my PhD and secured a great job with St Andrew’s Healthcare which suits me better and has taken me in a new direction. The memory of the rejection was still lingering, however, when my initial research application to the HMPPS NRC was rejected. I felt like they were kicking me in the teeth again. My initial application was for a piece of research with both a quantitative and a qualitative element. The feedback in relation to the quantitative project was so devastating that I scrapped this part of the research altogether and focused on the qualitative part only. To be fair, the feedback was justified and the re-written proposal is for a much more methodologically sound piece of research, but it still felt personal at the time.
Conducting research during a pandemic is not easy. By the time I had responded to the NRC feedback and was ready to re-submit, there was a resurgence in COVID-19 cases and the NRC were not accepting any further applications. I was advised not to re-submit until applications were being accepted again, otherwise I would have been rejected with no chance to re-submit again. This caused a three-month delay and I had to chase to find out when applications re-opened. When I finally re-submitted, my application was not rejected, but I did receive a long list of requests for further information. Some of these seemed very petty. Responding to them was a significant piece of work and the sense of personal rejection and being made to jump through hoops returned. I was very grateful to one of my supervisors who read my responses before I submitted them and helped to remove the irritation that was evident!
Having submitted the answers to these further questions I waited for ages for a reply. By this stage, I was on first name terms with the reviewer on the NRC. She had sent my application for further feedback from the HMPPS Interventions Team (one of whom I had worked closely with in the past). I was given further questions to answer (which seemed to miss the point of what I was trying to achieve). I tried my hardest to remain positive and to suppress the irritation. HMPPS had the power and were entitled to it. They had every right to reject my research application. Just because I had been a good employee in the past, did not oblige them to give me research access. I resigned myself to receiving a final rejection and started to think of other ways to explore my research questions. And as I reached this point of acceptance, I finally received a positive response saying that my application had been accepted!
Since then, things have moved quickly. I revised my University ethics application in the light of all the amendments I had made following the HMPPS feedback and this was quickly approved. I approached individual prisons for specific access and received positive responses from HMP Grendon and HMP Onley – in these cases possibly reaping the benefits of personal connections from my time in the Prison Service. I will be going into HMP Grendon in early January to start to recruit participants. I no longer work for HMPPS or for the Parole Board, but my status as a Forensic Psychologist and as a former HMPPS employee and Parole Board member will have an influence on my relationships with participants. They will have had experiences with psychologists (and may have had experiences with the Parole Board) which may be positive or negative and which may facilitate or hinder trust in me as a researcher. I will need to look out for these influences and reflect on them as the research progresses.
Share this:
The same old rhetoric, just another place
My sister phoned me the other day in great excitement. She’d just met a former criminology student from the University of Northampton, and she had an awful lot to say about it. She wasn’t in her hometown and had asked directions from a stranger to the river embankment. Having visited the embankment, she returned to town only to bump into the stranger again who enquired whether she managed to find it. They ended up chatting, my sister can do a lot of that, and she found out that the stranger was a police officer. My sister asked whether she knew me, why she would ask that I have no idea, it seems that she has formulated some notion in her head that all police officers must know each other or at least know of each other. This is the bit that my sister got so animated about, yes, the stranger did know me, I’d taught her at the University, and she was now in a budding police career. Apparently, I had done so much to help her. Now I don’t know about being that helpful and I suspect that many of my colleagues played a part in her success story, but it reminded me about what it is that we do and aspire to do as lecturers.
Whilst waiting to play my part in talking to school children the other day I started to read a new edition of a seminal piece of work on policing, The Politics of the Police (Bowling et al., 2019). The preface alone makes interesting reading and in ‘mentioning populist political reactions towards crime’, ‘zero tolerance of the marginalised and outsiders’ and ‘laissez-faire economics’ that promotes individual interests, my mind turned to the managerialist ideals that have dogged policing for over three decades. Those ideals saw the introduction of performance indicators, targets and the inevitable policing by objectives (Hallam, 2000), that resulted in some quite appalling manipulation of data and a diminution of service rather than an improvement. The problem was that the targets were never achievable and were simply put in place for managers to simplify the social world over which they had no control. What didn’t get measured, because it never could be, were the myriad of tasks that police officers and staff undertake daily. Dealing with people with mental illness, searching for missing persons and dealing with minor disturbances are an example of just a few such tasks. Bowling et al. (op cit.) subscribe to the notion that the job of the police is to help maintain social order, an ideal that does not lend itself to measurement. Counting the number of crimes committed in an area or the number of detected crimes is only an indication of failure, not success.
How does that policing narrative fit in with my opening paragraph? The former student was not an ideal student from a managerialist viewpoint. She didn’t attain so called ‘good grades’, I’m not even sure if she fully completed her studies. In terms of performance measurement, she doesn’t even feature and yet she, like so many others we have seen in Criminology, has flourished. Whilst concentrating on ‘retention and progression’ and ‘fails’ and ‘good grades’ we neglect the very reason we exist. Just as in policing where the figures were pored over by managerialist who had not slightest notion of the reality of the social world, so too are we in danger of simply seeking pleasing statistics to keep the wolves from the door because explanations of real success and failure are too complex for managers to understand or manage.
Imagine a world where the police just helped maintain social order, where probation were not plagued by notions of payment by results, where patients were just seen in A&E in a reasonable time and where lecturers just opened the minds of students and allowed them to think for themselves. Imagine the time and expense that could be saved and reinvested in providing real service and dare I say it ‘value for money’ if we stopped gathering meaningless data. Imagine managers casting aside the shackles of neoliberalist ideals and managing people, not using numbers as an indication of failure and impending doom. We can but dream, but my reality, as I’m sure is the reality of many of my colleagues, is the success stories that I occasionally hear and can reminisce about. No amount of number crunching can take that away and nor will it ever provide evidence of success or failure.
Bowling, B. Reiner, R. and Sheptycki, J. (2019) The Politics of the Police. (5th ed.) Oxford: OUP.
Hallam, S. (2000) Effective and Efficient Policing: Some Problems with the Culture of Performance, in Marlow, A. and Loveday, B. (eds.) After MacPherson: Policing after the Stephen Lawrence Inquiry. Lyme Regis: Russell House
Share this:
The number’s up for quantitative research! Or is it?
As my colleagues will no doubt confirm, I’m not a fan of numbers. Although, I always enjoyed maths, particularly algebra, my distaste for numbers comes when they are applied to people. Whilst I appreciate there are a lot of us, somewhere around 7.6 billion on the planet, the reduction of a human to a number doesn’t sit comfortably. This aversion to numbering people partly stems from academic study of the Holocaust, which was facilitated by the Nazi’s determination to reduce individual human lives first to digits and then to ashes. It also comes from my own lived experience, particularly in education, of knowing that individuals can and do change.
Criminologists such as Stanley Cohen (1988), Nils Christie (1997) and Jock Young (2011) have long recognised the fundamental flaws inherent in much quantitative criminology. They recognise that numbers are often used to obfuscate and confuse, taking readers down a route whereby they are presented as having their own intrinsic meaning, entirely distinct from the people whose data is being manipulated. Furthermore, those numbers are deemed scientific and authoritative, having far more sway than qualitative research predicated on finding meaning in individual lives.
Despite my antipathy to numbers, I have spent the last few months studying attempts to quantify a particular prison population; ex-servicemen. Much of this research is flawed in the same way as recognised by the eminent criminologists above. Instead, of answering what appears on the surface to be a straightforward question, many of these reports struggle to even define what they are trying to measure, let alone make sense of the measurements.
All this has made me think about the way we measure “engagement”. Last week, as you may have noticed from Manos’ entry, was the blog’s first birthday. Underneath, the professional front page lies, what WordPress rather hopefully describes, as ‘Stats’. From here, it is possible to identify the number of visitors per day, month and year, as well as the number of views. There is also a detailed map of the world, displaying all the countries from which these visitors are drawn. In essence, I have enough data to tell you that in our first year we have had 3,748 visitors and 5,124 views from 65 countries.
All of this sounds very encouraging and the team can make statements about how views are up on the period before, or make claims that we have attracted visitors from countries for the first time. If, so inclined, we could even have a leader board of the most popular contributor or entry; thankfully that doesn’t seem appeal to the team. If I wanted to write a report, I could include some lovely, bar or pie charts, even some infographics; certainly if you look below you will see a rather splendid Wordle which displays 233 different categories, used a total of 781 times.
However, what exactly do we know? I would argue, not a lot. We have some evidence that some people have visited the blog at least once, but as to how many are regular readers; we have no clue. Do they read the entries and do they enjoy them? Again, no idea. Maybe they’re just attracted by particular pictures (the evidence would suggest that the Yellow Submarine, Kermit, the Pink Panther and tattoos do exceedingly well).[i] There is some evidence that many of the visitors come via Facebook and Twitter and these offer their own illusion of measurable activity. Certainly, Twitter offers its own ‘Analytics’ which advises me that my tweet containing Manos’ latest blog entry earned an impressive 907 ‘Impressions’, and 45 ‘Engagements’ which equals an ‘Engagement Rate’ of 5%! What any of that means; your guess is good as mine! Does scrolling mindlessly on your newsfeed whilst waiting for the kettle to boil count as an impression or an engagement? Should I be impressed or embarrassed by a 5% engagement rate – who knows? If I add it to my imaginary report, at least I’ll be able to add some more colourful charts to accompany my authoritative narrative. Of course, it will still be largely meaningless but it should look splendid!
More concerning are the repercussions to any such report, which would seem to imply that I had total control over improving such metrics and if they didn’t improve, that would ultimately be down to my inertia, inability or incompetence. Of course, the blog is a voluntary labour of love, created and curated by a group of like-minded individuals… However, if we consider this in relation to criminology and criminal justice, things take a more sinister turn….the numbers may indicate something, but at the end of the day those numbers represent people with their own ideas, concerns and behaviours. Discussions around payment by results seem to miss this vital point, but of course it means that failure to achieve these results can be blamed on individuals and companies. Of course, none of the above denies quantitative data a place within Criminology, but it has to be meaningful and not just a series of statements and charts.
Now, I’ve got my anti-quantitative rant of my chest, I’ll leave the final words to Nils Christie (1997) and his command to make criminology exciting and passionate as befits its subject matter. In his words, avoid ‘[l]ong reports of the obvious. Repetitions. Elaborate calculations leading to what we all know’ (Christie, 1997: 13).
Instead we should always consider:
[h]ow can it be like this? How come that so much criminology is that dull, tedious and intensely empty as to new insights? It ought to be just the opposite, in a science based on material from the core areas of drama. Our theories are based on situations of conflict and heroism, danger and catastrophe, abuses and sacrifices – just those areas where most of our literary heroes find their material. And still so trivial! (Christie, 1997: 13).
Rather than mindlessly churning out quantitative data that looks and is perceived as sophisticated, as criminologists we need to be far more critical. If you don’t believe Christie, what about taking heed of Green Day’s command to ‘question everything? Or shut up and be a victim of authority’ (Armstrong et al., 2000).
References:
Armstrong, Billie Joe, Dirnt, Mike and Cool, Tre, (2000), Warning [LP]. Recorded by Green Day in Warning, Reprise: Studio 880
Christie, Nils, (1997), ‘Four Blocks Against Insight: Notes on the Oversocialization of Criminologists,’ Theoretical Criminology, 1, 1: 13-23
Cohen, Stanley, (1988), Against Criminology, (Oxford: Transaction Books)
Young, Jock, (2011), The Criminological Imagination, (London: Polity Press)
A revised and reworked submission of this entry was published on the British Society of Criminology blog on 1 June 2018.
[i] This point will be “tested under experimental conditions” by the gratuitous inclusion of a picture showing teddy bears “reading”.
Share this:
A help or a hindrance: The Crime Survey of England and Wales
I recently took part in the Crime Survey for England and Wales and, in the absence of something more interesting to talk about, I thought I would share with you how exchanging my interviewer hat for an interviewee one gave me cause to consider the potential impact that I could have on the data and the validity of the data itself. My reflections start with the ‘incentive’ used to encourage participation, which took the shape of a book of 6 first class stamps accompanying the initial selection letter. This is not uncommon and on the surface, is a fair way of encouraging or saying thank you to participants. Let’s face it, who doesn’t like a freebie especially a useful one such as stamps which are now stupidly expensive. The problem comes when you consider the implications of the gesture and the extent to which this really is a ‘freebie’, for instance in accepting the stamps was I then morally obliged to participate? There was nothing in the letter to suggest that if you didn’t want to take part you needed to return the stamps, so in theory at least I was under no obligation to participate when the researcher knocked on the door but in practice refusing to take part while accepting the stamps, would have made me feel uncomfortable. While the question of whether a book of first class stamps costing £3.90 (Royal Mail, 2018) truly equates to 50 minutes of my time is a moot point, the practice of offering incentives to participate in research raises a moral and/or ethical question of whether or not participation remains uncoerced and voluntary.
My next reflection is slightly more complex because it relates to the interconnected issues associated with the nature and construction of the questions themselves. Take for example the multitude of questions relating to sexual offending and the way in which similar questions are asked with the alteration of just one or two words such as ‘in the last 12 months’ or ‘in your lifetime’. If you were to not read the questions carefully, or felt uncomfortable answering such questions in the presence of a stranger and thus rushed them, you could easily provide an inaccurate answer. Furthermore, asking individuals if they have ‘ever’ experiences sexual offending (all types) raises questions for me as a researcher regarding the socially constructed nature of the topic. While the law around sexual offending is black and white and thus you either have or haven’t experienced what is defined by law as a sexual offence, such questions fail to acknowledge the social aspect of this offence and the way in which our own understanding, or acceptance of certain behaviours has changed over time. For instance, as an 18 year old I may not have considered certain behaviours within a club environment to be sexual assault in the same way that I might do now. With maturity, education and life experience our perception of behaviour changes as do our acceptance levels of them. In a similar vein, society’s perception of such actions has changed over time, shifting from something that ‘just happens’ to something that is unacceptable and inappropriate. I’m not saying that the action itself was right back then and is now wrong, but that quantitative data collected hold little value without a greater understanding of the narrative surrounding it. Such questions are only ever going to demonstrate (quantitatively) that sexual offending is problematic, increasing, and widely experienced. If we are honest, we have always known this, so the publication of quantitative figures does little to further our understanding of the problem beyond being able to say ‘x number of people have experienced sexual offending in their lifetime’. Furthermore, the clumping together of all, or certain sexual offences muddies the water further and fails to acknowledge the varying degree of severity and impact of offences on individuals and groups within society.
Interconnected with this issue of question relevance, is the issue of question construction. A number of questions ask you to reflect upon issues in your ‘local area’, with local being defined as being within a 10-15 minute walk of your home, which for me raised some challenges. Firstly, as I live in a village it was relatively easy for me to know where I could walk to in 10/15 minutes and thus the boundary associated with my responses but could the same be said for someone who 1) doesn’t walk anywhere or 2) lives in an urban environment? This issue is made more complex when it comes to knowing what crimes are happening in the ‘local’ area, firstly because not everyone is an active community member (as I am) therefore making any response speculative unless they have themselves been a victim of crime – which is not what these questions are asking. Secondly, most people spend a considerable amount of time away from home because of work, so can we really provide useful information on crime happening in an area that we spend little time in? In short, while the number of responses to these questions may alleviate some of these issues the credibility, and in turn usefulness of this data is questionable.
I encountered similar problems when asked about the presence and effectiveness of the local police. While I occasionally see a PCSO I have no real experience or accurate knowledge of their ‘local’ efficiency or effectiveness, not because they are not doing a good job but because I work away from home during the day, austerity measures impact on police performance and thus police visibility, and I have no reason to be actively aware of them. Once again, these questions will rely on speculative responses or those based on experiences of victimisation which is not what the question is actually asking. All in all, it is highly unlikely that the police will come out favourable to such questions because they are not constructed to elicit a positive response and give no room for explanation of your answer.
In starting this discussion, I realise that there is so much more I could say, but as I’ve already exceeded my word limit I’ll leave it here and conclude by commenting that although I was initially pleased to be part of something that we as Criminologist use in our working lives, I was left questioning its true purpose and whether my knowledge of the field actually allowed me to be an impartial participant.
Thoughts from the criminology team 