In recent years there has been a shift in the not for profit sector towards outcome measurement – understanding and reporting on what changes for service users as a result of the work of the organisation. The shift in focus from the number of interventions delivered, towards understanding the effects of these interventions is no doubt a positive step, increasing our knowledge of what works best for service users and providing evidence for good practice.
Before fully embracing this new paradigm, it is important to learn from the mistakes of our colleagues in other jurisdictions and other fields of work, particularly in relation to the issue of targets. Having a better understanding of outcomes can encourage organisations to set targets for themselves. The lure of targets is often tempting, and can create systems which prioritise completion of certain steps in the process, rather than aiming to fulfil the ultimate goal. Likewise, some targets are simply beyond the scope of possibility and may result in workers gaming the system in order to meet these targets. The following case study exemplifies how cheating the system becomes natural and in some cases almost necessary.
In the early 90’s, crime in New York, especially serious crime such as homicide, was at an all-time high. The city was gripped by fear of crime, crime rates were high and detection rates were low. Rudy Giuliani’s election in 1994 was won partially on the basis of his tough on crime stance, and his use of “zero tolerance” rhetoric. Shortly after his election, Giuliani assigned William Bratton as New York’s Police Commissioner – ushering in drastic reforms in both police practice and protocol.
While most people are familiar with the increased stop and search nature of Zero-Tolerance, the major changes to how police did their business was at the back end. Introducing an increased accountability process and computer system: CompStat. Precinct commanders became directly accountable for the crime rates, detection rates and indeed happiness and wellbeing indicators in their geographic areas. This accountability was far more than symbolic, with a Precinct Commander turnover rate of around 60% during the first 5 years of CompStat.
While the effects of zero-tolerance / CompStat upon the crime rate is not straightforward, the years that followed did see a drop in the city wide rates of serious crime. The systems however, required that each fortnightly briefing given by a Precinct Commander showed a decrease in their regional crime rates. The problem is that crime can only be influenced to a certain extent. There is a wall, beyond which the crime rate cannot be influenced by the police. CompStat did not recognise this wall. The system called for continued decreases in crime stats and increases in detection rates, and Precinct Commanders knew their jobs were on the line if they could not demonstrate this.
In New York, serious crimes were classified as minor ones. Police officers were informally encouraged not to record certain types of crimes and crimes with a higher likelihood of detection (such as drug related crimes) were prioritised over crimes with a lower chance of detection (such as assault or homicide). The figures were massaged, manipulated and misappropriated, in some cases significantly impacting on police officers’ ability to respond to crime. While the targets used were ultimately unrealistic, the consequences of missing those targets were all too real.
This tale of caution shows how target measures can influence practice in a way which undermines the performance of duty. This principle can be applied to a variety of other settings, and underlines how poorly considered target measures may negatively affect service delivery to vulnerable groups in society, through cherry picking of service users or the massaging of numbers.
For more information on the potential danger of targets, see the work of John Seddon and his company, Vanguard Consulting.
