Police recorded offences involving knives or sharp instruments: methodology changes

• Implementation of a new cross-police force methodology for counting the number of recorded offences involving knives or sharp instruments.

• Revisions to the coverage and guidance of the offences involving knives or sharp instruments collection.

Police recorded crime data are supplied to the Office for National Statistics (ONS) by the Home Office, who are responsible for the collation of recorded crime data supplied by the 43 territorial police forces of England and Wales, plus the British Transport Police. These data are based upon the notifiable offence list. These include all offences that could possibly be tried by jury (these include some less serious offences, such as minor theft that would not usually be dealt with this way), plus a few additional closely-related summary offences dealt with by magistrates’ courts, such as common assault.

To provide additional context to the main police recorded crime series, which is an offence-based collection, the Home Office have a number of additional collections. These collections allow forces to provide further details on the nature and circumstances of recorded offences, such as whether the offence was domestic abuse-related, involved a weapon or was a hate crime.¹ How these extra details are collated and sent to the Home Office will vary by police force but for many, the adding of additional information is dependent on a police officer or support staff adding an identifier or tag to a respective crime record. Counts of these identifiers or tags are then sent to the Home Office where they are quality assured by statisticians before the final data are sent to the ONS to be published.

Concerns have been raised by forces with the Home Office that some of the special collections, based upon these identifiers, were likely to be of variable quality since they relied on tags being manually assigned to a crime record. With many other priorities, it was possible that some recorded offences may not have had the tags correctly applied in the returns to the Home Office. For example, while the number of robberies recorded will have been a correct count of those on the force record management systems, the aggravating factors – such as whether the offence involved a weapon – may not have always been correctly identified. Some forces carried out extensive manual reviews of their crime records, used automated database extraction queries to ensure the data returns were accurate as possible, while others largely relied on officers and staff remembering to tag a record accurately. Because of the issues with existing collections, the Home Office has been developing a new approach using a computer-assisted classification tool. This tool scans the free text fields of a crime within force record management systems, which include the detailed circumstances of a crime as recorded by a call handler or an investigating officer. This new approach was piloted on the offences involving knives or sharp instruments collection. The project had three broad aims:

• to improve the accuracy of the offences involving knives or sharp instruments data collection
• to increase the consistency and comparability of data across forces
• to reduce the burden on forces in supplying high quality data

This new method has been deployed via a tool called the National Data Quality Improvement Service (NDQIS). The tool was piloted with forces and shown to improve data quality and comparability, leading to a national roll out of the new process.

Offences involving knives or sharp instruments data processed via NDQIS were first published for 12 forces² in Crime in England and Wales: year ending December 2020.

Forces with the highest volumes of offences involving knives or sharp instruments were prioritised for the roll-out with additional volunteer forces coming on board. In the year ending March 2020, these forces accounted for 64% of the offences recorded by the police across England and Wales that involved a knife or a sharp instrument. The aim is for the new methodology to be rolled out to all forces in England and Wales between 2021 and 2022.

Notes for: Overview

1. The annual data requirement (ADR) lists the requests for data made to all police forces in England and Wales under the Home Secretary’s statutory powers.

2. These forces were: Derbyshire, Dyfed-Powys, Greater Manchester Police, Leicestershire, Lincolnshire, Merseyside, Metropolitan, Northamptonshire, South Wales, South Yorkshire, West Midlands and West Yorkshire.

The National Data Quality Improvement Service (NDQIS) uses a computer-assisted classification tool designed to improve data quality reported to the Home Office. The tool reviews force records held by the police, including free text fields, to determine whether an offence involved a knife or sharp instrument. Firstly, a list of candidate records is created by a series of filtering rules, to ensure that:

• the offence code is in scope for the offences involving knives or sharp instruments collection
• the record is valid and has not been cancelled or transferred to another force for investigation
• a free text field contains words and phrases that are indicative of a crime carried out with a knife or a sharp instrument, for example “knife” “stab” or “puncture”

Once created, the free text records are reviewed using a bespoke data dictionary and ruleset, which lists words and sentences associated with these offences.

Each crime record processed by NDQIS is allocated to a category. These are summarised below:

• high confidence: offences which NDQIS determines have involved a knife or sharp instrument and do not need further review
• low confidence: offences which need to be manually reviewed by police forces to confirm whether or not the offence involved a knife or sharp instrument
• rejected: NDQIS determines no knife or sharp instrument was involved in undertaking of the offence, or the offence was possession only¹, and therefore not counted in this collection

This ruleset was developed by testing the tool on synthetic data created for the project and on real data supplied by forces involved in the initial pilot project. The feasibility of the project was proven from testing on these data. A first developed ruleset was then tested on the synthetic data and data supplied by six pilot forces.² From these six forces, 62% of the records were classified as high confidence. Home Office statisticians audited these records and agreed with the categorisation in 95% of offences.

A further 28% of offences were classified as low confidence and required a manual review. The remaining 10% of records were rejected, of which the Home Office statisticians agreed with the result in 97% of cases.

Auditing by Home Office statisticians identified a small number of records in this first ruleset where the NDQIS decision was deemed to be incorrect. These included:

• records allocated to high confidence where a knife had been used to self-harm, or threaten to self-harm
• records allocated to high confidence where the use of the knife had been for a different offence than the one recorded – often related to a different, previous offence
• rejected records that included the use of a knife or sharp instrument and should have been considered in scope
• low confidence records that clearly involved a knife or sharp instrument and could have been allocated to high confidence
• low confidence records that could have been rejected outright

From discussions with the pilot forces, it became apparent that the existing guidance on threats for the current collection were interpreted in different ways. To resolve this, Home Office statisticians discussed with police forces an agreed approach to dealing with threats. This generated questions about the weapons in scope of the collection. Following these discussions, a number of changes to the coverage of the collection were made (see Section 4: Changes in the coverage and guidance).

A second ruleset was developed based upon the revised coverage of the collection and to address the offences where Home Office statisticians had not agreed with the allocation decision made by NDQIS. There were two broad aims for this second ruleset:

• to increase the percentage of records passed by audit
• to lower the “manual review rate”, that is, the proportion of low confidence records that required force resources to review

This second ruleset, following the change in coverage and amended rules, was tested on seven forces.³ In this second test, 57% of records were classified as high confidence and 20% rejected, meaning a fall in the records classified as low confidence, which required a manual reviewed by the force from 28%, to 23%.

The proportion of records that were deemed to be correct by Home Office statisticians increased compared with the first ruleset. The percentage of correct high confidence records increased from 95% to 98% and rejected records from 97% to 98%.

This ruleset was judged to be suitable for implementation and the rollout to all forces in England and Wales. For the first 12 forces, the figures show an increase in the proportion of records classified as high confidence to 66%, and a fall in the number of records classified as low confidence to 21%. A summary can be found in Table 1.

Statisticians will continue to audit a sample of NDQIS records to ensure the data dictionary is up-to-date and records are being correctly classified. The NDQIS tool does not employ artificial intelligence techniques and does not learn from manual review decisions and update its ruleset. Changes that need to be made to the ruleset will be subject to decision making by statisticians in collaboration with the users of the tool in police forces before being rolled out. Changes to the ruleset will also be tested before release and clearly communicated in advance to the users of the resulting official statistics.

For the year ending March 2020, around half (51%) of the records in the 12 forces classified as low confidence were deemed to have involved a knife or a sharp instrument after manual review. This figure reflects the ambiguity of many of the low confidence records, with an even split of records being accepted or rejected.

Once forces have reviewed their low confidence records, data are submitted to the Home Office for quality assurance.

Notes for: Methodology

1. The collection focuses on offences where a knife or sharp instrument was involved in the commissioning of an offence, for example, used to threat or stab during an assault or a robbery. Simple possession offences are therefore excluded.

2. The forces were: Derbyshire, Merseyside, South Wales, South Yorkshire, West Midlands and West Yorkshire.

3. Greater Manchester Police joined the list of pilot forces.

The offences involving knives or sharp instruments collection is based on selected offences. These are:

• actual bodily harm and grievous bodily harm or assault with injury and assault with intent to cause serious harm
• attempted murder
• robbery
• threats to kill
• rape
• sexual assault
• homicide (figures taken from the Home Office Homicide Index)

Data have been collected for these offences since April 2008, although there have been some changes to the offences covered by the Home Office Counting Rules in this time period.

The pilot work for the National Data Quality Improvement Service (NDQIS) identified that not all forces were interpreting the guidance in the same way. This led to a review led by the National Police Chiefs' Council (NPCC) Lead for Knife-Enabled Crime, Deputy Assistant Commissioner Graham McNulty, resulting in four broad changes to the coverage and guidance of the collection. These are outlined below.

Clarification on when the use of a knife or sharp instrument as a threat should be recorded

The pilot showed this was an area where forces were interpreting the current guidance inconsistently. Some forces were including offences where a knife was not present (for example, the offender states “I will get a knife and stab you”) while other forces excluded these offences. New guidance has added a series of rules to ensure consistency between forces. These are:

• the knife or sharp instrument should be present at the time of the offence – or believed to be present by the victim. This includes cases where the sharp instrument is not seen but the threat is believed, or only a knife handle is seen¹
• “remote threats” should not be recorded, for example, threats over social media to use a knife
• “future threats” should not be included – that is, offences where the perpetrator threatens to use a knife or sharp instrument in the future (for example “I will get a knife and stab you”)
• “idle threats” should not be recorded, such as where an offender says they will stab someone in passing but there is no evidence that a knife was present when the threat was made

The removal of broken bottles and glass from the collection

While sharp, these are weapons that are generally used in the heat of the moment without a premeditated motive. They tend to be involved in a different type of context to those involving knives or other sharp instruments. Removal of these type of offences align the collection closer to what the public would consider to be “knife crime”. Similarly, pens and pencils, while potentially sharp instruments, have also been removed. The collection still covers other sharp instruments that may be used in a similar way to knives, such as scissors, syringes and axes.

The addition of sharp tools, such as chisels and screwdrivers

The police have told us that these tools are carried and used in a similar premeditated way to knives.

The inclusion of sharp instruments when used as blunt instruments

This recognises that the use of sharp instruments, even as a blunt instrument, carries an implicit threat of serious harm. The victim will not know whether the offender will use the weapon to stab with the blade, or to strike with as a blunt instrument. This change also brings the collection in line with offences involving firearms, where firearms used as blunt instruments are included in the data.

Notes for: Changes in the coverage and guidance

1. For example, if an offender threatens to stab a victim during a robbery and the threat is believed and belongings handed over, this would be considered an offence involving a knife or sharp instrument whether the weapon was seen or not.

For some police forces, the change in methodology has had little effect on the number of offences. These forces may have already had a relatively robust system in place for identifying offences involving knives or sharp instruments. Other forces saw a modest uplift in offences, while two forces saw a relatively large increase in the volume of offences tagged.

For some forces, the effect of the changes to the coverage of the collection had a greater impact than the change in methodology. Some forces under the new rules had a lower level of offences than previously published.

Three sources of data were used to estimate the effect because of the changes in coverage:

• the old manual aggregated data returns (12 forces)
• National Data Quality Improvement Service (NDQIS) results based on the first rule set (6 forces)
• NDQIS results based on the second rule set (12 forces)

Under the first ruleset, screwdrivers, chisels and other sharp tools were not considered to be in scope of the collection. Following the consultation with police forces, these offences were included in the second ruleset. A comparison of the two sets of data showed that the adding these weapons to the collection increased the volume of offences by around 2%.

Conversely, the removal of broken bottles or glass was estimated to reduce the level of offences by 2%.

The inclusion of sharp instruments being used as a blunt instrument had a minimal effect on the number of offences.

It is not possible to assess the effect of the change in the rules around threats on the level of offences recorded by the police. How forces interpreted the previous guidance around threats, especially for threats to kill, varied considerably. As most forces were submitting aggregated data prior to NDQIS, it’s not possible to see how each respective force treated these offences. However, 7 of the 12 forces had a lower level of threat to kill offences than the previously supplied data based on the old coverage of the collection, while 4 forces had a higher level.¹

Data from April 2019 to March 2020 were collected for the 12 forces² on both the old force and new NDQIS methodology and coverage. These data show:

• an overall increase in the number of offences identified involving a knife or sharp instrument of 2%
• a 1% increase in assault with injury and assault with intent to cause serious harm involving a knife or sharp instrument
• a 4% increase in robbery involving a knife or sharp instrument
• an 8% decrease in threat to kill offences involving a knife or sharp instrument.

Changes at the force level can be seen in Appendix table 1.

Notes for: Impact of methodology and guidance changes

1. One force had the same number in both sets of data.

2. Because of the implementation of a new crime recording system, NDQIS data for GMP are only available from 9 July 2019.

The implementation of the new National Data Quality Improvement Service (NDQIS) methodology and the change in coverage for the offences involving knife or sharp instruments collection means that data following these changes will not be comparable with those collected prior to April 2019.

The collection of two sets of data for the year ending March 2020 meant that comparisons could be made between the two. These data were used to create a set of ratios between the two datasets which could be used to adjust the back series.

In Table 2, the dummy force recorded 200 offences of assault with injury or assault to cause serious harm under the previous methodology and coverage in the year ending March 2020, and 220 via the NDQIS process and new coverage. These figures give a ratio of 1.10 between the figures – under the new process the force identified 10% more offences than previously. This ratio would then be used to adjust the previously supplied quarterly figures prior to April 2019. Figures for the other two volume offences in this collection – robbery and threats to kill – will also be adjusted by the ratios shown in the table – 1.13 and 1.08 respectively. Numbers for adjusted quarterly figures prior to April 2019 were then rounded to the nearest whole number.

For the other offences in Table 2 the ratios were fixed to 1. This was because of the low volumes of these offences as it is not appropriate or indeed sometimes possible to adjust these low numbers.

The new historical time series is then created by combining:

• the new adjusted figures for assaults, robbery and threats to kill
• the unadjusted figures for attempted murder, rape and sexual assault
• homicide data from the Home Office Homicide Index

The new time series were sent to police forces to be signed off for publication. The new adjusted figures can be found in the Open Data Tables for Police recorded offences involving knives or sharp instruments.

Over the last six years, improvements to recording processes and practices by the police have made substantial contributions to rises in recorded crime, including offences involving knives or sharp instruments.¹ Therefore, while the previous time series has been adjusted to take account of the new methodology and the change in coverage for the collection, time series data over the longer-term should be treated with caution. Increases in these offences since 2015 will be partly because of a genuine increase in these offences and partly because of improvements in recording.

As well as general improvements to crime recording, some forces have made specific improvements to their recording of offences involving knives or sharp instruments. As part of the sign-off process for a new time series, police forces were invited to explain changes in trend before the implementation of NDQIS. These are given in the Open Data Tables for Police recorded offences involving knives or sharp instruments.

Notes for: Trends in offences prior to implementation of the National Data Quality Improvement Service (NDQIS)

1. See Crime in England and Wales statistical bulletins.

The National Data Quality Improvement Service (NDQIS) methodology for the offences involving knives or sharp instruments collection will be rolled out to all police forces in England and Wales and the British Transport Police. Figures from the new NDQIS methodology will be published in the Crime in England and Wales statistical bulletins.