Background
In July 2021, the Office for Statistics Regulation (OSR) published a report assessing the Living Cost and Food Survey's (LCF) compliance with the Code of Practice for Statistics.
This report identified two requirements and three considerations for improvements to be made to ensure that the LCF and its outputs continue to be fit for purpose and for Family Spending in the UK to retain its National Statistics status.
This document presents updates on our work to meet the OSR's requirements, as of 15 November 2021, and planned future actions.
Requirement 1
To meet this requirement, we will demonstrate a positive direction of travel by making some short-term gains by the end of 2021.
Short-term gains
1. The Office for National Statistics (ONS) needs to take remedial action to improve the stability of the existing Living Cost and Food Survey (LCF) processing system and to develop a new system that meets the needs of users and the staff running the system.
2. The ONS should enhance its understanding of the value of the statistics by improving its engagement with users, within and outside the ONS, to capture a wide range of views and use these to drive its priorities for development. The ONS should reflect on the Government Statistical Service's User Engagement Strategy for Statistics to help determine the best methods for engaging with users.
3. The ONS should provide a mechanism and relevant access for other teams in the ONS who make use of LCF data to be able to contribute to the quality assurance of the data.
Actions taken as of December 2021
The ONS has set up a dedicated and ring-fenced improvement project, the Living Cost and Food Survey (LCF) and RPI improvement project, which has delivered:
detailed process maps, covering the current LCF data processing and identifying the "pain points" - recommendations for improving the processes and their governance
a commitment to the full rebuilding of the LCF data processing system module by module; the re-build will be complete by March 2023, but single modules will be deployed one by one into operations throughout the course of the development
the enhanced Data Quality Assurance Module in R and the new Data Aggregation Module in Python, which are being implemented in the BAU (business as usual) operations of the LCF Research team; the formal handover is scheduled for January and February 2022, respectively
the start of work on rebuilding the annual data outputs and associated quality-assurance checks; the code will be completed by the end of March 2022
the start of work on rebuilding the weighting module using the ReGenesees R package, aiming at integrating the weighting, estimation and calculation of standard errors and coefficients of variance in one module; the code is expected to be completed by the end of March 2022
The LCF research team, responsible for the LCF outputs, continues to host regular "curiosity meetings" with the internal users of the data. In those meetings, the team shares their findings about and interpretations of the LCF data, giving the chance for data users outside the LCF Research team to contribute to the quality assurance of the data. These meetings have been well attended and highly appreciated.
The LCF Research team also provides access to the micro-level unweighted and weighted data, alongside the aggregated data, to the Economic Statistics Group (ESG) data users.
The LCF Research team has initiated discussions with the existing LCF steering group members to refresh its terms of reference, membership list, and purpose as part of a plan to strengthen our engagement across the LCF user community.
Next steps
We will design and implement a streamlined business process for the Living Cost and Food Survey (LCF) data processing and its improved governance structure. Timeline: by the end of March 2022.
We will continue improving and updating the legacy modules of the LCF processing system. Over the years, LCF data processing system have organically grown, with additions of subroutines, derivations, and modules that have outlasted their use. The complete re-development of the system aims at streamlining the data processing, pruning unnecessary code, and improving code transparency by aligning it with the best coding practice in developing reproducible analytical pipelines. Though the system re-development is mostly tactical, it follows the Office for National Statistics' (ONS) policy to move away from the use of SAS and strives to exploit opportunities for using strategic tools and existing frameworks and standards to facilitate a future strategic uplift. Timeline: by the end of March 2023.
We will develop and deliver training for the LCF Research team to upskill them in the use of the updated modules of the LCF processing system. Timeline: by the end of March 2022.
We will review the governance around the LCF and its development, both internally and externally, and consider different channels of engagement to capture a wider range of views and uses. Timeline: continuous.
Requirement 2
We will publish a plan that includes specific actions, deliverables, and a timetable by the end of March 2022. This plan will explain how we will address the following strategic improvements.
Firstly, the Office for National Statistics (ONS) needs to develop a solution to address user need for more granular breakdowns of data, so that the devolved administrations and other key users can use the statistics in the ways that they need to for the public good.
Secondly, the ONS needs to invest time and resource to pursuing initiatives to improve the quality and robustness of Living Cost and Food Survey (LCF) data. The ONS should be open to creative solutions to improve the response rate, such as continuing to explore the use of different short-form and long-form questionnaires or diaries, alternative sampling strategies, and linking with other data sources, rather than focusing only on increasing the existing sample.
Actions taken as of December 2021
The Office for National Statistics (ONS) has appointed and commissioned NatCen to conduct an external review aimed at informing the ONS' statistical blueprint for its future expenditure data collection, including a high-level survey design. The project includes:
a user-requirement exercise to identify what household expenditure data our users need and for what purpose (work package 1)
an assessment of alternative, non-survey data that may be available to meet part of these needs (work package 2)
recommendations for the best design for a household expenditure survey to meet user needs, including quality and cost requirements (work package 3)
The review officially commenced at the beginning of November 2021 and will conclude by the end of March 2022.
Progress achieved by mid-January 2022
Work package 1: a draft report has been produced, summarising the requirements of the interviewed household expenditure data users. NatCen are following up on queries raised by the ONS on the draft report. All relevant contacts have been recontacted and the draft report is being updated as replies are received.
Work package 2: interviews have been scheduled for the review of alternative data sources, to be completed during January 2022. A draft report containing an overview of alternative sources of data for household expenditure is underway, which details the availability and maturity of data, as well as an assessment of suitability against the user needs highlighted in work package 1. The final report from NatCen is due on 26 January 2022 for the ONS to review.
Work package 3: the review of expenditure surveys in other countries is progressing well, and a number of countries have responded. There has been a meeting with the Household Income and Expenditure Survey in Australia, but others are yet to respond. Two draft tables summarising the 10 expenditure surveys and two diary surveys have been received for the ONS to review.
A project for testing expenditure data collection through a mobile app, developed by Statistics Netherlands as an alternative to the current LCF diary, has been set up. An internal one-week trial of the app, consisting of approximately 30 volunteers, is planned to run in late January and early February 2022. The aim is primarily to obtain feedback on the useability and functionality of the English offline version of the app as-is. Options for building an in-house hosting platform for the app are also being explored.
Next steps
Going forward, we will develop and implement a plan about how to transform the collection of survey-based expenditure data in line with the review recommendations, as well as for the wider transformation of the Household Finances Survey. Our aim is to complete this by the end of March 2022.
We will continue to implement the Living Cost and Food Survey (LCF) National Statistics Quality Review 2016 recommendations as outlined in the latest update here.
Consideration 1
The Office for National Statistics (ONS) should consider extending the scope of its Living Cost and Food Survey (LCF) and RPI improvement project work to include input from some of its key external users, such as those in the devolved administrations, where additional intelligence could be gathered on the use and issues faced by the government in its use of LCF data.
Actions taken as of December 2021
As part of the LCF external review by NatCen, a detailed user-requirement exercise has been carried out to inform the future of its household expenditure data collection. A draft report has been produced, summarising the requirements of the interviewed LCF data users. NatCen are following up on queries raised by the ONS on the draft report. All relevant contacts have been recontacted, and the draft report will be updated when replies are received.
Next steps
Please see the next steps to address Requirements 1 and 2.
Consideration 2
The Office for National Statistics (ONS) should consider the management of risks throughout the end-to-end production process as part of the Living Cost and Food Survey (LCF) project's medium-term work and ambitions.
Actions taken as of December 2021
The ONS has expanded the scope of the LCF and RPI improvement project to consider wider aspects of the end-to-end process. We have established a continuous improvement project aiming at streamlining the LCF receipts processing and data coding and editing processes.
Work on producing an LCF Quality Assurance of Administrative Data (QAAD) has started. The plan is to be ready with it by the end of January 2022.
Next steps
We will complete a quality assurance of the LCF data report using the Quality Assurance of Administrative Data framework at A3 level of the QA matrix, to fully assess the end-to-end risks to the survey. Our aim is to complete this by the end of January 2022.
Consideration 3
The Office for National Statistics (ONS) should determine a longer-term solution for the Living Cost and Food Survey (LCF), which draws on a broader base of data, international best practice, and wider transformation initiatives.
Actions taken as of December 2021
Work packages 2 and 3 from the external review are focusing on assessment of alternative, non-survey data, which may be available to meet part of the LCF data user needs, and on analysing the international practices for conducting expenditure surveys. As of the end of December 2021, progress has been achieved in several areas.
For work package 2, interviews have been scheduled for the review of alternative data sources, to be completed during January 2022.
For work package 3, the review of expenditure surveys in other countries is progressing well, and a number of countries have responded. There has been a meeting with the Household Income and Expenditure Survey in Australia, but others are yet to respond.
The ONS Household Finance Surveys team had two more meetings with colleagues from the Household Expenditure team at the Australian Bureau of Statistics to share experiences around collecting and processing household expenditure data. They also had a meeting with Stats Canada and Statistics Netherlands to catch up on developments with their field testing of the mobile app to collect household expenditure data.
Next steps
We will develop and implement a plan about how to transform the collection of survey-based expenditure data in line with the review recommendations, as well as for the wider transformation of the Household Finances Survey. We aim to complete this by the end of March 2022.
We will continue our collaboration and exchange of knowledge with the Australian Bureau of Statistics and restart our collaboration with other national statistical institutes (NSIs) (for example, Statistics Canada and Statistics Netherlands). This will be a continuous process.