Consumer prices development plan: updated July 2022

The Office for National Statistics (ONS) has a programme of research and development aimed at improving and maintaining its range of consumer price inflation statistics. The programme will ensure that the statistics continue to meet user needs, make use of new and innovative methods and data sources, and follow international best practice.

The article, Measuring changing prices and costs for consumers and households, proposed updates: March 2020, provides more detail on the "use cases" for each of our main inflation measures, including:

• Consumer Prices Index including owner occupiers' housing costs (CPIH)

• Consumer Prices Index (CPI)

• Household Costs Indices (HCIs)

• Retail Prices Index (RPI)

The UK statistics strategy, Statistics for the public good (2020 to 2025), sets the collective mission for the official statistics system as, "High quality data and analysis to inform the UK, improve lives and build the future". It is based on four pillars: radical, ambitious, inclusive, and sustainable.

Our development plan reflects how our consumer prices development work contributes to the strategy. We contribute by efficiently producing quality and relevant price statistics to meet users' needs, while keeping pace with evolving methods, sources, and digital processes. 

Items on the development plan are prioritised through discussion with our Advisory Panels on Consumer Prices. We expect to make good progress in the high priority areas, where work streams would only be stopped if it were necessary to ensure the publication of our main outputs.  

Medium priority items will be pursued as resource allows; however, some work may be slowed down or stopped altogether to allow progress on high priority items. Low priority items will only be taken forward where resource is available. This will not prevent high or medium priority workstreams from progressing. 

The work programme for consumer price statistics has been updated to reflect the views of the Advisory Panels on Consumer Prices, following discussion with both the Technical and Stakeholder Panels, and, indirectly, feedback from wider stakeholders.

Some aspects of the current work programme reflect the outcome of the 2015 Johnson review on consumer prices. They also reflect the priority placed on the use of alternative data sources by Professor Sir Charles Bean's review of economic statistics (the Bean Review). 

Recent work, since our last update of the consumer prices development plan, includes:

• June 2022 - as part of our biannual consumer price statistics research series, published our first research indices using new data for rail fares and new data for second-hand cars in consumer price statistics

• May 2022 - produced an historical series for Consumer Prices Index including owner occupiers' housing costs (CPIH) 

• May 2022 - published fourth preliminary estimates of the Household Costs Indices, reflecting the period between 2005 and 2021

Historical changes to consumer price statistics can be found in Appendix 1 of our consumer prices technical manual.

High priority items 

High priority items are the main element of our development programme and, if necessary, will be prioritised over the delivery of medium and low priority items.

Transformation of consumer price statistics 

We are currently working through a comprehensive transformation programme for consumer price statistics, to modernise their measurement and make better use of data and methods that become available to us. This is currently our highest priority, beyond producing monthly outputs, and will deliver the largest change to consumer price statistics in a generation.

At a high level, this involves obtaining robust sources of alternative data and the development of statistical systems to work with these data. It also involves methodological research to effectively classify, validate and construct high quality price indices from new data sources. We will also consider how these new processes can be effectively embedded into the monthly business process. These new data sources will be used in conjunction with traditionally collected data to improve the accuracy, efficacy, and representativity of consumer price inflation statistics. 

We are investigating a range of data sources for different item categories. See more information in our Transformation of consumer price statistics: April 2022  article.

Our roadmap sets out our plans to incorporate alternative data sources into our headline measures of consumer price statistics from 2023.

Since 2020, we have worked on developing the methods and systems for alternative data sources. Our research and developments in the transformation of UK consumer price statistics articles series, released biannually, provides an update to users on our work. The redevelopment of private rental prices statistics are anticipated to be used to measure owner-occupiers' housing costs in the Consumer Prices Index including owner occupiers' housing costs (CPIH).

The current phase of the project involves the publication of aggregate research indices, using new data and methods, as well as a range of impact analyses. Following feedback from our stakeholders, we have decided on a publication schedule to allow for user scrutiny and feedback.

Our publication schedule and a more detailed timeline can be found in our Transformation of consumer price statistics: April 2022 article.

While we expect this publication model to be followed in future years, we will continue to gather feedback from users on this approach to ensure we are best meeting stakeholder needs. We are currently working through this publication schedule for rail fares and second-hand cars, and in June 2022 published our latest research on using new data sources.

Developing the Household Cost Indices (HCIs) 

Work to develop the HCIs was first suggested in the Johnson Review (2015) before being proposed by Astin and Leyland (as the Household Inflation Index). The HCIs were first published in 2017 with the HCI fourth preliminary estimates published in May 2022.

We are committed to developing and delivering the HCIs as a quarterly measure in the future. With the current sharp rise in the cost of living and an increased stakeholder need for more timely estimates, we will develop a sustainable system to produce the existing HCIs on a quarterly basis in 2023.

We continue to refine the methods of the HCIs, and there are several concepts that will need to be explored further. One of these concepts is the method used to measure owner occupiers' housing costs on a payments basis, more specifically the method in which we measure mortgage interest payments.

Once our work on the HCIs has progressed, we will consider development options for other versions of HCIs that may be of use to users, such as the HCI-Capital (HCIC) variant.

See our article Developing the Household Costs Indices for more information.

Cost of living

With recent rises in inflation and the cost of living, we have prioritised work to provide additional insights for users using new, innovative analysis and the development of our current measures such as the HCIs.

Recent new insights include:

• a personal inflation calculator

• new experimental analysis such as the lowest-cost grocery items

• analytical analysis such as our recent cost of living synthesis article

We are continuing work to produce further insights and engaging our stakeholders to ensure we meet user need.

COICOP 2018 implementation

Classification of individual consumption by purpose (COICOP) is the classification structure that underpins CPIH and the Consumer Prices Index (CPI). It is also used to construct Household Final Consumption Expenditure estimates as part of the System of National Accounts. These data are subsequently used to weight the CPIH and CPI basket of goods and services. Revisions to the COICOP structure were finalised by the United Nations in 2018. Work will therefore be required to ensure we can restructure CPIH and CPI to account for these revisions by 2025, to remain in line with international best practice and standards.

Medium priority items 

Medium priority items form an important part of the work programme but, if necessary, delivery may be delayed for high priority items. 

Developing measures of accuracy for CPIH 

As a result of the complex survey design, calculating standard errors for CPIH and, specifically, the growth in CPIH, is challenging with two dimensions to the sampling: selection of items and selection of outlets.

We published our article on the effect of variance in the weights of CPIH in the Autumn 2017 edition of the Survey Methodology Bulletin Work on the effect of variance in the prices was reviewed by the Advisory Panels on Consumer Prices in 2018

As a result of this work, we published an academic review on producing standard errors for consumer price indices and plan to publish further work on:

• publication of interim estimates of components of the CPI sampling variance based on CPIH data 

• an academic paper presenting estimates of components of the CPI sampling variance based on CPIH data 

• extending variance estimation to new forms of data to be included in CPIH and CPI (rail fares transaction data, second-hand cars web-provided data, grocery scanner data, rental microdata, and web-scraped data)

Regional indices 

Regional price indices are an important interest for our stakeholders and will improve granularity of other high priority work, such as the HCI and subgroups on a CPIH-consistent basis.

The regular collection of prices for consumer price inflation statistics is optimised for measuring inflation at the UK level. This means that the number of locations visited per region is small and the prices for many products are collected centrally. The viability of using the existing price data and small areas estimation to produce regional expenditure estimates was explored in November 2017 and February 2019.

As a result, we are now looking at producing an experimental subnational consumer prices pilot for Northern Ireland. The project will look at the feasibility of improving the granularity of the statistics by boosting the CPIH sample in this region, with the longer-term aim to supplement this with alternative data sources. Further information is detailed in a case study in the Government Statistical Service (GSS) subnational data strategy.

As part of our transformation work, we are also restructuring the consumer prices hierarchy from 2024 so that, in future, we can more readily produce regional price indices. More details of how we plan to weight and aggregate consumer price statistics can be found in our Introducing alternative data into consumer price statistics: aggregation and weights article.

Improvements to owner occupier housing costs methodology 

Alongside the work to redevelop our private rental prices statistics, there are other ways in which the owner occupiers' housing costs (OOH) component of CPIH could be improved.

The Johnson Review (2015) suggests a flow measure may be worth considering; that is, only new lets. We investigated the feasibility of measuring the flow of rents and concluded that we do not currently have data sources available to us to disaggregate new rents from existing rents. We will review this as new data sources become available.

Northern Ireland rent prices are currently collected as part of the wider CPIH price collection, but we are investigating the potential of using data from the Northern Ireland Housing Executive to improve the sample.

Further quality assurance of CPIH 

As part of the work to seek re-accreditation for CPIH as a National Statistic, we developed a Quality assurance of administrative data (QAAD) document for our consumer price statistics. The QAAD highlighted several areas where the quality assurance for some data sources could be improved. We continue to seek the required assurance for these sources and will aim to update the QAAD biennially. 

Barcode pilot

Since June 2021, we have run a pilot collection of Global Trade Item Numbers (GTINs or barcodes) when collecting prices from physical outlets. We aim to use this to match products and prices more effectively between our alternative and traditional data sources. This will enable improved quality assurance procedures and the potential to directly substitute traditional data with alternative data if, in the future, we are unable to collect prices in stores. From January 2023, we will begin to collect barcodes for all products in all stores.

Improvements to elementary aggregate indices 

In 2010, the Office for National Statistics (ONS) made several changes to the methodology used to collect clothing prices. These changes meant that the gap between Retail Prices Index (RPI) and CPI, which use different formulae at the lowest level of aggregation, widened.

We will consider recommendations from the Johnson Review to review and publish the criteria for formula selection at the lowest level of aggregation.

Low priority items 

The delivery of low priority items may be delayed or stopped to ensure the delivery of high and medium priority items. 

Review of quality adjustment and monitoring of quality change 

In response to the Johnson Review, the Bean Review, and stakeholder engagement, we continue to develop tools for monitoring quality adjustment methods used in consumer price inflation statistics and provide more detail on how quality adjustment is monitored. 

Improvements to owner occupiers' housing costs net acquisitions (OOH(NA)) 

We currently produce an experimental net acquisitions index for Eurostat, as part of a pilot to incorporate OOH costs into the Harmonised Index of Consumer Prices (HICP). We will explore data sources that could be used to improve this experimental index.

Inclusion of financial intermediation services indirectly measured (FISIM) in CPIH 

FISIM are included in the National Accounts measure of household final consumption expenditure (HHFCE) but are not currently included in consumer price indices. The scope of CPI, which is governed by European legislation, is drawn from the same source as HHFCE. The exclusion of FISIM is one of the biggest differences between the two.

As CPIH is not bound by the same legislation, we will consider the suitability of including FISIM as a proxy for the service charge that households pay to banks. We will consider conceptual appropriateness, international practice, and methodology used to calculate a price index for FISIM. 

Discounting pilot

We have been collecting data on multibuy discounts alongside our consumer prices for several years. Multibuy discounts are not currently included in consumer price statistics as there is little information as to how many consumers take-up the available offers. In future, we will look to understand the take-up rates of multibuy discounts using data from our scanner retailers, and whether this can generalise to the discounting information we receive as part of the traditional price collection.

We will continue to keep users updated with the progress of our consumer prices development plan and publish an update of this article annually.