Coronavirus and the effects on UK prices

This document sets out our plans for data collection, compilation and publication of our various prices statistics following the implementation of social distancing policies and movement restrictions brought into effect on 23 March 2020 as a result of the coronavirus (COVID-19) pandemic. This is in line with international guidance on the compilation of the Harmonised Index of Consumer Prices (HICP) during the COVID-19 pandemic. Our range of consumer price statistics includes the Consumer Prices Index including owner occupiers’ housing costs (CPIH), the Consumer Prices Index (CPI), the Retail Prices Index (RPI) and experimental publications, such as the Household Cost Indices (HCIs), which are published annually. We also collect business prices – input and output Producer Price Indices (PPIs) – as well as house prices and private rental price data.

For consumer price statistics, approximately 80% of the price quotes in our CPIH sample (excluding administrative data sources) – or 45% by weight – are collected by price collectors who would usually visit outlets locally across the country every month. Because of the current movement restrictions, we were unable to field a local price collection for the April index (collected around 21 April) due to be released 20 May 2020. This situation is likely to continue for as long as social distancing policies are in place.

We have therefore instructed Kantar UK, which are contracted to carry out price collection on our behalf, to ask their price collectors to collect as many of the price quotes in our sample as they can from retailers’ websites, making early comparable and non-comparable replacements (see Field collection of consumer prices for more detail) if it is necessary to maintain the sample size. The collection period will also be extended to the week before and around “index day” (the second or third Tuesday of the month, on which most of our prices are normally collected) to allow time to collect as many price quotes as possible.

Under the plans set out in this article, for retailers without a website, Office for National Statistics (ONS) staff will phone outlets to ask them to provide price quotes or, if more than five price quotes are placed in that outlet, to return prices through an email form. Some prices are usually collected in-house by ONS staff through websites, phone calls and emails, and these collections will continue as normal. In general, some goods and services are not available at all (such as drinks at a pub), and neither ONS staff nor Kantar UK will attempt to collect these prices.

For goods and services that remain available to consumers to purchase, any missing prices will be excluded from the calculation, in line with our usual procedures. In cases where the sample is less than 50% of its usual size, we will include a marker in the bulletin to indicate this.

We consider two scenarios: “available” items, where goods or services can still be purchased by consumers (although it may not be possible to collect any prices for them if they are out of stock in all of the outlets in our sample), and “unavailable” items, where consumers can no longer access the market because it has effectively been shut down (for example, hairdressers or leisure centres). The scenario that applies to a particular item will dictate how we treat missing data for that item.

For available items that we have not been able to collect, we aim to make an imputation that reflects the price movement that we have missed by not being able to collect the data. Where a sample is less than 20% of its usual size, we will review on a case-by-case basis. In extreme cases, we may consider replacing the data with an imputation. An imputation will also be made for categories in which it has not been possible to collect any price quotes at all. The imputation will be based on the most appropriate choice of:

• imputing from the index immediately above it in the classification structure

• imputing based on the price movement of a similar item

• carrying forward prices

For unavailable items, where consumers can no longer access the market because it has effectively been shut down under current movement restrictions, we aim to make an imputation that has no impact on the all-items index, so that the calculation reflects only the price movements of those goods and services that consumers can purchase. We will therefore impute the index based on the all-items price movement of available items for which we have data. This will be either the annual growth or the monthly growth depending on whether the historical series tends to exhibit repeating seasonal patterns or not. This approach allows the all-items index calculation to be relatively unaffected by unavailable items. There are 92 items in our basket of goods and services that we have identified as unavailable for the April 2020 index (see Annex B), which accounts for 16.3% of the CPIH basket by weight. The list of unavailable items will be reviewed on a monthly basis.

Under Section 21 of the Statistics and Registration Services Act 2007, the Bank of England must make a determination on any changes to the coverage or basic calculation of the RPI that we propose, to establish whether such a change “constitutes a fundamental change in the index which would be materially detrimental to the interests of the holders of relevant index-linked gilts”. If so, the Chancellor of the Exchequer would need to consent to the change. We have shared this plan with the Bank of England, and they have determined that none of the planned temporary changes outlined “were both fundamental changes to the coverage or basic calculation of the RPI, and also materially detrimental to the holders of relevant index-linked gilts”. As a result, the Chancellor’s consent is not required to implement these temporary changes. The correspondence is available.

The plan for consumer price statistics will be carried out with immediate effect for the April index, with publication on 20 May 2020, and it will continue indefinitely until such time as it is considered safe to resume an in-the-field local price collection and goods and services are once more available to consumers. We will regularly review the situation to identify when these goods and services become available again. The procedures described in this plan should be considered temporary. Implicit in this proposal is the expectation that existing procedures will be resumed at some point in the future.

Our plans are consistent with Eurostat guidelines on the compilation of the HICP during the COVID-19 pandemic.

We also publish a range of price statistics that do not form part of our range of consumer price statistics: housing market indices and business price indices'.

The ONS and the joint producers have taken the decision to temporarily suspend the UK House Price Index (HPI) publication from the April 2020 index (due to be released 17 June 2020) until further notice. The impact of COVID-19 is expected to greatly reduce the amount of housing transactions that took place in April 2020, making it very difficult to produce a measure of UK house prices that would be representative of any true transaction activity within the housing market. We will continue to closely monitor the flow of transaction data with the view to reinstating the UK HPI as early as is practicable. Further details will be available in the March 2020 UK HPI, due to be released 20 May 2020.

The UK HPI is used to calculate several of the owner occupiers’ housing costs components of the RPI. The procedures described in this plan apply to those components of the RPI that are based on the suspended UK HPI data.

The PPIs and Services Producer Price Indices (SPPIs) are business surveys and therefore are reliant on businesses providing data via postal questionnaire and telephone data entry. The COVID-19 pandemic has meant we have had to consider alternative approaches to contact sampled PPI and SPPI businesses to collect price data, such as via email. To date, there has been a small impact on the survey’s response rates and therefore the quality of these statistics. We intend to continue to publish both the PPI and SPPI, and we will keep on monitoring quality closely, informing users of any further impact on the published figures.

The first case of the coronavirus (COVID-19) was reported to the World Health Organization (WHO) in December 2019 and was subsequently declared a public health emergency of international concern (PHEIC). This global pandemic is now expected to impact on the economic outlook for some time to come.

The latest available business surveys in the UK and real-time indicators point to a significant decline in economic activity, while the initial estimates for other countries highlight the extent of the impacts so far on the real economy. The response to COVID-19 will also impact on the ability of National Statistical Institutes (NSIs) to compile estimates of gross domestic product (GDP), inflation and the labour market as well as estimates for the economy more generally. We are responding to these challenges so that we capture the economic activity of the UK in line with the latest international guidance.

The UK government’s social distancing policies and the subsequent movement restrictions, implemented on 23 March 2020, are likely to have caused unusual price movements because of the high demand and low supply and significant shifts in the expenditure distribution, with many items in the basket of goods and services no longer available to consumers to purchase. This latter issue means that many of the items in our “basket” no longer have a “price” associated with them. Moreover, the expenditure patterns for items that consumers can still purchase are likely to look very different to our base data.

We have considered treating this from two perspectives: adjusting expenditure weights or adjusting the treatment of prices to reflect the economic reality. However, adopting either of these treatments in this way is impractical for a number of reasons. There is also a balance to be struck between reflecting economic reality and ensuring consistency both between countries and over the time series, where a particular consideration is yearly comparisons.

Since other countries are experiencing similar issues with their price collection and compilation, National Statistical Institutes have worked together to develop consistent international guidelines, in order to maintain international comparability and ensure the application of best practice under the circumstances. We have also taken advice from our Technical Advisory Panel on Consumer Prices.

However, recognising that there is some value to users in understanding how shifts in the expenditure distribution would affect our measures of price change, we will also publish supplementary analysis alongside our Consumer Price Statistics, UK bulletin. This will reflect the reality that some areas of the basket are unavailable to consumers by removing unavailable items from the basket and reweighting the remaining indices. This analysis will help users to understand how current expenditure might impact on estimates of price change; however, the lack of a reliable source of current expenditure data would make this approach inappropriate for our core outputs. The approach would also not be consistent with the fixed basket concept on which our consumer price statistics are based, and therefore could not be incorporated as part of the time series, particularly as we cannot rule out further shifts in the expenditure distribution.

Consumer price indices are complex statistics and are compiled from a range of different sources and types of data, many of which are harder to collect given current movement restrictions. This plan sets out a high-level description of what we will do to maintain our suite of consumer price statistics (which incorporates data from our housing market indices) in the face of the ongoing coronavirus pandemic; however, pragmatic ad-hoc decisions in some cases will be necessary to deal with situations as they unfold. In these cases, the plan sets out the basic principles that we will follow but may not provide detail for every eventuality.

The Statistics and Registration Service Act (SRSA) requires us to compile and publish the RPI every month. Under European regulations we are also required to supply the Harmonised Index of Consumer Prices, which is the same as our CPI, to Eurostat each month. The contingency plan allows us to meet both of these obligations.

Operational challenges

Where their baskets overlap, the Consumer Prices Index including owner occupiers’ housing costs (CPIH), Consumer Prices Index (CPI) and Retail Prices Index (RPI) are all calculated from the same price data. Approximately 80% of price quotes collected (excluding administrative data sources) are from stores around the country, but this only accounts for approximately 45% of the CPIH basket by weight (56% of the CPI basket and 45% of the RPI basket), as many of the items carrying the biggest weight are collected centrally. The collection is carried out by a third-party supplier, Kantar UK. The remainder are collected centrally by Office for National Statistics (ONS) staff through websites, phone calls, and other means such as email correspondence and brochures. Prices are collected approximately one month before publication, on or around “index day” (the second or third Tuesday of the month), and are published with a lag of one month, so that the monthly Consumer price inflation bulletin refers to prices in the previous month (for example, the April 2020 bulletin reflects prices as observed in March 2020).

The local price collection for the March index released 22 April 2020 was carried out in stores as usual. The sample obtained was around 90% of its usual size and was complemented by a central collection where the sample was nearly 100% of its usual size. Coverage levels were not even across items because consumers stockpiled certain non-perishable and essential items, so these were not available in stores for their prices to be collected. There were three subclasses where the sample was less than half of its usual size: flour, pasta and eggs. We have published experimental research separately on price changes for certain high-demand products (HDPs), such as hand sanitiser, long-life milk, toilet paper and antibacterial wipes.

For the April index due to be released 20 May 2020, as a result of social distancing and movement restrictions it will not be possible to send Kantar UK price collectors out into the field to carry out the local price collection in physical outlets across the country. This may extend beyond April, as the length of the restrictions is currently uncertain.

As with March’s collection, there may be ongoing stock availability issues. There are also a number of goods and services that are temporarily unavailable, for example, hairdressers or leisure centres.

Contingency plan

We therefore propose a contingency plan of action to address these issues. The first part of the plan details the steps that we will take to maintain a local price collection. The second part of the plan details how we will treat unavailability of goods and services when calculating the index.

The plan will be carried out for the April price collection, with publication on 20 May 2020, and will continue until field collection can be undertaken again. The procedures described in this plan should be considered temporary. Implicit in this proposal is the expectation that the existing procedures will be resumed once goods and services become available on the market again and it is safe to collect prices in the field, in line with UK government advice.

Field collection of consumer prices

This section of the plan refers to the scenario where a field collection of local prices cannot be carried out because of social distancing policies and the movement restrictions implemented by the UK government on 23 March 2020. This part of the plan has already been carried out for the April index; however, the details equally apply to what we will do in future months should movement restrictions continue.

Under our price collection plans, Kantar UK price collectors will collect as many of their prices as possible through alternate means. The ONS will provide a list of items that should continue to be collected; items that are currently unavailable have been removed. An item¹ is defined as available if it is possible for a consumer to purchase it either from outlets that remain open or through alternative means, such as the internet.

As part of this exercise, the ONS will also decide which items it should continue to collect centrally each month, prior to the collection day. The full list of unavailable items for the April 2020 index is provided in Annex B. Of this list, 92 items are classed as unavailable. It should be noted that some items are only in the CPIH and CPI basket and some are only in the RPI basket, so the true number for each measure is slightly lower: 90 items for the CPIH and CPI and 89 items for the RPI. This accounts for 16.3% of the CPIH basket by weight (20.2% of the CPI basket and 17.7% of the RPI basket). The list will be reviewed on a monthly basis, as it is possible that different goods and services may become available again at different times.

The ONS has identified the list of items by consulting trade websites, GOV.uk advice, retailers’ websites and other relevant sources.

For those multiple stores (any retailer that has 10 or more outlets) and independent stores (any retailer that has fewer than 10 outlets) that have a website, price collectors will use retailers’ websites to identify price quotes. Items that are out of stock on the website are not priced, and price collectors may need to navigate to the basket to confirm this. While there is an argument for including prices for out-of-stock items, it would not be possible to apply this on a consistent basis, as there will not always be a price available. Moreover, if out-of-stock prices are extreme, their inclusion in the calculation may be hard to justify. However, it may not always be possible to confirm whether a product is in stock and in these situations, we assume that the item is available and it will be priced.

For retailers without an online presence, this will involve contacting as many outlets as remain open through alternative means. ONS staff will contact these outlets by phone and ask them to provide price quotes. If more than five price quotes are placed in an outlet, we will ask them to return a price list by email. Consideration will be given to the burden placed on retailers with this approach.

The ONS will confirm with Kantar UK which items they should attempt to collect prices for, based on the list in Annex B. Kantar UK will then use this item list to provide intelligence ahead of “index day” on what prices they will be able to collect from retailers’ websites and which stores will need to be contacted by phone.

As far as possible, pricing will follow the existing price chain. Kantar UK collectors will have access to the usual handheld devices that they use to record prices. They must attempt to price the same product as in the previous month; however, under the contingency conditions described here, an immediate replacement may be drawn for website collections if this is not possible (under normal conditions, a collector would wait three months before replacing a product).

Where possible, this should be of the same quality as the previously priced product (that is, a comparable replacement, “C”), but a change in quality (a non-comparable replacement, “N”) is also permitted if no comparable replacement exists (see Table 1 in Annex A for a description of marker codes and their meaning). Under our usual procedures, a base price for non-comparable products is imputed based on the price movement of similar products (for more information on our replacement procedures, please refer to the Consumer Prices Indices Technical Manual, 2019). Kantar UK price collectors will mark all immediate replacements to indicate that they are because of the COVID-19 pandemic. Note that this procedure only applies to website collections, as it will not be possible to identify a replacement product through a phone call or email. Once normal service resumes, the immediate replacements must be switched back to the product that was originally priced, once it becomes available again.

Price collectors will not switch between outlets – they must draw a replacement from the same outlet or, if this is not possible, treat it as temporarily missing (a “T” marker). More detail about these codes and their meaning can be found in Table 1 in Annex A.

The collection period will also be extended to cover the full week containing “index day” as well as the preceding week. This will allow time for as full a collection as possible, and it will hopefully mitigate for any low samples. This step is not without precedent, as many prices are collected the day before or the day after “index day”, and some central items are collected in different weeks altogether. This approach effectively favours maximising the sample size over keeping the comparison period consistent through time.

Under existing collection procedures, prices for fruit and vegetable items are collected twice: on the Friday preceding “index day” and again on “index day” itself. Under the contingency arrangements described here, the additional Friday collection will only be conducted by visiting retailers’ websites. The index day collection, however, will be carried out as described earlier, through a combination of website collections and phone calls to retailers.

Central collection of consumer prices

For the central collection, we will continue to attempt to collect prices in line with our usual procedures, for example, through websites and phone calls and by email correspondence. However, we will not collect prices for any items that we have deemed to be unavailable (Annex B).

There may be issues with some administrative data sources, where the third-party agency collecting the information chooses not to collect the required information or does not transmit it to us. In these cases, we will still make every effort to ensure that the data are supplied.

Housing market price data feed into many components of our consumer price statistics. Rental data feed into private and social rental price indices (approximately 6.5% of the CPIH), and the private rental data also feed into the rental equivalence measure of owner occupiers’ housing costs used in the CPIH (approximately 16% of the CPIH). House prices feed into various aspects of owner occupiers’ housing costs used in the RPI, notably the measures of depreciation and mortgage interest payments (approximately 8% and 2% of the RPI respectively).

It should be noted that the methodologies used in our consumer price statistics for many of these measures tend to give smoothed estimates of price change and will therefore change slowly. This will help to mitigate the impact of any missing data. For treatment of such items, please refer to Imputation for missing available items.

Limitations to the approach for collection of consumer prices

ONS staff and Kantar UK collectors are aware that an unusually high number of comparable replacements will increase the volatility in the index, since there will inevitably be a price change (whether upwards or downwards) from switching from one product to another. However, this procedure mitigates against loss of sample size that would arise from not being able to collect the original product’s price. On the other hand, a large number of non-comparable replacements will work to reduce the sample size in the first month of introduction. However, this does allow us to maintain the sample size for later months, once it starts to affect the index calculation (for more information on imputation for non-comparable replacements, please refer to Section 9 of the Consumer Prices Indices Technical Manual, 2019). If a comparable or non-comparable replacement is drawn, care will be taken to make sure that the replacement product is as closely comparable as possible. In particular, the formula used at the lowest stage of aggregation in the RPI (the Carli formula) is known to give poor estimates of price change when the sample is volatile, which may be the case with the issues described earlier.

We may therefore need to remove some large outlying price changes. This will be done using existing outliering methods, such as the Tukey algorithm or percentage change checks (for more information please refer to Section 7 of the Consumer Prices Indices Technical Manual, 2019). Any extreme price changes resulting from an immediate replacement will then be removed at the scrutiny stage (unless the prices analyst has good reason to believe that the price should be retained). We will also provide Kantar UK with clear guidance on making replacements, to mitigate this scenario as much as possible.

Care will also be taken where base prices are used for small convenience stores from multiple retailers. In these circumstances, we might expect that online prices for the same product are lower, and therefore the price relative will not reflect a genuine price change. Moreover, this would introduce a downward bias into the index. Exploratory analysis suggests that these types of convenience store represent approximately 12% of price quotes in the food and drink basket (although items placed in convenience stores are not limited to food and drink). As these stores represent a significant proportion of the basket, Kantar UK will be asked to mark these price quotes with an “N” marker, to reflect the non-comparability of multiple convenience store and web-based prices.

A similar scenario arises for the items, “Pub – hot meal”, “Restaurant – main course” and “Restaurant – sweet course”. Movement restrictions have led to many food outlets offering their meals to consumers as takeaway items instead. While it is possible to continue collecting prices for these items, the price is not representative of the same thing because the service that a consumer gets from eating a meal in a restaurant, is not of a comparable quality to consuming a takeaway at home. Therefore, we will continue to collect these items, but they will be treated as non-comparable to control for the differing levels of quality.

Once we are able to resume a field collection for convenience stores and eat-in food outlets, we will revert to collecting the originally priced products.

A further consideration is that websites will most likely reflect national prices rather than regional price variations. Under our usual price collection procedures, prices are collected in different regions across the UK, ensuring that any regional variation in prices is captured in our all-items consumer price indices. This will not be the case for website collections.

There may be further unintended biases in our measures. For example, an article published by the Institute of Fiscal Studies (IFS) suggests that retailers are reducing the number of buy-one-get-one-free-type offers. Our consumer price statistics do not account for these types of offers, but consumers who would normally have taken up the offer now have to pay more to purchase the same quantity as before.

Finally, if the locally collected sample is deemed insufficient, then we would also consider trying to extend the price series using any of the research data sources we currently hold, such as web-scraped or scanner data. This was not necessary for the April price collection.

Housing market indices

The ONS publishes monthly measures of housing market inflation. These measures are the UK House Price Index (HPI) and the Index of Private Housing Rental Prices (IPHRP).

The UK HPI is a joint publication with other government departments and agencies, and it measures the change in the transaction value of residential properties using sales data collected on residential housing transactions, whether for cash or with a mortgage. Records only appear in the UK HPI once the transaction has been completed. The index includes all residential properties purchased for market value in the UK via data sources from HM Land Registry for England and Wales, Registers of Scotland, and HM Revenue and Customs’ (HMRC’s) Stamp Duty Land Tax data for Northern Ireland.

The IPHRP measures the change in the price tenants face when renting residential property from private landlords. It represents all rents, so it includes newly let properties (the flow of rents) and existing rents (the stock of rents). The index is constructed using administrative data on private rental prices from the Valuation Office Agency (VOA), Scottish Government, Welsh Government and Northern Ireland Housing Executive (NIHE).

The COVID-19 pandemic appears to have had a significant impact on the UK housing market. Anecdotal evidence suggests that in response to the government measures to reduce the spread of COVID-19, very limited actual house transactions are taking place; some estimates suggest less than 10% of the UK’s usual house sales.

Following discussions with data suppliers, it is clear that very limited housing transactions are expected so it would be very difficult to produce a meaningful measure of UK house prices that would be representative of any true transaction activity within the housing market. As a result, we have taken the decision to temporarily suspend the UK HPI publication from the April 2020 publication (due to be released 17 June 2020) until further notice. This decision has been made following detailed discussions with our publication partners. We will continue to closely monitor the flow of transaction data with the view to reinstating the UK HPI as early as is practicable. Further details will be available in the March 2020 UK HPI, released 20 May 2020. We will also be considering the most suitable approach for retrospectively publishing data for those months that have been suspended, should later data prove to be of a high enough quality, and we will provide users with details in due course.

While data availability as a result of the COVID-19 pandemic does pose a potential risk to the publication of the IPHRP over the coming months, discussions with our data suppliers have indicated that the supply of data is expected to be relatively unaffected as rental transactions continue to take place, and these rental data are being collected via electronic means (email and telephone). This also applies for the rental equivalence measure used to measure owner occupiers’ housing costs in the CPIH. We therefore expect to continue with the publication of the IPHRP and other private rental measures, but we will continue to review the impact of the COVID-19 pandemic on the price data being received.

The procedures for imputing missing data in our consumer price indices, described in Imputation for missing available items, apply equally to the treatment of UK house price data for use in the calculation of the owner occupiers’ housing elements of the RPI.

Notes for: Price collection

1. Note that price quotes for an item are collected from a range of outlets; therefore, the closure of one particular retailer does not necessitate the need to treat that item as unavailable.

Under the contingency plan for price collection, described earlier, Kantar UK will give all missing locally collected prices T markers and treat them as temporarily missing. When a T marker is applied, the price relative is excluded from the index calculation. This is consistent with existing collection procedures for consumer price statistics.

Imputation for missing available items

For those items defined as “available” – that is, they are still available to consumers to purchase in some form – we are in theory able to collect prices through some means. However, low stocks or lack of access to price data may lead to small sample sizes, which may cause quite volatile index movements. This could affect the all-items Consumer Prices Index including owner occupiers’ housing costs (CPIH), Consumer Prices Index (CPI) and Retail Prices Index (RPI). For both locally and centrally collected items, we will therefore review all indices at the lowest aggregation stage for available items whose sample has fallen below 20% of their usual size and assess their reliability. In extreme cases, we may consider making an imputation rather than using the data collected. This will be done only if strictly necessary and on a case-by-case basis.

If we have been unable to collect any price data for an item, or if the sample of price quotes is less than 20% of its usual size and we have decided to remove the price data, then an imputation will be made based on the method that best reflects the price behaviour of the missing item. This is likely to mean imputing based on the parent index movement at the next stage of aggregation.¹ In effect, this means elementary aggregate indices will be imputed from items, and so on up the hierarchy. For example, the item index for apples feeds into the subclass for Fresh or chilled fruit. Therefore, if the price index for apples needs to be imputed, we would use the price movement of Fresh or chilled fruit excluding apples. We may also impute based only on similar items if the parent index is not representative, or we may carry forward prices if the price development tends to be stable over time.

This approach relies on the principle that there is a price movement associated with these items; however, we have been unable to capture it under the price collection arrangements described in this article. We will therefore make an imputation based on the most reasonable assumption. A flow chart of the decision-making process is provided in Determination of imputation methods, later in this section. A worked example of how the imputation will be applied is provided in Annex C.

A particular consideration is where it is not possible to collect any prices from independent retailers in an entire stratum. In this scenario, imputing from the parent index would mean imputing based on the price movement of multiple stores. It may therefore be more appropriate to carry forward prices, if multiple price movements are not a good proxy for independents and the price development is stable.

Options in the event of market shutdowns (unavailable items)

This subsection details how we will impute for items where the good or service is not currently available and it has been agreed that we will not be collecting prices. The aim of this approach is to reflect the situation where a market has shut down and is unavailable to consumers; therefore, there are no prices and expenditure falls to zero. Our consumer price statistics are Laspeyres-type indices, where basket quantities are fixed at some point prior to the price reference period. This is to avoid price movements becoming compounded by shifts in expenditure patterns. In theory, therefore, the best approach would be to stop the index and begin a separate index with a new basket that has current expenditure patterns as the base. However, this is inconsistent with the requirements of both domestic and European legislation and with current procedures. In addition, it would place extra burden on already stretched resources, and the current expenditure patterns required are unavailable. It is possible that they may change again. We therefore have decided to extend the existing price series, even though it is not possible to record prices. This approach is consistent with international guidelines.

Imputation for unavailable items

For the production of the Office for National Statistics’ (ONS’) consumer price statistics, we will impute price movements for unavailable items based on the monthly growth in the all-items index for available, non-imputed items. This approach is preferred as it means the monthly development of the all-items index is unaffected by unavailable items, for which there is no longer any price. This aligns with Eurostat guidelines.²

To support our preferred methodology, we will also produce a supplementary economic analysis of the CPIH and CPI. This analysis will aim to reflect how the CPIH and CPI would differ if we were to reweight the basket to exclude unavailable items. This experimental analysis is to provide users with a greater insight into how the changes in consumer expenditure during this period are likely to affect our measures of consumer price inflation. This analysis is intended as a complement to our main measures; our preferred approach to measuring inflation remains that described more widely in this article. This is because we do not have a reliable source of current expenditure data for those items that are available on which to base the supplementary analysis. The supplementary approach would also not be consistent with the fixed basket concept on which our consumer price statistics are based and therefore could not be included as part of the time series without incorporating additional linking procedures; this is worsened by the fact that we cannot rule out further shifts in the expenditure distribution.

Imputation for unavailable seasonal items

There is also the question of how seasonal prices should be treated. If we aim to reflect the reality that some items are unavailable, then just as the price no longer exists, there is also no longer any seasonality, and imputing a seasonal pattern would be inconsistent with the reality. However, a consequence of this would be that the annual growth rates would be affected and that these effects also would feed through into the annual growth rates for next year.

The imputation procedure for unavailable items described in the previous subsection aims to preserve the integrity of the all-items index’s monthly price development by ensuring that the resulting calculation reflects only the price movements of those items that are available to purchase and for which, therefore, a price exists.

We have considered the choices involved in maintaining the integrity of either monthly growth or annual growth. For seasonal items, imputing the monthly growth would lead to a break in seasonal structures that will affect annual growth rates in subsequent years. Therefore, for any such seasonal items, we will instead impute based on the annual all-items growth for available items. This ensures the seasonal pattern from the year before is generally maintained, with price growth between last year and this year coinciding with the price growth for all available items. A worked example of how the imputation will be applied is provided in Annex C.

Of the unavailable items, 13 have been identified as seasonal. TThis accounts for 2.1% of the CPIH, by weight (2.6% of the CPI and 1.8% of the RPI). The list of seasonal and unavailable items for April 2020 is provided in Annex B.

Determination of imputation methods

The decision tree in Figure 1 sets out the process that we will follow in determining which imputation method we will use. A series of statistical tests will be used to establish whether series are seasonal and stable and which indices are a good proxy for the price behaviour.

Figure 1: Decision tree for imputation of items

Source: : Office for National Statistics – Coronavirus and the effects on UK prices

Download this image Figure 1: Decision tree for imputation of items

.png (47.6 kB)

The imputation being carried out depends in the first instance on whether the item has been defined as “available” or “unavailable”.

If it is “unavailable”, then we identify whether the item is seasonal. If it is seasonal, then we impute using the aggregate annual growth of all available, non-imputed items. If it is not seasonal, then we impute using the aggregate monthly growth of all available, non-imputed items.

If it is “available”, then we first need to identify whether the sample size is zero or whether it is less than 20% of its usual size and we have made the decision to impute. If neither of these conditions holds, then we calculate the index as usual.

If an imputation is required, then we first establish whether the parent index is a suitable proxy for imputation. If it is, then we use the monthly growth of the parent index, based on available, non-imputed items to impute the growth for the missing series. If it is not, then we look at whether the series is stable over time. If it is, then we carry forward the missing index values. If it is not, then we consider whether other similar items would make a good proxy. If there is a suitable donor, then we impute based on the monthly growth. If there is not, then in the final case we impute using the monthly all available, non-imputed items growth.

Notes for: Calculation of consumer price statistics

1. This is the same method that is used to impute price movements for out-of-season items in our consumer price statistics (Section 9.5.1.1).

2. The International Monetary Fund (IMF) guidance recommends imputation from the parent index. It also suggests that prices should not be carried forward. However, the guidance does not make a distinction between imputation for items that are available to consumers but that we were unable to collect and those items that are unavailable to purchase.

For both locally and centrally collected items, where a component index is based on a large number of imputations or missing products, we will add a marker to the bulletin to indicate if an estimate is based on less than 50% of its usual sample or if it has been imputed. We will also provide response rates (relative to the February 2020 index).

One issue not discussed in this article is expenditure data. We would expect social distancing policies and movements restrictions to have a large impact on consumer expenditure. Unfortunately, since weights are lagged by two years, we would see no effect until we calculate the 2022 weights¹. This means that the current weights are not likely to be reflective of current expenditure and that the 2022 weights are unlikely to be reflective of 2022 expenditure. This may be mitigated to some extent by the fact that the weights are annual and will reflect expenditure across the year rather than in the affected months. We will develop a strategy for treating 2022 weights once conditions return to normal. This will be better informed once 2020 expenditure data become available.

This article has also not discussed how we will return to measuring price change once a normal collection can be resumed. We will update users on this matter in due course.

Notes:

1. For the Retail Prices Index (RPI) weights, this is 2021, since the expenditure used for that year covers July 2019 to June 2020.

The Office for National Statistics (ONS) is making several changes across its consumer, business and house prices bulletins with the aim of making them more concise and easier for users to navigate. For completeness, we have also included one change related to the coronavirus (COVID-19). This process has been informed by web metrics and user feedback, and we aim to stagger these changes over 2020. The proposed changes are outlined by month in this section.

22 April 2020

The Private rental growth measures, a UK comparison publication has now moved to an annual publication, with a short ‘Comparisons with private sector rental growth measures’ section available every third month in the Index of Private Housing Rental Prices statistical bulletin from April 2020 onwards. The first annual publication is planned for July 2020.

17 June 2020

The House Price Index (HPI) will be suspended from April onwards because of the limited number of transactions recorded because of COVID-19. The HPI will restart once sufficient transaction data are available to produce meaningful statistics.

The Measures of owner occupiers’ housing costs quarterly release will also move to an annual release, with the first annual release planned for March 2021, while a small section will also be made available every third month in the Index of Private Housing Rental Prices (IPHRP) statistical bulletin from June 2020 onwards.

19 August 2020

The Consumer Prices Index including owner occupiers’ housing costs (CPIH) sub-groups quarterly bulletin will move to data tables only; the coverage of the data will remain unchanged, with the quarterly economic commentary being used to undertake a more detailed analysis of these statistics at least once a year or when interesting trends emerge.

The Construction Output Price Indices (OPIs) bulletin will be stopped and the data will form part of the quarterly Construction output in Great Britain bulletin.

16 September 2020

The Consumer price inflation bulletin will continue to be published on a monthly basis with limited changes, but the detailed briefing note will be stopped and replaced by data tables covering the same content, including the monthly records and the outlook section.

21 October 2020

The Producer Prices Index (PPI) bulletin will continue to be published on a monthly basis. The Services Producer Prices Index (SPPI) will no longer be a standalone quarterly publication; instead, it will form part of the PPI bulletin each quarter. The coverage of the data published will remain unchanged.

Late 2020

Our longer-term development plan is to utilise the newly acquired data from the Valuation Office Agency (VOA) to replace the UK IPHRP and private rental market statistics (PRMS) for England with one unified release covering rental growth, indices and levels.  

This plan describes our approach to maintaining data collection for our consumer price statistics, in line with international guidance, in the context of the UK government’s social distancing policy and the movement restrictions in force during the coronavirus (COVID-19) pandemic.

For as long as our price collectors are unable to visit outlets in the field, we will collect as many prices as possible through alternative means: where possible, this will be though retailers’ websites, and where retailers do not have a website, we will attempt to collect prices through phone calls and by email. We will also make immediate replacements for out-of-stock items to maintain a sufficient sample size.

We define items in the basket of goods and services as either “available” or “unavailable”, depending on whether they are still available to consumers to purchase or whether they are unavailable under the UK government’s movement restrictions. For missing available items, we will impute a price movement based on whichever approach best captures the missed price movement. For unavailable items, we will impute a price movement that has relatively little impact on the all-items calculation – reflecting that these items do not contribute to inflation.

Because of the limited number of housing transactions taking place, our UK House Price Index (HPI) will be temporarily suspended from the April index until further notice. Where this is used for the calculation of the Retail Prices Index (RPI), we will impute based on the principles described in Imputation for missing available items.

The procedures that we have described in this article will be put into place for our April consumer price indexes (released 20 May 2020) until such a time as we are able to resume field collection from local outlets.

The latest expectations point to a significant decline in economic activity in the first half of this year at least, reflecting how the COVID-19 pandemic has led to a reduction in the demand for goods and services and the impact on the ability of businesses to supply those products as well as many businesses ceasing operating. Our range of consumer price statistics will help decision makers to understand how supply and demand issues impact on price changes for UK consumers, based on a fixed basket framework. Our supplementary analysis, based on a reweighted basket, will complement these measures by providing a clearer picture of changing prices for items that are available to purchase in the current context.

Additional information has been published on the challenges we face in compiling gross domestic product (GDP) and labour market statistics and how we are planning to respond to those.

Table 2 gives details of the list of items defined as “unavailable” for the April 2020 price collection. These items have been identified through consultation of trade association websites, UK government advice and retailer websites.

In some cases, the decisions on unavailability relate to how a particular item is defined. For example, the guidance for item 410509 “Electrician” explicitly states that this is for a non-essential service, whereas trade association advice suggests that these services should only be available for essential works.

In general, every item is a special case and we have had to consider the specifics of each item in making a decision.

This section briefly sets out how we will impute for two example cases: air fares (which we are treating as unavailable) and Flour, self-raising, 1.5kg (which we are treating as available). Annex B provides the complete list of unavailable items and the imputation method we will be using for each.

C.1. Air fares

UK government guidance advises against all but essential travel overseas. Many flights have been cancelled and many countries have banned entry to non-nationals, further limiting travel options. It is likely that the main use of air travel currently is for repatriation. So, the vast majority of flights in our basket will be unavailable to consumers.

We therefore treat air fares as “unavailable”. Following the approach described in section 4, we have tested the series for seasonality and identified some significant seasonal movements. We therefore impute the annual change in air fares prices based on the annual all-items movement of all available, non-imputed items. This approach allows us to make an imputation that has a relatively neutral impact on the all-items monthly price movement. In other words, the all-items index is predominantly based on the price movements of things that are available and is unaffected by those goods and services that consumers can no longer access. It also maintains consistency between seasonal and non-seasonal imputations.

The following example (based on real CPIH data) demonstrates how the imputation will be made:

To calculate the April 2020 index value for air fares, we take the April 2019 index and apply the aggregate all-items growth of available (non-imputed) items, which we denote by A:

However, because the annual movement crosses a chainlink, this gives us the chainlinked index value for April 2020, which is referenced to January of 2019 (for more information on chainlinking please refer to Section 3 of the Consumer Prices Indices Technical Manual, 2019). We therefore need to re-reference the April 2020 value to January 2020, consistent with other item indices. This is achieved by dividing through by the chainlinked index value in the base period, which for 2020 is January 2020:

Therefore, the April 2020 (Jan 2020=100) index value for air fares will be given by 127.265 multiplied by the annual all-available-items price movement

C.2. Flour, self-raising, 1.5kg

Under UK government advice, essential goods remain accessible to consumers. This includes larder staples such as flour. For this reason, we will continue to treat the item, “flour, self-raising, 1.5kg”, as available, reflecting that it can still be purchased. However, the sample for this item was less than 50% of its usual size for the March 2020 index, released in April. Therefore, this may be an item that is being stockpiled and may suffer from sample size issues in the April 2020 price collection.

Any collected prices will be used to construct the item index. In the event that our sample is less than 20% of its usual size, we will review the series and we may consider replacing it with an imputation if necessary. If the sample size is zero, we will need to make an imputation.

As there are still prices for this product, we want to choose an imputation method that best captures the missed price movements. Following the approach described in Determination of imputation methods, we will apply tests to establish the most effective imputation strategy. In the first instance, we will consider imputing from the parent index at the next stage of aggregation. If this is not an appropriate proxy, we will instead consider carrying forward prices. Again, if this is not appropriate, we will consider whether similar items could form a good proxy and, as a last resort, we would consider imputing from the all-items index.

The following example (Table 4, based on real CPIH data) demonstrates how the imputation will be made, if we were to use imputation from the parent index.

The Flour, self-raising, 1.5kg item feeds into the Flour and other cereals (1.1.1.2) subclass of the Classification of Individual Consumption According to Purpose (COICOP), which in turn feeds into the Bread and cereals class (1.1.1). As Flour, self-raising, 1.5kg is the only item that feeds into the Subclass 1.1.1.2, we therefore calculate the monthly price movement of the Class 1.1.1 for March 2020 to April 2020 based on matched price quotes. We denote this monthly movement by S. Then, we calculate the April 2020 price movement for Flour, self-raising, 1.5kg by multiplying the previous month’s index value by the parent movement S: