Coronavirus (COVID-19) and Consumer Price Inflation weights and prices: 2021

• Under normal circumstances, the Consumer Prices Index including owner occupiers' housing costs (CPIH) and the Consumer Prices Index (CPI) weights for 2021 would have been based on household final consumption expenditure (HHFCE) for 2019 data. However, 2019 predates the large shifts in household spending seen over the course of the coronavirus (COVID-19) pandemic. Therefore, for the calculation of weights for 2021, we have followed international guidance (PDF, 503KB) and made adjustments to better reflect 2020 expenditure for the most heavily affected spending categories.

• As HHFCE data are currently only available to Quarter 3 (July to Sept) 2020 we have had to estimate spending for Quarter 4 (Oct to Dec) 2020, based on the Quarter 4 2019 growth from Quarter 3 2019. Also, the data are in an earlier stage of compilation than would normally be used, and so we would expect that the uncertainty around the 2020 data is greater than usual. Nonetheless, there is a balance to be struck between using representative data and using the best quality data. Although this is usually satisfied by using HHFCE data at a two-year lag, in the context of the coronavirus pandemic, making adjustments to reduce the lag in the most heavily affected spending categories is the best way to strike this balance.

• For the collection of January's reference prices - used to index price quotes in subsequent months for the remainder of the year - we have necessarily conducted an online price collection. This may have impacts on the effective sample size later in the year if the mode of collection changes back to field resulting in a large number of replacements.

• For items that cannot be collected in the reference period, and for which we have imputed price movements, we will impute reference prices based on the imputed price movement once the item can be collected again. Some imputed items are defined as unavailable to consumers and are imputed in such a way that their price movement has a negligible impact on the all-items inflation rate. Others are available to consumers to purchase; however, we have been unable to collect a sample for them. In these cases a price movement has been imputed that best reflects the expected price movement. These imputation procedures have been in place since the April 2020 CPIH, CPI and the Retail Prices Index (RPI), which was the first date at which consumer prices were collected during a period of movement restrictions.

With the outbreak of the coronavirus (COVID-19) pandemic, and the introduction of movement restriction policies in late March 2020, it was necessary to make radical changes to the collection and compilation of consumer price data.

Firstly, price collectors were unable to visit stores and it was necessary to switch the collection mode to predominantly online and, where this was not possible, through telephone collections.

Secondly, some markets were completely closed to consumers rendering the concept of a price meaningless for such items. It was therefore necessary to design imputation strategies to deal with items that were unavailable to consumers.

Thirdly, movement restrictions led to large scale shifts in consumer spending that were not reflected in the fixed basket weights used for the Consumer Prices Index including owner occupiers' housing costs (CPIH), the Consumer Prices Index (CPI) and the Retail Prices Index (RPI, which is discussed separately in Section 3). We published supplementary analysis to aid users in understanding the impact that these distributional effects may have on our lead measures of consumer price inflation.

More detail is provided on our approach to producing consumer price indices in our articles Coronavirus and the effects on UK prices and Consumer prices: resuming a field-based price collection. This approach is consistent with international guidance (PDF, 503KB) on the collection and compilation of consumer price statistics over the course of the pandemic.

As these changes were introduced partway through the year, it was not appropriate to make changes to the weights or base prices (outside of the usual approach to replacements, as described in Section 9.3 of the Consumer Price Indices Technical Manual, 2019). However, for CPIH and CPI, new weights are introduced annually in the December and January index¹, and new base prices are introduced in the January index. This article therefore provides details of how we have calculated weights for the index year 2021, how we have approached the collection of base prices for 2021, and how this will be affected by the current period of movement restrictions in place across the UK.

Section 3 of this article describes the process for calculating weights for 2021 and Section 4 describes the collection of reference prices.

Notes for Overview of consumer price inflation weights and prices for 2021

1. For more information on the double weights update, please refer to the article Assessing the impact of improvements on the Consumer Prices Index.

CPIH and CPI

Under normal practices the weights of the Consumer Prices Index including owner occupiers' housing costs (CPIH) and the Consumer Prices Index (CPI) are based on household final consumption expenditure (HHFCE) data from two years' previous; that is, had 2020 been a typical year, 2019 HHFCE data would have formed the basis of the 2021 weights update for the CPIH and CPI. However, given the atypical expenditures that occurred in 2020 as a consequence of the coronavirus (COVID-19) pandemic, it is not clear that this is the most appropriate approach to calculating weights for 2021¹.

The guidance (PDF, 503KB) for compilation of the Harmonised Index for Consumer Prices (HICP)² suggests that:

"the weights for 2019 need to be reviewed and updated to make them representative for 2020. Typically, the first national accounts estimates of household consumption expenditure by ECOICOP³ for the year 2020 only become available in September 2021, which is too late ... Thus, preliminary national accounts data and other sources need to be employed."

We aim to adopt the same procedure for use in the CPIH and CPI (which is the UK's HICP) and update our expenditure estimates to reflect 2020 spending levels where possible. At the time of calculating the CPIH and CPI weights, the annual HHFCE data for 2020 are unavailable. The second estimate of Quarter 3 (July to Sept) 2020 was published on 22 December 2020, meaning that data for Quarters 1 to 3 are available to us, albeit at an earlier stage of compilation than we would normally use.

These estimates are likely to change over time as more comprehensive data sources become available. We have identified three sources of uncertainty over and above what would be expected in the traditional production of the CPIH and CPI weights:

1. The 2020 data are less comprehensive and likely to be revised relative to the dataset that would normally be used. This is because additional data sources are usually incorporated later in the year, which improve the estimates. Additionally, the challenges associated with the coronavirus pandemic have had an impact on the quality of expenditure data over 2020. For more information, please see the Consumer trends publication. However, these estimates are currently the best available data on household spending and better reflect the impact of changing consumer spending patterns as a result of the pandemic than the 2019 vintage of data.

2. It is necessary to estimate Quarter 4 (Oct to Dec) 2020 spending because of the lack of expenditure data.

3. It is unclear how the spending distribution will evolve over the course of the year ahead. If vaccines are successful in reducing the impact of the pandemic, then spending patterns may differ from 2020 spending. On the other hand, if the pandemic continues to necessitate the use of movement restriction policies then 2020 spending patterns may look a lot like 2021 spending. It is of course impossible to know what will happen over the course of the year ahead, and so this introduces a greater source of uncertainty around the weights.

The extent to which point 3 matters depends on the particular measurement objectives of the index in question. The Eurostat guidance (PDF, 503KB) makes it clear that:

"the HICP of year t is defined as a Laspeyres-type index where the weights refer to t-1 and the price reference period corresponds to December t-1."

If our aim in measuring consumer price inflation is to compare the annual change in prices of things that consumers bought last year, then the Laspeyres is an appropriate choice of index. However, many users - particularly in light of the coronavirus pandemic - have the reasonable expectation that the consumer prices basket is reflective of the things that they are currently spending their money on. This is better reflected through a Paasche index⁴; however, data availability rules out the calculation of such an index.

In normal times, expenditure does not change very much from year to year. With the advent of the coronavirus pandemic this is no longer the case. Should the spending distribution change again in 2021, the CPIH and CPI weights will not reflect what consumers are currently buying. They will reflect the change in price of things that consumers bought last year. The imputation strategy described in Coronavirus and the effects on UK prices mitigates for this. Additionally, recognising that there is a user need to understand the impact of the changing spending distribution on our measures of inflation, we will continue to publish our reweighted basket analysis on a quarterly basis. We will publish Quarter 4 (Oct to Dec) 2020 on 18 February 2021.

The following sections describe our approach to the calculation of expenditure for use in the weights for 2021 in more detail and will also explain the mitigations we have taken for the uncertainties in points 1 to 3.

Estimation of Quarter 4 2020

Before we can produce annual estimates of 2020 spending, it is first necessary to estimate consumer spending in Quarter 4 (Oct to Dec). We have applied the growth from Quarter 3 (July to Sept) to Quarter 4 2019, to Quarter 3 2020, to estimate spending in the missing period. We have used this approach because:

• it accounts for seasonal effects associated with Quarter 4 spending

• it accounts for coronavirus-related effects through application to Quarter 3 2020 data

We chose to use the coronavirus effects from Quarter 3 specifically as it was felt that other quarters were less representative of coronavirus-related policies in the final quarter of 2020. The UK was unaffected by the coronavirus for most of the first quarter (Jan to Mar) of 2020. Quarter 2 (Apr to June) saw the most severe movement restrictions across 2020. Quarter 3 saw some relaxation of movement restrictions and return to normal spending. This coincides best with Quarter 4, where movement restrictions were largely managed through regional tier systems⁵. Even the period of tighter restrictions in November was not on the scale seen in Quarter 2. Therefore, it is considered that Quarter 3 2020 data provide the best reflection of coronavirus effects in Quarter 4.

The CPIH and CPI estimates are aggregated according to the Classification of Individual Consumption by Purpose (COICOP). The current system for producing the CPIH and CPI weights involves acquiring HHFCE data at the published COICOP4 (class) level. Breakdowns are then applied using available data sources to derive estimates at the more detailed COICOP5 level (subclass). As these supplementary data sources were unavailable for 2020, we have estimated Quarter 4 spending at the class level.

Identification of significant shifts in the spending distribution

We have conducted analysis to identify the likely extent of differences between our 2020 spending estimates and the 2020 dataset that would normally be available for use in weights calculations (that is, for the 2022 CPIH and CPI). This involved calculating 2018 HHFCE based on the second estimate of Quarter 3 2018 and using the Quarter 4 estimation method described in the previous section. This was then compared with the 2018 annual dataset used in the compilation of weights for CPIH and CPI in 2020, which includes more comprehensive data sources. It should be noted that the analysis is indicative only, as 2018 predates the pandemic and therefore there are no coronavirus effects present in the 2018 dataset.

The relative proportional differences at class level (with signs removed) between the two analytical datasets for 2018 are calculated. It should be noted that the differences are partly driven by the estimation method for Quarter 4 and should not be considered as a reflection of the quality of HHFCE data.

The interquartile range lies between 6.7% and 23.4%, with a median value of 11.6%. However, the differences can be as large as 163.8%. This indicates that there is a great deal of uncertainty associated with the 2020 estimates. Given that, in normal times, we are comfortable to assume that spending data on a two-year lag are a reasonable proxy for spending patterns in the previous year, and given also that we are not confident that 2020 estimates that show a smaller change in spending are more accurate than the 2019 estimates, we have chosen to only use 2020 estimates for the more strongly affected spending categories. In other words, we consider that we do not have sufficient evidence to suggest that the smaller adjustments would give a better spending estimate than simply using the 2019 data. This is consistent with the Eurostat guidance (PDF, 503KB), which states that:

"[a]s a minimum, the expenditures of the most heavily affected segments of consumption should be re-estimated."

In order to identify the more strongly affected spending categories we need to establish a threshold for what should and should not be adjusted using our 2020 spending estimates. We have identified such a threshold using two approaches.

Firstly, we have used the interquartile range from the analysis presented previously, which suggests that a 25% shift in spending between 2019 and 2020 would be an appropriate threshold.

Secondly, we have independently scrutinised the class-level data to pick out the categories that we would expect to require an adjustment. This is to some extent anecdotal; however, we have also cross-referenced this against empirical analysis, such as our reweighted basket analysis and Retail sales data. The evidence from this second, qualitative approach similarly suggests that a threshold of 20% to 25% would be appropriate. Therefore, our approach is to use 2020 estimates for any classes that show a change of more than 25% from 2019 and, recognising that some judgement is required, to review any classes with a change between 20% and 25% on a case by case basis.

Using this approach, we have identified 25 classes to be adjusted using 2020 spending (from a total of 87). The 25 classes that have been identified are presented in the following list of COICOP classes and groups. The weights estimates for 2021 will be presented in the forthcoming article, Consumer price inflation, updating weights 2021, due to be published on 15 March 2021.

List of COICOP classes and groups for which weights will be based on 2020 spending estimates, UK 2021

3.1.4 Cleaning, repair and hire of clothing
5.1.3 Repair of furniture, furnishings and floor coverings (included in 5.1.1 Furniture and furnishings)
5.6.2 Domestic services and household services
6.1.2 Other medical products (included in 6.1.2/3 Other medical and therapeutic equipment)
6.2.1 Medical services (included in 6.2.1/3 Medical services and paramedical services)
6.2.2 Dental services
6.3 Hospital services
7.1.1A New cars
7.1.1B Second-hand cars
7.1.3 Bicycles (included in 7.1.2/3 Motorbikes and bicycles)
7.2.2 Fuels and lubricants
7.2.4 Other services
7.3.1 Passenger transport by railway
7.3.2 Passenger transport by road
7.3.3 Passenger transport by air
7.3.4 Passenger transport by sea and inland waterway
9.1.5 Repair of audio-visual equipment and related products
9.4.1 Recreational and sporting services
9.4.2 Cultural services
9.6 Package holidays
11.1.1 Restaurants and cafes
11.1.2 Canteens
11.2 Accommodation services
12.1.1 Hairdressing and personal grooming establishments
12.1.2 Electrical appliances for personal care (included in 12.1.2/3 Appliances and products for personal care)

Of the 25 classes, four fell into the range between 20% and 25%. These were 5.6.2 Domestic services and household services, 7.1.1A New cars, 7.1.1B Second-hand cars and 9.1.5 Repair of audio-visual equipment and related products. This last category was included for consistency with other similar categories, such as 5.1.3 Repair of furniture, furnishings and floor coverings.

Classes that fell in the 20% to 25% range, but which we are not using 2020 spending for, include 2.1.3 Beer. Although there is a possible justification for using 2020 expenditure in that we would expect consumers to have switched their spending on alcohol from licensed premises to food and drink outlets, both 2.1.1 Spirits and 2.1.2 Wine fell below the 20% threshold. Whilst this on its own is no reason to rule out using 2020 spending figures, it was felt that similar arguments could be made for consumers switching from restaurant and café spending; however, the food and drink expenditure was well below the 20% threshold.

Identification of sustained shifts in the spending distribution

We also conducted analysis to identify whether the changes we have seen in the spending distribution were sustained over the course of the year, or whether there were certain classes that saw a short-term, temporary change. Where spending changes have been temporary, it is not clear that it would be preferable to use 2020 expenditure. This approach has been used by the Australian Bureau of Statistics as part of their 2020 annual reweight.

We analysed the quarterly HHFCE data that are available for 2020, comparing quarterly growth against 2019 and identifying any large changes in Quarter 2 spending where expenditure subsequently fell back to under 25% of 2019 growth in Quarter 3. However, this analysis was unable to conclusively identify any classes where the change in spending was not sustained throughout 2020. This is because the variation between quarters was naturally high in the data - most likely as a result of points 1 and 2 at the start of Section 3.

Further scrutiny of the weights from our reweighted basket analysis also did not highlight any areas for consideration. We have therefore made no further changes on this basis.

Special cases

There are a number of CPIH and CPI spending categories that fall outside of the scope of HHFCE and which are therefore estimated using alternative sources of spending data.

One major category for this is package holidays. In the national accounts, spending on package holidays is included within the COICOP categories that reflect separate elements of the package (for example, flights). For CPIH and CPI, package holidays represent a spending category in their own right. To derive an estimate, a proportion of package holiday expenditure is first removed from 7.3.3 Passenger transport by air and 7.3.4 Passenger transport by sea and inland waterway using Living Costs and Food Survey (LCF) data on a two-year lag. Second, the LCF estimate of overall package holiday spending (again, at a two-year lag) is used to represent consumer spending on this category.

However, the LCF data on package holiday spending for 2020 are not yet suitable for the production of the CPIH and CPI estimates. But we do not consider that it would be appropriate to use 2019 spending estimates for package holiday weights in 2021. Therefore, we have estimated package holidays expenditure for 2020. This is done in two stages.

The first stage is to remove package holiday spending from the national accounts data. We have done this calculation using 2019 LCF data as usual. This relies on the assumption that the relative proportion of package holiday air and sea transport spending to other air and sea transport spending is unaffected by the coronavirus pandemic.

Using this assumption, the second stage is to then use the package holiday expenditure removed from the national accounts transport categories to derive a ratio estimator. This is used to pro-rate LCF package holidays (PH) spending. The estimator is defined as follows:

This requires the additional assumption that the change in transport package holiday spending is the same as the change in non-transport package holiday spending. This approach has the advantage that 2020 package holiday estimates are consistent in the magnitude of their change with transport estimates for 2020.

An adjustment is also made to 7.1.1A New cars to remove "cars in kind," which includes, for example, company cars. As per the list of COICOP classes and groups, we are using 2020 data to calculate weights for this class. However, we only have the cars in kind adjustment on a 2019 basis. Therefore, we must rely on a similar assumption to that made for package holidays transport - that the relative proportion of consumer spending on new cars, to cars in kind spending, is unaffected by the coronavirus pandemic.

For other affected categories it is reasonable to use 2019 data, as we would not expect there to be a significant change in spending because of the pandemic. This includes, for example, Council Tax, trade union subscriptions and Vehicle Excise Duty.

Quality assurance

We have conducted a range of quality assurance to ensure that our weights estimates are of the best possible quality.

We consulted on our proposed plans with the Advisory Panel on Consumer Prices - Technical (APCP-T) on 9 October 2020 and again on 18 December 2020 to discuss and refine our proposals, and get feedback on early estimates of the data. APCP-T were broadly supportive of the approach we have described in this article. Users should also note, however, APCP-T's submission to Eurostat (provided in Section 6), which recommends that national statistical institutes should have flexibility to update their weighting schema should the need arise in 2021. Nonetheless, we will not be updating our expenditure weights outside of the usual routine as this would present a number of challenges, both conceptual and practical. We will, however, continue publication of our reweighted basket analysis on a quarterly basis, with the Quarter 4 2020 estimates planned for publication on 18 February 2021. The imputation strategy described in Coronavirus and the effects on UK prices also mitigates for the effects of changes in the spending distribution.

We have also drawn on national accounts' expertise within the ONS. A quality assurance meeting was held with national accountants on 8 January 2021 to share details of our planned approach and to seek feedback on the estimates. Finally, we have compared our estimates against other available data sources, such as data from the Retail Sales Index, electronic point of sale data collected directly from retailers (known as scanner data) and our earlier estimates for the reweighted basket analysis.

These measures were held in addition to the usual procedures for preparing and quality assuring the annual CPIH and CPI weights. They ensure that the sources of uncertainty as defined in points 1 to 3 in Section 3 are mitigated for.

Price updating

The class-level expenditure estimates described previously are pro-rated to subclass using the HHFCE process described earlier in Section 3. Subclass estimates are then price updated. For classes based on 2019 data, the price movement from 2019 (average) to December 2020 is used, so that 2019 quantities are expressed in reference period prices. For classes based on 2020 data, the price movement from 2020 (average) to December 2020 is used, so that 2020 quantities are expressed in reference period prices. This is the functional form of a Lowe index. Users should note that some of the indexes that have been used for price uprating will be based on imputed data as specified in the article, Coronavirus and the effects on UK prices. This ensures that the aggregation of CPIH and CPI is structured in a consistent way.

The price-updated subclass estimates are then summed to get total CPIH and CPI expenditure. This is used to calculate spending shares at each stage of the COICOP hierarchy, as per the usual procedure. The final weights will be made available in the article Consumer price inflation, updating weights 2021, due to be published on 15 March 2021.

Summary of the weight updating process

In summary, the process that we have followed to prepare expenditure weights for CPIH and CPI in 2021 is:

1. Get household final consumption expenditure (HHFCE) data for Quarters 1 to 3 2020 at the COICOP4 level (class)

2. Estimate Quarter 4 2020 by applying the Quarter 4 2019 growth to Quarter 3 2020

3. Compare annual 2020 estimates with annual 2019 data and identify shifts in expenditure greater than 25% (allowing for an element of judgement in the range from 20% to 25%)

4. Use 2019 estimates (where the change is below the threshold specified in 3) and use 2020 estimates where it is above the threshold

5. Estimate package holidays spending for 2020 using available Living Costs and Food Survey (LCF) data and HHFCE transport spending categories

6. Make further adjustments for CPIH and CPI categories that are out of scope of HHFCE using 2019 data

7. Derive COICOP5 (subclass) estimates using auxiliary data as per usual

8. Price update subclass estimates such that the lagged quantities are expressed in reference period prices, using an index base of either 2019 or 2020 depending on the expenditure data used

9. Sum the resulting set of subclass expenditures to get total CPIH and CPI expenditure, then calculate relative spending shares to form the weights at each level of the COICOP hierarchy

RPI

The Retail Prices Index (RPI) weights are based on a different source of data at an 18-month lag, rather than the 24-month lag in CPIH and CPI expenditure. This means that for 2021, weights would normally be based on LCF expenditure estimates covering the period July 2019 to June 2020.

Unlike CPIH and CPI expenditure, this period includes some pre-pandemic expenditure (Quarters 3 and 4 2019, and Quarter 1 2020) and some mid-pandemic expenditure (Quarter 2 2020). We also expect that Quarter 2 of 2020 is the most significantly affected quarter of pandemic spending, given that this represents the period of strictest movement restrictions.

It should be recognised that there is trade-off between the accuracy of expenditure estimates, which would be negatively affected by trying to calculate 2020 RPI expenditure, and the relevance of expenditure estimates which, although may be preferable on a 2020-basis, may not change as significantly as for CPIH and CPI given that the current reference period for expenditure includes some pandemic spending. We therefore do not intend to make adjustments to the usual weights compilation process for RPI.

Revisions to 2021 weights

As described in the previous sections, the expenditure weights for CPIH and CPI are normally based on HHFCE data lagged by two years, and for RPI are based on LCF data with a slightly shorter lag: from July two years previous to June of the previous year.

For 2021, the approach to calculating CPIH and CPI expenditure weights is very different. We will unusually incorporate 2020 data for those spending categories that have been most heavily affected by the coronavirus pandemic. This allows us to incorporate more timely data, that are more reflective of current consumer spending, but also means that there is a greater level of uncertainty attached to these estimates. Estimates of consumer expenditure will evolve as more robust data and additional sources become available. However, all of the expenditure weights represent our best estimate of consumer spending at this point in time and will therefore not be revised later in the year (other than the usual second price update that is introduced with the February CPIH and CPI indexes⁶).

Notes for Weights for 2021

1. Note that the Retail Prices Index (RPI) is out of scope of this article, since the expenditure weights cover a different time period, which includes some of the pandemic period.

2. An internationally comparable measure of consumer price inflation, specified by Eurostat.

3. ECOICOP is a Eurostat-specific variant of the UN's Classification of Individual COnsumption by Purpose (COICOP)

4. In reality a Törnqvist or a Fisher index may be more appropriate, as these indices average the consumption basket across the two comparison periods; however, such an approach would also require timely data that are not available, and would also reflect a departure from international guidance.

5. For more information, see the England, Scotland, Wales and Northern Ireland guidance.

6. For more information on the double weights update, please refer to the article Assessing the impact of methodological improvements on the Consumer Prices Index.

Reference prices are collected with the January index. This set of prices is used as the basis of price comparisons for the remainder of the year. The prices overlap a great deal with the January index prices that are referenced to the previous year's January. The difference lies in the particular set of reference prices for the small number of items that are newly added to the basket. For these items, completely new reference prices will need to be collected. The upcoming article, Consumer price inflation basket of goods and services: 2021 describes changes to the basket of goods and services for 2021, including new additions.

January is therefore an important month for consumer price collection. As is widely known, January 2021 was a period of widespread movement restrictions across the UK. This has necessitated an online price collection, supported by telephone collections to businesses without a website (as described in our article, Coronavirus and the effects on UK prices). In fact, we have been operating an online price collection since November 2020, when a second period of national movement restrictions was implemented.

A consequence of changes to the mode of price collection is that sample sizes are reduced, as it is more difficult for price collectors to identify the price quotes that are being tracked. Figure 1 shows the monthly Consumer Prices Index including owner occupiers' housing costs (CPIH) response rates (relative to February 2020's sample) since movement restrictions began and excluding unavailable items. Response rates have been slightly higher in periods of field collection and were lowest in April 2020 when it was first necessary to switch to an online and telephone collection. Since April 2020, response rates have remained at over 80%. It should be noted, however, that response rates for online collections were higher than for telephone collections.

A further consequence of the coronavirus pandemic is that the sample refresh of around 30 locations that would normally be incorporated with the January price collection has not been possible this year. Section 5.2.3 of the Consumer Price Indices Technical Manual, 2019 gives more information on our usual location rotation and re-enumeration procedures. In the short-term, we would expect this to have little impact on the quality of our consumer price inflation statistics.

The remainder of this section describes how we have mitigated for the impact of missing prices in the reference period.

Policy for replacements in the price reference period

In the first period of movement restrictions, where it was necessary to move to a predominantly online price collection, we mitigated for losses of sample by allowing price collectors to replace missing items immediately. Under normal procedures, a missing item is treated as temporarily missing for three months before a replacement is sought. More detail is provided in our article, Coronavirus and the effects on UK prices. Note that this was not the case for price quotes collected by telephone, as it was not possible for price collectors to identify a replacement in this way.

When price collection was moved to a predominantly online collection in November 2020, we did not initially implement the immediate replacement policy. This was because the imposition of national movement restrictions was considered to be a temporary policy, and a return to stores in December would have meant that any immediate replacements would have had to have been replaced again, before they could contribute to the index.

However, in December it was necessary to continue with an online collection, and immediate replacements were introduced at this point. Consideration was given to the impact that this would have on January 2021 reference prices, as it was not known at that stage whether January's price collection would be conducted in outlets, or through websites and phone calls. Implementing an immediate replacement policy in December meant that, regardless of the mode of collection, the sample of January reference prices would have been no worse than the sample size had immediate replacements not been allowed.

A further consequence of having January reference prices from web and telephone sources is that, at the point where collection returns to the field, it is possible that a further replacement will be necessary (reflecting the switch back in collection mode). For replacements that are considered to be non-comparable with the base price, this will require the imputation of a new base price (for more information please see Section 9.3 of the Consumer Price Indices, Technical Manual 2019). This will reduce the effective reference price sample for the remainder of the year, since the sample will contain a greater proportion of imputed reference prices and a smaller proportion of observed prices (price comparisons between periods after the reintroduction will be unaffected).

Replacements that are considered to be comparable with the base price can be compared directly, although this may introduce some volatility into the index. Price collectors are therefore encouraged to price the original product where possible and, only if a comparable replacement cannot be found, to price a non-comparable replacement.

Depending on how the response to the coronavirus (COVID-19) pandemic evolves over the course of 2021, it is possible that there may be later returns to online and telephone collections, and back again. Any subsequent mode changes are likely to affect sample sizes. Where mode changes result in additional non-comparable replacements being made, a greater proportion of the reference price sample will be imputed. However, this effect is independent of the initial reference price sample that was drawn. We will continue to follow the procedures described in our coronavirus articles.

Impact of imputations in the price reference period

A further consideration is the consequence of having no reference prices where an imputation has been used for the item as a whole. In the article, Coronavirus and the effects on UK prices, we identified two types of item missingness in our consumer price data:

• missingness because of unavailable items; that is, where goods and services cannot be bought by consumers because of movement restriction policies

• missingness because of available items, where goods and services can still be bought by consumers in some way, however, we have been unable to collect any prices

Unavailable items

For unavailable items, we impute a price movement based on either the monthly movement of the all-available-items index (for non-seasonal items) or the annual movement of the all-available-items index (for seasonal items). This approach is used so that the indices for goods and services that consumers cannot buy have a negligible impact on the calculation of the all-items inflation rate. In the case of non-seasonal items this is the monthly rate. In the case of seasonal items this is the annual rate.

For January 2021, the national movement restrictions mean that there are a significant number of unavailable items in the price reference period. Figure 2 shows the proportion of the CPIH basket that has been unavailable each month since movement restriction policies began. April 2020, which coincides with the very first introduction of movement restrictions, saw the highest proportion of unavailable items at 16.3%. In January 2021, 13.9% of the basket was unavailable. This is very close to the proportions from the English national movement restrictions in November 2020, and the latter half of the first period of UK-wide movement restrictions in May and June 2020.

Therefore, nearly 14% of the basket will be missing January reference prices because of item unavailability. It would not be appropriate to attempt to collect prices for use in later periods, because the prices of items that cannot be purchased in January are, by definition, undefined. It would be a fiction to try and make use of them. Instead, once an item returns to "available," a notional reference price will be imputed based on the imputed unavailable price movement of that item up to the period of reintroduction. This ensures that the index remains consistent with the micro-level notional price data over the period of unavailability, and that observed price changes after the period of unavailability are appropriately reflected in the index.

Available items

In general, for available items where we have been unable to collect a sample for a particular item, we aim to impute a price movement that best captures the movement we would have observed, had we been able to collect the price data. This will be done by either carrying the previous month's prices forward, or by imputing the price movement from a higher aggregate. Usually this will be from the level immediately above, however, on occasions this will be taken from higher levels or the all-items index if this is considered to be a more appropriate choice.

Therefore, in the case where there are no reference prices for a particular item, once price quotes begin to enter the sample, we will need to impute reference prices for them. This is analogous to the case of non-comparable replacements in normal consumer prices collection, where we have to estimate a reference price using our best estimate of the price movement from the reference period. In the case of available items, our best estimate is the imputation that was made according to the principles described in the previous paragraph. Therefore, available imputations will be used to impute missing reference prices later in the year.

Impact of chain-linking

Price movements from the new price reference period (January 2021) need to be chain-linked to the existing CPIH, CPI and Retail Prces Index (RPI) series in order to create a continuous data time series and allow comparisons to be made between years. More information on chain-linking in the CPIH, CPI and RPI is provided in Sections 3.6 and 11.4.2 of the Consumer Price Indices Technical Manual, 2019.

In the article Consumer price statistics: resuming a field-based price collection, we described how our main principle in returning to a field collection was that:

"we aim for price comparisons in current and future periods to be unaffected by index levels over the period of movement restrictions."

However, the longer the pandemic continues, the more difficult it becomes to honour this principle, because over time the overlap between pre- and post-pandemic samples will shrink (a situation complicated by the system of regional tier restrictions). In particular, because the period of movement restrictions has crossed a chain-link in all of our indices, the imputations that have been made over this period (particularly for unavailable items) will be permanently reflected in the level of the index.

An alternative way to think of linking baskets is that an implicit quality adjustment is made between years, where it is assumed that the difference in quality of the two baskets being linked is determined by the difference in their cost. Because a quality adjustment is implied, the level of the 2021 index is effectively adjusted to match the quality of the previous years' basket. However, part of the cost of the previous years' basket is associated with imputed prices for things that are not available to consume. Therefore, the unavailable imputations will be reflected in future index values.

In this article we have described our approach to the calculation of weights and collection of reference prices for 2021. Expenditure weights for 2021 represent our best estimate of consumer spending at this point in time and will therefore not be revised later in the year (other than the usual second price update that is introduced with the February Consumer Prices Index including owner occupiers' housing costs (CPIH) and Consumer Prices Index (CPI)).

The details presented in this article follow on from the previous articles, Coronavirus and the effects on UK prices and Consumer prices: resuming a field-based price collection, in which we have described our approach to maintaining our consumer price statistics over the course of the coronavirus (COVID-19) pandemic.

It is likely that the procedures described in these articles will be required for as long as movement restriction policies persist. However, there is a great deal of uncertainty over the year ahead, and it is not possible to know how events will unfold and what the implications will be for our consumer price statistics. We will therefore continue to update users on our plans as and when circumstances require.

This note is on behalf of the Technical Advisory Panel for Consumer Price Statistics (APCP-T) and is intended to provide feedback on the Eurostat proposals for the Harmonised Index of Consumer Prices (HICP) weights in 2021.

The APCP-T's remit is to provide independent advice to the UK's National Statistician on technical aspects of consumer price indices. APCP-T has an international membership, and members are appointed based on their expertise in the field of consumer price inflation, rather than because of an affiliation to any particular organisation. More information can be found in the published Terms of Reference and membership list.

At the recent APCP-T meeting held on Friday 9 October, Panel members discussed weights for 2021 and the draft Eurostat guidance on compilation of weights for the HICP. Panel members agreed that weights should be developed that provide the best estimates of spending in the upcoming year, and that the default position should be stability within the year.

However, Panel members were also concerned that the approach for 2021 could be problematic. In particular, it is impossible to know for sure what will happen over the next 12 months, and for how long the coronavirus (COVID-19) pandemic will continue. If the pandemic ends relatively quickly we would be in the reverse position to 2020, and HICP estimates would be produced - perversely - based on lockdown spending patterns.

There is a larger amount of uncertainty around the year ahead than we would normally experience. The best way to manage this uncertainty is to allow flexibility in the guidelines so that national statistical institutes (NSIs) can adapt their weighting schema in real time to respond to events as they unfold, and therefore reflect relevant spending patterns. Flexibility of course should be applied carefully and within agreed boundaries, and changes should only be made where it becomes clear that weights are not reflective of consumer spending.

Over the past year Eurostat's guidelines have enabled NSIs to respond flexibly to lockdowns through appropriate imputations of prices for unavailable items. A similar principle should be applied to the weights. We also need a way to reflect the additional uncertainty because of the weights used.