Household Costs Indices for UK household groups QMI

• National Statistic: No
• Data collection: Survey data, Administrative data
• Frequency: Quarterly publication of monthly data
• How compiled: Representative sample based on around 700 goods and services
• Geographic coverage: UK
• Related publications: Consumer Price Inflation, UK

This Quality and Methodology Information (QMI) report contains information on the quality characteristics of the data (including the European Statistical System five dimensions of quality) as well as the methods used to create it. The information in this report will help you to:

• understand the strengths and limitations of the data
• understand the methods used to create the data
• help you to decide suitable uses for the data
• reduce the risk of misusing data

• The Household Costs Indices (HCIs) are measures of inflation as experienced by different types of households; these differ from our other measures of consumer price inflation, the Consumer Prices Index (CPI) and the Consumer Prices Index including owner occupiers’ housing costs, (CPIH), which are aggregate measures based on recognised economic principles.
• For some goods and services, a different measurement approach is used in the HCIs compared with CPIH and CPI.
• CPIH and CPI largely capture price changes as goods and services are acquired, whereas the HCIs capture price changes for some items as they are paid for; this includes owner occupiers’ housing costs, the inclusion of interest payments on debt, student loan repayments, and an alternative approach to measuring expenditure on insurance.
• HCIs use a “democratic” approach to weighting in that they utilise the average households’ expenditure shares; this differs from the “plutocratic” approach used in CPIH and CPI where expenditure weights reflect the total share of expenditure by households in the UK.
• Weights for different household subgroups are constrained to the all-household totals used in CPI and CPIH; data from the Living Costs and Food survey are used to pro-rate expenditure totals according to the spending distributions of household subgroups.
• The Household Costs Indices are experimental statistics and so a degree of caution is advised when using these statistics.

Overview

Consumer price inflation is the rate at which the prices of the goods and services bought by households rise or fall. Consumer price inflation is estimated by using consumer price indices. One way to understand a price index is to think of a very large shopping basket containing the goods and services typically bought by households. The price index estimates changes to the total cost of this basket. An overview of the methodology and an explanation of the main features of the suite of indices and their uses is available in our Consumer price indices, a brief guide article.

The Household Costs Indices (HCIs) are measures of consumer price inflation as experienced by different types of households. There are a number of differences between the way that the HCIs measure price change compared with our other measures of consumer price inflation.

As we spend more on some things than others, the components of a price index are “weighted”. This ensures that they reflect the importance of the items in the shopping basket. To better understand how households experience inflation, the HCIs weights are based on the average households’ basket.

This approach differs from that used in the Consumer Prices Index (CPI) and Consumer Prices Index including owner occupiers’ housing costs (CPIH). Because the target population of CPI and CPIH is the UK as a whole, the basket is based on total spending by households across the UK. However, this approach is implicitly more reflective of higher spending households, and therefore would not be appropriate for understanding the experience of a typical household.

The HCIs also seek to capture the change in prices of goods and services as households pay for them. This includes capturing the change in cost of repaying interest on various types of debt, as well as directly capturing the payments that households make to own and live in their own home, such as mortgage interest payments, stamp duty, and major repairs and maintenance.

This approach differs from the approach used in CPIH, which uses equivalent rental prices as a proxy for the cost of using housing services. More information on the concepts and development of the HCIs can be found in our Developing the Household Costs Indices article.

Use cases

We have illustrated our future approach to measuring changing prices and costs for consumers and households using three "use cases". The three cases refer firstly to the CPIH as our lead measure of inflation based on economic principles. The CPI is based on the same framework but excludes owner occupiers’ housing costs and Council Tax.

The use cases also refer to the HCIs as a set of measures to reflect the change in costs as experienced by different households. They refer to Retail Prices Index (RPI) as a legacy measure that is required to meet existing user needs. Our Shortcomings of the RPI as a measure of inflation article describes the shortcomings of the RPI.

Strengths and limitations

The HCIs are measures of inflation as experienced by different types of households. We produce measures for a range of different population subgroups, including:

• income decile
• expenditure decile
• tenure type (whether a household is a private renter, a non-private renter, an outright-owner occupier, or a partial owner occupier)
• retirement status
• whether a household has children or not

An aggregate all-households measure is provided for reference; however, we recommend that analysis should focus on subgroup measures. This is because different types of households will experience inflation in different ways and an aggregate measure will not necessarily be meaningful.

Price indices are calculated using price and expenditure data. Expenditure shares can be calculated using two different methodological approaches; these are plutocratic and democratic weighting. Plutocratic weights are the total share of UK expenditure on a given spending category and are used for CPIH and CPI. Democratic weights are the average households’ share of expenditure on a particular spending category and are used for HCIs.

Our Investigating the impact of different weighting methods on CPIH methodology compares the two approaches, alongside additional approaches to weighting a price index.

While the calculation of inflation rates for household groups is straightforward analytically, data constraints make their estimation challenging in practice. In particular, an analysis of household group specific inflation rates would ideally use price indices and expenditure weights specific to each household group. This would reflect the fact that different households will purchase different goods and services from different outlets and therefore face different prices. However, such data are not available and therefore we have had to use national price indices as an approximation.

Research published by Kaplan and Schulhofer-Wohl (2017), found that most of the variation in inflation rates for different US households was because of the prices that they paid for the same kinds of goods. This research is based on a panel dataset of transactions made by 50,000 US households (Inflation at the household level, Journal of Monetary Economics, volume 91(C)). We are currently exploring potential improvements in this area with our academic partners.

These limitations do not impede the validity of the chosen methodology and its robustness. For more information please see our related Methodology to calculate CPIH-consistent inflation rates for UK household groups.

Expenditure weights for each household subgroup are calculated by reapportioning annual total UK expenditure by each households' relative share of spending. The reapportioning is done at the Classification of Individual Consumption by Purpose (COICOP) "Class" level, with expenditures drawn from the CPIH and CPI weighting model.

Household-level expenditure distributions are taken from the Living Costs and Food survey. For example, if a household is responsible for 1% of total spending on carpets and other floor coverings, then 1% of the CPIH-consistent expenditure total for carpets and other floor coverings is assigned to that household.

The Living Costs and Food survey typically has a sample size of around 5,000 households. Users should therefore be cautious when analysing more granular breakdowns of the HCIs as survey sample sizes may be small. For example, decile weights are achieved by splitting the survey into ten parts. Similarly, the weights at the lowest stage of aggregation are likely to have fewer households reporting expenditure than at higher stages of aggregation.

Another important component of the HCIs is interest. Households pay interest on debt, but also benefit from interest received on savings. The HCIs capture the interest payments that households make on household debt, but they do not capture the interest earnt by households on savings.

It is therefore important to consider the HCIs alongside a matched income measure when analysing the impact of changing prices on different types of households. The income measure used in our HCIs publications is mean equivalised disposable household income, as presented in our Average household income, UK bulletins. Our The measurement of real income in the UK: options for a coherent approach article discusses this in more detail. The On Household Costs Indices paper (Andrew Aitken & Martin Weale, 2021) also discusses this issue, and the payments approach more broadly, in the context of a consumption framework.

Recent improvements

On 4 December 2023, we will publish the first experimental Household Costs Indices for UK household groups quarterly bulletin. Previously, we have published annual experimental estimates on an ad-hoc basis, reflecting methodological improvements that were made. The move to a quarterly publication has involved putting in place robust and reproducible systems and processes and applying a greater degree of consistency across methodologies. The new quarterly bulletins will provide users with more timely estimates of the impact of the rising cost of living on different types of households.

This section provides a range of information that describes the quality and characteristics of the data and identifies issues that should be noted when using the output.

Accuracy and reliability

The Household Costs Indices (HCIs) are experimental statistics and so methodologies are subject to further improvements. We have previously published annual updates on an ad-hoc basis. Differences in the estimates presented between publications reflect subsequent improvements to the methodology. From December 2023, our HCIs revisions policy is that we will not routinely revise previously published estimates. However, while the HCIs are experimental statistics, we may occasionally make revisions to incorporate major methodological improvements or for other special cases. In this case, we will notify users through our Household Costs Indices for UK household groups bulletin in advance of any planned changes.

Further improvements to the methodology that may be considered for future development include:

• the measure of student loan repayments, which is based on a basket of nominal graduate salaries; for reasons of consistency, a basket of real graduate salaries may be preferred
• the measure of student loan repayments, which only measures the change in Plan 1 repayments; future development work should consider incorporating newer payment plans into the calculation
• the stamp duty weight, for which the sources currently include a small amount of out-of-scope expenditure for buy-to-lets and commercial house purchases
• the current mortgage interest payments methodology (as used in the Retail Prices Index (RPI); note that we do not plan any changes to the calculation of the RPI on this basis)
• interest on loans and overdrafts, which are currently out of scope of the Living Costs and Food survey and are calculated manually from the available survey variables, and could be improved by adding direct questions into the survey questionnaire

There are no current plans to undertake this development work in the near future.

Coherence and comparability

There are a number of differences between how Consumer Prices Index including owner occupiers' housing costs (CPIH) and Consumer Price Index (CPI) measure inflation compared with the HCIs, which reflects their different use cases. Specifically:

• HCIs expenditure weights are based on the average households’ expenditure shares, rather than total UK expenditure shares
• Items in the CPIH and CPI basket are largely measured on an acquisitions basis (with some exceptions for practical reasons).

However, some items in the HCIs basket are measured on a payment basis, including:

• owner occupiers’ housing (OOH) costs, which are based on the direct payments that households make to consume housing services, notably mortgage interest payments, stamp duty, major repairs and maintenance, and other OOH-related payments; OOH are excluded from CPI and included in CPIH using the equivalent rent as a proxy for the cost of consuming housing services
• interest on debt, which is out of scope of CPIH and CPI, but is included in the HCIs as an additional class
• university tuition fees, which are captured through both the repayment of student loans, and tuition fees paid upfront whereas, in CPIH and CPI, actual tuition fees are captured
• the weights for insurance, which reflect total expenditure on insurance premiums – a “net” approach is used in CPIH and CPI, where part of the expenditure is removed to reflect the payments that are made back to households for insurance claims

Components of the HCIs that are measured on a payments basis accounted for approximately 10.0% of the all-households basket in 2023.

The On Household Costs Indices paper discusses the coherence of the HCIs within a consumption framework, and highlights a potential bias in the data from only including interest paid on debt, and not accounting for interest received from savings. For this reason, it is important to consider the HCIs in the context of a matched income measure.

The income measure used in our HCIs publications is mean equivalised disposable household income, as presented in our Average household income, UK bulletin. However, the interpretation of these results should be treated with caution, as there are elements within the two measures that mean they are not coherent.

For example, the HCIs include changes in the cost of Council Tax, while the disposable income measure provided in this article deducts direct taxes (including Council Tax). This means that any changes in the cost of Council Tax will be counted in both measures. Details of how measures of income and measures of changing prices and costs should be produced so that they are coherent are available in our The measurement of real income in the UK: options for a coherent approach article.

The HCIs measure price change on a “domestic” basis under the current methodology. This refers to expenditure within the UK economic territory. It includes spending by foreign households visiting the UK and excludes spending by UK households abroad. Conceptually, a “national” approach may be preferred. This refers to expenditure by UK households, and excludes spending by foreign households in the UK, but includes the spending of UK households overseas. This is considered for potential future development work.

A further consideration is that the survey data used to apportion the all-household expenditure totals to household groups, do not include spending by foreign visitors to the UK. The data also exclude non-private households, such as institutional households, which are in scope of HCIs expenditure.

CPIH and CPI are based on international best practice and CPI specifically is based on a harmonised methodology that allows for international comparisons to be made. The HCIs cannot be easily compared with other international CPIs, however, there are some countries that have developed similar measures.

Stats New Zealand publish Household Living Costs Price Indices (HLPIs) that provide insights into the inflation experienced by 13 different household groups. As with our HCIs, mortgage and other interest payments are included. Insurance weights are included on a “gross” rather than “net” basis and a democratic weighting scheme is used. HLPIs, however, have no exceptional treatment for education repayments.

The Australian Bureau of Statistics’ Selected Living Cost Indices (SLCIs) similarly adopt a payments approach. However, unlike HCIs and HLPIs, a plutocratic weighting scheme is used, rather than a democratic one.

In 2022, the US Panel on Improving Cost-of-Living Indexes and Consumer Inflation Statistics in the Digital Age published the Modernizing the US Consumer Price Index for the 21st Century report. The report recommended that, “because of the urgency of issues related to income and wealth inequality, social welfare, and poverty, developing price indexes for population subgroups along the income distribution should be a high priority for BLS. Identifying data sources that would ultimately allow production of price indexes by income quintile or, if possible, decile is a key part of this work.”

The HCIs are based on a consistent methodology from 2007 (or growth rates from 2008) to the present. Estimates for 2005 and 2006 are provided from historical calculations, but data limitations in this period mean that estimates may not be directly comparable.

How we collect the data, main data sources and accuracy

For the calculation of democratic subgroup weights, we make use of the aggregate household expenditure data that underpin the class and higher-level weights used in the construction of the Consumer Prices Index including owner occupiers' housing costs (CPIH) measure, which are largely derived from System of National Accounts: SNA 2008 estimates. For further details of these estimates, see our Consumer trends: UK bulletin.

Since these data draw on a range of the best available data sources, they are the most reliable estimates of household spending for the UK as a whole. These data are annual calendar year expenditure totals for each of the class-level categories.

To calculate democratic weights, we also need to know the classification of individual consumption by purpose (COICOP) expenditures at the household level. To achieve this, household-level expenditure data are taken from the Living Costs and Food Survey (LCF).

The LCF is a continuous survey of the expenditure patterns of UK private households based on a sample of around 5,000 responding households per year. Demographic information about each household is also collected, along with the components required to calculate expenditure for each of the 87 class-level categories.

The LCF contains the most detailed household-level expenditure data that are currently available. The LCF data we use for our estimates consist of around 5,000 households per year, surveyed between Quarter 1 (Jan to Mar) 2003 and Quarter 4 (Oct to Dec) of the most recent available year (for example, in 2019, we use data for 2017).

Price indices for each of the 87 class-level categories are consistent with the indices used to construct CPIH. For more details, see Table 56 of our Consumer price inflation dataset. For more information on how these indices are constructed, please refer to our Consumer Price Indices Technical Manual.

For a small number of components that are specific to the Household Costs Indices, calendar year expenditure totals, class-level indices and some household expenditure variables are calculated separately using a range of available data sources.

How we process the data

In order to construct household-level expenditure estimates that are consistent with the most accurate CPIH expenditure data, we effectively “allocate” the CPIH-consistent expenditure totals for each class across the observed LCF households based on their reported expenditure.

There are several methods to do this but following testing and advice from our Technical Advisory Panel for Consumer Prices (APCP-T), we divide reported total CPIH expenditure on each of the 87 COICOP classes among the households we observe in the LCF in proportion to their expenditure share on that class-level category. These expenditure shares are calculated using weighted household spending from the LCF, which ensures that the total expenditure is representative of the population rather than just the LCF sample.

The expenditure shares are then calculated for each household group, using the democratic method of weighting. This gives a set of HCI weights for each household group. To calculate the inflation rates for each household group, the unrounded class-level price indices for each month are combined with the appropriate expenditure weights to produce an aggregate price index, which is then chain-linked to form a continuous series. For more information about how the CPIH is constructed, please see our Consumer Price Indices Technical Manual.

The following is a simplified overview of the HCI compilation process:

Subgroup-specific index aggregation, components

• Democratic weights by subgroup.
• CPIH class-level indices.
• HCI-specific indices for owner occupiers’ housing (OOH), education and interest.

Democratic weights by subgroup, components

Total all-household expenditure by class:

• CPIH expenditure totals (based on HFCE)
• HCI-specific expenditure totals for OOH, education, insurance and interest

Household-level COICOP expenditures:

• Living Costs and Food Survey household expenditures
• supplementary variables for interest

How we quality assure and validate the data

Typically, there are a small number of households whose expenditure is concentrated on a single product type and some that include negative expenditure (possibly reflecting the un-winding of prior overpayments). We therefore remove any households who report negative expenditure, and households who spend 80% or more of their total expenditure on a single class-level category. Historically, this has removed around 0.5% of the total sample and has no clear impact on our results.

We also observe households who report very high total expenditure. However, as these are legitimate members of the sample and represent the very high expenditure households in the population, they are not excluded from this analysis.

There are a number of issues that can occur as a result of scaling survey data to match all-household spending totals. For example, where the expenditure weight within the CPIH is based on data other than the LCF data, the differences between the CPIH-consistent and LCF expenditure totals can be extremely large.

There are also some instances where the coverage of the LCF data is low. Therefore a large amount of CPIH expenditure is allocated to a small number of households for a particular class. For example, medical and paramedic services are a COICOP class where a very small number of households reported spending over the full dataset. In these cases, we adjust our methodology to avoid distorting the results.

For each year, COICOP classes were identified where these two conditions were both met:

• the ratio of CPIH to LCF expenditure is greater than two (that is, total CPIH expenditure is more than double the LCF expenditure)
• the percentage of households that report spending on that COICOP class over the year is less than 20%

The general methodology for consumer price indices is detailed in our Consumer Price Indices Technical Manual. For more information on the methodology underpinning the construction of subgroup inflation measures, please refer to our Methodology to calculate CPIH-consistent inflation rates for UK household groups and Investigating the impact of different weighting methods on CPIH methodology. Our Developing the Household Costs Indices (HCIs) article also provides more information on how the HCIs have been developed.

The HCIs are based on the index described by Astin and Leyland in the Towards a Household Inflation Index paper. Astin and Leyland also published the Measuring inflation as households see it: next steps for the Household Costs Indices paper in 2023. Aitken and Weale discuss the HCIs within the context of a consumption framework in the On Household Costs Indices paper published in 2021. This highlights areas of potential biases in the data.