Household Cost Indices, UK, methodology for third preliminary estimates: 2005 to 2019

The Household Cost Indices (HCIs) are a set of experimental measures, currently in development, that aim to more closely reflect UK households’ experience of changing prices and costs. More specifically, they will aim to measure how much the nominal disposable income of different household groups would need to change, in response to changes in costs, to enable households to purchase the same quantity of goods and services of the same quality. Put simply, the broad approach of the HCIs is to measure the change in the outgoings of households.

When considering the household experience of changing costs, it is informative to look at different groups within the population, as we would expect them to experience changes in prices and costs differently. Our analysis in 2014 , and related publications, such as our CPIH-consistent inflation rates for UK household groups, have demonstrated that different household groups (for example, retired households and low-income households) can experience different levels of inflation. Therefore, as with previous publications of the HCIs, the third preliminary estimates of the HCIs focus on measuring the experience of a number of different population groups, although an all-households index has been produced for reference.

The Consumer Prices Index including owner occupiers’ housing costs (CPIH), our lead measure of inflation, measures changes in the prices of goods and services as consumed by households. While measuring changes in the price of consumption is extremely important for measuring economic activity in the UK, it does not always reflect the changes in prices and costs that are directly observed by UK households. Please refer to a description of the different measures and approaches to inflation in the UK (Measuring changing prices and costs for consumers and households, proposed updates: March 2020).

For example, the CPIH measures owner occupiers’ housing costs (OOH) using a rental equivalence approach. This approach estimates the cost of consuming housing services by calculating the price that would need to be paid to rent an equivalent property. The HCIs look at using a measure of direct payments in place of rental equivalence (such as mortgage interest payments, dwelling insurance, ground rent and Stamp Duty Land Tax) to more closely reflect changes in costs as they are experienced by UK households.

A methodological article on the HCIs was released alongside the first publication. This article described the methodological differences between the CPIH and the first preliminary estimates of the HCIs and explored the impact of each of these differences. The second methodological article built on the former article, and provided the methodological framework for the second preliminary estimates of the HCIs following discussion with the Advisory Panel on Consumer Prices - Technical.

The present article expands on the former article by presenting additional methodological development, following discussion with the Advisory Panel on Consumer Prices – Technical. In particular, the article presents new improvements to the education class, which includes for the first time tuition fees paid upfront (in addition to student loan repayments), and to the interest on financial debt class, which takes into account further categories of interest payments compared with the previous release. In this regard, the article presents specific results for the education and the financial debt class.

Other methodological changes that have enhanced the definition of a household subgroup (tenure), as well as wider changes that impact on the HCIs (that is, changes in owner-occupier household costs), are also described. Furthermore, the article compares the HCIs estimates in the previous and in the current releases. The comparison will show how methodological changes have improved estimates.

For completeness and ease of reading, this article retains much of the information from the former methodological article.

Because the third preliminary HCIs series is the most up-to-date, it should be that referenced to for research purposes. Any future changes to the HCIs will be highlighted in future releases and, during the development phase, we will update the historical series accordingly. The development phase is due to conclude with the quarterly test running, currently planned for 2022.

It should be noted that the HCIs are experimental; therefore, we would caution against any use of the indices, other than for research and indicative purposes. We welcome feedback on the methodology presented in this article to cpi@ons.gov.uk.

The preliminary estimates of the Household Costs Indices (HCIs) differ to measurement of the Consumer Prices Index including owner occupiers’ housing costs (CPIH) in five different ways. These are:

• the use of democratic weighting
• the use of a payment approach for some items:
• owner occupiers’ housing costs (OOH)
• tuition loan repayments and tuition loans paid upfront (rather than the full cost of tuition fees)
• insurance, where gross expenditure is used to calculate the weight for insurance premia
• the inclusion of a measure of interest costs on financial debt

Section 3 outlines the proposal of the HCIs and the current development.

Section 4 provides information on the data used for the current development.

Section 5 describes the democratic weighting methodology that has been used to construct the HCIs and how it differs from the plutocratic method used by CPIH.

Section 6 has two sub-sections. Firstly, it details the results associated with the education and with the financial debt class of spending, which underwent a change in treatment between the former and the current release of the HCIs. Secondly, Section 6 shows how the changes in these classes and other methodological changes have impacted on the third preliminary estimates of the HCIs.

Sections 7 and 8 discuss limitations and strengths of the HCIs.

Back to table of contents

Consumer price indices measure price changes across the economy. However, this may not necessarily reflect the households’ month-on-month experience of changing prices and costs. Household Costs Indices (HCIs) are designed to fill this gap, that is, to measure price and cost changes as experienced by different household groups. The HCIs thus fulfil a different purpose compared with the Consumer Prices Index including owner occupiers’ housing costs (CPIH), which is the Office for National Statistics’s (ONS’s) lead measure of inflation. Therefore, the focus of this article is household subgroups.

The Consumer Prices Index (CPI) was the lead measure of inflation up until March 2017, when it was replaced by CPIH. CPIH incorporates a measure of owner occupiers’ housing (OOH) costs into the CPI framework, along with Council Tax. In line with CPI, CPIH has a largely acquisition-based approach with a few exceptions, depending on the goods and services in the basket.

With regard to OOH costs, CPIH adopts a “rental-equivalence” methodology (which is a use-based approach) similar to other international consumer price indices. The rental equivalence method consists of determining how much rent would be paid for an equivalent property in the private sector, as a proxy for housing services. In CPIH, OOH costs are thus imputed based on an estimate of equivalent costs for renting.

The HCIs build on CPIH with features that reflect their separate scope. Firstly, the HCIs aim to capture prices at payment, which leads to differences between CPIH and the HCIs for the classes of spending, with a time lag between payment and use. This applies, for example, to university fees paid through a student loan, since the actual payment for a household is deferred. It follows that, while CPIH captures student loans through the full price of tuition fees in the year of enrolment, the HCIs adjust for student loans (particularly for tuition loans) at the time of repayment.

CPIH and the HCIs also differ in their treatment of OOH costs, which are a rental-equivalent approach and a payment approach respectively.

Another distinction between CPIH and the HCIs lies in the measurement of insurance premia. In fact, CPIH expenditure weights are based on the service charge element of an insurance package, whereas the HCIs expenditure incorporates the full cost, including money used to reimburse households.

Alongside these, the measurement of changing prices and costs as experienced by households requires the inclusion of items that impact on a household budget but are not within scope of CPIH. This is the case with interest payments, which are not part of CPIH because interest on loans is neither a good nor a service consumed by households. They are, however, included in the HCIs, since they are part of the payments made by households to consume goods, and a change in interest rates impacts on households’ expenditure as much as a change in price of ordinary goods and services (Astin and Leyland, 2015 (PDF, 546KB)).

The differential scope of the HCIs from CPIH similarly informs the weighting methodology for their estimation. While CPIH deploys a “plutocratic” weighting mechanism, the HCIs use a “democratic” method, as illustrated later in the article (Section 5) and described in Investigating the impact of different weighting methods on CPIH.

Features of the HCIs

The preliminary estimates presented in the second release of the HCIs differed to CPIH in five ways:

• a democratic weighting method
• the use of a payments approach for measuring owner occupiers’ housing costs (OOH)
• gross expenditures for dwelling, transport and health insurance
• the inclusion of interest payments on credit card debt
• the inclusion of student loan repayments rather than the full cost of tuition fees

In the third release of the HCIs, there are further changes to the education class to reflect more closely household expenditure for tertiary education. In particular, the third preliminary estimates adjust for UK tertiary education through tuition repayments (often referred to as student loan repayments) and for tuition fees paid upfront, which are included for the first time in the HCIs.

Additionally, in the new release, the class of interest payments has been expanded to include mail orders, secured and unsecured loans, and overdrafts, following discussion with the Advisory Panel on Consumer Prices. Another substantial change refers to improvements in the OOH cost (payments) approach, originally presented in the article Measures of owner occupiers’ housing costs, UK: July to September 2019.

Further changes, compared with the former release, refer to the definition of a new household subgroup (households with or without a disability), and to the separation of subsidised and private renters into unique classes (whose total provides the actual rentals class) with the aim of enhancing the expenditure profile of the tenure subgroup.

In the following paragraphs, the rationale for the improvements to the HCIs is discussed. Data for the items specifically designed for the HCIs (for example, mail orders) were either drawn from available data sources (for example, Wealth and Assets Survey) or derived indirectly for the purpose of the HCIs, as will be illustrated in Section 4.

Owner occupiers’ housing (OOH) costs (payment approach)

Housing costs represent a large proportion of households’ expenditure. Households incur OOH costs as a result of owning, maintaining and living in their own home. Under the payment-based approach, OOH costs attempt to capture what households pay out as owner occupiers to consume housing services.

At present, the costs refer to:

• mortgage interest payments
• dwelling (building) insurance
• ground rent
• Stamp Duty
• estate agent fees
• Home Buyers’ Survey
• major repairs and maintenance
• house conveyancing

Mortgage capital repayments are excluded as they do not represent consumption expenditure; however, in the future, a variant HCIs measure will be developed, which will incorporate such payments. For more information please refer to the National Statistician’s statement on the future of the Household Cost Indices.

The payment-based approach for OOH in the HCIs is the same as that published in Measures of owner occupiers’ housing costs, UK: January to March 2020.

Compared with the second preliminary estimates of the HCIs, the OOH payment approach has undergone some changes through the implementation of a new methodology for the Stamp Duty Index and the corresponding weights. They will be described in Section 4.

Tuition loans and tuition fees paid upfront

University tuition fees are either paid upfront or through a loan, which is repaid at a deferred date upon earning above a certain amount. Therefore, at any point in time, expenditure for university education consists of upfront payments, made by some households with current students, and of student loan repayments, made by households with ex-students.

Based on acquisition principles, CPIH accounts for total university tuition fees, whether paid up front or through a loan, and regardless of additional interest costs. For the HCIs, however, a distinction between the type of payment is necessary since only present expenditure impacts on a household budget. Therefore, the HCIs are designed to account for both upfront tuition fee payments and student loan repayments, which are weighted according to their share of expenditure.

The second preliminary estimates of the HCIs included student loan repayments for the first time, with a view to including both types of payment in the next release. Accordingly, the third preliminary estimates also account for tuition fees paid upfront. Since loan repayments may also cover former-students’ subsistence costs, with potential double-counting of expenditure as discussed in the former methodology article, these latter have been removed for the aim of the new release. Therefore, in the latest HCIs, the cost of tuition is estimated through tuition loan repayments and tuition fees paid upfront.

Insurance premium (gross costs)

While life insurance is excluded from consumer price indices because it is regarded as a financial asset rather than a good or service consumed by households, insurances for the dwelling (house contents), transport and health are in scope and so are measured by both CPIH and the HCIs.

Households pay an insurance premium as a guarantee of compensation for loss (for example, through theft). The cost of an insurance premium results from the service charge paid to the company and from a payment into a “claims pool” from which claims are met. Based on their alternative purpose and acquisition or payment approach, CPIH and the HCIs capture insurance expenditure differently.

Under the acquisition approach, expenditure reflects the net impact on the household sector as a whole. In particular, insurance payments allocated to meeting claims are regarded as a transfer in the household sector (that is, payments from households claim-free are received by the households making a claim). Accordingly, the service charge is the only element captured by CPIH (net premium) for expenditure.¹ Conversely, under the payment approach, expenditure is represented by the full cost of the package. Therefore, the HCIs adopt a “gross premium” approach for insurance.

Interest payments on financial debt

Interest payments are the amount paid to borrow money, or for delaying the repayment of a debt, with costs increasing when interest rates rise. While interest payments are not part of CPIH, which measures the changing cost of consumption goods and services, they are measured by the HCIs, which treat interest payments as a household outgoing.

Clearly, for owner-occupier households, interest payments on a mortgage are the major outgoing within the category. Nevertheless, a household budget is also impacted by other forms of debt, such as those incurred when taking out a loan to buy a car.

In the former release, the HCIs comprised mortgage interest repayments (included in OOH costs), interest on maintenance and tuition loans (captured in the student loan repayments component), and interest on credit card debt. This latter has undergone a revision to also incorporate mail orders, loans other than student loans broadly defined as secured and unsecured loans, and overdrafts. Therefore, the third preliminary estimates of the HCIs comprise, in addition to the interest payments already captured in the former release, interest on tuition loans exclusively (rather than on tuition and maintenance loans, in principle captured under unsecured loans) and interest on the previously-defined larger class of financial debt (credit cards and mail order, secured or unsecured loans, overdrafts).

Notes for background

1. CPIH has a net premium approach for expenditure and a gross premium approach for prices (see Section 4).

Price indices are estimated from the quantity of products purchased and the prices of these products, where expenditure (or weight when expressed as a share of total expenditure) is defined as price times quantity. The Household Costs Indices (HCIs) are estimated for class-level categories in accordance with the Classification of Individual Consumption according to Purpose (COICOP). Hereafter, class of spending and COICOP are used interchangeably.

For the estimation of the HCIs, the price data are for the most part taken from the Consumer Prices Index including owner occupiers’ housing costs (CPIH). Expenditure consists of both household-level (or micro-level) data and aggregate household spending data in the UK. The former is facilitated by the Living Costs and Food Survey (LCF), which gathers detailed expenditure variables from a sample of around 6,000 representative households per year.

For each interviewed household, the LCF provides spending information, sample weight and demographic variables. Through sample weight, household expenditure in the UK is estimated, whereas demographic information enables the computation of price indices by type of household (for example, retired versus non-retired). The LCF data used for the latest HCIs release refer to households surveyed between Quarter 1 (January to March) 2003 and Quarter 4 (October to December) 2017.

CPIH and HCIs share the aggregate household spending data (or total expenditure) for those items that are identically derived by the two measures. The data are extracted, for every class of spending, from the System of National Accounts, which compiles them from the LCF and other sources. As with the micro-level data, for the latest HCIs release, the time period for total expenditure spans from 2003 to 2017.

A downside of expenditure data from surveys is that household spending estimates might not align with the System of National Accounts, because of typical surveys limitations (for example, low response rate). To address this issue, the final household data informing the construction of CPIH and of the HCIs are obtained by reconciling micro-level data with national accounts aggregates.

For the additional items included in HCIs but not CPIH, price indices are separately computed. Corresponding household-level expenditure is extracted or estimated from the LCF, whereas total expenditure data are drawn from the most suitable available source, whether administrative sources or other survey-based sources. Details on these additional data are provided in the next subsections.

Education component

The education component of the HCIs is the same as CPIH for primary and secondary education, and for tuition fees paid by international students. Differences apply for the tuition fees paid by UK domiciled students as they are entitled to apply for tuition loans. Therefore, for the aim of the HCIs, the cost of tuition fees for UK students is accounted for by both tuition fees paid upfront in each year of the HCIs time series and by tuition loan repayments over the same time period, but for student loans borrowed in the past.

The following sub-sections describe the methods to derive expenditure for tuition loan repayments and for tuition fees paid upfront, and their price indices.

Tuition loan repayments (Student Loans Plan 1)

There are currently two repayment plans for graduates who have received student finance within the UK.

Plan 1 relates to students from Scotland, Northern Ireland, England and Wales who started their undergraduate course before 1 September 2012.

Plan 2 refers to English and Welsh students who started their undergraduate course after 1 September 2012, with mandatory repayments starting from April 2016 if gross income exceeds the annual threshold.

A further repayment plan (Plan 3 for postgraduate studies) was introduced in the academic year ending 2017, with repayments starting from April 2019. The HCIs currently adjust for Plan 1, with a view of including Plan 2 and Plan 3 in future years once repayment expenditure becomes more significant.

The second preliminary HCIs estimates, which incorporated for the first time a measure of student loan repayments, included both tuition and maintenance loan repayments, which are not strictly part of the education costs and may cause double counting. In the absence of a breakdown in the published statistics of student loan repayments (SLC), a method was designed for the removal of maintenance loans, which has enabled the education component to become more refined for the latest release. The rationale and steps of the method are outlined in the following.

Although repayments for tuition fees and maintenance loans are not available separately, it can be reasonably assumed that the breakdown by type of loan (tuition fees versus maintenance loans) at the time of repayments reflects that at the time of borrowing, which is available. This latter can be drawn from the SLC for England, Wales and Northern Ireland and from the Scottish Government (statistics on Higher Education Student Support). Based on this assumption. the method is as follows:

1. Compute proportion of maintenance loans for each student cohort (“C”) (data source: SLC’s publications)

2. Derive repayment cohort from student cohort (based on the assumption of a typical three-year undergraduate course, see Income Contingent Repayment by cohort)

3. Estimate expenditure for maintenance loans by applying C to student loan repayments for each repayment cohort (data source: SLC’s publications)

4. Estimate expenditure for tuition fees loans as the difference between student loan repayments and maintenance loans for each repayment cohort

5. Aggregate expenditure for tuition fees loans, for each calendar year, across all the repayment cohorts

Table 1 shows the application of the method in a generic calendar year T.

Although the second and the third preliminary estimates differ for the removal of maintenance loans, the index for repayments with maintenance loans excluded can be reasonably assumed to be the same as the index for total repayments. This assumption is analogous to the insurance case where gross and net insurance have the same price index (see Insurance premium, later in this section).

The price index, extensively described in the second methodological article, is derived from a threshold model, which consists of estimating monthly student loan repayments from graduate salaries (without bonuses) and the repayment threshold, taking into account that individuals pay 9% of their income over a given threshold. As the index does not use prices, it could be defined as a “quasi-price” or “costs” index and January graduate salaries are fixed over the calendar year to bring it closer to a price index.

The unchained index is obtained as follows:

where s_Jan,y is the salary in January of the calendar year y, h (h=1, ..., N) is the household with a graduate salary above the threshold, t_i is the threshold income for month i (which is constant throughout a financial year), and w_h,y is the survey weight for household h. January graduate salaries are fixed over the calendar year to allow for direct “cost” comparisons to be made.

Monthly graduate salaries (without bonuses) are extracted from the Office for National Statistics (ONS) Labour Force Survey (LFS) and are derived in line with the Graduates in the UK labour market publication. A graduate is defined as a person aged 21 to 64 years who has left education with a qualification above “A” level standard, which includes higher education or a degree.

Notably, the estimation of monthly repayments has changed to better capture repayments: while in the second preliminary estimates the repayment threshold was applied to monthly mean graduate salaries, in the new release the threshold is applied to the salary of each graduate in the LFS, which leads to a price index that is based on the full survey information.

Tuition fees paid upfront

Since university fees are paid either through a loan or upfront, the HCIs are designed to also include payments for upfront tuition fees. These latter, however, are not directly available from the Higher Education Statistics Agency (HESA), which publishes total university fees. Therefore, for the inclusion of upfront fees, a method was specifically developed as follows: based on data availability on total fees (HESA) and on fees borrowed (SLC), for each academic year fees paid upfront were derived as a difference between the former and the latter figures.

While upfront tuition fees for Wales and Scotland were similarly derived, an alternative method was required for Northern Ireland (NI) where tuition loans also include the fees borrowed to study in the Republic of Ireland (RoI), whereas RoI is out of scope as it is not part of the UK. The evidence that university fees in NI are comparatively similar to Scotland and Wales informed the derivation of upfront fees in NI as a product of total university fees in NI and the average proportion of fees borrowed in Scotland and Wales:

The price index for tuition fees is the same as the price index for total tuition fees, which is used in the Consumer Price Indices (CPI/CPIH).

OOH costs (payment approach)

Owner occupiers’ housing (OOH) costs have undergone some methodological improvements, starting with the release Measures of owner occupiers’ housing costs, UK: July to September 2019 and further with the release Measures of owner occupiers’ housing costs, UK: January to March 2020.

The main improvement relates to changes in the measurement of the Stamp Duty Index. Under the previous methodology the expected Stamp Duty was computed for each price band as the product of the mid-point value and the corresponding number of houses sold. This was subject to a certain degree of arbitrariness and required assumptions, such as the uniform distribution of sales within the price band. In addition, the uprating was based on the UK House Price Index, which did not enable the Stamp Duty Index to capture regional variation.

Since Quarter 3 (July to September) 2019, the Stamp Duty Index is obtained from more granular data, consisting of final residential property transactions prices across England, Scotland and Wales. The data also include an experimental measure for first-time and former buyers, which allows the index to reflect first-time buyer’s relief, which was not accounted for in the previous methodology. Since Quarter 1 (January to March) 2020, Stamp Duty only refers to residential receipts.

The OOH changes could impact on the new HCIs estimates, as compared with the second preliminary estimates.

Prices and expenditure for OOH costs used are the same as those used in CPIH(payments).

Insurance premium

The types of insurance captured by the HCIs relate to dwelling (house contents), transport and health, and additionally buildings insurance, which is included within OOH costs.

CPIH captures expenditure on the service charge (net premium) whereas the HCIs expenditure is the full insurance premium (gross premium), including the “claims pool”. They are called “net weights” and “gross weights”, respectively.

In CPIH, aggregate household expenditure for the net weight is calculated as the difference between expenditure on insurance premia and the amount paid out in claims. In the HCIs, aggregate household expenditure for the gross weight is drawn from the LCF.

The price index for gross premium is based on the gross cost of the premium in accordance with the HCIs design of capturing the full costs faced by the households. The same price index is adopted by CPIH because of the infeasibility of deriving prices for the service charge only.

Interest on financial debt

While interest payments are out of scope for CPIH, which focuses on consumption goods and services, the HCIs adjust for them. The second preliminary estimates of the HCIs included interest payments on credit card debt, alongside mortgage interest (class 4.2) and student loans (class 10), with the view to capturing further types of financial debt in future releases. This is achieved in the third preliminary HCIs estimates, which also comprise interest payments on mail orders, secured and unsecured loans, and overdrafts.

For the former release, interest payments on credit cards were assigned to the class of spending 12.6.1 (financial services not elsewhere classified (n.e.c)). Following the expansion of the class of interest on financial debt, interest payments are assigned to the class 12.6.9, which allows us to keep all of the items together in the same class. This follows from UK practice where a residual class (“9”), within the COICOP structure, is introduced when the existing classification does not include a new specified class of spending. Another example is Council Tax for which the 4.9 class of spending is used.

Each sub-component of the new interest on financial debt class is described in detail in the subsequent sections.

Price indices of the new interest on financial debt class

The index for each sub-component of the financial debt class was derived through a “revaluation approach”, which is used internationally to capture changes in the cost of interest payments. The method, which is based on the Consumer Price Index Manual: Theory and Practice (IMF, 2004, paragraph 10.27), is currently used by Statistics New Zealand in their Household living-costs price indexes.

The rationale is that, to obtain a “price” for interest payments, the effective interest rate has to be applied to a stock of debt, which is a monetary value, where the (purchasing) value of money varies with inflation. Since a fixed stock of debt, on which interest is charged, changes in value through time, an adjustment for the changing value of the stock of debt because of inflation is required. This gives a price estimate which is comparable through time.

For the HCIs, the adjustment is carried out through a re-evaluation of a change in interest rate based on the corresponding change in CPIH; that is, they are multiplied together to provide an adjusted series for inflation. The method thus requires the CPIH time series alongside interest rate for each financial debt sub-component over the same time period.

For credit cards, the base data consisted of the following Bank of England (BoE)’s interest rates:

• the not seasonally adjusted weighted average interest rate on credit card loans to households (CFMHSDG), available from January 2005 to December 2015 and later discontinued

• the not seasonally adjusted weighted average interest rate on credit card loans to individuals (CFMZ6IR), available from January 2016 onwards

The BoE series for interest rates on credit cards was also used for mail orders, for which BoE interest rates are not available.

For secured and unsecured loans, and overdrafts the base data were:

• CFMHSDE – monthly average of UK resident monetary financial institutions sterling weighted average interest rates, loans secured on dwellings to households (in per cent) not seasonally adjusted

• CFMHSDI - monthly average of UK resident monetary financial institutions sterling weighted average interest rates, other loans to households (in per cent) not seasonally adjusted

• CFMHSDH - monthly average of UK resident monetary financial institutions sterling weighted average interest rates, overdrafts to households (in per cent) not seasonally adjusted

Financial debt expenditure

National expenditure for credit cards was obtained similarly to the second preliminary estimates of the HCIs, that is, total expenditure was obtained as a product of the UK number of households with the average weekly expenditure per household (scaled up to an annual average) for credit card interest debt (data source: Family Spending in the UK).

National expenditure for interest payments on mail orders and overdrafts was estimated through three steps:

• first, monthly payments on mail orders (with interest) and monthly overdrafts amount were drawn from the Wealth and Asset Survey (WAS)

• secondly, monthly interest payment expenditure was obtained by applying monthly BoE interest rates to expenditure for mail orders and to overdraft amounts from WAS; this is because every mail order payment includes a portion of interest and an overdraft incurs an accrued interest that the household pays

• thirdly, the resulting monthly interest expenditure for mail orders and for overdrafts was aggregated on a calendar year basis

Household-level expenditures, normally extracted from the LCF, were not available for overdrafts. Therefore, a specific method was applied to estimate household interest payments on overdrafts.

The method consisted of applying multiple linear regression to overdrafts from WAS and using the regression coefficients to estimate overdrafts from the LCF. The explanatory variables were selected amongst those available both in WAS and the LCF and based on their correlation with overdraft amount. In the LCF, first the households with an overdraft were identified based on a flag variable, and then the predicted value of overdraft amount was estimated. To derive interest payments on overdrafts, the interest rate from BoE was applied to the predicted values for each household.

Expenditure for interest payments on secured and unsecured loans was also derived from WAS; however, in this case interest payments were derived by applying a ratio estimator based on LCF data (since LCF contains additional variables not present in the WAS dataset that allow us to derive the interest paid) to total WAS repayments. Details on the calculation of the ratio estimator are provided in Annex A.

This estimator has LCF interest payments as the numerator and LCF total payments as the denominator and reflects the fact that the majority of loan repayments are amortised, meaning that households pay a fixed total amount every month, but the amount of interest paid decreases each month.

Since the LCF does not include a question on whether the loan is secured or unsecured, interest payments in the estimator are derived by using the average rate for both secured (CFMHSDE) and unsecured loans (CFMHSDI):

where y is the calendar year (y=2003, ..., 2017) and j refers to whether the loan is secured or not (j=1,2). For example, if the ratio is 0.05 and expenditure from WAS in a calendar year is £0.9 billion, the corresponding interest paid is £ 0.045 billion.

Loans with 0% interest were excluded from the calculation. Amongst the remaining loans, student loans and loans on second homes were excluded from the analysis as they are adjusted in other sections of the HCIs (education and OOH respectively). Since record-level estimates of interest repayments are required for the calculation of democratic weights, the ratio estimator described previously is also applied to the total loan repayments for each household in the LCF dataset. This assumes that the interest to repayment ratio is constant for every household.

Other changes

The third preliminary estimates of the HCIs include disabled households as a further household subgroup. The categorisation is based on the LCF which, from the financial year ending 2014, includes the following questions (PDF, 3.72MB), which reflects the definition of disability under the Equality Act 2010:

• Do you have any physical or mental health conditions or illnesses lasting or expected to last for 12 months or more? Yes/No

• Does your condition or illness reduce your ability to carry out day-to-day activities? Yes, a lot/Yes, a little/Not at all

As a further change, the third preliminary estimates of the HCIs introduce a breakdown of the actual rentals class of spending (4.1) into private and subsidised rentals. This is so that price movements for subsidised renters are not influenced by private rental movements and the other way around.

The split of rental expenditure by type of tenure required a slight change in the methodology used in CPIH to reconcile household and national expenditure. There are instances where the CPIH expenditure differs from reported LCF expenditure and a very small number of households report expenditure within a particular class. In these instances, a proxy is used where expenditure on a COICOP class is reapportioned to households using a higher aggregate (for example, group or division). However, because subsidised renters and private renters are distinct categories, the proxy methodology is not effective for subsidised rentals and therefore was suppressed. The potential impact of this is that rental expenditure may be overstated for these households.

The Consumer Prices Index including owner occupiers’ housing costs (CPIH) could be broadly considered a Lowe price index, which is the method used by most statistical agencies to construct their headline CPIs.

The Lowe price index in CPIH “uses current-period price information alongside expenditure weights that are price-updated” (Flower and Wales, 2014, page 12). With household data, the Lowe index “weights the price experience of different households by their share of expenditure” (Flower and Wales, 2014, page 13). Put otherwise, households contribute to the index with a weight proportional to their expenditure share, which leads to high-spending households having a larger weight than low-spending households. This weighting mechanism, defined as “plutocratic weighting”, is appropriate for macroeconomic indicators such as the CPIH.

An alternative mechanism is the so-called “democratic weighting”, which assigns instead equal weight to each household’s share of expenditure and, consequently, is deemed to be more appropriate for the Household Costs Indices (HCIs). Democratic weights are obtained as follows:

• first, each household’s expenditure on a class of spending is divided by the household’s total expenditure

• secondly, the average expenditure share for this class is calculated, which provides a democratic weight

• this procedure is then repeated for each class of spending within the basket of goods and services

Note that in populations where all households purchase goods in equal proportions, a price index is the same independently of the selected approach (plutocratic versus democratic) since the weights are homogeneous across the households. On the other hand, the more the expenditure basket differs across households, the larger the difference between a price index under the two approaches.

The formulae for the plutocratic and democratic weighting can be found in the first methodological article. Additional information on the two methods can be found in the following articles: Investigating the impact of different weighting methods on CPIH, and Methodology to calculate CPIH consistent inflation rates for UK households; Variation in the inflation experience of UK households: 2003 to 2014.

This section presents the results associated with the new education and financial debt classes and with other methodological changes from the former Household Costs Indices (HCIs) historical series.

Education and financial debt

Consistent with previous methodological articles, the potential impact of a change is inferred from the Consumer Prices Index including owner occupiers’ housing costs (CPIH) using impact analysis, which allows us to understand the impact in isolation from other HCI-specific effects. In particular, impact analysis compares the all items CPIH (as published) with the CPIH, which would stem from expenditure and price indices associated with the HCIs components of education and of interest on financial debt.

Differences are larger for the financial debt component, which is an additional class compared with CPIH. In particular, Figure 1, for the education component, shows a difference between 2007 and 2008, when the HCIs Education Index pushes the all items CPIH (as published) upwards. The direction of change is similar for most periods.

Figure 2, for the financial debt component, shows an upward movement on CPIH (as published) in most periods through time, more notably between 2007 and 2009 and between 2013 and 2015.

Third (2020) versus second (2019) preliminary HCIs estimates

In the following, the impact of methodological changes between the second preliminary HCIs estimates in 2019 and third preliminary HCIs estimates in 2020 is presented for retired and non-retired households, second-lowest, middle and second-highest income deciles, and households classified by tenure type, as these classifications were estimated for both series. The disabled households group was not available last year and so we are unable to provide a decomposition for this series.

A common approach to reporting inflation is through the year-on-year growth rate, which compares prices for the latest month with the same month a year ago. The growth rate is thus “determined by the balance between upward and downward price movements of the range of goods and services included in the index” (Consumer price inflation, UK: December 2018, page 2). By extension, the impact of methodological changes between two similar series could be explored through the contributions to the difference in their growth rate (alternatively defined as decomposition).

The second and the third HCIs preliminary series are not comparable in the strict sense because of a number of changes in the underlying component indices.

First of all, compared with the former release, the education class of spending has undergone three changes: the addition of tuition fees paid upfront, the removal of maintenance loan repayments, and the computation of the student loan repayments index using a microdata-based threshold model.

Secondly, while in the former release the financial debt class of spending consisted of credit cards only, in the new release mail orders, secured and unsecured loans and overdrafts are also included.

In addition, to better capture changes in rental expenditure, the actual rentals class of spending is broken down by type in the new release.

On the other hand, these improvements specific to the HCIs are not the only classes of spending with changes between the second and third preliminary series. As described in the Background section, the Stamp Duty Index of the owner occupiers’ housing (OOH) costs has undergone improvements between the former and the latest release of the HCIs, with a potential impact on the differences in the 12-month growth between the two series.

Nevertheless, the decomposition of the difference in the HCIs’ growth rate between the second and the third series does not reduce to the sum of the contributory effects of the methodological changes to the HCIs and of OOH costs. This is because a change in a component modifies overall expenditure, resulting in changes in both household shares and in the distribution of total expenditure across the full spectrum of the Classification of Individual Consumption according to Purpose (COICOPs). This results in democratic weights partly differing between the former and the new HCIs series.

In the following, the impact of the improvements, measured as a contribution to the difference in the growth rate, is plotted for each subgroup.

The impact is measured through the decomposition of the difference (between the two series) into five components:

• OOH costs, which shows the impact of the new Stamp Duty Index

• the interest on financial debt class of spending (“debt”), which shows the impact of adding, to the former class consisting of credit cards only, interest payments on mail orders, secured and unsecured loans, and overdrafts

• the education class of spending (“education”), which shows the impact of adding tuition fees paid upfront, of removing maintenance loan repayments and of improving the threshold model for student loan repayments using microdata instead of aggregated data

• the private and subsidised rental classes of spending, compared with the aggregate class (“actual rentals”)

• the contributions of other classes of spending resulting from weight adjustments (“residual scaling differences”).

The cross-sectional contributions to the year-on-year growth rate for the second and the third preliminary HCIs estimates are presented in Annex B.

Retired and non-retired households

For retired households, the breakdown of the actual rentals class of spending represents the main contribution to the change in growth rate (Figure 3). The breakdown indeed has the effect of separating out the impact of private and subsidised rent. The rentals contribution in Figure 3 is mainly attributable to the subsidised rent as a combination of prevalence (21% of retired households) and increased spending, until 2015, compared with the actual rentals category.

For non-retired households, the decomposition of the difference in growth rate between the two series shows an impact for each class of spending that has undergone a methodological change, particularly for OOH and education (Figure 4). For OOH costs, this finding might reflect the greater presence of first and subsequent buyers amongst non-retired households, as a result of a change in the Stamp Duty Index. The upward contribution of education could be explained with the payment of upfront tuition fees, which is included in the third release.

Second-lowest, middle and second highest income deciles

For the second-lowest income decile, the change to the rental class of spending, broken down into private and subsidised rentals in 2020, dominates the difference in growth rate between the two series (Figure 5). The change could be attributable to the access to social housing for the households in a lower socioeconomic status as well as to limited access to university education and alternative finance arrangements (see Figures B5 and B6).

The second-highest income decile shows contributions from multiple classes of spending (Figure 6). The larger impacts come from OOH and education with positive and negative contributions up to 15%. The role of OOH can be explained by the effect of better capturing the cost of Stamp Duty. The positive contribution of education can be partly ascribed to adding tuition fees paid upfront in the third preliminary estimates.

Whilst middle-income households are not included in the results article, it is useful to consider the methodological impacts on this group nonetheless as it helps to show what the impacts would be for a more “average household”.

The middle-income decile is defined by increased spending for rent until 2015 (Figure 7), which is attributable to subsidised rents increasing over the time period (see Figure B10). Other leading classes are OOH and education, whose impact is however smaller compared with the second-highest decile. This can be explained with a smaller proportion of owner-occupied households and of households accessing university education in this subgroup.

Households by type of tenure

Figures 8 to 10 show the impact of the methodological changes for households living in social housing, privately renting or in owner-occupied accommodation. It is noticed that the category of owner-occupiers includes variables related to the purchase transactions, which can lead to households appearing both in rented and in owner-occupied accommodation (see Figure B16).

A major contribution is provided by the decomposition of the actual rental category into the accounting classes (private and subsidised rentals) whose effect was combined in previous releases. To this purpose, Figure 8 indicates that subsidised rentals had a major upward impact on the 12-month growth until 2015 for households in social housing. The other way around, over the same time period, private rentals had a major downward impact on the 12-month growth for households in private renting (Figure 9).

These findings reflect the differential pattern of subsidised and private rent against the aggregate actual rentals, with subsidised rentals growing fastest and private rentals slowest until 2015, when the pattern reversed. Households in private renting also point to a positive impact for the class of education, partly attributable to the payment of upfront fees. On the other hand, owner-occupier households show an impact of OOH, which can be ascribed to the changes in the Stamp Duty Index, alongside an impact for education (Figure 10).

There are limitations to the construction of the Household Costs Indices (HCIs) that warrant discussion. These refer both to the data, and to limits inherent in the way that democratic weighting has been applied. The discussion is reported separately for data input and method.

Data (prices and weights)

The data input’s limitations are as follows.

The HCIs are intended to measure the experience of changing prices and costs for UK households; hence, household-level spending data should refer to all UK households. However, micro-level data are survey-based and, therefore, they miss a proportion of the population that live in institutional households (that is, care homes and student halls). For example, expenditure related to the use of health services for people in care homes is not captured by the Living Costs and Food Survey (LCF).

This limitation is partly overcome through the redistribution of household expenditure based on aggregate expenditure from the System of National Accounts. In fact, this latter makes adjustments to ensure that the estimated expenditure covers the entire UK population. Another limitation is that national accounts data omit UK household spending overseas and reflect the spending of foreign households within the UK, which future developmental work will address.

Other limitations arise for HCIs-specific items:

• double-counting expenditure may also be considered a drawback for insurance premia, with respect to the goods and services that are purchased from insurance claims

• interest payments on overdrafts were estimated from regression whereas direct estimates from household surveys are preferable; to improve the estimates, for next year’s release, further potential explanatory variables will be identified from the Wealth and Assets Survey (WAS) and the LCF datasets

• graduate salaries (without bonuses) are used for the threshold model in the education class, however, some bonuses might be included, this is because there is no variable for earnings excluding bonuses in the Labour Force Survey (LFS). For the purpose of the analysis a proxy measure was thus derived which uses the respondents’ usual gross pay if they state that their gross pay in the reference period was higher than usual because of receiving a bonus payment

Limitations of the weighting method

One of the primary limitations of the democratic weighting approach relates to the intended coverage of the HCIs. The price indices that inform the method are either extracted from CPIH or are constructed based on aggregated data. However, the calculation of “true” household group-specific price indices would require the use of household-specific prices. As price data are collected from retailers rather than by asking households the prices they have paid for each item, separate price indices are not available for different types of household. The methodology thus assumes that households all experience the same changes in price.

The “democratic” weighting gives an equal weight to each household, rather than to each individual in the economy as would be the case in a “truly” democratic index (Flower and Wales, 2014, page 13). This latter would pose, however, major challenges for data representativeness and implementation. Furthermore, in a “true” democratic approach, individuals would be followed longitudinally to capture their expenditure trajectories through time, which is not feasible. More information could be found in the article Variation in the inflation experience of UK households: 2003 to 2014.

In spite of the limitations described in Section 7, the Household Costs Indices (HCIs) have a major strength. Indeed, they address the requirement, outlined in the international manual of consumer prices (PDF, 5.2MB), of measuring “price inflation as experienced and perceived by households in their role as consumers” (page 17). The improvement to the education and to the financial debt classes of spending represent a further step towards developing such an index in the UK. Results by type of household and income decile enable the identification of the differential experience of inflation across the economic spectrum. This is supplemented by the CPIH-consistent results for UK household groups that are constructed through recognised economic principles.

It is recognised that further development should be in consideration for the HCIs, such as mortgage interest payments, which are a substantial component of a household budget and the inclusion of student loans repayment under more recent Plans (for example, Plan 2 from September 2012).

For an overview of the inflation measures in the UK including the HCIs, and their proposed updates please refer to Measuring changing prices and costs for consumers and households, proposed updates: March 2020.

Retired and non-retired households

A retired person is defined as anyone who describes themselves in the Living Costs and Food Survey (LCF) as “retired” or anyone over minimum National Insurance pension age describing themselves as “unoccupied” or “sick or injured but not intending to seek work”. A retired household is defined as one where the combined income of retired members amounts to at least half the total gross income of the household.

Second low- and second high-income households

The reasons for choosing the second and ninth deciles are described in the CPIH-consistent inflation rate estimates for UK household groups.

Households classified by tenure type

We categorise households into three tenure types: subsidised rented, privately rented and owner-occupied.

Subsidised renter households are defined as any household that either rents their property from a council, a registered social landlord or live in their property rent-free. Privately rented households are defined as any household that rents their property from a private sector landlord. It excludes households who live in their property rent-free.

Owner-occupier households are defined as any household in which the residents own the property outright or are buying the property with a mortgage.

HESA

Higher Education Statistics Agency

SLC

Student Loans Company

The work has benefitted from the input of a range of Office for National Statistics (ONS) experts including, but not limited to, Tanya Flower, Philip Gooding, Michael Hardie, Chris Jenkins, Andy King and Helen Sands. We also acknowledge the useful advice provided by external experts on the Advisory Panels on Consumer Prices – Technical. In particular, we are grateful to John Astin and Jill Leyland for their continued support towards the development of this work. We also would like to express our gratitude to Paul Bloomfield, David Bovill, Joanna Bulman, Joel Jones, Clara Kidd and Lyndsey Williams.

As described in Section 4, the interest on secured and unsecured loan items are calculated by applying a ratio estimator based on Living Costs and Food Survey (LCF) data, to total Wealth and Assets Survey (WAS) repayments on secured and unsecured loans. This was necessary because of the prevalence of amortised repayment schedules amongst this type of borrowing. The LCF dataset provides variables that allow the calculation of amortised interest payments.

The computation of interest payment expenditure on secured and unsecured loans took into account compound interest as follows.

Let:

D_h (m) be the amount of debt D outstanding for household h in month m (to simplify the notation we reference a household with, say, two loans as two separate households)

A_h be the total amount borrowed by household h

P_h be the amount of a monthly repayment for household h (note that because the loan is amortised there is no time-dependence)

r_h(t_h) be the (monthly) interest rate on household h’s loan taken out t months prior to month m (note that both the rate itself, and the amount of time since the loan was granted are dependent on the household h)

We can write the amount of debt outstanding for a household h in month m as:

where the derivation is readily available in the literature.

Then the interest payment on the outstanding principal will be:

This provides the LCF interest payments at the household level in month m, with the total amount over a calendar year being the sum of the LCF interest payments across 12 months.

Therefore, the total amount of interest repaid by all households H in month m is given by:

Since data on interest rates charged for loans are not directly available in the LCF, the following two simplifications are made in the calculations:

1. we use the BoE interest rate for time (m-t):

where the interest rate was the average of CFMHSDE and CFMHSDI

2. we use the average loan age for month (m-t):

This second simplification allowed us to run our calculations at a more aggregate level and avoid the possibility of encountering negative interest payments at the household level in the absence of information on the specific interest rate. Since the interest paid each month is different, this calculation was compounded for each month of the year to get an annualised amount of interest paid.