Introducing the single official house price index

This article presents early historic estimates for the new official UK House Price Index (UK HPI). The new UK HPI will be published in its entirety in June 2016 and then monthly thereafter, replacing the existing house price indices currently published by the Office for National Statistics (ONS) and Land Registry for England and Wales.

The 2010 National Statistician’s Review of Official House Price Statistics recommended the production of a “single definitive house price index produced by the official statistics producer community”. Addressing this recommendation, the Office for National Statistics (ONS), Land Registry for England and Wales (Land Registry), Registers of Scotland and Land & Property Services Northern Ireland (LPSNI) have worked together and developed a new UK HPI that will fulfil this. This article provides users with an early look at a historical time series (1997 to 2011) for England and Wales based on the new methodology and an overview of the improvements made.

The 2010 National Statistician review concluded that neither of the current official house price measures, namely those currently published by ONS (which at the time was published by Department for Communities and Local Government and inherited by ONS in 2012) and Land Registry met all the main user needs. Given these limitations, there was scope to review the methods and sources being used and to make improvements that would provide users with a comprehensive UK HPI, available consistently at a sub-regional level and for sub-sets of transactions.

The main limitations with the current measures published by ONS and Land Registry can be summarised as follows: Coverage:

• the current ONS HPI is based on a sample of properties purchased using a mortgage - as such, it does not include any properties purchased using cash
• the current Land Registry HPI uses repeat sales methodology, meaning that only those properties that have transacted more than once are included - therefore it does not include any new build properties and may under represent properties that are seldom transacted

Price methodology:

• the calculation of average prices used in the current ONS HPI is carried out arithmetically - given the distribution of house prices (where typically the average price can be skewed by high value property), international best practice has evolved over time to recommend the geometric averaging of price data
• the calculation of average prices in the current Land Registry HPI is based on a standardised set of properties from April 2000, which are then uprated by the movement in the price index - this means that prices in the later periods are not reflective of current trends in the housing market
• the current ONS HPI approach to publishing average price levels does not allow for comparison between years

Therefore the new UK HPI methodology directly addresses these limitations.

In the most recent period published (December 2011), the new UK HPI shows an average price level of £185,000 for England and Wales. This is lower than the price recorded by the current ONS HPI for England (£222,000) for the same period but still above the equivalent price levels recorded by Land Registry (£157,000) and by way of further comparison, the main privately produced house price indices by Halifax (£157,000) and Nationwide (£163,000). It should be noted that the Nationwide and Halifax are UK averages.

The results published in this article are early estimates to provide users with an understanding of the likely impact of introducing the new UK HPI. The published data will be subject to further quality assurance before the live publication of data in June 2016 and therefore could be subject to change. England and Wales data are presented in this article for ease of comparison. It is expected that a similar impact will be present when UK estimates are published.

The main reason for the reduction in the average price in December 2011 for the current ONS HPI compared with the new UK HPI is the use of a geometric average, as opposed to an arithmetic average, to calculate prices. For example, if the new UK HPI December 2011 price was recalculated using an arithmetic average, it would be £222,000 (as opposed to £185,000). This reasoning will hold across the length of the timeseries. The improvements to other aspects of the methodology will also contribute to the difference (for example the inclusion of cash sales), although the impact of these is more difficult to quantify.

Figure 1 shows the impact of improvements by comparing average price data for England from ONS, England and Wales from Land Registry and the England and Wales series from the new UK HPI (Note: England and Wales has been used for comparison purposes as Land Registry does not publish UK estimates. ONS do not publish an England and Wales aggregate, as such, the ONS England series has been used).

A divergence can be seen between Land Registry and new UK HPI estimates from 2000. The average price from the current Land Registry HPI is computed by taking a geometric average price in April 2000 and applying estimated growth rates back to 1995 and forwards in time. Land Registry base averages are those of sale prices within a single month (April 2000) rather than a weighted average. The new UK HPI uses a year’s worth of sales to construct weights and uses these to construct the average price. This mix-adjustment allows for different houses being sold in different periods by annually updating the fixed basket of properties and is a difference in methodology that is driving the divergence from 2000.

Figure 2 presents the annual percentage change in the new UK HPI index for England and Wales compared to ONS (England series) and Land Registry (England and Wales series) measures. The trends in the series can be seen to be reasonably consistent over time in the size and direction of house price changes. There are some small differences when looking at the current ONS and Land Registry data compared with the new UK HPI, which are due to a combination of factors:

(i) Differences in estimation techniques - both the new UK HPI and ONS HPI employ a hedonic regression while Land Registry apply a repeat sales method.

(ii) A difference in coverage of the data – ONS measure is based on mortgage transactions, as such excludes properties that are cash sales. Under a repeat sales methodology as used by Land registry new properties are not taken into account as they have not been previously traded. Cash sales and new properties are both included under the new UK HPI methodology.

Given that all three series show very similar trends, these differences are not having a large impact on house price growth over the period presented.

Between January 2002 and December 2011 the average annual growth was 4.9% for the ONS HPI (England), 4.6% for the Land Registry HPI (England and Wales), and 6.1% for the new UK HPI (England and Wales).

This section gives more detail of the main methodological improvements implemented in the new UK HPI. Further information on these improvements can also be found within the methodological article published in February 2016.

Model characteristics

Consistent with the ONS HPI, the new UK HPI applies a hedonic regression model. Further analysis was conducted to assess the suitability of characteristics feeding into the new hedonic regression model. In the case of the new UK HPI, the price-determining characteristics are as follows;

• local authority district in Great Britain and housing market area in Northern Ireland
• ACORN area classification variable
• property type (that is, detached, semi-detached, terraced, flat)
• floor area (metres squared)
• number of rooms
• new or old property

Buyer status (first-time or former owner-occupier) and financing status (cash or mortgage) were considered as price- determining characteristics in the model. However, they have not been included as they accounted for very little of the variation in price. It has also been argued by some users that these are characteristics of the buyer, rather than the property. Nonetheless, it is recognised that there is interest in these categories, as such results by buyer status and financing status will be published each month.

Weighting the price index

The new UK HPI is mix-adjusted to allow for the fact that different houses are sold in different periods by annually updating the fixed basket of properties. The resulting mix-adjustment weights will be calculated annually and will be derived using the latest complete year’s worth of transactions. This proposal is a change to the methodology used in the current ONS HPI, where a rolling 3 years' worth of transactions are used, but is in line with the Land & Property Services Northern Ireland (LPSNI) HPI which uses 1 year.

Seasonal adjustment

A seasonally adjusted HPI will continue to be produced and published, if seasonality is found to be present in the data. The level of geography at which the seasonally adjusted series will be calculated and published has yet to be decided, and will be assessed once a sufficient period of new data is available from the new HPI to carry out a formal seasonal adjustment review.

Method for calculating average prices

There are different ways of calculating average prices. These include the arithmetic mean, the geometric mean and the median. The current ONS HPI uses the arithmetic mean, whereas Land Registry and LPSNI use the geometric mean, and Registers of Scotland publish results for both arithmetic mean and median.

International experts and the Government Statistical Service (GSS) Methodological Advisory Committee were consulted and decided that the geometric mean was the best measure, as it is less distorted by high values and is in line with international best practice.

Comparable average price levels

The new UK HPI will publish a comparable set of price data by up-rating the price at the reference period by the index, a similar approach to that used in the current Land Registry HPI. To ensure price levels remain representative, the reference period for the price series will be updated every five years, and the whole of the average price series rescaled to align with the new reference period.

The new UK HPI will be produced by ONS and published monthly by Land Registry on GOV.UK. This central point of publication will be managed by Land Registry and provide users with headline prices for UK, country and regional data, along with sub-regional data (to a local authority level) for England, Scotland and Wales. Northern Ireland results will be published quarterly and will be held constant in the UK calculations for the first 2 months of each quarter due to the volatility that is likely to be present in monthly estimates. Northern Ireland data will be available through the LPSNI release on the DFP website, although users will be linked to this data directly from GOV.UK.

The input data and new methodology will allow for the publication of the following breakdowns at all levels of geography detailed above:

• type of dwelling (detached house, semi-detached house, terrace or flat)
• type of property (new build or old)
• volume of sales
• cash or mortgage purchases (not available for Northern Ireland)
• type of buyer (first-time buyer or existing owner; not available for Northern Ireland)

The monthly publication of data will be accompanied by an accessible monthly report providing commentary and background information for the latest published data. This will also be accompanied by full tables of data for breakdowns detailed above (Northern Ireland detail will be available on a quarterly basis) along with interactive analysis tools.

Where possible, a full back series for the new UK HPI will be published. A commitment has been made to produce back series for the new UK HPI to reflect the availability of data that is currently published. This will need to be calculated in two stages:

First, historic data will be produced using the data and new methodology. However, this approach can only be used to produce a timeseries based on the availability of data (for example, Land Registry data for England and Wales is only available from 1995).

Second, where there is insufficient data available to produce historic data using the new methodology, a suitable method will be applied to derive a backseries, for example, prior to 1995 the path of the current ONS HPI will be used to construct the period from 1968 to 1995.

It should also be advised that a full back series will not be available for each of the various breakdowns available in the new UK HPI – for example, the distinction between cash and mortgage sales is only available from 2011.

Finally, the ongoing development of the new HPI has been assessed by the Statistics Authority for compliance with the Code of Practice for Official Statistics. The assessment report was published in July 2015. The Statistics Authority will carry out a second phase assessment after the new index is published, which will involve an assessment of the statistics in light of users’ views , consider how the Working Group have responded to the findings from the first report and cover compliance with remaining practices of the Code. It is hoped this assessment process will enable the new UK HPI to attain National Statistics status in due course.