Private rental growth measures, a UK comparison: January to March 2017

The Index of Private Housing Rental Prices (IPHRP) measures the change in price of renting residential property from private landlords. The same private rent price indices are also used for the rental equivalence measure of owner occupiers’ housing costs (OOH) in CPIH (Consumer Prices Index including owner occupiers’ housing costs).

With CPIH now being put first in the consumer prices statistical bulletin, the quality of private rental data is paramount. This is because OOH are based on a rental equivalence (RE) approach; specifying that the cost of housing services is estimated by looking at the cost of renting an equivalent property to the one you own.

This RE method is conceptually strong and the approach used in Germany and the USA, but requires a large amount of good quality data to be implemented. OOH in the UK is based on rental data drawn primarily from the Valuation Office Agency (VOA) and consists of around 500,000 separate rental estimates per year, giving approximately 40,000 prices per month.

VOA is one of a range of sources of rental data across the UK, but is the one that provides the highest quality and most robust data for measuring consumer inflation. The remainder of this article assesses the merits of the different data sources, highlighting why the VOA data best meets the requirements for a monthly Consumer Prices Index.

Superficially there is a considerable difference between the Office for National Statistics’s (ONS’s) measures of rents, the Index of Private Housing Rental Prices (IPHRP) and owner occupiers’ housing costs (OOH), and comparable indicators from the private sector. However, once adjustments are made for differences in what the indices are measuring, the ONS measure of rents is more closely aligned with the other sources of rental prices.

Some of the differences can be explained by compositional changes and quality changes in the stock of rental properties. Both of these are specifically excluded from IPHRP to ensure that only pure price change is captured in the index. Put another way, the aim of the IPHRP is to compare “like with like”.

In addition there are methodological differences in the way rental prices can be measured. One approach is to measure the stock of rents; that is, aiming to capture the price of all properties in the rental market. The second approach is to measure the flow of rents, in other words, to capture the price of new lettings made in the reference period. Both approaches yield different results.

A number of private sector organisations – estate agents and property companies – produce estimates of changes in rental prices. These include Countrywide PLC, Homelet and LSL Property Services. In addition, other companies such as Zoopla, are a rich source of rental data. More details about these and the Office for National Statistics’s (ONS’s) measure, the Index of Private Housing Rental Prices (IPHRP), are given in this section.

3.1 Index of Private Housing Rental Prices

Index of Private Housing Rental Prices (IPHRP) measures the change in price that tenants face when renting residential property from private landlords, thereby allowing a comparison between the prices tenants are charged in the current month as opposed to the same month in the previous year. The index does not measure the change in newly advertised rental prices only, but reflects price changes for all private rental properties.

The IPHRP is constructed using administrative data. That is, the index makes use of data that are already collected for other purposes in order to estimate rental prices. The sources of rental prices are the Valuation Office Agency (VOA), Scottish government and Welsh government. All three organisations deploy rental officers to collect the price paid for privately rented properties. The sources of expenditure weights are the Department for Communities and Local Government, Scottish government, Welsh government and the VOA.

3.2 Countrywide PLC

Countrywide has over 1,300 offices nationwide operating under locally recognised brands. Countrywide regularly publishes a monthly new lets average rental price, which is based on 4,000 to 5,000 properties let in the reference month by letting agencies under the Countrywide umbrella. The data are weighted and stratified by regions and number of bedrooms. The regional weights are derived from the English Housing Survey (EHS) and the “number of bedroom” weights are calculated using Countrywide data. Countrywide also manages a portfolio of 75,000 properties and publishes an occupied lets and a renewal average rental price based on this portfolio of properties.

3.3 HomeLet

HomeLet is a private company that provides tenant referencing and insurance for letting agents, landlords and tenants. Since January 2009, it has published a UK new lets monthly index called the HomeLet Rental Index, which is based on achieved private rental prices collected from its tenant referencing services (around 29,000 per month). In August 2016, HomeLet re-launched this index incorporating a methodology that stratifies by property type and geography and a historic series was also made available.

3.4 LSL Property Services

LSL Property Services is a provider of residential property services, including residential lettings. Its estate agency arm trades under a number of brands. LSL has commissioned a monthly private housing rental index since January 2011 called the Buy-to-Let Index, which is based on around 7,000 new lets in a month, collected from the mix of rental services that it provides across England and Wales. The sample is stratified and weighted by “government regions” and by property type. The weights for regions are derived from the EHS, whereas property type weights are derived from LSL’s database of privately rented properties spanning a number of years.

3.5 Zoopla

Zoopla is a property website covering the residential property market, which focuses on providing users with the resources they need to make better-informed property decisions. Launched in 2008, the Zoopla website is a searchable directory of UK residential properties and is one of the leading online destinations for property consumers to search for homes and do their market research.

All of these sources provide an extensive range of rental data, but a closer look at the various rental measures published by them over recent years provides a diverse picture, as shown in Figure 1.

It is notable that the rent increases measured by the private sector have tended to be higher than those shown in ONS’s IPHRP. Also, the private sector measures are more volatile, which initially caused concern, but this can be explained further in this section.

This higher level and volatility, of the private sources can mainly be explained by what the different data are showing. Firstly, almost all of the private sector measures primarily focus on newly let properties, whereas IPHRP picks up a mixture of newly let properties and existing lets (Countrywide also covers both, but Figure 1 just shows their new lets series).

This is important, as evidence obtained from VOA rental officers suggests that the greatest price rises occur when properties are newly let, as against existing tenants, who tend to see smaller price increases.

In early 2016, the VOA placed a question in the BDRC Landlord Panel survey whose respondents comprise National Landlord Association members with a wide range of portfolio sizes and geographies. The question asked “What was the typical level of rent you were able to achieve?” broken down by “new tenants” and “renewals with existing tenants”, in 2015. It was reported that 23% of new lets were charged at the same rate as the previous letting, however, 46% of renewals saw no increase. For those whose rent did increase, the average increase for new lets was 5% while for renewals this was lower at 3.4%.

To summarise, landlords were less likely to increase rents for existing tenants, and when rents were increased, it was by a smaller percentage. This is thought to happen because landlords make a considered business decision, preferring a small increase from a known reliable tenant, rather than experiencing a void period with no income and the associated cost of re-letting. Consequently, larger price increases tend to occur when the existing tenant moves out and the property is advertised.

In addition, analysis of contract length statistics finds that rental contracts tend to last for around 12 months. During this contract period a tenant’s rent is unchanged. IPHRP, as a measure of the stock of rental properties, includes all new lets, those renewing their contract and those within their existing contract. This results in a smoother series than those that simply focus on new lets, where there is more volatility between each month.

So, differentiating between newly and existing lets is crucial to analysing the differences. Unfortunately, not all of the private sector providers separate out these, which makes it more difficult. However, analysis has been possible with one of the private sector data sources, namely by ONS on the Countrywide data. Also, ONS is in the process of obtaining data from Zoopla and the results from this will be included in updated versions of this article.

Looking at the analysis of Countrywide data, Figure 2 shows that newly let properties tend to have both higher and more volatile growth patterns compared with its “occupied lets” series, which captures the position for existing tenants.

This leads onto a few important questions – what is the percentage split between new and occupied lets in the Countrywide data and more generally in the UK as a whole? And then how does this compare with the make-up of the IPHRP dataset?

Countrywide data broadly consists of a 90 to 10 split between occupied lets and new lets. The split within the VOA series is not currently available, although plans are in place to address this through transformation of their data collection system. We are currently discussing this with VOA and once the split is available, it will be incorporated in future analysis. In the meantime, as the raw data collected used in IPHRP is representative of the full range of local market rents in each Broad Rental Market Area it seems acceptable to assume that the IPHRP series consists of a broadly similar split.

So, on the basis of this assumption, there are three analyses to consider:

• a growth comparison of the Countrywide occupied lets series with the all-IPHRP dataset; whilst this is strictly not comparing “like with like” (as the IPHRP also includes new lets), it is a reasonable assumption to start with, to assess overall quality comparisons between the two datasets

• repeat the first point but expressing both series as an index

• undertake a hedonic regression with all-Countrywide – (both new and occupied lets) data, to enable a comparison with the all-IPHRP dataset; this will create more of a “like with like” basis for the comparison

The results of this are shown in Section 4.

Figure 3 compares the Countrywide occupied lets series with the all-Index of Private Housing Rental Prices (IPHRP) dataset (which is a mixture of new and occupied lets). Whilst they are on a different basis, the two datasets follow a very similar pattern. This is not that surprising, as occupied lets account for over 90% of the stock. Some other small disparities will also still remain, which are caused by other differences between the two sources, such as coverage and the methodology applied.

On coverage, Countrywide only covers properties that are let or managed by them, so whilst being a large data source, they don’t necessarily have data for all areas, for example, they might not have data for all local authorities. While Countrywide use data from the English Housing Survey (EHS) to weight their transactions at a regional level, an adjustment is not made below this level. This may cause a bias in the estimates for a region if the data collected does not appropriately represent that of rental properties for the region.

IPHRP is also weighted at the regional level, but biases below the regional level are minimised in IPHRP by the large sample size of the Valuation Office Agency (VOA) data. Rent officers aim to keep this sample representative within each broad rental market area by utilising various data sources (such as the census) and their local knowledge of the rental sector. Further information of the VOA data collection can be found within the article published in January 2015.

As for methods, IPHRP takes a sample at the start of the year, which is followed for a year to calculate a price index, whereas Countrywide follows the properties they manage and calculates an average price. There are also some differences in the weights applied, although these have a minimal impact, as essentially both are derived from Department for Communities and Local Government data (more details on the weighing can be found in Section 3).

Whilst the annual growth rates of both series are very similar, how do the levels of the series compare? The two series cannot be compared in monetary terms, as IPHRP is only published as an index; however, the Countrywide data can also be expressed as an index, so that a comparison can be made. Figure 4 again compares the Countrywide occupied lets series with IPHRP, with both series expressed as an index referenced to January 2013 equals 100.

The comparison shows minimal drift between the series and therefore Figures 3 and 4 indicate a positive outcome in showing that the two datasets follow a similar levels trend. However, a better “like with like” comparison would be beneficial; to reinforce the conclusion that IPHRP is correctly measuring the stock of private rents.

This can be achieved by undertaking a hedonic regression on the full set of Countrywide data and comparing this with the IPHRP dataset (which covers all rents). For this, Countrywide transactions data have been used, which consists of three separate datasets:

• new lets – a property let with a new tenant moving in (around 4,000 to 5,000 properties)

• renewals – a property where the tenant’s contract is expiring and the tenant agrees a new contract (around 6,000 properties)

• occupied lets – all the properties with active tenancies – these include historic new lets and all renewals (around 75,000)

A hedonic regression similar to that used in the production of the UK House Price Index has been developed by methodologists in the Office for National Statistics (ONS). Further information on the method used can be found in Annex A. Given that the Countrywide occupied lets data is only available as a full year from 2014, this hedonic regression series can only currently be calculated for 2015 (with 2016 analysis to be published later in 2017). The resulting hedonic price index reflecting the full stock of rental properties (new and existing lets) is presented against IPHRP in Figure 5.

Figure 5 shows that the two series, whilst based on different sources, produce an index with a similar trend pattern, albeit with slightly greater disparity in the summer months. Once more timely (2016) data is available, this analysis will be re-run for subsequent articles, incorporating a focus on lower geographic levels, such as regional.

Therefore, when considering this latest chart and the earlier comparisons with the Countrywide occupied lets data, it is clear that the IPHRP shows very similar patterns – thus providing additional confidence in the IPHRP as a robust measure of the growth in the stock of rental properties.

To add further evidence to this conclusion, we are investigating the potential acquisition of rental data from Zoopla and once available will undertake analysis of this data to compare how a price index calculated using Zoopla data compares with the IPHRP. More details regarding the method, results and conclusions will be presented later in 2017.

In evaluating measures of rental price, some users have focused on the difference between the average rental prices published by the Valuation Office Agency (VOA), as part of their Private Rental Market Statistics (PRMS) publication, and the Index of Private Housing Rental Prices (IPHRP). Both are based on the same underlying private rents data collected by VOA rent officers for England.

Figure 6 presents the 12-month growth rates for the VOA PRMS series and IPHRP (using the England IPHRP series for comparability). Two additional series are also presented, which adjust for changes in the composition of the VOA PRMS sample over time.

There are clearly some major differences between the VOA PRMS series and IPHRP. This raises the question; “How can we have confidence in IPHRP if it diverges so much from VOA private rents series?" Well, these differences can be explained.

IPHRP is a “price index”, in that it seeks to make pure price comparisons, whereas VOA’s PRMS is a simple “average of transactions” collected during the period – the latter is designed to provide a “snap shot” of the rental market over the previous 12 months.

Over the period presented, the average annual growth rate of the VOA PRMS is 4.2% compared with an average annual growth rate in IPHRP of 2.2%. The difference of 2 percentage points (pp) can be accounted for as follows:

• geographical compositional effects (0.9pp) – as shown by the yellow line in Figure 6

• compositional changes relating to the number of bedrooms of properties in the VOA PRMS sample (0.6pp) – as shown by the red line in Figure 6

• coverage differences between PRMS and IPHRP (0.2pp) – as shown in the article Explaining private rental growth

The remainder (around 0.3 percentage points) of the difference can be explained by:

• the application of price index methodology to calculate price indices (a matched sample and mix adjustment); this is discussed further in the article Explaining private rental growth

• changes in composition below the local authority level

• a general improvement in the quality of the private rental sector, changes in quality would impact an average price measure but are intentionally removed from a price index; in recent times, the size of the private rented housing stock has more than doubled and some of this supply has come from the owner occupied market, which is generally in better repair than the rental market – evidence from the English Housing Survey suggests that rented properties are now better maintained than they were a few years ago

We are therefore confident that we understand and can explain the differences between the tracks of the IPHRP and VOA PRMS, and are therefore confident that IPHRP is a fair reflection of the increase in rental prices over time. Additional work will be taken forward to further break down and explain the remaining difference.

A simple example illustrating how compositional change impact an average price measure is available in Annex B.

The analyses in this article have shown the wide range of sources available for rental data and highlighted the extent of similarities, and differences, that exist between them. This has all pointed to the Index of Private Housing Rental Prices (IPHRP) being a high quality and robust measure that is fit for purpose in measuring UK consumer price statistics.

However, there is still an ongoing requirement to ensure that IPHRP keeps abreast of any developments in the private rental market and that the statistics keep pace with the rapidly developing big data agenda. Section 7 expands on this.

To address the ongoing challenge of ensuring that the rental data used for consumer prices is of the highest quality, the Office for National Statistics (ONS) is producing a specific work programme for this, sitting within the wider Prices work programme. The core resources for the work are in the Housing Market Indices (HMI) branch, but these are supplemented by the central Prices Development team.

Some of the aspects of the work programme that provide the ongoing assurance and development of rental price statistics include:

• creating an additional dedicated statistical resource within the HMI branch – responsible for the production and development of the Index of Private Housing Rental Prices (IPHRP)

• continuing regular meetings with the Valuation Office Agency (VOA), Welsh government and Scottish government – as the main sources of the private rental data for England, Wales and Scotland; this communication allows ONS to provide assurance of the data being used, understand any developments in the rental market and to gather anecdotal evidence behind any price changes as they occur; quality assurance processes applied to this data can be found within the OOH Quality Assurance of Admin Data

• building on the analysis presented in this article to provide regular updates of the comparisons between IPHRP and other published measures of rental price statistics – developing more detailed analysis of comparisons at a lower geographical level, such as by region

• actively pursuing the acquisition of VOA rental micro-data, once the Digital Economy Bill is in place; this gives ONS access to a much richer dataset, to further explore and develop rental analyses

• following on from this, continually explore the availability and suitability of other sources of private rental market data for potential inclusion in the production and assurance of IPHRP

• ONS will continue to work with the Northern Ireland Housing Executive to develop a robust source of Northern Ireland rental price data so that the coverage of IPHRP can be extended to UK

This section provides further information on the methodology used to produce the series presented in Figure 5.

Transaction level data was received from Countrywide containing rental price information on all properties they had let in England, Scotland and Wales between April 2013 and February 2016. This was provided as three datasets:

• renewals – all contracts renewed within the period

• new lets – all new contracts within the period

• occupied lets – all occupied units for each month within the period

The countrywide data was used to create a rental price index for 2015 by using a hedonic regression model. In a hedonic regression the properties are defined in terms of a set of features or characteristics, each of which contributes to the rent paid for a property. For example, the presence of a garden, the number of bedrooms, or the location of the property will all contribute to the amount of rent paid, but none of these features can be priced in isolation. A regression model is used to estimate the value of each of these features from the set of properties during a particular period. Both new and existing lets were used in this regression model so that the resulting price index represents the stock of rental properties, consistent with IPHRP. The reason a hedonic regression is used (rather than comparing simple average) is that a hedonic regression can adjust for changes in the composition of the sample over time to make sure we are comparing “like with like”.

The Countrywide data was combined with some other data sources and the following property characteristics were used as “predictor” variables:

• local authority

• CACI ACORN area classification

• number of bedrooms

• furnished or unfurnished

• house type (flat or house)

A hedonic regression model of the following form was used:

where:

• pi is the monthly rent of a property

• k is a constant

• xij indicates whether property i has the characteristic j

• Bj is the coefficient associated with characteristic j

• ei is the statistical error

Using this model 83% of the variation in price (R squared) can be explained by the predictor variables. A predicted price was then obtained for each property based on the property characteristics used in the model.

As transactions for every local authority were not available for each month regional indices were used to impute for missing local authority rental prices. To create an index for each month the modelled rent in each month was compared to the modelled rent in the base period (Jan 2015). Transactions from the previous year were used as weights in order to aggregate the data to higher levels, that is, 2014 transactions were used to calculate weights in order to aggregate 2015 data. The consequence of this is that we are only able to present data for 2015 at this stage.

Figure 5 presents this series against IPHRP and shows that whilst based on different sources of data, the two series produce an index with a similar trend pattern.

Section 5 made reference to change in the composition of the VOA sample, but why does this matter? Well, this has a large impact on the “average rents” measure, in increasing it. However, there is not a similar impact on the index, as this does not happen with a price index measure – these effects are intentionally excluded as the index aims to compare “like with like”. A simple illustration of this is presented in Table 1.

This shows two local authorities (LAs) – LA1 has a monthly rent of £500 whereas LA2 has double this, at a monthly rent of £1,000.

The monthly rent is the same in both years 1 and 2 for each of the two LAs. However, the relative weights between the 2 years have changed, with both LAs still accounting for 50% of the total market in year 1, but the split between LA1 and LA2 changing to 40 to 60 in year 2.

The weighted average rent has thus increased from £750 in year 1 to £800 in year 2 (a 6.7% increase), all due to a change in composition in the population. However, there has been no actual increase in the rental prices, so a rental price index would show no increase in price between periods 1 and 2, whereas the average rents approach does.

The article Explaining private rental growth includes historic analysis on changes in the composition of the VOA PRMS sample and its impact on the resulting series.