1. Introduction
Statistics on the number of new and existing residential dwelling transactions are needed to monitor the level of activity in the housing market. This helps the planning process and helps policymakers and businesses make informed decisions.
Housing is a localised area of policy, and so these statistics are needed for small geographic areas. These statistics are available in the house price statistics for small areas (HPSSAs), which report the number of transactions of new and existing dwellings down to the Middle-layer Super Output Area (MSOA) level in England and Wales.
The number of transactions of new dwellings reported in the HPSSAs does not cover all new dwellings that were sold. The HPSSAs are based on HM Land Registry's Price Paid Data (PPD). These are not created specifically for the production of statistics, and it is not possible for them to identify all transactions of new dwellings as new.
We want to get the best possible coverage of sales of new dwellings, and we have explored experimental data from Ordnance Survey (OS) that could help us do this. OS has analysed data from multiple sources to identify when a new dwelling has been completed. This approach has the potential to validate the allocation of transactions of dwellings as new in the HPSSAs and to identify transactions that might relate to new dwellings that were not previously identified as new.
This report describes analysis undertaken to create a new "adjusted" version of the HPSSAs that takes account of the additional transactions of new dwellings identified in the OS data. The report also compares the published HPSSAs with the adjusted version of the HPSSAs.
For more information about how the data were created and the methodology we used, see the Data and methodology section.
The analysis presented in this report focuses on the number of transactions of new dwellings. This is a subset of the number of new dwellings built because not all new dwellings are sold; some new dwellings are built by a housing association, local council or developer that makes them available for rent rather than selling them.
Back to table of contents2. Summary of research conclusions
The adjusted version of the house price statistics for small areas (HPSSAs) covers more transactions of new dwellings than the published HPSSAs. The effect of this adjustment was that the number of transactions of new dwellings in England increased by 12.9% and the number of transactions of existing dwellings decreased by 1.8%.
In terms of quality assurance, the number of new dwellings identified in the Ordnance Survey (OS) data is similar to official statistics on the number of new dwellings built, both at the national and local level. Our analysis also showed that many of the records identified as new dwellings in the OS data were also listed in the HM Land Registry data as transactions of new dwellings. This provides reassurance the OS data provides a good coverage for new dwellings.
Adjusting the HPSSAs means that we could report the median price paid for new dwellings in 15.7% more areas. Overall, the adjustment has a small impact on prices for both new and existing dwellings. It affects the median price paid for existing dwellings in 32% of areas compared with 44% of areas for new dwellings. The majority of prices in these areas changed by less than £30,000.
Back to table of contents3. Disclaimer
We have published this Research Output to provide an indication of the effect of using data from Ordnance Survey (OS) to adjust the house price statistics for small areas (HPSSAs). Research Outputs are produced to provide information about new methods and data sources being investigated.
While some new analysis is presented here, these are not the official statistics on HPSSAs, which should still be used for data on the number and price of transactions of residential dwellings at the small area level.
Back to table of contents4. Research findings
Understanding the coverage of Ordnance Survey data
The data provided by Ordnance Survey (OS) aim to capture every dwelling that has been completed in recent years. Before using these to adjust the house price statistics for small areas (HPSSA), we wanted to check the validity of the OS data against other sources by ascertaining the level of coverage. We compared the number of completed dwellings in the OS data with housing supply statistics from the Ministry of Housing, Communities and Local Government (MHCLG).
Figure 1 shows the total number of dwellings identified as completed in the OS data as well as MHCLG statistics on the number of new dwellings (XLS, 734KB) (including new, converted and properties that have had a change of use).
Figure 1: The experimental Ordnance Survey data broadly agree with official statistics on new dwellings
Number of new dwellings, financial year ending 2014 to financial year ending 2018, England
Source: Ministry of Housing, Communities and Local Government - Table 123, Housing Supply: net additional dwellings; Ordnance Survey experimental data
Download this chart Figure 1: The experimental Ordnance Survey data broadly agree with official statistics on new dwellings
Image .csv .xlsFigure 1 shows that the overall number of new dwellings identified in the OS data is broadly comparable with the MHCLG's official statistics. For the financial year ending 2018, the latest year where both figures are available, the number of new dwellings in the OS data is 4.8% less than the number reported in the official statistics.
Local authority level data from the MHCLG can also be used to assess the coverage of the OS data. Figure 2 shows a comparison of the number of new dwellings in the OS data and in the MHCLG statistics for local authorities.
Figure 2: The experimental Ordnance Survey data are consistent with official statistics at the local authority district level
Number of new dwellings identified, financial year ending 2018, England
Source: Ministry of Housing, Communities and Local Government - Table 123, Housing Supply: net additional dwellings; Ordnance Survey experimental data
Download this chart Figure 2: The experimental Ordnance Survey data are consistent with official statistics at the local authority district level
Image .csv .xlsAlthough there are differences between the two sources for individual local authorities, there is a clear positive relationship between the two datasets. Data for previous years also show a similar pattern. Some difference between the two sources is to be expected. For example, the two sources may identify the same completed dwelling but with a different date recorded, meaning they could fall within the yearly total for different years.
Based on the validation checks reported in Figure 1 and Figure 2, we are confident in using the OS data to create an adjusted version of the HPSSAs for this research.
Transactions analysis: impact of adjusting dwelling transaction data based on OS data
The main analysis presented in this report is an assessment of the impact of the OS data on the number of new dwelling transactions reported in the published HPSSAs.
The HPSSAs use Price Paid Data (PPD) from HM Land Registry to provide statistics on the price paid and number of residential dwelling transactions for dwellings that were sold in England and Wales (the analysis in this report relates to England only).
Figure 3 shows the number of transactions that are recorded as:
- new in both PPD and OS data
- new in OS data but existing in PPD
- new in PPD but not found in OS data
Figure 3: In most cases, the HM Land Registry Price Paid Data new dwelling allocation is supported by the Ordnance Survey data
Comparison of records by source and allocation as new or existing dwellings, financial year ending 2014 to financial year ending 2018, England
Source: HM Land Registry – Price Paid Data; Ordnance Survey experimental data
Download this chart Figure 3: In most cases, the HM Land Registry Price Paid Data new dwelling allocation is supported by the Ordnance Survey data
Image .csv .xlsMost transactions of new dwellings are present in both the PPD and the OS data. There are also transactions in the PPD that are not recorded in the OS data.
The analysis in this Research Output looks at the effect of recategorising the transactions that are recorded as existing dwellings in the PPD but as new in the OS data, to create an adjusted version of the HPSSAs. This will result in the number of new dwellings in the adjusted dataset being higher and the number of existing transactions being lower. Table 1 summarises the different results possible when comparing the PPD and the OS data and the action taken when creating the adjusted HPSSAs in each case.
| As recorded by Land Registry | As recorded by Ordnance Survey | Explanation | Action taken on adjusted HPSSAs |
|---|---|---|---|
| New | New | OS confirms the LR ‘new’ allocation. | Retain the LR ‘new’ allocation. |
| Existing | New | OS suggests the transaction was for a new dwelling. | Adjust record of transaction to ‘new’. |
| New | Not listed | Land Registry records a transaction as new when they have evidence which suggests it is. | Retain the LR ‘new’ allocation. |
| Existing | Not listed | Transaction is likely to be an existing dwelling. The OS analysis tracks new developments only. | Retain the LR ‘existing’ allocation. |
Download this table Table 1: Action taken for different combinations of allocation by HM Land Registry and Ordnance Survey when creating the adjusted HPSSAs, England
.xls .csvFigure 4 shows the overall number of new dwellings sold in England as published in the HPSSAs compared with the number of new dwellings in the adjusted HPSSAs.
Figure 4: Adjusting records based on Ordnance Survey data increases the number of transactions of new dwellings
House price statistics for small areas (HPSSA) and adjusted HPSSA transactions of new dwellings, financial year ending 2014 to financial year ending 2018, England
Source: Office for National Statistics – House price statistics for small areas; HM Land Registry – Price Paid Data; Ordnance Survey experimental data
Download this chart Figure 4: Adjusting records based on Ordnance Survey data increases the number of transactions of new dwellings
Image .csv .xlsIn Figure 4, the number of transactions of new dwellings in the HPSSAs and in the adjusted HPSSAs are relatively stable over time and follow similar trends. In more recent years, the difference in the number of transactions of new dwellings between the two series has slightly increased, reaching 12.9% in the financial year ending 2017 and financial year ending 2018.
There are many more dwellings sold from existing stock than are sold new. The impact of adjusting the HPSSAs is lower for existing transactions, which reduced by 1.8% in the financial year ending 2018. Table 2 shows the resulting change in the number of transactions by comparing the published HPSSAs with the adjusted HPSSAs for the financial year ending 2018.
| Number of Transactions, England, FYE 2018 | HPSSAs | Adjusted HPSSAs | Change |
|---|---|---|---|
| Transactions of new dwellings | 104,950 | 118,532 | 13,582 (12.9%) |
| Transactions of existing dwellings | 742,715 | 729,133 | -13,582 (-1.8%) |
| Total | 847,665 | 847,665 |
Download this table Table 2: Summary of the difference in the number of transactions, England, financial year ending 2018
.xls .csvThe effect of adjusting the HPSSAs on the number of transactions reported at the small area level is shown in Figure 5.
Figure 5: Adjustment of new dwelling allocations has larger effect in areas that had a small number of transactions
Distribution of transactions of new dwellings by Middle-layer Super Output Area (MSOA) for financial year ending 2018, England
Source: Office for National Statistics – House price statistics for small areas; HM Land Registry – Price Paid Data; Ordnance Survey experimental data
Download this chart Figure 5: Adjustment of new dwelling allocations has larger effect in areas that had a small number of transactions
Image .csv .xlsIn the HPSSAs, price data are only published for areas where five or more transactions have been recorded in the reporting period. Figure 5 shows that the adjusted HPSSAs have more Middle-layer Super Output Areas (MSOAs) with enough transactions to report a median price paid for new dwellings. The published HPSSAs have 2,586 (38%) MSOAs with enough transactions to report a median price paid; this increases to 2,991 (44%) in the adjusted HPSSAs. The impact of this on prices is covered in the next subsection of this report.
Having considered the effect of adjusting the HPSSAs on transactions of new dwellings, we can also review the effect on transactions of existing dwellings. The number of transactions of existing dwellings is a lot larger than new dwellings, so the impact of the adjustment on the number of existing dwellings is smaller.
In the financial year ending 2018, all MSOAs in England had enough transactions of existing dwellings to allow comparisons of median prices. Figure 6 shows the distributions of transactions for existing dwellings by MSOA, for the published HPSSAs and for the adjusted HPSSAs.
Figure 6: Adjustment of existing transactions has small effect on distribution
Distribution of transactions of existing dwellings by Middle-layer Super Output Area (MSOA) for financial year ending 2018, England
Source: Office for National Statistics – House price statistics for small areas; HM Land Registry – Price Paid Data; Ordnance Survey experimental data
Download this chart Figure 6: Adjustment of existing transactions has small effect on distribution
Image .csv .xlsSummarising the effect of adjustment on new and existing dwellings in Figure 5 and Figure 6, we can see that the effect of adjustment for transactions of existing dwellings is less pronounced than for new dwellings, as there are many more transactions from existing stock than for new dwellings. Table 3 shows the numbers of MSOAs with five or more transactions for the HPSSAs and the adjusted HPSSAs.
| Number of MSOAs with five or more transactions, FYE 2018 | HPSSAs | Adjusted HPSSAs | Change |
|---|---|---|---|
| Transactions of new dwellings | 2,586 | 2,991 | 405 (15.7%) |
| Transactions of existing dwellings | 6,791 | 6,791 | 0 (0%) |
Download this table Table 3: Summary of Middle-layer Super Output Areas (MSOAs) that had five or more transactions, England, financial year ending 2018
.xls .csvAdjusting the HPSSAs to recategorise some transactions of existing dwellings to new dwellings has increased the number of MSOAs for which we can publish median prices for new dwellings. This is because there are now more MSOAs with five or more transactions of new dwellings. Out of the 6,791 MSOAs in England in the financial year ending 2018, there are 2,991 MSOAs with a publishable median price for new dwellings in the adjusted HPSSAs. This is an increase of 405 (15.7%), compared with the published HPSSAs.
For transactions of existing dwellings, the adjustment to the HPSSAs does not reduce the number of MSOAs for which a median price can be published.
Having shown that the number of transactions of new dwellings increases in the adjusted HPSSAs, the following subsection looks at the impact on MSOA median prices, which is the main focus of the HPSSAs.
Prices analysis: impact of adjusting house price data based on OS data
The overall median price paid for transactions in MSOAs is not affected by the adjustment, but it does affect the median price of new and existing dwellings separately in some areas. This is because some areas have a different split of transactions of new and existing dwellings from which to derive a median. Also, as covered in the previous subsection, when more transactions of new dwellings are available, some additional MSOAs have enough transactions for an MSOA-level median price paid for new dwellings to be published (see Table 2).
Figure 7 shows the number of MSOAs that had five or more transactions of new dwellings for the HPSSAs and the adjusted HPSSAs over the last few years.
Figure 7: Adjustment increases the number of MSOAs for which a median price paid could be published
Number of publishable Middle-layer Super Output Areas (MSOAs) for median price paid for new dwellings, financial year ending 2014 to financial year ending 2018, England
Source: Office for National Statistics – House price statistics for small areas; HM Land Registry – Price Paid Data; Ordnance Survey experimental data
Download this chart Figure 7: Adjustment increases the number of MSOAs for which a median price paid could be published
Image .csv .xlsFigure 7 shows that more MSOAs could have a median price paid published for new dwellings following adjustment than are currently published since 2014.
Many more dwellings are sold from the existing stock than are sold new, and these are more evenly distributed geographically than new dwellings, which can be clustered in multi-unit developments. These factors mean that MSOA-level transactions of existing dwellings are impacted far less by reallocation than transactions of new dwellings, with all areas reamining above the threshold to publish existing dwelling prices. Although all MSOAs still have a median price for existing dwellings, the calculated median price for an MSOA may differ through the reallocation of some transactions to new dwellings.
When comparing the overall price distribution of MSOAs across median price paid bands for transactions from both new and existing dwellings, the adjustment made little difference to the broad picture.
Additional comparisons can be made to look at each MSOA to see how the price is affected by the adjustment. In the financial year ending 2018, there were 2,586 MSOAs in both the HPSSAs and the adjusted HPSSAs where comparison of the median price of new dwellings is possible.
Figure 8 shows the extent to which the median price paid for new dwellings in these 2,586 comparable MSOAs differs after new dwelling allocations are adjusted.
Figure 8: Adjusting the HPSSAs had no effect on median price paid for new dwellings in most areas
Difference between house price statistics for small areas (HPSSA) and adjusted HPSSAs new dwellings median price by Middle-layer Super Output Area (MSOA) for financial year ending 2018, England
Source: Office for National Statistics – House price statistics for small areas; HM Land Registry – Price Paid Data; Ordnance Survey experimental data
Download this chart Figure 8: Adjusting the HPSSAs had no effect on median price paid for new dwellings in most areas
Image .csv .xlsIn more than half (56%) of comparable areas, the adjustment did not affect the median price paid. Of the remaining 1,130 MSOAs, most (84.7%) differ by up to £30,000, with a few MSOAs having price differences of over £50,000 following adjustment. When we look at the difference in percentage terms instead, the distribution is similar.
While the difference made to new dwelling prices can only be calculated for 2,586 comparable areas, the larger number of sales of existing dwellings means that median price paid can be compared for all 6,791 MSOAs. This is because they are calculated from more transactions, making them less sensitive to the recategorising of some transactions to new dwellings.
Figure 9 shows the distribution of differences, by MSOA, to existing median price paid following adjustment of records.
Figure 9: Adjusting the HPSSAs had no effect on median price paid for existing dwellings in two-thirds of areas
Difference between house price statistics for small areas (HPSSA) and adjusted HPSSAs existing dwellings median price by Middle-layer Super Output Area (MSOA) for financial year ending 2018, England
Source: Office for National Statistics – House price statistics for small areas; HM Land Registry – Price Paid Data; Ordnance Survey experimental data
Download this chart Figure 9: Adjusting the HPSSAs had no effect on median price paid for existing dwellings in two-thirds of areas
Image .csv .xlsMore than two-thirds (68%) of the existing median prices stayed the same after adjustment of the HPSSAs. A further 28% were different by £10,000 or less after adjustment, leaving 262 (3.9%) areas with differences in excess of £10,000. On average, prices of new dwellings tend to be higher than for existing dwellings, which may explain why more MSOAs showed a decrease in price after adjustment.
Back to table of contents5. Data and methodology
This section summarises the two main sources used in this report and gives an overview of how we have used them.
House price statistics for small areas (HPSSA)
The house price statistics for small areas (HPSSAs) use Price Paid Data (PPD) from HM Land Registry to provide statistics on the price paid and number of transactions of dwellings that are sold in England and Wales. This also includes information about whether a transaction is of a new dwelling or being re-sold from the existing stock.
HM Land Registry processes records of transactions for new and existing dwellings slightly differently. Those known to be new dwellings go through a separate process to those not identified as such. If a transaction is not specifically identified as being in a new development, it is recorded as "existing". Transactions relating to new dwellings in small developments and conversions are the most difficult to identify so are thought to be more likely than those from larger developments to be recorded as "existing".
This report uses PPD downloaded in April 2019, which is the same data used to create the HPSSAs that were published on 26 June 2019.
Ordnance Survey Data
Ordnance Survey (OS), as part of a project for the Ministry of Housing, Communities and Local Government (MHCLG) looking at the rate of house building, has used OS data products and a range of other sources to attempt to identify if each dwelling has been completed and, if so, when it was completed. The version used in this report covers England.
This analysis from OS provides information on new dwellings from April 2008 onwards. It was created from a range of OS, HM Land Registry and Office for National Statistics (ONS) products.
Knowing when a new dwelling has been built is complex, and there is no single definitive administrative dataset that summarises what new dwellings have been completed. OS has used numerous sources of information such as whether an address is registered for mail or Council Tax and whether a change of ownership has been made. In doing so, agreement of different sources can provide evidence for whether and when a new dwelling was completed. Contradicting or incomplete information may indicate a dwelling is yet to be completed.
OS is aware of important dates and quality of input datasets and reviews its process and decision logic accordingly.
The data used for the analysis presented in this report are based on a file supplied to the ONS in July 2019.
Methods
We carried out the following checks to ascertain the validity of the OS data:
- we aggregated record-level data on new dwellings from OS and compared the number of records against a summary of the number of records provided by OS; these matched, confirming that our processing of the OS data was correct
- We compared our aggregated version of the OS data on new dwellings to official statistics on house building produced by the MHCLG (Table 123 "New build" + "Conversion" + "Change of use", published 15 November 2018); these two data sources were broadly comparable
As a result of our investigations of the record-level OS data and our subsequent discussions with OS, we made decisions about how to process the data to create an adjusted version of the HPSSAs. We used these decisions to create a list of transactions that would be edited in the creation of the adjusted HPSSAs, which were produced as follows:
- in the OS data, we retained only the dwellings that have transacted (that is, where the OS data include price paid information matched from HM Land Registry's PPD)
- we retained a list of transacted dwellings from the OS data where HM Land Registry recorded the transactions as being for an existing dwelling
- we retained only transactions that are within the HM Land Registry standard PPD to ensure consistency with current HPSSA processing
- we retained transactions where the date on which the dwelling was believed to have been completed ("ready date") in the OS data was derived from the PPD
- we retained transactions where the "ready date" identified in the OS data is within a year of other important sources of information contained in the OS data (that is, date the address existed, date registered for Council Tax or date built); this reduces the chance of relating the transaction of an existing dwelling with the same or similar address to a new dwelling
Having completed this processing of the data, we then created the adjusted HPSSAs as follows:
- we matched the list of new dwelling transactions in the OS data to the PPD for transactions of existing dwellings, changing the identified transactions from being recorded as existing to new
- we calculated the median price paid and number of transactions for each area and financial year
The adjusted HPSSAs were then ready for comparison against the published HPSSA.
Back to table of contents6. Feedback
We would like to receive feedback on this Research Output and the methodology used to produce these estimates, including how they might be improved and potential uses of the data. Please email your feedback to better.info@ons.gov.uk.
Back to table of contents7. Next steps
As a result of the research presented here, we will explore the feasibility of integrating the Ordnance Survey (OS) data into the house price statistics for small areas (HPSSAs).
We will do this by:
- investigating the possibility of securing ongoing data supply
- conducting further quality assurance
- considering the best way to incorporate the adjustments described in this report into the HPSSA production process
We will provide updates on our progress and other developments in the HPSSAs.
Back to table of contents