A new framework for UK GDP: progress, challenges and the future

This article is part of a series of articles outlining the developments we are implementing to the UK National Accounts. It is focused on the improvements we made to gross domestic product (GDP) in Blue Book 2019, the challenges we faced, and the further improvements we plan to make over the coming years. Published alongside it is an article outlining the scope of all the improvements to be made in the 2020 edition of the Blue Book.

In Blue Book 2019, we introduced important elements of a new framework for producing our annual estimates of GDP, but did not implement double deflated industry gross value added (GVA) in the UK National Accounts as planned. When we outlined our ambitious plans for Blue Book 2019, we noted that the new framework would not only lead to better quality estimates of GDP but would also provide a framework to identify inconsistencies in the compilation of GDP.

The results from the full application of the framework identified three key challenges:

• Cases of incongruence, particularly pre-2010, between deflators derived from the Consumer Prices Index (CPI) compared with the Producer Price Index (PPI), two primary deflator sets used in the UK National Accounts.

• New information in the new framework gave cause to question previous decisions taken to balance the three approaches to GDP at a detailed product and industry level.

• Issues around implementing some of our existing statistical methods in the new framework; an example of the application of chain-linking is outlined here.

In response to these challenges, we have developed a research programme that will lead to improvements in the use of deflators and approach to the GDP balancing process in the UK National Accounts. In time, these will lead to experimental double deflated estimates of industry GVA and integration into the UK National Accounts.

In 2016, the Independent Review of UK Economic Statistics identified opportunities for the ONS to develop its UK National Accounts in line with international best practice. In October 2018, following investigation of alternatives, we outlined ambitious plans for the scope of Blue Book 2019, including the transformation of the UK National Accounts by using new data sources and methods to produce gross domestic product (GDP).

This new framework (the “H-Approach”) involves confronting the estimates from the three approaches to estimating GDP – production, income and expenditure – using the Supply and Use Tables (SUTs) framework. This framework, since 1991, has been applied to set the annual current price level of UK GDP, but would be extended to also set the level of volume GDP, bringing the UK in line with international best practice and with the aim of introducing double deflated estimates of industry gross value added (GVA).

In September 2019, we introduced important elements of this new framework, but did not implement double deflated industry GVA in the UK National Accounts. Given the importance of fully understanding the impacts of double deflated GVA, we opted to use our pre-existing approach to produce industry-level estimates.

This article presents previously unpublished results from intermediate stages of the Blue Book 2019 production process to illustrate the challenges faced in implementing double deflation. The article goes on to outline a research agenda to address these challenges and, in time, lead to the full implementation of double deflation in the UK National Accounts. The article assumes an understanding of the “H-Approach” as outlined as part of our Blue Book 2019 article series; for ease an overview is provided in Annex A.

The remainder of this article is structured as follows:

• Section 3 – outlines what double deflation is and its importance
• Section 4 – outlines the progress made as part of Blue Book 2019
• Section 5 – outlines the challenges faced and what this means for our future work plan
• Section 6 – outlines our future research and communication, and planned roadmap to double deflation in the UK National Accounts
• Annex A – provides an overview of the “H-Approach”

Double deflation is internationally accepted as the best approach to producing volume estimates of industry GVA. It has been the focus of important recommendations from the National Statistics Quality Review and the Independent Review of UK Economic Statistics.

Producing SUTs both in current prices and in volume terms using the “H-Approach” provides a framework to enhance data coherence, consistency and quality of GDP estimates, as well as improved links to other parts of the national accounts framework such as productivity, regional accounts and environmental accounts. One of the main benefits of introducing the “H-Approach” is the introduction of double deflated estimates of industry-level GVA. One of the most significant improvements of this approach is to ensure that the deflation of the outputs produced in the economy, the intermediate consumption of goods and services used up to produce those outputs and the final uses of those outputs are all on a consistent basis. There are other ways to implement double deflation in volume GDP estimation but the “H-Approach” provides the most rigorous framework for reconciling the components of GDP at a granular level and is considered international best practice.

The other main benefits of double deflation are:

• greater coherence in the national accounts, as the growth rates of volume GDP will be equal when measured from both the production and expenditure approaches

• reconciliation across the SUTs both in current prices and in volume terms to generate better quality estimates of volume GVA at the industry level

• better measures of total factor productivity growth (and labour productivity growth) at the industry level

Blue Book 2019 was the first major step in the journey outlined in October 2018, putting in place the foundations of the new framework for estimating GDP and delivering a number of benefits including:

• increased data confrontation in the balancing of current price and volume estimates through the SUTs framework; this approach produces more coherent estimates of GDP as a more holistic view is taken of all the information available in the production of these estimates

• the expenditure approach has traditionally been the determinant of annual benchmarked volume GDP estimates but, while consumption deflators are thought to be of high quality, the same is not true of the deflators of other components of final expenditure. It is possible that these can be enhanced by making more use of PPIs and research has therefore been undertaken to identify the best available deflator at a product level for each transaction in the UK National Accounts

• the compilation of the institutional sector accounts (goods and services account and the production account) are now compiled alongside the supply and use framework, allowing this information to feed into the balancing process, providing a much broader dataset to support the quality assurance process and thereby improving coherence within the UK National Accounts

• the provision of an integrated framework for identifying the main areas for further development, which can be targeted as we move through our transformation journey

Furthermore, in recent years, improvements in the UK National Accounts have been introduced in a layered manner, in which these have essentially been added on top of previous estimates of GDP. As such, it has not been possible to review the way in which balancing has been carried out in previous years and whether this could be improved. The introduction of a new framework has allowed for a full balancing process to be undertaken, which is based on the latest estimates available, in which unbalanced estimates from each compilation area of the UK National Accounts were confronted in the supply and use tables (SUTs) for 1997 to 2016.

In addition to the new framework, we also introduced important data sources in Blue Book 2019 including the new Annual Survey of Goods and Services (ASGS) and the Annual Purchases Survey (APS).

The ASGS estimates the goods and services produced by each industry, providing a more complete picture of output by product – particularly for the service industries. The APS provides a much richer understanding of the structure of intermediate consumption by industry, thereby providing more accurate estimates of current price gross value added (GVA) for each industry. These new survey estimates are integral to the production of double deflated estimates of GVA, which require high-quality information on the output and intermediate inputs into the production processes for each industry.

Despite these significant improvements to the UK National Accounts, we did not implement double deflated industry GVA in the UK National Accounts. The next section outlines some of the challenges faced, and the next steps for our research agenda aimed at delivery of our ambitious plans.

The decision not to publish double deflated GVA estimates

In the run-up to Blue Book 2019, we had noted the potential for significant revisions to the industry composition of GVA and to headline GDP. When introducing changes to the national accounts, which are likely to lead to revision, we take account of two simple principles, the extent to which the:

• changes reflect a conceptual and quality improvement to data sources and methods

• resulting outputs give a coherent economic story within the national accounts framework when compared with a wider range of economic indicators

Figure 1 shows volume GDP estimates as published in the 2018 and 2019 Blue Books. While the 2019 vintage of the data shows GDP at a higher level (in large part reflecting improvements to the measurement of consumption of fixed capital, introduced in Blue Book 2019) it follows a broadly similarly growth profile to the 2018 vintage.

An additional line shows the level of GDP at an intermediate stage of the Blue Book 2019 production process, based on a theoretical application of our planned methods and data source improvements. This line shows a slower rate of growth in the run-up to the 2008 to 2009 financial crisis, with comparatively smaller revisions to overall GDP during and after the financial crisis.

It should also be noted that both the Blue Book 2019 final published data and the Blue Book 2019 theoretical application of the “H-Approach” are based on very similar current price datasets, meaning that the significant difference is a result of the application of the new framework for deflation. When moving from the previously used single extrapolated approach of estimating volume GVA growth, and in turn, GDP growth to the application of the “H-Approach” and double deflation, changes to the estimates are expected, in particular to the GVA estimates in volume terms by industry. This reflects international experience as well as previous UK initiatives producing SUTs both in current prices and in volume terms. However, because of the economic significance of a change of this magnitude, it was decided that more time, resource and quality assurance was needed to understand whether these initial estimates were economically coherent and the sources of the divergence identified and explained.

Given the significant change made to our methods, systems and data as part of Blue Book 2019, initial investigations focused on validating that the methods and new systems had been properly applied and implemented. This was a process which proved more resource and time consuming than anticipated. Once this had been completed, attention moved to verifying the new dataset and understand the drivers of any revisions.

Given the complexity and vast volumes (approximately 63 million data points) of data produced in the “H-Approach”, we undertook detailed end to end investigations across a number of industries. These almost uniformly revealed two primary causes for revisions, which will be explored in more detail in this article:

• the difference was driven by the transition items within the new framework to convert from “basic” and “producer” prices not adequately reflecting the different source datasets (an explanation of the transition between basic and producer prices and its importance in the “H-Approach” is provided in Annex A); this could also be symptomatic of a challenge with the basic or producer price deflators themselves

• The “H-Approach” enables the confrontation of data across dimensions not previously considered in the compilation of UK GDP (for example, simultaneously confronting current price and volume estimate) and this new lens provides insight that could lead to different balancing decisions at a detailed level; as these data are then used to weight the different deflators used in the deflation process, the underlying weighting structure could change; however, whilst the impact of different balancing decisions is likely to have only a modest impact on headline GDP, it may more significantly affect the industry and product distribution

In summary, our conclusions were that the revisions in the early years of the time series did not necessarily represent an improvement as they moved headline GDP away from previous results that were derived by reliance on the Consumer Prices Index (CPI) deflators. This was of particular concern because when considering the trinity of PPIs, CPIs and the transition items between the two, the CPI is considered higher quality.

To overcome this, we developed an approach to apply balancing adjustments to the items in the supply and use framework that drive the transition between basic and purchasers’ prices, given these are a lower quality component in the ‘trinity’ outlined above. This involved making balancing adjustments to approximately 20 per cent of cells in the supply and use matrix to put more weight on outcomes that would have been derived from a more direct use of consumer prices, in short reverting back to outcomes in line with our previous approach. In order to retain a supply and use balance in volume terms and reconcile GDP, counteracting adjustments were made to intermediate consumption – meaning that we had to delay implementation of double deflation in our industry estimates of GVA until we could more fully understand the impacts of this treatment.

Differences in deflators

One of the fundamental principles of the “H-Approach” is that deflation initially takes place at basic prices using a common deflator across both the supply and use of related products. This feature means for the first time we are considering the weight to place on supply side deflators such as the ONS Producer Price Indices (PPIs) and Services Producer Price Indices (SPPIs), in our long-term estimates of GDP compared with estimates deflated at purchasers’ prices using, for example, the CPI.

Traditionally, as the expenditure approach set the annual level of volume GDP, the compilation of volume GDP has, therefore, made greater use of deflators from the expenditure approach. The largest share of the expenditure approach is driven by households, where we have predominantly used the CPI deflator set. The logic of using the expenditure approach to set the level is partly driven by the fact that, statistically, the CPI holds several advantages over the PPI. The sample size is far larger – around 180,000 price quotes a month compared with approximately 7,500. We also have a greater control over the specific items priced in the CPI basket compared with the PPI, meaning we can better ensure that we are comparing like-for-like products month-on-month.

The CPI basket of goods and expenditure weights have also been updated annually since their inception, so that their composition reflects household consumption at that point in time. Until the introduction of annual chain-linking, the PPI composition and expenditure weights are only updated every five years, with significant historical changes. That said, it should be noted we have plans to introduce chain-linking of these series as part of the package of improvements in Blue Book 2020.

In the “H-Approach” these two deflator sets are confronted. If it is assumed that the transition from purchasers’ prices to basic prices is correctly accounted for and measured, the deflator choice should make very little difference. In other words, if the various trade and transport margins and taxes and subsidies on products applied to basic prices to move to purchasers’ prices are properly understood and estimated, it should not matter which you start with. If, after doing this, one cannot observe consistency between the basic prices and purchaser’s prices, this implies either a measurement issue or a problem in the method used to convert between price bases. Detailed analysis identified that in many industries there was a divergence between CPIs and PPIs that was particularly pronounced before 2010.

While this can be illustrated within a number of products, Figure 2 shows that this divergence is visible even at an aggregated product level, when considering the CPI for goods and PPI for manufacturing products (while coverage differences exist between these two aggregates, they are broadly comparable). Whilst we always expected to need to make balancing adjustments to provide a reconciliation, we did not expect these to be on the scale required in the pre-2010 period.

The reason for this divergence is currently being explored, but there are several economic, measurement and practical considerations that could lead to a change in the relationship between the CPIs and PPIs. These include:

• a genuine change in the relationships between basic prices and producer prices around the time of the financial crisis, which is plausible, for example, trade margins are likely to be more volatile

• the move to the 2007 version of the Standard Industrial Classification (SIC) and the 2008 version of the Classification of Products by Activity (CPA), which determine the product and industry definitions used in the SUTs

• the transition of the Annual Business Inquiry (ABI) to the Annual Business Survey (ABS), which is the backbone of data collection in the national accounts and one of the sources underpinning the calculation of weights in the PPI

• modernisation and re-engineering of national accounts covering the systems, methods and processes at the ONS, leading to the loss of some processes that account for some of the divergence in the national accounts

A further consideration identified in our analysis is the level at which deflation takes place in the SUTs. A feature of the “H-Approach” is consistent deflation across all products regardless of final use. However, this is a concern if there is not a detailed level of product information in the SUTs framework. For example, in product 29 "manufacture of motor vehicles, trailers and semi-trailers" the grouping represents at least three heterogenous products, with different prices, weights and primary users. The deflation of the supply of these products at this level would match the deflation of the use of these products. Clearly, households do not consume significant quantities of trailers and semi-trailers compared with the amounts used for capital formation or export, thus the weighting of the deflator applied to the household component would be inappropriate. However, a more detailed SUTs framework would allow for more specific price deflators to be applied to the specific type of product category and by type of purchase. Over time we will look to increase the level of product detail used to compile the National Accounts to better account for these types of differences but what can be published is subject to the availability of data to underpin such increased detail.

In order to overcome these challenges, we have initiated the following workstreams:

• Workstream 1: Review of deflators before the financial crisis to ascertain the cause of the divergence between CPI and PPIs pre-2010 with a view to explain, or if appropriate, resolve the divergence

• Workstream 2: Given the increased reliance on a wider range of deflators in the “H-Approach”, begin a long-term development programme to increase the quality of the deflators used in the national accounts, including improved capture of quality change outlined in the Independent review of UK economic statistics

• Workstream 3: Review the optimal product and industry level of detail to produce the UK National Accounts and the minimum additional products required to produce better quality double deflated estimates of industry GVA

• Workstream 4: Review the methods and data sources used to populate the different price bases

Challenges in reconciling the different approaches to measuring GDP, at a detailed level

Our analysis of the initial Blue Book 2019 results implied that there were some cases where the weights put on different deflators based on the current price GDP balance produced incoherent outcomes. Tracing this back through the production process identified that this may be as a result of balancing decisions.

In general, we tend to put more weight on the annual estimates of the income and production approaches in current prices; they tend to have higher quality data with the production approach dominated by the Annual Business Survey (ABS) and the income approach by comprehensive administrative sources from Her Majesty’s Revenue and Customs (HMRC). In more recent years a focus has been put on income data given the lack of detailed purchases data since 2004 until the re-introduction of the Purchases Survey in 2015, and very strong growth rates indicated by the ABS, which were incongruous with other sources.

These challenges are illustrated in a case study for industry 74 – other professional, scientific and technical activities which includes specialised design, photographic and translation services. Figure 3 shows increased divergence between income and production in recent years with balancing decisions keeping the level close to that of income.

In years where the balanced level is outside the bounds of the two unbalanced series, this reflects two primary factors:

• use of information from other parts of the national accounts or external sources

• decisions taken because of incomplete data coverage or to account for data quality, the latter do not reflect pure balancing decisions but address known issues with the underlying data

In order to overcome these challenges, we have initiated the following workstreams:

• Workstream 5: Produce and publish a metadata catalogue for all sources used in the compilation of GDP to drive a review of historical balancing decisions and establish published balancing principles based on the latest data landscape

• Workstream 6: Link and confront the micro-dataset at a firm level to understand differences between the production and income approaches of GVA, making recommendations for quality adjustments and changes to data collection where appropriate

Chain-linking

In addition to the challenges outlined above, as an early user of the "H" Method globally, we have encountered several methodological challenges in implementing the SUTs in current prices and in volume terms as produced using the “H-Approach". Many of these were resolved as part of our development in Blue Book 2019 but the chain-linking of double deflated industry estimates of GVA requires further work.

In the UK National Accounts, volume estimates are calculated using a method known as chain-linking. They are produced as the result of joining together two indices that overlap in one period by rescaling one of them to make its value equal to that of the other in the same period. In 2016, we published an article outlining our approach to chain-linking.

A property of chain-linking is that the volume estimates are not additive. As such, the combining of component estimates, through addition or subtraction, can only be done on estimates expressed in the prices of the previous year. The resultant chained volume estimates are then calculated by linking two or more series together using chain-linking.

However, this process can cause incoherent results when series are negative or particularly volatile. While we did not introduce any changes to this chain-linking method, incoherent results are more likely when operating double deflation, where the prices of the inputs and outputs could move differently and exaggerate any differences in the underlying current price data. Outside of double deflation, there are cases where there is a precedent in other parts of the UK National Accounts for treating series as if they are additive to overcome some of these oddities.

Figure 4 shows an example for industry 19 (coke and refined petroleum products) and illustrates the differences that can result from using the current ONS approach to chain-linking, and by taking the output CVM minus the intermediate consumption CVM.

In order to overcome this challenge, we have initiated the following workstream:

• Workstream 7: Work with the international community and domestic users to outline the best approach to chain-linking double deflated estimates of volume GVA, that balances conceptual considerations and user needs

Operational challenges

The introduction of the “H-Approach” has involved a vast increase in the number of observations that are being used to reconcile and balance GDP in volume terms. This has been accommodated in some part by the increased use of automated balancing software that optimised the supply and use framework in line with pre-defined constraints based on evaluation of the quality of the data sources used in GDP.

However, the scale of the challenge is vast. In order to overcome this challenge, we have initiated the following workstream:

• Workstream 8: We will make operational changes to better handle the increases in the volume of data that we are now processing; this work package will consider increased use of automation and / or operating the “H-Approach” more flexibly in a production context

From the challenges outlined here we have developed a series of workstreams that in time will enable the production of quality estimates of double deflated industry gross value added (GVA). Table 1 outlines our plans for communicating the results of this research as it emerges. The dates may well change depending on the results of our research, but this reflects the current thinking.

In addition to the research, we plan to publish experimental estimates of double deflated GVA alongside Blue Book 2020, reflecting initial progress on the research areas identified here. As these are developed further, we aim to incorporate double deflated estimates of volume GVA into the UK National Accounts in Blue Book 2021. Given this ambitious research agenda, it is likely that research will continue beyond Blue Book 2021, with improvements to our double deflated industry GVA estimates and gross domestic product (GDP) more widely happening as part of our continuous development in future Blue Books.

The scope of the experimental estimates and the proposed implementation will be conditional on the findings of our research programme and wider consultations, which will provide further insights into some of the feasibility considerations, particularly to historical estimates. Once fully implemented, its impact is expected to lead to improved quality estimates of volume GDP and productivity analysis.

Rob Kent-Smith and Rachel Meyrick, Office for National Statistics.

The “H-Approach” refers to the full framework set out in the UN Handbook on Supply, Use and Input-Output Tables with Extensions and Applications (2018). The application in this article only refers to the top part of the full “H-Approach”.

A high-level conceptual schematic of the approach, identifying five main stages relating to the SUTs in current prices and in volume terms (preferably, in previous years’ prices) can be found in Section 5 of the Transformation of gross domestic product in Blue Book 2019 article published on 11 October 2018. While these stages are presented in order, they are in practice iterative, with several iterations likely before gross domestic product (GDP) is balanced – for the first time this will include an element of automated rules-based balancing. The balancing process can be completed covering all stages simultaneously or sequentially.

Stage 1

This shows the SUTs currently used to produce annual GDP in current prices. Apart from adding the institutional sectors to the balancing framework to allow earlier and more coherent data production for institutional sector accounts, the method was unchanged in the Blue Book 2019 approach. In the supply table, domestic supply includes domestic output by product and imports (represented by “I”). The use table includes intermediate consumption, final consumption expenditure, gross capital formation and exports.

Domestic supply is measured in basic prices and the components of the Use Table are measured in purchasers’ prices, except GVA, which is at basic prices (Table 2). To ensure that the tables are presented on the same conceptual price basis, trade and transport margins and taxes (less subsidies) on products (also known as the valuation matrices) are included in the supply table, represented by the “I”, “M” and “T”.

The SUTs at purchasers’ prices can be balanced independently but the balance may be revisited at a later stage in the process, reflecting the iterative nature of the “H-Approach”. The imports of goods and services matrices and the valuation matrices are balanced, and so the move between the basic price balance and the purchasers’ price balance will always remain balanced. The number of iterations will depend on several issues, including whether the tables are balanced sequentially or simultaneously.

Stage 2

One of the main features of the “H-Approach” is to allow consistent deflation across different types of national accounts transactions where they are related to the same product, following the principle that the buyer and seller pay the same price in a transaction at basic prices. The implication of this is that the core elements of both the SUTs need to be on the same price basis. As the supply and use tables are measured in basic prices and purchasers' prices respectively (see Table 2), adjustments need to be applied. To do this we remove the imports and re-allocate the trade and transport margins and taxes (less subsidies) on products from within the use table, allowing the entries at purchasers’ prices in the Use Table to be moved to basic prices (or producers’ prices, if appropriate). It is important to note, if the identities (for example, product supply equals product use) are in balance in the SUTs at purchasers’ prices, then they will remain balanced at basic prices.

To ensure that we fully use available data, these reallocations are done at the product level separately for Value Added Tax (VAT), other taxes, subsidies, trade and transport margins, imports of goods (EU), imports of goods (non-EU) and imports of services. If the current price SUTs at purchasers’ prices are balanced, then the SUTs at basic prices will also be balanced as the imports use matrix and the valuation use matrices are self-balanced – another important feature of the “H-Approach”.

Stage 3

Now that both tables are on the same price basis, they can be deflated to previous years’ prices (PYPs) using a common set of deflators for each product, the approach to this can be seen in section 5 of the Transformation of gross domestic product in Blue Book 2019 article published on 11 October 2018.

In practice, each "row" represents 112 rows within that transaction, with each representing a product category. A deflator is available for each of these rows, with the underlying principle that the buyer and seller face the same price for that transaction. This allows for the consistent and coherent deflation in the SUTs framework.

Therefore, PYPs are used within the SUTs framework and chain-linked once the final balance is achieved. Time series in PYPs are derived from multiplying the volume series in the current period, by the average price of the previous year. PYPs are additive and are the method by which volume series can be aggregated together.

It is worth noting that the headline measure of volume GDP is a chained volume measure (CVM) – the effects of inflation are removed by considering changes in quantity between consecutive periods and holding the prices in the previous period constant. However, the components of CVMs are not additive, meaning that they cannot be meaningfully balanced in a SUTs framework.

In addition to imports, margins and taxes, there are some elements of output at basic prices that, when separately identified map directly across to the use table, benefit from deflation using their own set of deflators. These are identified in Section 5 of the Transformation of gross domestic product in Blue Book 2019 article published on 11 October 2018. These are generally done to maximise the data used in the process, for example, non-market output produced by government is identified and deflated separately to allow the use of the rich range of direct volume indicators used in the UK. Once deflated, this results in SUTs at PYPs expressed at basic prices. As previously detailed, if the current price SUTs at purchasers’ prices are balanced, so are the tables at basic prices in previous years’ prices.

Stage 4

To move to SUTs in PYPs at purchasers’ prices, imports of goods and services added and trade and transport margins and taxes (less subsidies) are reallocated within the use table – reversing the steps taken between Stage 1 and Stage 2. As previously detailed, if the current price SUTs at purchasers' prices are balanced, by definition, so are the SUTs at purchasers’ prices in previous years’ prices.

Stage 5

These results are then quality assured at detailed granular level, including against a range of external indicators and estimates deflated using transaction-specific deflators. If adjustments are to be made there are a number of options available:

• revisit the current price SUTs balance at purchasers’ prices

• make adjustments to the items that form the transition between purchasers’ prices and basic prices

• exceptionally, change the deflators used for a specific product

This process would be done iteratively to achieve the final balance.

It is important to stress here, that the “H-Approach” framework makes the best use of the deflators from both the supply and use side, rather than the univariate approach used currently. Hence, the expenditure deflators presently used will not simply be discarded in the “H-Approach”, but will be used where appropriate, based on quality assessments of the deflators.

As a final step, the PYP estimates can then be chain-linked into a measure of volume GVA for the total economy and by industry.