Output in the Construction Industry: A comparison of construction output and Markit CIPS data

The Office for National Statistics’ (ONS’s) estimates of output in the construction industry and the Markit construction purchasing managers index (PMI) have painted a different picture of the underlying performance of the construction industry; raising questions about the quality and accuracy of both sources.

This interim report looks at some of the possible reasons for the divergence between the 2 surveys and recommends further work in the area.

There are clear conceptual differences between the underlying methodologies of the 2 measures that make a direct comparison very difficult but, post 2013, both show an underlying upwards trend for output in the construction industry.

It is not surprising, given the conceptual differences explained in this article, that the 2 measures sometimes give different signals. Both measures offer the user something different; the Markit PMI is the more timely estimate, but our slower release of data enables us to provide a more comprehensive coverage of the sector.

In order to attempt a direct comparison of the 2 measures, we have created a set of balance statistics (otherwise known as diffusion indexes) using the Monthly Business Survey (MBS) returns for the construction industry.

A close match between diffusion indices was only found when manipulating the MBS data to remove volatility and seasonality, considering 3 month on year growths, in the adjusted MBS diffusion index.

The monthly Output in the Construction Industry release provides users with estimates on the performance of the construction industry. These estimates can, at times, appear to differ with the Markit/Chartered Institute of Purchasing and Supply (CIPS) Purchasing Managers Index (PMI).

Previous articles, such as Chamberlin (2008) and Alhassan, Ayoubkhani and Hardie (2012) have monitored the statistical coherence between ONS and Markit/CIPS series. In these studies, a form of standardisation was used in order to compare the series. This article sets out the differences in the underlying methodology of the 2 measures and then presents work we have completed to replicate the Markit/CIPS methodology using MBS data, making it possible to determine whether there is a relationship between the 2 once the methodological differences have been removed.

The methods used to compile the output in the construction industry and Markit’s construction PMI are conceptually different and are outlined in Table 1.

There are some clear conceptual differences between the 2 methodologies as Table 1 highlights. These include:

• the variable measured by Markit PMI is a directional variable that relies on the purchasing managers reporting business conditions for the current period based on the previous period, whereas the MBS variable we collect is a quantitative variable that measures the value of work

• the Markit PMI variable is a comparison of the current period with the preceding period, whereas our data measure the current period only and growth rates are calculated from the resulting time series

These conceptual differences can lead to the 2 surveys publishing different pictures of the performance of the construction industry as section 4 describes.

The first stage of our comparison is to examine the headline figures published by Markit and ONS, which are shown in Figures 1 and 2. It is worth noting that these series are both seasonally adjusted and that the ONS series uses chained volume measures, which therefore take account of price changes within the construction industry.

Both series show that in 2013, the underlying trend in the construction industry was one of growth. It is worth noting that since April 2013, the PMI, as shown in Figure 1, has consistently been above its neutral threshold of 50.0, suggesting that there has been continuous growth within the industry since this point. Output in the construction industry also shows a similar underlying trend from 2013 onwards.

The movement within the PMI time series, to a casual observer, could suggest that there have been dips in performance which would be a misinterpretation of the data. Using the months of March and April 2015 as an example, in March, the PMI stood at 57.8 and in April at 54.2, this indicates that the construction industry was still growing in April but at a slower rate.

In contrast, the movement in the output in the construction industry series can, however, be directly interpreted as a rise or fall within the industry and show that there is a level of volatility surrounding the industry’s performance. Therefore, the 2 series will contradict each other, as long as Markit CIPS is above 50 then the construction industry is expanding, whereas a fall in output in the construction industry can be interpreted as the industry contracting. However, ONS would advise that one month’s movement does not imply a contraction and that a longer time span of data should be used to make this judgement.

Some of the differences described above are due to the use of a diffusion index by Markit/CIPS. This would be the case regardless of the sample size differences, the timeliness differences and the removal of seasonality. We can show this using a worked example. By the very nature of the sample designs of the 2 surveys there will be some businesses who are responding to both surveys. Even in these cases it is possible that the conceptual differences described in section 2 can lead to the resulting indices portraying different pictures of the performance of the construction industry. Tables 2 and 3 provide an illustrative example how this might occur using hypothetical data provided by 4 businesses which are common to both surveys. This example uses fictitious data, and does not incorporate any of the weighting processes.

For output in the construction industry we sum the value of work for a given month for each business to provide the total value of work in the construction industry. This is then compared with the total value for the previous month to provide a month on-month growth rate. Thus, in this example the value of work has decreased by 1.0% compared with the previous month.

When the same businesses are reporting to Markit/CIPS, they only report the direction of output compared with the previous month. Table 3 summarises how these data would therefore be considered, with 2 businesses reporting an increase, 1 reporting a decrease and the other remaining the same.

The Markit/CIPS index is calculated by the following formula:

I=(1× P_U )+ (0.5× P_S )+ (0 × P_D)

where P represents the percentage of businesses, and U,*S* and D represent the directions of ‘up’, ‘same’ and ‘down’.

Thus for our example, the index is calculated as follows:

I=(1× 50)+ (0.5× 25)+ (0 × 25)=50+12.5+0=62.5

Using the formula an index value of 50 indicates that there has been no change in the performance of the industry that month. In this hypothetical example we obtain a value of 62.5 which indicates quite strong positive growth for the month.

So using just the 4 common returns we have 2 very different conclusions regarding the performance of the construction industry during October 2015. The primary cause in this case is that Markit/CIPS respondents do not provide an order of magnitude for their output movement. So while 2 of the 4 businesses are increasing, once the relative size of the movements is taken into account the smaller gains in businesses C and D do not negate company A’s fall, resulting in an overall decrease for the industry.

That example focuses on the month-on-month movement but, when comparing the two sources, it is more appropriate to analyse the underlying trend of the data. In order to try and negate some of the theoretical and conceptual differences we have created a diffusion index of the MBS data. This should allow for a more direct comparison of the 2 construction industry indicators.

The data source for the diffusion index is the MBS. Individual returns have been assessed for the change between one period and another and the PMI methodology was replicated to calculate the percentage of businesses who were higher, lower or exactly the same and to create a resulting diffusion index value. The data are in nominal or current price terms and thus the effects of any price changes are not removed. The resulting diffusion index is shown in Figure 3.

Both diffusion indices are non-seasonally adjusted, however, the MBS diffusion index expresses more volatility and seasonality than the Markit PMI. It is clear that there is a very strong seasonal pattern in the ONS series, with December standing out as the month in which a greater quantity of companies report a lower value of work than in either November or January.

We can address this by seasonally adjusting the series, as shown in Figure 4.

In Figure 3, it can be seen that the Markit series remained above 57 from July 2013 to October 2014, which does not follow its usual seasonal pattern, resulting in the sustained period of growth in Figure 4.

Despite following the Markit methodology, there are clear differences between the 2 series and the divergence between the two is more apparent after seasonal adjustment. The MBS series after seasonal adjustment is relatively flat and no longer shows a sustained period of growth.

Further, the ONS output series is often seen to be volatile on a monthly basis, with the quarterly series showing a greater indication of trend. To determine if a better fit could be found, rather than seasonally adjust we can create an alternative adjusted MBS diffusion index which, rather than seasonally adjusting at an index level, addresses seasonality and volatility at a company level, by using the 3 month on year growth rates rather than the month on month growth rates. Figure 5 shows the comparison between this adjusted MBS index and the seasonally adjusted PMI index.

After controlling for volatility and seasonality within our MBS dataset, the diffusion indices now show a similar trend. This leads us to question whether PMI responders may consider the wider picture of their company when judging their performance level; looking at more than just the 2 most recent months, are they accounting for the seasonal factor themselves – that is, if December is always lower than November in terms of output, but this December is better than last December, then maybe they would respond that performance levels are higher.

For the Monthly Business Survey (MBS) diffusion indices created above, a business was only deemed to have remained the same where their reported value of work was exactly the same as in the previous period. Thus a small increase of just £1,000 would be classed as an improvement. However, a purchasing manager of a large construction firm, when responding to Markit, may overlook or average out such a small gain and report that their output has remained the same.

It is possible to incorporate a boundary into the MBS diffusion index, such that a level of change below a certain margin would be considered as remaining the same. Figure 6 takes the adjusted diffusion index, and shows how it would change when using margins of 5%, 10% and 20%, but there are no significant differences between the series.

The main conclusions from this report are:

• The Office for National Statistics’ (ONS’s) estimates of output in the construction industry and the Markit construction purchasing managers index (PMI) have painted a different picture of the underlying performance of the construction industry; raising questions about the quality and accuracy of both sources.

• This interim report looks at some of the possible reasons for the divergence between the 2 surveys and recommends further work in the area.

• There are clear conceptual differences between the underlying methodologies of the 2 measures that make a direct comparison very difficult but, post 2013, both show an underlying upwards trend for output in the construction industry.

• It is not surprising, given the conceptual differences explained in this article, that the 2 measures sometimes give different signals. Both measures offer the user something different; the Markit PMI is the more timely estimate, but our slower release of data enables us to provide a more comprehensive coverage of the industry.

• In order to attempt a direct comparison of the 2 measures, we have created a set of balance statistics (otherwise known as diffusion indexes) using the Monthly Business Survey (MBS) returns for the construction industry.

• A close match between diffusion indices was only found when manipulating the MBS data to remove volatility and seasonality, considering 3 month on year growths, in the adjusted MBS diffusion index.

There are a number of different questions that arise from this work which require further investigation so our next steps will be to:

• look to determine why the ONS headline and Markit CIPS month on month measures diverge, particularly from 2013 onwards

• analyse the effects of small, medium and large companies on both measures

• take a wider look at other sentiment surveys on the construction industry, for example the State of Trade survey published by the Federation of Master Builders (FMB)

• explore how VAT data on the construction industry compares with MBS data

We will provide an update on this work in July 2016.