Comparison of labour market data sources

This page is designed to be a working reference document. We intend to update this page as we develop a better understanding of our sources and the differences between them.

The update published on 27 April 2022 includes:

• addition of Validation and quality assurance of labour market sources (Section 11)

• addition of Annex 8: Validation and quality assurance documentation (Section 20)

• update to other annexes (Sections 13 to 19)

The update published on 23 February 2022 includes:

• Challenges of Measuring the Labour Market

• Jobs

• Earnings

• Vacancies

• Redundancies

The Employment, Unemployment, Public Sector Employment and Coronavirus Job Retention Scheme sections will be updated after the Labour Force Survey reweighting exercise in June 2022. These were last updated on 11 December 2020.

For a full list of labour market datasets see A guide to labour market data.

Labour market statistics measure many different aspects of work and jobs, and provide an insight into the economy.

A framework for labour market statistics is used to describe the major concepts that exist within the labour market and their relationship to each other. The framework is based on the concepts of labour supply and demand, which together provide a comprehensive picture from the perspective of both households and businesses. This approach is long-established and has wide international acceptance, including by the International Labour Organization (ILO).

Labour supply consists of people who are employed, as well as those defined as unemployed or economically inactive. It is measured using surveys of households. Labour demand is represented by employers who have a need for work to be done and offer compensation (earnings) to the employees who do it. Labour demand is mainly measured using surveys of businesses. There is also some administrative data used as part of labour market statistics, and information about earnings is collected from both households and businesses.

As part of wider transformation plans, we have been seeking to identify and introduce new data sources, such as those from government departments and the private sector. This is to give a quicker and/or more complete picture of our economy.

The coronavirus (COVID-19) pandemic and the government response had a significant effect on the UK labour market. Many businesses ceased operating or had to change their working practices because of the coronavirus pandemic, while government interventions allowed for workers to be paid while away from work. The introduction of social distancing also changed the way individuals work, and their ability to look for and find employment.

The combined effect of the impacts has not only changed the UK labour market, but also our ability to measure it. For more information, see our Coronavirus and the effects on UK labour market statistics article. Most of our information relies on data collected from surveys of households and businesses. The temporary closure of businesses, increase in remote working, and introduction of social distancing continue to have an impact on collecting these survey data.

Providing necessary information through multiple data sources, changing how our surveys operate, and reviewing methodology can cause challenges when giving a comprehensive picture of the labour market.

This page compares the strengths and weaknesses of the main data sources used to produce the labour market figures. This includes the advantages of new administrative data sources and limitations of some of our published figures. We intend to update this page as we develop a better understanding of the new sources.

The concept of employment, which is about people and whether or not they have work, differs from the concept of jobs, since a person can have more than one job and some jobs may be shared by more than one person.

The Labour Force Survey employment versus HMRC’s Pay as You Earn Real Time Information

There are two main sources of information about employment:

• Labour Force Survey (LFS) employment

• Experimental Statistics from Real Time Information (RTI) payrolled employees (Experimental Statistics)

The LFS and HM Revenue and Customs’ (HMRC’s) Pay As You Earn (PAYE) RTI measure different things. The LFS follows the International Labour Organization (ILO) definition of employment, which consists of people aged 16 years and over who did one hour or more of paid work in the previous week and those who had a job they were temporarily away from (for example, because they were on holiday, off sick or, now, on furlough).

RTI measures the number of people who are being paid through the PAYE system via company payrolls. It could include self-employed people if someone has both an employee job and is self-employed (regardless of which is their main job) and does not include those whose jobs are not part of a company payroll or those whose company does not pay them through a PAYE scheme.

The RTI data used for publication come from a dataset processed for use in statistics, rather than the raw operational data, a separate article details the methodologies used.

Main differences

There are a number of differences between these sources, and more detail of these differences can be found in Annex 1.

Coverage is different since RTI only includes individuals who are being paid through the PAYE system via company payroll, whereas the LFS employment includes all forms of employment. LFS employment is estimated by a sample from UK residential addresses that excludes people living in some communal establishments (NHS accommodation is included, while residential care homes, prisons and defence establishments are excluded). The LFS will cover those earning below the lower earnings limit as well as the self-employed, those on government training schemes (some of which will be included in RTI), unpaid family workers and those who were employed, but not paid, during a particular reference period.

The LFS is collected via an interviewer, giving opportunity to clarify answers and grouping respondents into the ILO definition of employment. Responses are therefore subjective based on whether a responder feels attached to the labour market, for example, some of those temporarily away from work that they expect to return to will consider themselves as employed, while others might not. Additionally, a respondent self-defines their employment status between employee or self-employed; the definition of self-employment is those who in their main employment work on their own account, whether or not they have employees. Some of those who are actually paid through payroll might classify themselves as self-employed as they own the business. RTI data are collected through employers’ legal obligation to provide PAYE information to HMRC; they are based on individuals receiving pay through payroll and do not fully align with international definitions of employment. RTI classifies any person with an employee job as being an employed person, whereas LFS only classifies a person as being an employee if their main job is an employee job.

There are differences in reference periods: RTI is an average of the estimated number of employees in paid employment on each respective day of the month, while the LFS series is based on interviews regarding a respondent’s labour market situation within a reference week. The LFS estimates are then calculated based on interviews from reference weeks across a three-month period.

Since the LFS is a survey, bias from non-response and sampling variability is another explanation for some differences seen, although the Office for National Statistics (ONS) invests significant effort and resources into obtaining high response rates. The LFS is weighted to a version of already published population projections, rather than being used to determine the size of the population. Consequently, it can only ever apportion to an existing population projection, with estimates always summing back to that original population total. The LFS is not designed to measure short-term changes in the population. In October 2020, the LFS was reweighted to account for the impact of the coronavirus (COVID-19) pandemic on survey interviewing methods. There may also be further refinements to the methodology, that lead to further improvements and revisions in the future. We will continue investigating these options and implement them if they are of sufficiently significant benefit.

RTI payments are imputed to forecast tax submissions expected (but not yet received) for each employment. These are then adjusted using calculated grossing factors reflecting the expected probability those imputed payments will eventually be received. However, this process predominantly affects the more recent history and, consequently, affects the most recent periods much more than earlier months.

Comparison of latest data

The two sources are related and would often have similar properties and move in similar ways. However, because they measure different things, there are times when you would not expect them to follow similar paths. Here, we look at the differences between employment and employees from the LFS and payrolled employees from RTI.

Figure 1 shows a comparison of a three-month moving average of the headline RTI employees series and the headline LFS employment and employee series. As shown, the LFS employment series is consistently higher as this includes those self-employed, unpaid family workers and those on government training schemes. Between 2014 and 2019, the three series follow a similar trend. Since March 2020, while RTI has seen a sharper fall, LFS employment has also fallen. However, we are not seeing the same falls in the number of LFS employees.

Figure 2 shows the growth rates of employment and employees in LFS and RTI payrolled employees compared with the same period of the previous year. Over the past five years as a whole, the two series show a broadly similar rate of employment growth. However, larger differences are seen since April 2020.

The self-employed group goes some way to explaining the difference between the LFS employment and LFS employees. Since April to June 2020, we have seen record falls in self-employment, coinciding with large flows of people moving from self-employment into being an employee (as can be seen in Figure 3), although the number who had changed jobs had not increased from normal levels.

Consequently, some of the fall in self-employment comes from an increase in the number of people who have changed to classifying themselves as an employee, even though they have not changed jobs. As mentioned earlier, this categorisation is self-defined in the LFS and it is possible that recent coronavirus (COVID-19) government support schemes, such as the Coronavirus Job Retention Scheme (CJRS), have influenced respondents’ views on whether they are in fact self-employed or an employee. More work is required to understand the self-employed group, and those that are reclassifying themselves as employees.

Government legislation around off payroll working was also due to change in April 2020, bringing more alignment between different employment statuses. While implementation of this legislation has been delayed until 2021, it is possible that some firms may have already made preparations to be compliant by April 2020, which could also possibly have an impact on the numbers of employees.

Since February 2020, the number of RTI payroll employees has fallen by around 767,000; the larger falls were seen at the start of the coronavirus pandemic. In comparison, the number of people in employment according to the LFS has fallen by around 566,000 between December to February 2020 and July to September 2020 (over the same period LFS employees fell by 9,000). There were also around 210,000 people away from work because of the pandemic and receiving no pay in September 2020.

Those who are not receiving pay through payroll should not be included in the RTI figures; the RTI definition states that a person must not have missed more than one period of pay (that is, for someone paid weekly, not to miss more than one week’s pay, for someone paid monthly not to miss more than one month) to be included in the count of payrolled employees. However this group would remain within the LFS while they still reported having jobs, which goes some way to explaining the greater fall in the RTI figures.

Figure 4 shows the difference in LFS employees since February 2020 and those away from jobs and not being paid from the weekly LFS data, alongside the fall in RTI payrolled employees. The falls to LFS employees have been relatively small, in comparison with the fall in RTI payrolled employees, even when adding the number of people away from jobs and not being paid, this difference between the sources persists.

Similarly, Figure 5 shows the fall in LFS employment and those away from jobs and not being paid from the weekly LFS data, alongside the fall in RTI payrolled employees. While the weekly LFS employment data are volatile, they show a similar story since February 2020 to that of RTI.

Summary

We go some way to explaining some of the differences between LFS and RTI in this article but there is more work to be done. While a full reconciliation will not be possible, the ONS will work closely with HMRC to further understand the differences and quantify where possible.

Both the LFS and RTI provide a measure of people who are working, and each have their own uses. RTI provides a timelier estimate of payroll employees for both UK and regionally but only includes those who are paid through the payroll. Also, it does not have the detail that is available in LFS; however, HMRC are continuing to develop their outputs and may include more breakdowns in future.

LFS employment includes employees and the self-employed as well as people on government training schemes and people who work unpaid for family businesses; therefore, it gives a more comprehensive picture of the labour market. The LFS employment data are available by many different breakdowns including sex, age, disability, ethnicity, occupation and industry.

The LFS-based employment levels and rates are also available back to 1971 and can be used when considering changes over longer periods of time.

There are two measures that are commonly used to understand spare labour supply. Unemployment, as measured through the Labour Force Survey (LFS) and the Claimant Count, based on administrative data from benefits systems. Both important measures of spare labour capacity in the UK economy however they each record different aspects of the non-utilisation of labour.

LFS unemployment versus Claimant Count

The official estimates of unemployment for the UK are provided by the Office for National Statistics (ONS). These are produced from the LFS in line with the International Labour Organization (ILO) definition of unemployment. This states that a person is unemployed if they are either:

• without a job, want a job, have actively sought work in the last four weeks, and are able to start work within the next two weeks

• out of work, have found a job and are waiting to start it in the next two weeks

Respondents' status is determined by their responses to a number of questions about their labour market activity. Only those people whose answers indicate that they satisfy these conditions are included in the UK unemployment estimates, which are published monthly in the Labour market overview statistical bulletin. Since the LFS is a survey, bias from non-response and sampling variability is another explanation for some differences seen.

The Claimant Count seeks to measure the number of people claiming benefit principally for the reason of being unemployed.

The Claimant Count has generally been a count of the appropriate benefits within the UK's current benefit regime that best meet that criteria, and therefore sensitive to any changes in the benefit regime. Currently, this is a combination of claimants of Jobseeker's Allowance (JSA) and claimants of Universal Credit (UC) who fall within the UC "searching for work" conditionality.

Those claiming unemployment-related benefits (either JSA or in the UC searching for work conditionality group) may be:

• wholly unemployed and searching for work
• employed but with low income and/or low hours, that make them eligible for unemployment-related benefit support
• under certain circumstances, not required to seek work and therefore economically inactive

UC, which has been rolled out to replace the previous income-based element of JSA, along with a range of other income-based benefits, is designed to be a broader benefit, covering more people with underlying eligibility than those who claimed JSA. As a household benefit, the holistic situation of all members of the household are considered, and their obligations to look for work considered. Therefore, the roll-out itself has caused notable changes to the level of the Claimant Count, significantly closing the previously existing gap between the count and unemployment levels. Consequently, the measure is currently an Experimental Statistic, as movements are not wholly caused by changes in the labour market.

The Department for Work and Pensions (DWP) also publishes an alternative Claimant Count; this Official Statistic seeks to model a data time series of the number of people that would be claiming unemployment-related benefits if UC had been fully rolled out in 2013. As such the series should better reflect changes in the labour market, by eliminating the impact of the rollout.

As with unemployment, the gap between the Claimant Count and the Alternative Claimant Count has narrowed as the UC rollout has progressed. In theory, when the rollout and migration to UC has been completed, the Claimant Count and Alternative Claimant Count should merge to the same statistic. However, there currently are some definitional differences preventing this, because of information becoming available later than the timetable for producing the Claimant Count allows. Hence the Alternative Claimant Count is lagged by an additional month

Main differences

The main differences between the two definitions can be summarised as:

• people who are unemployed are not necessarily eligible for unemployment benefits or may choose not to claim even when they are eligible; further details can be found in Annex 2

• people who are eligible for unemployment-related benefits may not fit the definition of unemployment (for example, it is possible to be employed or inactive while claiming benefits included in the Claimant Count)

• Claimant Count statistics are based on a particular "count date" falling on the second Thursday of the month, whereas the LFS unemployment figures are based on interviews across the whole of a three-month period

• the headline Claimant Count statistics do not include 16- to 17-year-olds because of historical changes in coverage of the benefits involved; also, people above State Pension Age are far less likely to fall within this part of the benefit regime

• LFS unemployment statistics are based on all people meeting the unemployment definition aged 16 years and over (under certain circumstances, 16- to 17-year-olds may fall within the Claimant Count and are included in some breakdowns but not in the headline Claimant Count)

Comparison of latest data

The ONS publishes a regular comparison of the difference between LFS unemployment and the Claimant Count as part of the monthly Labour market overview statistical bulletin (see Table X05). This comparison allows for the differences in age and time coverage of the two series, and for the ineligibility of full-time students, but still reflects other definitional differences.

Figure 6 shows that over time, the two measures have both converged and diverged. It is noticeable that in 2009 and 2010, even though both measures increased, the gap did not narrow. The levels of LFS unemployment remained consistently higher than the Claimant Count from early 1997 to mid-2019, although they moved closer together over the course of the rollout of UC. During the coronavirus (COVID-19) pandemic, Claimant Count figures have seen record increases while LFS unemployment levels have increased less and far more gradually.

The claimant count statistics are an accurate count of the number of people claiming unemployment- related benefits (and remain so under COVID-19).

A number of enhancements and easements have been made to Universal Credit, which impact the statistics:

• temporary suspension of the claimant commitment and work search requirements
• for the self-employed, relaxation of the Minimum Income Floor requirements
• issuing of COVID isolation notes to replace fit notes

In addition, claimants are accessing UC as a “top-up” to government support packages (such as the Coronavirus Job Retention Scheme and Self-Employment Income Support Scheme) to legitimately claim unemployment benefits whilst “furloughed”. A proportion of unemployed claimants will be “employed” – furloughed, or with low earnings or hours of paid work.

During the pandemic, we have seen a significant increase in people identifying themselves as temporarily away from paid work that they expect to return to. There are also a number of people who have said that they are temporarily away from work for COVID-19-related reasons and are not receiving any pay. Given that these people are not getting paid, it is likely they would be eligible for benefits, but they have stayed within the LFS definition of employed. During April to June, there were over 470,000 of these people, who would have remained in the LFS definition of employment, many of whom would have started claiming benefits. In July to September there were still an estimated 215,000.

During the early months of the pandemic, there was an unprecedented shift from LFS unemployment to economic inactivity. Many of these started returning to unemployment in later months, having not met the ILO definition of searching for and being available for work during the national lockdowns. While this move into economic inactivity would have suppressed the increase in unemployment during this time, it is likely that they would have continued to claim benefits and stayed within the Claimant Count.

We have also looked in a little more detail at people on the fringes of unemployment for July to September 2020, those that under ILO definitions are not LFS unemployed but who may be included in the Claimant Count. These include the inactive group who may begin to seek work and those temporarily away from work for coronavirus-related reasons, without earnings. This group decreased from 2.18 million to 1.72 million between April to June 2020 and July to September 2020 (see Figure 7).

For July to September 2020, an estimated 1.62 million people were unemployed, up by 318,000 on the year. The annual increase in unemployment is driven by those unemployed for up to six months, up by 224,000 on the year to 1.04 million. However, the number of those unemployed for over 12 months has also increased by 30,000 on the year. To estimate duration of unemployment, LFS respondents are asked how long they have been looking for work. Respondents are unlikely to discount short periods where they were not looking from this. Consequently, those who briefly stopped looking for work in the earlier stages of the pandemic, and were therefore classified as economically inactive, are likely to return to unemployment duration estimates in longer-term categories. It is unlikely that these people who became inactive for a while would have flowed out of the Claimant Count.

Summary

LFS unemployment, Alternative Claimant Count and Claimant Count measure different things. However, unemployment and the measures of the Claimant Count will normally perform similarly under similar labour market conditions. This means that while they do not measure unemployment, the Claimant Count measures can be useful indicators for how unemployment moves over time, or compares between different geographic areas.

The LFS-based headline unemployment estimates are best used to describe national and regional levels and rates of unemployment at any one point in time and when considering changes over longer periods of time.

Because of not being subject to sampling variability, Alternative Claimant Count data are generally best used to better understand short-term changes in the labour market but aren’t published as frequently or timely as the Claimant Count. The Claimant Count is also not subject to sampling variability but can be used to help understand the relative unemployment position within and between local areas. However, changes in the Claimant Count measures seen at the start of the pandemic do not wholly reflect changes in unemployment, because some people turning to the benefit system as support would not necessarily have lost a job at that time.

As stated earlier, the number of people with jobs is not the same as the number of jobs.

This is because a person can have more than one job.

Estimates of jobs can be compiled from a number of sources, including short-term employment surveys (STES), the Quarterly Public Sector Employment Survey (QPSES), the Business Register and Employment Survey (BRES) and the Labour Force Survey (LFS).

STES is a group of surveys that collect employment information from private sector businesses that are used to compile workforce jobs (WFJ). In December of each year, WFJ benchmarks the private sector employee component to BRES private sector employee estimates. The private sector employee component of WFJ is not publicly available, therefore it is not possible to reconcile the WFJ and BRES totals that are published. WFJ is a quarterly measure of jobs in the UK and is the preferred measure of short-term employment change by industry. A variety of outputs are produced, including industry and region of workplace.

BRES is an annual survey and the official source of employee and employment estimates by detailed geography and industry.

WFJ compared with HMRC PAYE RTI

Main differences

Similar to differences between the LFS and Real Time Information (RTI), most differences between the datasets can be attributed to coverage and content.

WFJ can be broken down into employee and self-employed jobs; the latter is sourced from the LFS.

WFJ employees are the number of employees, aged 16 years and over, that an organisation directly pays from its payroll; collected through a survey with a specific count date. RTI counts anyone who has a paid employee job during the reference period; it is a monthly average of daily counts of the number of payrolled employees and not the number of jobs.

Some differences are seen partly because of imputation and weighting applied to the WFJ employees and sampling variability as a result of being collected from surveys. RTI payments are imputed to forecast tax submissions expected (but not yet received) for each employment. These are then adjusted using calculated grossing factors. However, this process predominantly affects the more recent history and, consequently, affects the most recent periods much more than earlier months.

Comparison of latest data

Figure 8 shows that the series generally follow a similar trend, with RTI showing stronger growth at the start of the series and then levelling off to be more in line with WFJ growth. The employee jobs (EJ) element of WFJ has tracked well with RTI throughout the coronavirus (COVID-19) pandemic period; from June 2020 to the most recent period, September 2021.

WFJ compared with LFS

The concept of employment (measured by the LFS as the number of people in work) differs from the concept of jobs, since a person can have more than one job and some jobs may be shared by more than one person. WFJ and LFS series are reconciled annually. The LFS, which collects information mainly from residents of private households, is the preferred source of statistics on employment.

The LFS can also be used to produce estimates of the total number of jobs in the UK by adding together the headline employment figures (which are equivalent to main jobs) and those for workers with a second job. However, the LFS only collects main and second jobs and will exclude those with more than two jobs. The WFJ series, which is compiled mainly from surveys of businesses, is the preferred source of statistics on jobs by industry since it provides a more reliable industry breakdown than the LFS.

Main differences

The 2006 National Statistics Quality Review of Employment and Jobs Statistics (PDF, 4.35MB) identified a number of reasons why the LFS and WFJ estimates of jobs can differ from each other. One reason is that the LFS excludes those living in communal establishments, whereas the WFJ does not. Another reason is that the LFS collects information on main and second job, but no information is collected on people who have three or more jobs but these would be counted within WFJ.

Some of these factors can be quantified approximately using information from the LFS and other sources, while others are more difficult to measure. The measurable factors causing differences between the LFS jobs and WFJ estimates are available in Dataset X03.

Comparison of latest data

Figure 9 shows headline WFJ data, LFS jobs data (main jobs plus second jobs) and two further series adjusted to take into account the measurable factors causing differences between LFS jobs and WFJ (as published in November 2021). Over the latest comparable three-month periods, the published LFS series shows a quarterly increase of 205,000 jobs (0.6%) and the published WFJ series shows an increase of 293,000 jobs (0.9%). Once accounting for the measurable factors, the adjusted LFS estimate of total UK jobs is lower than the adjusted WFJ estimate by 621,000 (1.8% of the LFS total).

BRES

BRES is an annual survey of around 85,000 businesses. It provides employee and employment estimates at detailed geographic and industrial levels. BRES is the primary source for employee estimates at a detailed workplace regional and industrial level. Timelier, less detailed, employment estimates are available in WFJ.

Summary

RTI data published as part of the monthly Labour Market release counts the number of employees on payrolls and is not a measure of jobs.

WFJ is the preferred source of jobs broken down by region of workplace and industry when considering all jobs in a timely manner. BRES should be used where more detailed analysis of region of workplace and/or industry is required or where less timely data for employees is of interest.

Both WFJ and BRES industry data are recommended in preference to industry data from household surveys, such as the Annual Population Survey (APS) and LFS, where the industry information would be obtained from the survey respondent rather than the business. However, LFS and APS are useful when looking at more detailed breakdowns not available from the WFJ and BRES business surveys, such as analysis by occupation.

There are four measures of public sector employment:

• Public Sector Employment (PSE), mainly collected from the Quarterly Public Sector Employment Surveys (QPSES)
• public sector employee job estimates, collected from the Business Register and Employment Survey (BRES)
• residence-based public sector employment estimates, collected from the Annual Population Survey (APS)
• residence-based public sector employment estimates, collected from the Labour Force Survey (LFS)

The Office for National Statistics (ONS) PSE statistical bulletin provides the UK's headline official estimates of public sector employment. As well as estimates of total public sector employment, breakdowns are published by government sector, broad industry and region of workplace, together with Civil Service employment by government department.

PSE estimates are based on a complete census of local government and civil service and cover all public bodies with 20 or more employees. The primary source is the QPSES. PSE estimates count every individual with an employment contract who is being paid in the public sector, including those where public sector employment would be their second job.

BRES provides workplace-based public sector employee job estimates down to local authority level. BRES collects comprehensive information from businesses representing the majority of the economy. BRES produces estimates of employees, rather than workforce jobs. Self-employed jobs, HM Forces and government-supported trainees are therefore excluded. Individuals are classified to the public sector dependent on the legal status of the organisation for which they work, as classified on the Inter-Departmental Business Register (IDBR), and follows UK National Accounts concepts and definitions. Since BRES is a business survey, jobs are classified on a workplace basis (that is, a public sector job is counted within the local authority where the organisation is located).

The APS and LFS provide residence-based PSE. The APS enables PSE estimates to be broken down to local area region of residence. Individual respondents to the APS and LFS classify themselves as working within the public or private sector. Estimates of the number of people working in the public sector from the APS and LFS are generally much higher than the ONS' official quarterly estimates of PSE. This is partially because many people who work within public sector premises, while being employed by private sector organisations, will classify themselves as working in the public sector, for example, cleaners or security guards employed by a contractor to work at public sector premises.

As can be seen from the figures in Table 1, PSE and the BRES estimates are reasonably comparable, whereas APS and LFS estimates show a much higher proportion of people employed in the public sector; this is mainly because of the self-classification of sector on the APS and LFS.

Summary

PSE, mainly collected from the QPSES, is the official source of public sector employment and the best source to use for both headline estimates and comparisons over time. As PSE is not available below regional level, local area region of workplace can be derived from BRES.

The APS and LFS generally over-estimate the number of people employed in the public sector employment but can be used when required to be broken down by personal characteristics and region of residence.

There are several data sources on earnings and income. Each has its uses, strengths and limitations. The majority of the Office for National Statistics (ONS) sources are detailed in A guide to sources of data on income and earnings. We aim to summarise these analyses within this article.

Average weekly earnings (AWE) is the ONS' lead indicator of short-term changes in earnings. It is designed to capture monthly changes in the AWE of employees in Great Britain. AWE is based on the Monthly Wages and Salaries Survey (MWSS), which covers employees working in businesses with 20 or more employees in all industrial sectors in Great Britain (an adjustment is made for smaller businesses, using factors derived from the Annual Survey of Hours and Earnings (ASHE)). Aggregate data are collected from businesses on total amount paid and number of people being paid. This gives timely information on short-term trends in earnings but lacks the detail available from ASHE.

Alongside AWE, ASHE is another important ONS source of earnings statistics. ASHE is our main measure of earnings in terms of structural statistics; it provides information about the levels, distribution and make-up of earnings and hours paid for employees in the majority of industries and occupations across the UK. It is also the lead source of estimates for the gender pay gap and the number of jobs paid below the National Minimum Wage or National Living Wage. The level of detail available (such as full-time versus part-time, weekly pay, hourly pay, annual pay, by occupation, industry, low-level geography, sex and age) reflects a large sample size and resource-intensive data collection, which means it is published only on an annual basis. ASHE uses a reference date in April and is published in the autumn of the same year.

HM Revenue and Customs' (HMRC's) data include information on people paid through the Pay As You Earn (PAYE) Real Time Information (RTI) system. Statistics are based on people who are employed in at least one job, and monthly estimates of aggregate pay reflect the average of such people for each day of the calendar month. As is the case for ASHE, published figures focus on median monthly pay; however, mean pay is also available and so are breakdowns by industry, age-band, various geographies and pay distribution.

The Labour Force Survey (LFS) also collects information on the earnings of employees. While the earnings data are known to be underestimated in the LFS they are a useful source of breakdowns not available in our other sources such as ethnicity and education. The data on individual's earnings captured by the LFS is thought to be of a lower quality than ASHE or AWE as LFS information is self-reported by employees. Furthermore, LFS responses can be given by proxy (by other individuals in the same household) when an individual is unavailable for interview. This gives further scope for recall error from respondents.

AWE compared with HMRC PAYE RTI

Main differences

Because of the different methodologies of AWE and RTI, the headline statistics for average pay levels are not directly comparable. In a distribution such as earnings, where the higher end is skewed because of a small percentage of very high earners, the mean (which is the focus of AWE estimates) will be higher than the median (which is the focus of RTI estimates). Both RTI and AWE will include anyone who has been added to a payroll for just a month, for example, to receive a bonus or similar; however, these are treated differently between the two sources. If people joined halfway through the month, RTI calendarises them (they are counted as part of an employee), whereas AWE counts them as a whole employee (which contributes to a smaller average wage).

Both RTI and AWE estimates include the earnings of those employees whose earnings were reduced for any reason, for example those on the Coronavirus Job Retention Scheme (CJRS). Additionally, the RTI statistics cover Northern Ireland, HM Armed Forces and government-supported trainees paid via PAYE, pay-rolled redundancy payments and signing-on fees, all of which are excluded from AWE.

Another important difference is that RTI estimates are calculated on a person basis, while AWE estimates are calculated on a job basis. As people can have more than one job each, the pay from their multiple jobs would be summed together in RTI; this difference will also cause the RTI estimates to be higher than the AWE estimates.

As with LFS, AWE will be subject to sampling variability, which will also lead to short-term differences between the RTI and AWE estimates.

Comparison of latest data

Figure 10 shows both sources have a similar trend over time when based on mean earnings. The RTI series is consistently higher than AWE, with the factors described earlier contributing to this pattern. In the months to December 2021, growth recorded in RTI is stronger than for AWE. This reflects, in part, lower percentage growth in RTI in 2019. In addition, if people being made redundant receive substantial payments, RTI estimates may increase.

The composition of the full set of jobs affects pay growth. AWE has identified that changes in the proportion employed within individual industries has had very little impact on pay growth, whereas RTI has identified that a fall in the number of job inflows has had a notable upward impact. Neither source is able to estimate the impact of changes in job by occupation within industry, and reference to LFS estimates may provide insight on this aspect.

Median RTI is a less comparable measure here, since AWE figures include bonus payments that generally create data extremes, which are not reflected in median as an estimator of average.

Interpreting average earnings data throughout the coronavirus (COVID-19) pandemic has been difficult, especially when measuring growth on-the-year. In July 2021, we published a blog, which explains the complexities of interpreting these data for AWE. The complexities were also evident in RTI earnings.

There were temporary factors that we refer to as base and compositional effects, which increased the headline growth rate in earnings above the underlying rate. The base effect refers to the comparison of recent data with a low base period 12-months previously, for example at the height of the coronavirus pandemic. The compositional effect refers to the changing composition of employee jobs. These temporary factors have largely worked their way out of the latest growth rates by October to December 2021, but a small amount of base effect for certain sectors may still be present.

Figure 11 shows both sources have a similar trend over time, with both sources being impacted by the base and compositional effects when analysing growth rates. The AWE series decreased more rapidly than RTI in the summer of 2020, and both saw massive increases in their growth rates 12-months later in the summer of 2021.

ASHE compared with HMRC PAYE RTI

Main differences

ASHE is based on a 1% sample of employees, sampled from HMRC PAYE data in January of the same calendar year. The January sample is updated with HMRC data supplied to the ONS in April to take account of new entrants to the labour market and those who have changed jobs since January. While the sample is employees, ASHE is completed by the employer in relation to the selected employees. RTI includes anyone who has a paid employee job during the reference period (that is, it is not based on a sample).

As ASHE uses a reference period within April each year and RTI is a monthly average of daily estimates, the most comparable published figures are ASHE's median gross weekly pay of all employee jobs, for employees on adult rates of pay and whose pay was unaffected by absence in the reference period and RTI's non-seasonally adjusted median weekly pay for April of each year. Note that the ASHE definition was revised for 2020 and 2021 only, to include any furloughed employees whose pay was affected by absence, reflecting the unusual labour market situation and existence of the Coronavirus Job Retention Scheme (CJRS).

Comparison of latest data

Figure 12 shows the distribution of weekly pay from ASHE and RTI, including the median (50th percentile). The series trend well over time, but ASHE is consistently higher than RTI for the median and up to and including the 75th percentile, and RTI is consistently higher than ASHE for the top 10%, 5% and 1% of earners. This is caused in part by the exclusion of workers with absences and those not on adult pay rates in the published ASHE estimates; analysis of ASHE including these employee jobs increases the ASHE median by close to £20 (of an observed difference between RTI median and ASHE median of approximately £65) in 2020. Another important driver of difference - as yet not quantified - is that RTI figures measure pay per person, while the ASHE data measure pay per job. As a person can have several jobs, the pay from each job is summed for a person in the RTI data and pay per person will be higher than pay per job. Another factor, especially the difference for the highest earners, is that ASHE represents a 1% sample of the population whereas RTI is a census of those paid through Pay as You Earn and so will capture the UK's highest paid employees.

ASHE compared with AWE

A common misconception of the headline ASHE and AWE figures is that they are measures of pay rises. However, they are both designed to estimate earnings of all employees in the economy at a single point in time and therefore measure the change in average pay, rather than the average of changes in pay. This is the reason why average pay can increase without anyone having had a pay rise.

Main differences

Although ASHE and AWE measure similar aspects of the labour market, they are two different measures of employee earnings that can exhibit notably distinct differences when compared with one another. ASHE benefits from a large sample size that, coupled with the array of individual and geographic characteristics that are also gathered and a longitudinal aspect, allows for more detailed analysis than other sources of earnings data. However, as it is an annual survey, it is not as timely as other measures of earnings and there are known coverage issues of bonus and incentive payments resulting from the reference period (the highest bonuses are paid in March, rather than April).

AWE does not include information on businesses with fewer than 20 employees or detailed characteristics that ASHE contains, which (although compensated for by the inclusion of estimates calculated using factors extracted from ASHE) means some changes in earnings resulting from compositional effects could be missed. However, it has better coverage of bonus and incentive payments and provides a much timelier estimate of trends in earnings.

A detailed comparison of these sources, their strengths and their weaknesses was published in An overview of and comparison between ASHE and AWE: 2017.

Comparison of latest data

Figure 13 shows that the two series track each other closely up to 2010 but there is a notable divergence between the series for a few years before returning to general stability and narrowing from 2016. Some potential reasons for divergence in the two series are changes between occupational and industrial classifications, methodological change for AWE in the move from the Average Earnings Index (AEI) to AWE in 2010 and reclassification of banks, further education and large companies between public and private sectors.

Labour Force Survey

Earnings questions on the LFS are only asked of responders in their 1st and 5th interviews. Therefore, estimates of gross weekly and hourly earnings from the LFS are subject to high sampling variability.

The data on individual's earnings captured by the LFS is thought to be of a lower quality than ASHE or AWE as LFS information is self-reported by employees. ASHE and AWE gather information from the employer which is thought to be more accurate as employers can consult payroll records. Individuals may not have such records to hand and their responses may therefore be subject to higher levels of recall error. Furthermore, LFS responses can be given by proxy (by other individuals in the same household) when an individual is unavailable for interview. This gives further scope for recall error from respondents. Because of this recall error, estimates of earnings based on the LFS that are published by the ONS typically exclude those who earn more than £100 per hour as a quality assurance measure. These factors combined mean that gross weekly and hourly pay are known to be underestimated on the LFS.

The main advantage of LFS earnings estimates are the breakdowns as they can be estimated for a high number of demographics collected on the survey.

Summary

More work is required to align the sources where possible to better understand the differences and where possible quantify.

ASHE is the preferred series for measuring levels of pay and detailed breakdowns because of its granularity, whereas AWE is more suitable for measuring rates of change as well as impact of bonus payments as it is a more frequent series. The frequency of AWE also allows for comparison with measures of inflation to calculate changes in real earnings.

RTI is the best source for timely earnings data by geographical breakdowns, industry, age band, and distribution.

We recommend that LFS earnings estimates only be used for demographics unavailable in other sources.

The Office for National Statistics (ONS) vacancy survey compared with Adzuna

Vacancies are defined as positions for which employers are actively seeking recruits from outside their business or organisation. Vacancies over time are indicators of the demand for labour. You can find out more about our vacancy survey in our guide to labour market statistics.

Official vacancy estimates are based on the Office for National Statistics (ONS) Vacancy Survey; this is a survey of businesses designed to provide estimates of the stock of vacancies across the economy, excluding agriculture, forestry and fishing (a small sector for which the collection of estimates would not be practical) and private households. Headline vacancies estimates are published for rolling three-monthly periods, monthly vacancies estimates are also available but should be interpreted with caution as they are experimental and non-seasonally adjusted

In response to the coronavirus (COVID-19) pandemic and the consequent demand for more timely data, the ONS has started publishing experimental figures for job adverts provided by Adzuna, an online job search engine that collates information from thousands of different sources in the UK. These range from direct employers' websites to recruitment software providers to traditional job boards, thus providing a comprehensive view of current online job adverts. More information about the methodology is available.

Main differences

The Vacancy Survey covers businesses in Great Britain only, although estimates for the UK are derived by grossing up the data for Great Britain, along with information about employment in Northern Ireland businesses.   Businesses in Northern Ireland are not surveyed because of the risk of overlap with responses to other surveys conducted by Northern Ireland departments.

Adzuna data has coverage of all job adverts advertised through their platform in the UK. As this source is limited to online vacancies, there will be some vacancies missed such as casual work advertised through word-of mouth or localised small enterprise tradespeople seeking temporary assistance or offering apprenticeships, which are filled using other headhunting methods. While we are not able to estimate how many adverts are missed in the online job adverts, analysis has shown that larger businesses are more consistent between the two sources. It is also possible that the circumstances of the coronavirus pandemic have caused a behavioural shift in businesses' recruitment practices, with a greater share of roles being advertised online than would otherwise have been the case.

The Vacancy Survey, as a sample survey, is designed to produce estimates of vacancies which are representative of the overall business population, both by industry and by the size of business, whether those vacancies are advertised online or otherwise. However, like any sample survey it is subject to sampling and non-sampling error.

A further difference between the two sources is companies placing single online advertisements which cover multiple vacant jobs. On Adzuna this would count for one advert, whereas the vacancy survey would capture the number of vacancies.

Whereas estimates from the Vacancy Survey are produced using the Standard Industrial Classification 2007 (SIC 2007), Adzuna estimates are produced by Adzuna Category which is not directly comparable. The Vacancy Survey collects vacancies from an employer at a specific point in time whatever the role, and these are allocated into whatever industry the employer is determined to be in.  For example, a large retailer may advertise for lorry drivers, cleaners and retail assistants, and these would all fall within the retail industry even though they are different occupations. This differs from Adzuna where the category is determined by the job description or job title so the vacancies may fall across many industries.

The Vacancy Survey is based on a reference date - the first Friday of the month unless it is the first day of the month, in which case it is the second Friday of the month -- and is published within six weeks of the reference date. Industry and organisation size breakdowns are available at UK level; however, no geographic data are available.

Adzuna weekly online job adverts publish vacancies with a lag of six days. Any vacancies that are not advertised online would not be included in the online job advert estimates. Data are available by Adzuna category and by UK country and International Territorial Level (ITL) region.

Comparison of latest data

The online Adzuna job advert estimates displayed a decline in total vacancies in the four weeks after Christmas 2020 after which it entered a period of rapid growth. The growth continued through to the beginning of June 2021 when it began to slow down. Notably, the vacancy survey remained reasonably flat from November 2020 to February 2021, when it began to follow the growth trend seen in the online statistics.

In July to August 2021 growth on the Vacancy Survey surpassed that of Adzuna; while the latter had become more stable the vacancy figure continued to rise more rapidly through to October 2021.

While both series are above pre-pandemic levels, the divergence between the two, between August and October 2021, is similar to that seen the same time in 2020, despite increases in the volume of vacancies on both series. A possible factor that may have contributed to this is large companies placing single online advertisements which cover multiple vacant jobs, as described above. Alongside this, possible seasonal patterns in certain types of recruitment activity (for example graduate recruitment) where larger business may post more multi-job adverts, could explain the divergence seen in this period in the last two years.

Estimates for December 2021 and January 2022 have seen the two series converge once again.

Summary

The Office for National Statistics (ONS) Vacancy Survey is the best source to use over time, and for headline estimates by industry and business size. However, Adzuna has additional breakdowns, such as regional data. The combined use of these sources is being considered for future development, using the Adzuna regional information to produce estimates of a more detailed breakdown of the Vacancy Survey data.

Labour Force Survey (LFS) redundancies compared with Insolvency Service HR1 data

There are two potential sources on the number of redundancies. One is the Labour Force Survey (LFS) and the other is an administrative data source, HR1 forms submitted to the Insolvency Service (IS) by employers when intending to make collective redundancies.

The IS regularly publish management information on advance notification of redundancies. Since January 2022, the Office for National Statistics (ONS) publish estimates for potential redundancies based on IS data.

Main differences

The LFS asks people whether they had left a job in the three months prior to interview and if so, why. Answer categories in this survey include "made redundant" and "took voluntary redundancy". The LFS estimates are based on interviews that take place across a three-month period, each asking about the previous three months (therefore, a recorded redundancy in the LFS would have happened somewhere between three and five months ago). Therefore, there is a lag between when redundancies occur and when they are reflected in the survey estimates. These data cover the United Kingdom (UK).

The Advance notification of redundancies form, or known as the HR1 form, is collated through the Insolvency Service. HR1s are forms businesses must fill out when they are proposing to dismiss 20 or more employees at a single establishment within a period of 90 days. A separate form is required for each establishment when it is proposed that 20 or more employees will be dismissed. The data therefore reflect proposed redundancies that may not result in actual redundancies. Companies are also required to submit a HR1 form when making changes to employees' terms and conditions, pension changes or change of location. Published HR1 data from the Insolvency Service includes the latter in their estimates, however the ONS cleans these cases from the data before publication to make estimates definitionally comparable with the LFS. These data only cover Great Britain (GB). Estimates for Northern Ireland are available from Northern Ireland Statistics and Research Agency (NISRA).

Comparison of latest data

As the LFS asks a respondent if they were made redundant in the last three-months, our weekly HR1 data have been aggregated to three-monthly periods for comparability, as shown in Figure 15. Over time, the two sources have followed a similar trend, with the LFS series slightly lagged and both displaying record low redundancies in December 2021.

Summary

LFS redundancy data are currently published monthly and are the most appropriate source for looking at actual redundancies. Data are available by age band, sex, region of residence and industry.

HR1 data will now be published weekly at GB-level in our Economic Activity and Social change in the UK: real-time indicators bulletin and monthly broken down by regional and industry level in HR1 potential redundancies. The HR1 ONS data will be a month ahead of LFS data, and although it only covers potential dismissals, will be a good indicator of the trend for official LFS estimates.

There are a number of sources available that estimate the number of furloughed jobs.

HM Revenue and Customs (HMRC) publish the number of furloughed jobs in their Coronavirus Job Retention Scheme (CJRS) statistical bulletin. This information is sourced from administrative data and where possible linked to further sources to provide more detailed information.

The Labour Force Survey (LFS) collects information on people who are temporarily away from a job that they expect to return to. This would include people who are on annual leave, maternity or paternity pay and more recently those furloughed. In response to the coronavirus (COVID-19) pandemic, a number of questions were added to the LFS to better understand the impact of COVID-19. A follow-up question was asked to those temporarily away from work or working less hours because of COVID-19 on the amount of pay they were receiving.

The Business Impact of Coronavirus (COVID-19) Survey (BICS) was launched in April 2020 in response to the coronavirus pandemic. It included a question asking businesses to estimate the proportion of jobs furloughed. These proportions can be applied to existing jobs data to estimate the number of furloughed jobs. These estimates are publish bi-weekly in the BICS results.

Figure 16 shows a comparison of the sources that collect information on furloughing.

The sources differ greatly in terms of levels, but all show the same trend throughout, including a downward trend into September 2020.

Summary

The administrative data as published by HMRC are the best source of furlough data, and they are available by region, sector, age, gender and employer size. Alternative sources can offer useful insight but should be used with caution.

Accuracy

(The degree of closeness between an estimate and the true value).

One aspect of measuring accuracy is reliability, which can be measured using evidence from analyses of revisions to assess the closeness of early estimates to subsequently estimated values. Revisions spreadsheets containing the data behind these analyses are made available with statistical bulletins. Links to the revisions analyses from each source can be found in Annex 8.

Another way of indicating the accuracy of the data is to measure sampling variability. Most of the labour market estimates published in the statistical bulletins are based on statistical samples and, as such, are subject to sampling variability. Links to the sampling variability analysis from each source can be found in Annex 8.

Details of the sample sizes and response rates for the Labour Force Survey (LFS), a major component of labour market statistics, are published in our regular Performance and Quality Monitoring Reports; these reports also cover steps taken to improve LFS response rates through the coronavirus (COVID-19) pandemic. Like many social surveys, LFS response rates have been declining over the last decade and fell significantly during the pandemic.

Other forms of non-sampling error are difficult to measure. These include coverage in surveys, processing and measurement errors and non-response.

Coherence

(The degree to which data that are derived from different sources or methods, but which refer to the same phenomenon, are similar.)

Estimates for payrolled employees, employment and for jobs could be expected to behave in a similar way. Similarly, it could be expected that unemployment and the claimant count would match one another's movements. However, in the short-term at least, many of these series can move in opposite directions, especially in times of economic uncertainty. Although each measure similar phenomena, each are derived from separate sources and have a number of basic differences. This page helps to explain the differences, and we intend to update as we develop a better understanding of the sources.

Each month analysis is undertaken to check the coherence of published sources, including in curiosity meetings with discussions between production teams and economic analysts. These activities act as a secondary quality assurance process and help analysts to better understand differences between sources to explain to stakeholders and data users.

Each month a comparison dataset between unemployment and the Claimant Count is published alongside the monthly release. In March 2022, our income and earnings statistics guide was published to better understand differences in definitions and sources of income and earnings statistics. This article supplements the more detailed overview of and comparison between the Annual Survey of Hours and Earnings and Average Weekly Earnings. Each year our article on reconciliation of estimates of employment and jobs is published.

Comparability

(The degree to which data can be compared over time and domain.)

Labour market estimates come from a wide variety of sources. These different series were introduced at different times and therefore have differing start dates. However, when methodological changes or other effects such as benchmarking, are made to the latest data, every effort is made to ensure that all previous data are amended to make them directly comparable. In addition, data can be modelled using more than one source to create a longer comparable data time series, for example, estimates of employment and unemployment going back to 1971.

Although the LFS is carried out using International Labour Organization (ILO) guidelines in many countries, not all interpret or apply those guidelines in exactly the same way. For example, a common term is "working age". Countries have different statutory school-leaving and retirement ages. Until August 2010, the UK used a "working age" definition based on men aged from 16 to 64 years and women aged from 16 to 59 years. In August 2010, the UK dropped this "working age" definition and introduced new series based on those aged from 16 to 64 years for both men and women. The Organisation for Economic Co-operation and Development (OECD) gather data from different countries and where possible adjust to publish comparable estimates.

Extensive information on the quality and methodology of each source, along with definitions, can be found in the relevant methodology documentation. These are linked in Annex 8.

Timeliness

(Describes the time between the date of publication and the date to which the data refers.)

Each source of labour market statistics has a different level of timeliness. The lag between date to which the data refers and date of publication depends on a number of factors including: data source (survey or administrative source), data owner (Office for National Statistics (ONS) or external supplier), data collection mode (for example, paper questionnaire, interviewer, online questionnaire), issued sample size and processing methodology.

Timeliness of estimates has always been important to users of Labour Market data, especially during the coronavirus pandemic where timely data were important to inform policy-makers and the public on labour market trends. Our monthly headline labour supply and earnings estimates are published as three-monthly rolling estimates at a two-month lag, our vacancies estimates are published provisionally at a one-month lag and our annual Business Register and Employment Survey, and Annual Survey of Hours and Earnings estimates are published at a 12- and 6-month lag respectively. To supplement these, and publish more timely estimates, the ONS and HM Revenue and Customs introduced Real Time Information flash estimates at a one-month lag in April 2020 and the ONS also publishes administrative sources of potential redundancies and online job adverts with less than a one-month lag.

This page summarises the work done so far on understanding how different data sources can be used to help understand the labour market. We will continue to work with data providers and stakeholders to develop a better understanding of the sources and, where possible, work to quantify differences. We intend to update this page when greater understanding of the data sources is achieved and if new sources are introduced into our reporting of the labour market.

Recently, regional breakdowns of payroll employees and their earnings have been included in the Real Time Information (RTI) bulletin. From December 2020, the RTI bulletin will also include industrial breakdowns of payroll employees and their earnings. The next steps will be to further explore the regional and industrial differences between RTI and existing data sources and provide an update on this page.

Labour Force Survey (LFS)

Published data

• Monthly publication of three-month rolling average estimate of employment data from the Labour Force Survey.

• Available broken down by type of employment (employee, self-employed, full-time and part-time), age, sex, ethnicity, disability, occupation, industry, socio-economic classification, country of birth, and nationality.

• Much of the LFS data are available back to 1992; headline estimates available back to 1971.

Main differences

• Definition - anyone aged 16 years and over in paid work and those who had a job that they were temporarily away from in the reference week.

• Reference period - one week.

Inclusions

• UK resident population (not foreign residents),

• Family workers who were employed but not paid (undeclared economy),

• Those earning below Pay As You Earn (PAYE) threshold,

• Self-employed,

• Some communal establishments (private households, NHS accommodation), and

• People living away from parental home in a student hall of residence or similar institution during term time.

Exclusions

• People aged under 16 years (although included in the survey they are excluded from employment estimates),

• Communal establishments (other than those listed previously),

• Temporary foreign workers, and

• Employees not paid during the reference period (for example, for certain types of seasonal work).

When to use

• For detailed analysis of the labour market using breakdowns not available on RTI.

• For long-term time series or historical estimates.

Real Time Information (RTI)

Published data

• Monthly payrolled employees for the UK and its regions, published data includes a flash estimate for the latest month.

• Available broken down by NUTS1, NUTS2, NUTS3, local authority, age, and industry.

Main differences

• Definition - anyone who has a paid employee job during the reference period.

• Reference period - a monthly average of daily estimates.

Inclusions

• All individuals receiving pay through a PAYE scheme,

• Employees on a training scheme if paid through PAYE scheme,

• People aged under 16 years people living in communal establishments,

• Some foreign residents,

• Those temporarily absent but still being paid (for example on maternity leave), and

• People whose main job is self-employed, but second job is as an employee and paid through PAYE.

Exclusions

• Employed individuals in the undeclared economy whose income is not reported to HM Revenue and Customs via PAYE,

• Self-employed, and

• Members of PAYE schemes where no employee earns above the Lower Earning Limit for National Insurance or has another job.

When to use

• For timely estimate of payroll employees (UK and regional breakdown available), lacks the detail and longevity available on LFS.

LFS unemployment

Published data

• Monthly publication of three-month rolling average estimate of unemployment data from the Labour Force Survey.

• Available broken down by age, sex, ethnicity, disability, country of birth, nationality, previous industrial sector, and unemployment duration.

Main differences

• Definition - anyone without a job, who have been actively seeking work in the past four weeks and are available to start work in the next two weeks or anyone out of work, have found a job and are waiting to start in the next two weeks.

• Reference period - one week.

Inclusions

• All aged 16 years and over.

• All satisfying International Labour Organization unemployment definition, whether they are claiming unemployment-related benefits or not.

Exclusions

• Those employed or economically inactive but still eligible to claim unemployment-related benefits.

• It is also possible that a person may work full--time and claim Jobseeker's Allowance fraudulently.

When to use

• To describe levels and rates of unemployment at any one point in time and when looking at changes over time.

• For long-term time series or historical estimates.

Claimant Count

Published data

• Monthly publication of the number of people claiming benefits principally for the reason of being unemployed.

• Data are available broken down by age, sex, region, local and unitary authority, and Parliamentary constituency.

Main differences

• Definition - measure of people claiming benefits principally for the reason of being unemployed.

• Reference period - count date, second Thursday of every month.

Inclusions

• Those aged 18 years to State Pension age,

• Those employed or economically inactive but still eligible to claim unemployment-related benefits, and

• Fraudulently claiming unemployment-related benefits.

Exclusions

• Unemployed but not eligible to claim for various reasons. Examples include:

• not made a minimum National Insurance contribution across the previous two years,

• receiving more than a particular level of income,

• their spouse's income disqualifies them,

• they work more than 24 hours a week,

• in full-time education, and

• they voluntarily left employment.

• Anyone who is eligible but chooses not to claim unemployment-related benefits.

When to use

• To understand short-term changes in the labour market and to describe the relative unemployment position between small areas.

Workforce Jobs (WFJ)

Published data

• Quarterly estimates of the number of jobs in the UK.

• Data are available broken down by employee jobs and self-employed jobs for breakdowns by industry and region.

Main differences

• The number of employees aged 16 years and over that an organisation directly pays from its payroll.

• The reference period varies between surveys included in WFJ. This includes:

• Short-Term Employment Survey -- count date, the first Friday after the second Thursday in the quarter month.

• Labour Force Survey - one week.

Inclusions

• All workers paid directly from the business's payroll(s),

• Employees on a training scheme if paid through payroll organisations not on Pay As You Earn (PAYE),

• Those temporarily absent but still being paid (for example on maternity leave), and

• Those earning below PAYE threshold.

Exclusions

• People aged under 16 years,

• Self-employed agency workers paid directly from the agency payroll,

• Voluntary workers,

• Unpaid family workers, and

• Working owners who are not paid via PAYE.

When to use

• WFJ is the preferred source of jobs broken down by region of workplace and industry when considering all jobs in a timely manner.

Real Time Information (RTI)

Published data

• Monthly payrolled employees for the UK and its regions, published data includes a flash estimate for the latest month.

• Available broken down by NUTS1, NUTS2, NUTS3, local authority, age, and industry.

Main Differences

• Definition - anyone who has a paid employee job during the reference period. It is not a count of the number of jobs.

• Reference period - a monthly average of daily estimates.

Inclusions

• All individuals receiving pay through a PAYE scheme,

• Employees on a training scheme if paid through PAYE scheme,

• People aged under 16 years,

• People living in communal establishments,

• Some foreign residents,

• Those temporarily absent but still being paid (for example, on maternity leave), and

• People whose main job is self-employed, but second job is as an employee and paid through PAYE.

Exclusions

• Self-employed,

• Employed individuals in the undeclared economy whose income is not reported to HM Revenue and Customs via PAYE, and

• Members of PAYE schemes where no employee earns above the Lower Earning Limit for National Insurance or has another job.

When to use

• For timely estimate of payroll employees (UK and regional breakdown available), lacks the detail and longevity available on LFS or WFJ.

Labour Force Survey (LFS)

Published data

• Monthly publication of three-month rolling average estimate of employment data from the Labour Force Survey.

• Available broken down by type of employment (employee, self-employed, full-time and part-time), age, sex, ethnicity, disability, occupation, industry, socio-economic classification, country of birth, and nationality.

• Much of the LFS data are available back to 1992; headline estimates available back to 1971.

Main differences

• Anyone aged 16 years and over in paid work and those who had a job that they were temporarily away from in the reference week.

• The reference period is one week.

Inclusions

• Employee main jobs,

• Self-employment main jobs,

• Government-supported trainees,

• Unpaid family workers,

• Workers with second jobs,

• Those not paid via PAYE,

• Those temporarily absent but still being paid (for example on maternity leave), and

• Those earning below PAYE threshold.

Exclusions

• People aged under 16 years,

• Temporary foreign workers,

• Armed forces not living in private accommodation,

• Workers living in communal establishments, and

• Third and subsequent employee jobs.

When to use

• Use when looking at more detailed breakdowns not available from WFJ or BRES.

Business Register and Employment Survey (BRES)

Published data

• Annual publication of the number of employees in the UK.

• Includes breakdowns by full-time and part-time, sector, industry, country, and English regions.

Main differences

• An employee is defined as anyone aged 16 years or over who is paid directly from the payroll, in return for carrying out a full-time or part-time job or being on a training scheme.

• The reference period is a count date on the first Friday after the second Thursday in September.

Inclusions

• All workers paid directly from the business' payroll(s),

• Employees on a training scheme if paid through payroll,

• Organisations not on PAYE,

• Those temporarily absent but still being paid (for example on maternity leave), and

• Those earning below PAYE threshold.

Exclusions

• People aged under 16 years,

• Self-employed (where identified),

• Any agency workers paid directly from the agency payroll,

• Voluntary workers and unpaid family workers, and

• Working owners who are not paid via PAYE.

When to use

• BRES should be used where more detailed analysis of region and/or industry is required or where less timely data for employees are of interest.

Public Sector Employment (PSE)

Published data

• Public Sector Employment bulletin published quarterly.

• Available by government sector, broad industry, and region of workplace, together with Civil Service employment by government department.

Main differences

• A complete census of local government and Civil Service and cover all public bodies with 20 or more employees.

• Counts every individual with an employment contract who are being paid in the public sector, including those where public sector employments would be their second job.

• Classified to the public sector on the legal status of the organisation.

• Estimates based on region of workplace.

When to use

• UK's headline official estimates of public sector employment.

• Use for total headcount, and breakdowns by government sector, broad industry, and region of workplace.

• Available back to 1999 so best source to use for comparisons over time.

Business Register and Employment Survey (BRES)

Published data

• BRES published annually provides workplace-based public sector employee jobs estimates available down to local authority level by industry and full-time or part-time status.

Main differences

• Sample of business therefore information provided by businesses.

• BRES produces estimates of employees, rather than workforce jobs.

• Self-employed jobs, HM Forces and government-supported trainees are excluded.

• Individuals are classified to the public sector dependent on the legal status of the organisation for which they work.

• Estimates based on region of workplace.

When to use

• Use when public sector employment estimates required by local area region of workplace.

Labour Force Survey (LFS)  and Annual Population Survey (APS)

Published data

• LFS estimates of public and private sector employment published quarterly.

• APS estimates of public and private sector employment, broken down by sex available on NOMIS.

Main differences

• Sample of households.

• Individual respondents to APS and LFS classify themselves as working within the public or private sector.

• Estimates based on region of residence.

When to use

• LFS and APS known to over-estimate public sector employment.

• Use when public sector employment estimates required by region of residence or other characteristics not available in PSE.

Average weekly earnings (AWE)

Published data

• Monthly publication of rolling three-month estimates of mean weekly pay.

• Estimates are available for total, bonus, regular and real pay (using inflation estimates) and broken down by sector and industry.

Main differences

• Average weekly earnings for anyone with a live employment on a PAYE scheme in the reference period.

• The reference period is the pay period that covers the last week of each month; information is collected based on pay frequency.

Inclusions

• Regular pay,

• Overtime pay,

• Bonus pay

• Shift premium pay,

• Allowance pay,

• Employees on trainee or junior rates of pay,

• Employees whose earnings were affected by absence, and

• Employees working abroad if they are paid from UK payroll.

Exclusions

• Employer National Insurance contributions,

• Employer contributions to pension schemes,

• Benefits in kind,

• Expenses,

• Redundancy payments,

• Signing-on fees,

• Stock options not paid through payroll,

• Director fees,

• Arrears (although arrears are collected, and reported separately),

• Accrued holiday pay,

• Self-employed,

• Northern Ireland,

• HM Armed Forces, and

• Government-supported trainees.

• Reporting of bonuses: captured in every month of the year, identified separately to regular pay.

• Statistical adjustment: using ASHE to estimate pay for businesses with under 20 employees.

When to use

• For measuring rates of change as well as impacts of bonus payments.

Annual Survey of Hours and Earnings (ASHE)

Published data

• Annual publication of employee earnings.

• Estimates on the structure and distribution of earnings by a range of different breakdowns covering sex, occupation, industry, age, and geography.

Main differences

• Median earnings for those who are employed for the reference date in April, based on a 1% sample of jobs from the HM Revenue and Customs Pay As You Earn register from the January (and supplemented in April) of the same year.

• The reference period is the pay period that covers a specified date in April each year.

Inclusions

• Regular pay,

• Overtime pay,

• Bonus pay,

• Shift premium pay,

• Allowance pay, and

• Northern Ireland

Exclusions

• Redundancy payments and expenses,

• National Insurance contributions,

• Employer pension contributions,

• Benefits in kind,

• Stock options not paid through payroll,

• Arrears,

• Self-employed,

• HM Armed Forces,

• Government-supported trainees paid via PAYE,

• Employees working offshore (for example, oil rig workers),

• Employees on trainee or junior rates of pay (those not on adult rates of pay)

• Employees whose earnings in the pay period were affected by absence (for example because of sickness), and

• Earnings for members of schemes where no employee earns above the Lower Earning Limit for National Insurance or has another job.

• Reporting of bonuses: some concern about their inclusion as the information might not be available to respondents when they submit information.

When to use

• For measuring levels of pay and detailed breakdowns, in terms of employee and employer type and forms of pay (hourly, weekly, annual).

Labour Force Survey (LFS)

Published data

• Quarterly publication of calendar quarter estimates of mean and median pay.

• Estimates are available for gross weekly pay broken down by region, occupation and industry, and the distribution of gross hourly pay.

Main differences

• Mean and median earnings for those aged 16 years and over, who classify themselves as employees in the reference week.

• The reference period is one week.

Inclusions

• Regular pay,

• Overtime pay,

• Bonus pay,

• Shift premium pay,

• Allowance pay, and

• Northern Ireland.

Exclusions

• Expenses,

• National Insurance contributions,

• Employer pension contributions,

• Benefits in kind,

• Stock options not paid through payroll,

• Arrears, and

• Self-employed.

When to use

• For measuring levels of pay and detailed breakdowns, in terms of demographics unavailable in other sources.

Real Time Information (RTI)

Published data

• Monthly mean and median pay for the UK and its regions; published data includes a flash estimate of median pay for the latest month and shows the distribution of pay.

Main differences

• Average monthly earnings of anyone who has a paid employee job during the reference period.

• The reference period is a monthly average of daily estimates.

Inclusions

• Regular pay,

• Overtime pay,

• Bonus pay,

• Shift premium pay,

• Allowance pay,

• Northern Ireland,

• Arrears,

• Employees on trainee of junior rates of pay,

• Employees whose earnings were affected by absence,

• Payrolled redundancy payments,

• Payrolled signing-on fees,

• Payrolled expenses,

• Government-supported trainees paid via PAYE, and

• HM Armed Forces.

Exclusions

• Stock options not paid through payroll,

• Employer National Insurance contributions,

• Employer contributions to pension schemes,

• Benefits in kind (except payrolled benefits in kind),

• Earnings for members of schemes where no employee earns above the Lower Earning Limit for National Insurance or has another job, and

• Self-employed.

• Reporting of bonuses - captures all bonus payments paid via PAYE but they can be difficult to differentiate from normal payments in the data.

• Statistical adjustment: calendarisation of those people joining payroll part way through the month.

When to use

• Timely earnings data by region and distribution.

ONS Vacancy Survey (VACS)

Published data

• Monthly publication of three-month rolling average of vacancies, and experimental monthly data time series.

• Data available by business size and industry (Standard Industrial Classification (SIC) 2007).

Main differences

• Measures the number of vacancies that are actively seeking recruits from outside the organisations.

• A single advert published for multiple jobs would capture multiple vacancies.

• Reference period is the first Friday of the month unless this is the first day of the month. In this case, the reference date moves to the second Friday of the month.

Exclusions

• Excludes employment agencies, private households and agriculture, forestry and fishing industries.

• Cannot be broken down by further demographics.

When to use

• Analysis of vacancies over time and for breakdowns by industry and business size.

• Analysis of the total number of vacancies.

Adzuna

Published data

• Weekly indices of online job adverts.

• Data available by region and Adzuna category (not SIC codes).

Main differences

• Point in time estimate of all online job adverts.

• A single advert published for multiple jobs would count as one advert.

• Reference period is once a week, date noted on dataset.

Exclusions

• Excludes adverts not posted online (word of mouth or notice boards).

• Cannot be broken down by further demographics.

When to use

• Analysis of regional breakdowns of vacancies.

• Analysis of short-term, weekly movements.

LFS redundancies

Published data

• Monthly publication of three-month rolling average of level and rates of redundancies from the Labour Force Survey (LFS).

• Data are available by age, sex, industry and region.

Main differences

• LFS redundancies include respondents who were not in employment and who had been made redundant within the current month or last two months.

• They also include respondents who were recently re-employed but had been made redundant within the current month or last two months.

Inclusions

• People in employment, International Labour Organization unemployed or economically inactive who left their job within the last three months, and

• People who were made redundant or took voluntary redundancy.

When to use

• To look at redundancies over time by age, sex, region of residence and industry.

HR1 potential redundancies (as published by the Insolvency Service)

Published data

• Currently available monthly through the Management information on Advanced Notification of Redundancy Scheme.

Main differences

• Where employers of potential redundancies of 20 or more employees at one establishment (where an employee is assigned to work) notify the Insolvency Service's Redundancy Payments Service within a period of 90 days or less.

• This notification needs to be within 30 days of the first dismissal (for up to 99 employees) or within 45 days (for 100 or more employees).

Inclusions

• By law, each company has a statutory obligation to complete a HR1 form, which registers the intention to make 20 or more employees redundant.

• HR1 forms are also used to register changes to employees' terms and conditions.

• Does not include businesses intending to make less than 20 redundancies or Northern Ireland.

When to use

• To gain understanding of potential planned redundancies.

HR1 potential redundancies (as published by the ONS)

Published data

• Currently available weekly through our Economic Activity and Social change in the UK, real-time indicators bulletin.

• Also available monthly, broken down by regional and industry level, in our HR1 potential redundancies dataset.

Main differences

• Where employers of potential redundancies of 20 or more employees at one establishment (where an employee is assigned to work) notify the Insolvency Service's Redundancy Payments Service within a period of 90 days or less.

• This notification needs to be within 30 days of the first dismissal (for up to 99 employees) or within 45 days (for 100 or more employees).

Inclusions

• By law, each company has a statutory obligation to complete a HR1 form registering the intention to make 20 or more employees redundant.

• Before publication, the ONS removes any cases that mention:

• Pension,

• Contractual,

• Terms and conditions,

• Transfer or move, and

• TUPE (transfer of undertakings (protection of employment)).

• HR1 forms are also used to register changes to employees' terms and conditions.

When to use

• To gain understanding of potential planned redundancies.

Labour Force Survey (LFS) and Annual Population Survey

Quality and Methodology Information (QMI)

• Labour Force Survey QMI

• Labour Force Survey -- user guidance

• Performance and Quality Monitoring Reports

Revisions

• EMP05: Employment rate revisions triangle

• UNEM04: Unemployment rate revisions triangle

• S02 Regional labour market: Sampling variability and revisions summary

Sampling variability

• A11: Labour Force Survey sampling variability

• S02 Regional labour market: Sampling variability and revisions summary

Real Time Information (RTI)

Quality and Methodology Information (QMI)

• New methods for monthly earnings and employment estimates from Pay as You Earn Real Time Information (PAYE RTI) data

Revisions

• Earnings and employment from Pay As You Earn Real Time Information, revision triangle

Claimant Count

Quality and Methodology Information (QMI)

• Claimant Count QMI

Revisions

• CLA03: Claimant Count revisions triangle

• Revisions to the Universal Credit component of the Claimant Count

Workforce Jobs (WFJ)

Quality and Methodology Information (QMI)

• Workforce Jobs QMI

Revisions

• Revisions to workforce jobs

• JOBS06: Workforce jobs revisions triangle

Sampling variability

• JOBS07: Workforce jobs sampling variability

Business Register and Employment Survey (BRES)

Quality and Methodology Information (QMI)

• Business Register and Employment Survey (BRES) QMI

Revisions

• Published BRES results for the most recent period are provisional and are revised to final in the following release.

Sampling variability

• Tables of coefficients of variation (CVs) are published alongside each of the six main BRES publication tables.

Public Sector Employment (PSE)

Quality and Methodology Information (QMI)

• Public sector employment QMI

• Table 13 of the Public Sector Employment data tables includes response rates.

Average weekly earnings (AWE)

Quality and Methodology Information (QMI)

• Average weekly earnings QMI

Sampling variability

• A table of sampling variability for headline estimates can be found in EARN01: average weekly earnings.

Annual Survey of Hours and Earnings (ASHE)

Quality and Methodology Information (QMI)

• Annual Survey of Hours and Earnings (ASHE) methodology and guidance

• Guide to interpreting Annual Survey of Hours and Earnings (ASHE) estimates

• Annual Survey of Hours and Earnings, low pay and ASHE pension results QMI

Revisions

• Published ASHE results for the most recent period are provisional and are revised to final in the following release.

Sampling variability

• Tables of coefficients of variation (CVs) are published alongside each of the main ASHE publication tables.

ONS Vacancy Survey (VACS)

Quality and Methodology Information (QMI)

• Vacancy Survey QMI

Revisions

• Published ONS Vacancy Survey results for the most recent period are provisional and are revised to final in the following release.

Adzuna

Quality and Methodology Information (QMI)

• Using Adzuna data to derive an indicator of weekly vacancies

HR1 potential redundancies (as published by the ONS)

Quality and Methodology Information (QMI)

• Developing an indicator of potential redundancies using Insolvency Service data