Labour Market Survey: research and results overview

There are three reports in this release:

• Labour Market Survey Statistical Test technical report

• Labour Market Survey Statistical Test characteristics report

• Labour Market Survey Statistical Test comparative estimates report

All three reports should be considered together when reading and interpreting the results as each provides important context to the findings.

These Research Outputs are not official statistics relating to the labour market. Rather, they are published as outputs from research into an alternative prototype survey instrument (the Labour Market Survey (LMS)) to that currently used in the production of labour market statistics (the Labour Force Survey (LFS)).

It is important that the information and research presented in the comparative estimates report and characteristics report is read alongside the accompanying technical report to aid interpretation and to avoid misunderstanding. These Research Outputs must not be reproduced without this disclaimer and warning note.

As part of the Census and Data Collection Transformation Programme (CDCTP), the Social Survey Transformation (SST) Division has been looking at the end-to-end survey designs for the current household survey portfolio including the Labour Force Survey (LFS). This includes establishing the feasibility of introducing a default online mode of collection for household surveys.

To collect labour market data via an online-first collection design, a new prototype product is being developed called the “Labour Market Survey” (LMS). The LMS has a mixed-mode design (online and face-to-face) and for this test the focus has been on core data collection requirements needed to produce labour market estimates. The current design of the LMS is still in development and as such is currently a prototype. Notable design differences between the LMS and LFS include:

• modes of collection – the LMS is mixed-mode but is online by default with face-to-face follow up

• reference week – the LMS has a rolling reference week, which means the recall period for questions is always closer to when respondents complete the questionnaire (no more than one week)

• question wording and order – transformation of questions for the LMS rather than a direct translation from the LFS

• different field collection periods – eight-week field period on the LMS (two weeks online; six weeks online and face-to-face)

• communication materials – a redesigned suite of letters, information leaflets, and branded envelopes to help respondents understand the purpose of the LMS and how they can take part online

For more a more comprehensive explanation of these design features and others, please see the technical report.

The test detailed in the reports here is known as the “Labour Market Survey (LMS) Statistical Test” as it has been designed to collect data that will allow important labour market statistical estimates to be produced. This will be the first instance in which the prototype LMS has be used to produce such estimates. This test marks the first time the Office for National Statistics (ONS) has tested a mixed-mode design (online and face-to-face) at scale for a voluntary household survey.

Data collection for the LMS Statistical Test was outsourced to Ipsos MORI, which hosted the online questionnaire and supplied the field force for the face-to-face interviews.

Comparisons in this collection of reports are made between the prototype LMS and an equivalent Labour Force Survey (LFS) dataset that has been reweighted and adjusted to account for design differences between the LMS and LFS.

There were five main objectives of the Labour Market Survey (LMS) Statistical Test:

• to measure the online engagement rate and mixed-mode response rates (see LMS technical report)

• to produce important labour market estimates using a prototype new survey data source (see LMS comparative estimates report)

• to compare the estimates to the existing data source (that is, the LFS) (see LMS comparative estimates report)

• to compare the socio-demographic characteristics of responding households or individuals (see LMS characteristics report)

• to evaluate all aspects of the test (including design, operations and results) and incorporate the results into future research and testing cycles

Engagement and Response

The engagement rate for the LMS test (that is the proportion of the households sampled that accessed the survey) was 60.7% overall; online engagement was 31.1% and face-to-face engagement was 29.6%.

The response rate (that is the proportion of the households sampled that provided usable data) was 56.5%. The online (28.4%) and face-to-face (28.1%) response rates were very similar.

Socio-Demographic Characteristics

There were minimal differences between the LMS test and the LFS in terms of the characteristics of the responding sample; the proportions of responding males and females were the same. Some of the age distributions varied slightly.

The responding sample for the LMS test was more evenly distributed in terms of the Index of Multiple Deprivation (IMD) deciles than the LFS over the same period.

Employment, unemployment and economic inactivity

No statistically significant differences were found between the LMS test and the comparative LFS dataset for any of the important headline labour market estimates.

Educational status and labour market status

Significant differences were observed in the estimates of people aged 16 to 24 years in full-time education.

Significantly more men and women in this age cohort were estimated to be in full-time education and also in employment, based on findings from the LMS test.

These differences need further investigation in the context of the changes made to the design of the prototype LMS. Specifically, the questionnaire design will be reviewed and the approach to sampling will be investigated further.

Full-time and part-time workers

No significant differences were observed between estimates for the number of employees, both full-time and part-time, produced from both comparative datasets.

There were significantly fewer people who were classified as self-employed in the estimates produced by the LMS compared with those from the LFS. This difference in self-employment was also found to be significant for women.

There are a number of potential explanations for the estimation of fewer self-employed respondents on the LMS including differences in the survey design, operational delivery of the LMS and questionnaire design. Further work is required to understand the impact of these explanations and to ensure the LMS produces reliable estimates that meet users' needs.

Actual weekly hours worked

Significantly fewer total hours were worked by the total workforce on the LMS compared with the LFS comparative dataset.

There are several questionnaire and operational differences that may be driving these statistical differences. This may include the use of the rolling reference week on the LMS, which was implemented to try and reduce any potential recall error that may have occurred with the online collection mode. This design change will be evaluated alongside users' needs to ensure the LMS produces reliable estimates.

Data from the Labour Market Survey (LMS) Statistical Test provide initial evidence towards validating the approach taken by the Office for National Statistics (ONS) towards transforming survey design to optimise for online-first, mixed-mode data collection. The findings are encouraging, considering the breadth of differences in the design of the LMS and given the challenges other statistical organisations have faced when transforming surveys to include an online collection mode. However, it should be acknowledged that these are initial findings and represent a single set of comparative estimates.

Further research that builds upon the findings from this test, and further investigations into the factors driving some of the differences described, is required to progress the development of the LMS.

We would like your feedback on this publication. Please email LMS.Transformation@ons.gov.uk (and include the subject line, “Research Outputs feedback”).