Economic activity and hours worked question development for Census 2021

In December 2018, the government presented to Parliament a White Paper Help Shape our Future: The 2021 Census of Population and Housing in England and Wales (PDF, 967KB). This outlined the Office for National Statistics' (ONS') proposal to collect information on economic activity and hours worked to meet the needs of central and local government to develop strategies to promote economic growth and to identify and address economic inequalities.

This report details the development of the questions on employment activity last week, reasons for not working, looking for work, availability to work, waiting to start a job already accepted, ever worked and hours worked.

As in previous censuses, the question will only be asked of people who are aged 16 years or older.

Census 2021 will be an online-first census, with a target of 75% online returns. We have reviewed and tested each question online and have recommended additional design and functionality amends specific to the electronic questionnaire. We have also recommended additional changes to the question designs following our stakeholder engagement, evaluation and testing programmes. These changes will be detailed in this report.

The main recommended changes to question designs are:

• the removal of the write-in box for year last worked; respondents no longer need to enter this information

• to simplify and shorten questions; for example, the question "In your main job, how many hours a week (including paid and unpaid overtime) do you usually work?" has been shortened to "How many hours a week do you usually work?" with a separate instruction "include paid and unpaid overtime"

• to replace incomplete question stems, for example, "Last week, were you:", with complete questions, for example, "In the last seven days, were you doing any of the following?"

Any additional changes to question designs, including online functionality and accessibility, will be detailed in the sections Research that led to the 2018 White Paper and Research that led to the recommended Census 2021 question designs.

Since the publication of the White Paper, the Office for National Statistics (ONS) has conducted and concluded the final phase of testing on economic activity and hours worked. Occupation, industry, travel to work, economic activity and hours worked are all part of the labour market topic. We have also published a report covering the questions on occupation and travel to work.

This report details the development of questions on:

• employment activity last week

• reasons for not working

• looking for work

• availability to work

• waiting to start a job already accepted

• ever worked

• hours worked

The report on occupation, industry and travel to work details the development of questions on:

• employment status

• organisation name

• job title

• job description

• main activity

• supervisory status

• method of travel to work

• place of work

• workplace address

This report provides links to previously published research and the findings of additional testing that led to the final recommended questions for Census 2021 in England and Wales. The questions and response options for Census 2021 have now been finalised through the census secondary legislation: the Census (England and Wales) Order 2020, and Census Regulations for England and for Wales.

The evidence base for the question designs for Census 2021 is discussed in the section Research that led to the recommended Census 2021 question design.

Annex 4 provides details of the definitions and terms used in this report.

In June 2015, the Office for National Statistics (ONS) held a formal, 12-week consultation process asking census users for their views on the topics that were required in the questionnaire in England and Wales. The aim of the consultation was to promote discussion and encourage the development of strong cases for topics to be included in Census 2021.

In May 2016, we published our response to the 2021 Census topic consultation. This set out our updated view on the topics to be included in Census 2021, including:

• a summary of proposals for new topics

• next steps

• an overview of our plans

This response included a commitment to continue to collect data on economic activity and hours worked and to not to collect data on year last worked, but to explore the potential of administrative data sources for these data.

The 2021 Census topic consultation revealed a clear requirement for information on economic activity and hours worked to underpin local strategies, understand and address inequalities, and promote economic growth.

A detailed summary of the consultation responses relating to the labour market topic, that includes economic activity and hours worked, can be found in the labour market topic report (PDF, 809KB). In this report, we made clear commitments to the public.

Our commitments regarding economic activity and hours worked were:

• to ensure that the questions align with changing international definitions (ILO's 2013 Resolution concerning statistics of work, employment and labour underutilization) and the labour market concepts remain consistent with those collected in the Labour Force Survey (LFS)

• to use the responses to the ONS Consultation on revising the Standard Occupational Classification 2010 (SOC2010) to inform the detailed quality requirements for National Statistics Socio-economic Classification (NS-SEC) and whether this identifies any further scope for reducing the sub-topics needed to derive the current version of NS-SEC used in census outputs

• to explore how the online questions can improve the collection of data while still meeting user needs

We provide an update on how we met these commitments in Annex 1.

Following this, we began a comprehensive programme of research and development. We provide a full list of the tests used in the development of the questions on economic activity and hours worked in Annex 2. Further details are provided in the summary of testing for Census 2021.

The tests used a range of qualitative and quantitative research methods. A short description of the different research methods and sampling techniques is given in the question and questionnaire development overview for Census 2021. Testing included respondents from a wide range of backgrounds.

In December 2018, the government presented to Parliament a White Paper Help Shape our Future: The 2021 Census of Population and Housing in England and Wales (PDF, 967KB). This outlined our proposal to keep collecting data on economic activity and hours worked.

The questions for Census 2021 are now finalised. We have evaluated the questions for their potential impact on data quality, public acceptability, respondent burden, financial concerns and questionnaire mode. We present details of this evaluation in Annex 3.

In the White Paper Help Shape our Future: The 2021 Census of Population and Housing in England and Wales (PDF, 967KB), we stated that we were finalising the questions for Census 2021.

The starting point for testing the questions on economic activity and hours worked was the 2011 Census question designs. You can see these questions on the 2011 Census Household Questionnaire (PDF, 1.8 MB).

Employment activity last week

In the 2011 census the question was:

Last week, were you:

Tick all that apply

Include any paid work, including casual or temporary work, even if only for one hour

[ ] working as an employee?

[ ] on a government sponsored training scheme?

[ ] self-employed or freelance?

[ ] working paid or unpaid for your own or your family's business?

[ ] away from work ill, on maternity leave, on holiday or temporarily laid off?

[ ] doing any other kind of paid work?

[ ] none of the above

Changes to Government Digital Service Standards since 2011 meant that all partial question stems were changed to full questions. So, we used full questions in all our testing, such as "In the last seven days, were you doing any of the following?".

The International Labour Organisation (ILO) definition of employment has changed since 2011 to exclude unpaid training as a form of economic activity. The Labour Force Survey (LFS) collects government sponsored training schemes under the "Employee" response option. Therefore, we removed the response option "On a government sponsored training scheme" from the question.

To comply with the Equality Act 2010, "Paternity leave" has been included in the response alongside "Maternity leave".

Testing a two-stage question with binary response options

We collected views on two variants of the question in informal interviews (2018:1) with Office for National Statistics (ONS) staff. Each participant was given one of three versions of the questionnaire and asked to answer all questions on behalf of a member of their household, relative or friend. This provided a wide range of responses for different educational and employment backgrounds.

• Variant 1: A two-stage question. The first question was "Last week, did you have a job, either as an employee, self-employed, or freelance?" and the second question was "Last week, were you away from work ill, on maternity or paternity leave, on holiday, or temporarily laid off?" The response options to both questions were "Yes/No" answers. Respondents who answered "Yes" to the first question skipped past the second question and around the looking for work questions. Those who answered "No" to the first question went to the second question. Respondents who answered "Yes" to the second question were routed around the questions on looking for work.

• Variant 2: A one-stage question, "Last week, were you working or temporarily away from work?" Three lines of instructions were provided to clarify what should be counted. The responses were binary "Yes/No" answers. Respondents who answered "Yes" were routed around the questions on looking for work.

Respondents to both questions thought there were too many instructions to read and consider, with Variant 2 being considered particularly difficult to read. The Variant 2 question stem was misread by several respondents who interpreted it as:

• choosing between working or being temporarily away from work

• asking if they were temporarily away from work only

• asking if they were temporarily away from work but for work purposes (for example, on a business trip or for training)

Participants who answered the Variant 1 question were unsure how to count maternity leave. People on maternity or paternity leave have a job, so would answer "Yes" to the first question and would be routed around the question explicitly asking if they were on maternity leave. Some respondents questioned why they were asked two questions with the same routing instructions.

Participants were unclear if they should include volunteering work, as other forms of unpaid work were included in the instruction "Working unpaid for your own or family's business".

A variation of the two-stage question was brought forward for cognitive testing (2018:6) with members of the public at a jobs fair. The first question retained the "Yes/No" response options and routing. The second question included a list of "Economically active" tick-box response options, plus a "None of the above" option for those who are not working.

Last week, were you doing any kind of paid work, including casual or temporary work, even if only for one hour?

• Yes

• No

Which of the following were you doing last week?

• Away from work ill, on maternity or paternity leave, on holiday, or temporarily laid off

• Working unpaid for your own or your family's business

• Working in a paid apprenticeship

• None of the above, for example, retired or not working

Respondents reported no problems with the two-stage approach. However, many respondents found the first question stem and second question responses too long and complicated.

We conducted a test (2018:14) to assess the revised labour market questions for comprehension, accuracy and ease of completion. Cognitive interviews were held with participants with non-standard work circumstances. Testing was conducted across three rounds, and respondent feedback from each round was used to amend the question for the next round.

The first round used the two-stage question from the previous test. The line about including casual or temporary work was moved out of the question stem and into an instruction to shorten the question stem. The response option to the second question "Working in a paid apprenticeship" was removed because of space restrictions and a low response rate.

As with previous tests, there was some disagreement in how respondents defined the terms used:

• some participants were confused about the use of the term "paid work" in the question stem

• "last week" was not always understood to mean "in the last seven days"

• some participants who were not in work interpreted "None of the above, for example, retired" as not applying to them, as they were not retired

For the second round of testing, the two questions were combined into one question. The question stem and response options were changed to reduce ambiguity around the term "paid work". "Last week" was changed to "last seven days" to clarify the time period the question refers to. Finally, "for example, retired" was removed from the "None of the above" response option.

In the last seven days, were you doing any of the following?

Tick all that apply

Include casual or temporary work, even if only for one hour

• Working as an employee

• Self-employed or freelance

• Temporarily away from work ill, on maternity or paternity leave, on holiday or temporarily laid off

• Doing any other kind of paid work

• None of the above

The change from "Last week" to "In the last seven days" led to an improvement in data quality. However, not all respondents referred back to the reference date, so there was still some difference in interpretation.

Many respondents found the "Temporarily away from work" response too long and did not read all of the examples. In the next round of testing, this response option was split into two response options to address this issue.

Electronic questionnaire testing

We conducted a test (2018:37) on the electronic questionnaire in an online survey with 2,715 participants in England, Wales and Northern Ireland.

The response option for "Temporarily away from work ill, on maternity or paternity leave, on holiday or temporarily laid off" was split into two separate tick-boxes: "Temporarily away from work ill, on holiday or temporarily laid off" and "On maternity or paternity leave".

Analysis showed that respondents changed their response to this question frequently before submitting their final response. This could indicate that some terms were causing confusion.

This version of the question was tested during the 2019 Rehearsal (2019:15) and no issues were found.

Reasons for not working

In the 2011 Census, this question came after the set of questions on looking for work and availability to work.

In 2011, this question was:

Last week, were you:

Tick all that apply

[ ] retired (whether receiving a pension or not)?

[ ] a student?

[ ] looking after home or family?

[ ] long-term sick or disabled?

[ ] other

Because of changes to the Government Digital Service Standards, partial question stems that appeared in 2011 were changed to full questions during testing; for example, "Which of the following best describes what you were doing last week?".

Merging this question with "Waiting to start a job already accepted"

We collected views on three variants of the question in a qualitative test (2018:1). Informal interviews were conducted, with each participant given one of three versions of the questionnaire and asked to answer on behalf of a member of their household, relative or friend.

• Variant 1: The same response options as the 2011 question, with an additional response option "In unpaid training".

• Variant 2: The same response options as the 2011 question, plus "Waiting to start a job already accepted". This was to investigate the potential to combine this question with the final availability to work question. Similar to Variant 1, the response option "In unpaid training" was added to the response options.

• Variant 3: Included the "Waiting to start a job already accepted" response option, excluded the "In unpaid training" response option, and the "Retired" response option was shortened by removing the line "whether receiving a pension or not".

Many participants found the placement of this question burdensome, as they had to answer the questions on availability to work before they could provide a reason for not working.

Participants answering on behalf of someone who was retired found the instruction "whether receiving a pension or not" in the first and second variants helpful.

Following this test, we decided not to combine this question with the availability to work question. The "Waiting to start a job already accepted" response option required different routing instructions to the other response options. As this is a multi-tick question, if a respondent ticks two or more response options with different routing, it would not be possible to follow both instructions.

Testing the revised question

We commissioned an external research agency to conduct multi-round testing (2018:14) to assess the revised question for comprehension, accuracy, ease of completion and applicability across groups with different education and employment circumstances. Cognitive interviews were held with participants with non-standard qualifications and work circumstances. Testing was conducted across three rounds, with respondent feedback from each round being used to amend the question for the next round.

The question tested in the first round excluded the "In unpaid training" and "Waiting to start a job already accepted" response options from the previous test. The full "Retired (whether receiving a pension or not)" response option was reinstated.

As in the previous test, many participants felt negatively about the position of the question. They had to answer three questions on availability for work before they could justify their reason for not working.

In the second round of testing, the question was moved after "In the last seven days, were you doing any of the following?" This allowed respondents who were retired, long-term sick or disabled to provide a reason for not working before moving on to the next questions.

The question stem was changed from "last week" to "in the last seven days" to make it consistent with changes made to the timeframes of other questions identified during testing. The word "best" was removed from the question stem, as this implied only one response is expected, when the respondent can tick multiple responses.

Which of the following describes what you were doing in the last seven days?

Tick all that apply

• Retired (whether receiving a pension or not)

• Student

• Looking after home or family

• Long-term sick or disabled

• Other

The new position of the question showed a clear improvement in respondent acceptability and responses received.

Electronic questionnaire testing

We tested (2018:37) the online version of the question in an online survey. The final version of the question from the multi-round testing was presented. The response option "Student" was changed to "Studying" so it made grammatical sense as a response to the new question stem.

No problems were encountered with this question during this testing or the 2019 Rehearsal (2019:15).

Looking for work

The data from this question are used to help classify unemployed respondents as economically active or inactive.

Testing conducted on other questions (2018:14) showed that the timeframe should be moved to the beginning of the question stem, as respondents found this easier to read and understand. The question was updated with this in mind:

In the last four weeks, were you actively looking for any kind of paid work?

• Yes

• No

Further testing did not identify any problems with the question design or the amended question stem.

After the reasons for not working question was moved to before this question, additional guidance was added to this question in an accordion on the electronic questionnaire. This guidance explains why we ask the questions on looking for work and availability to work to people who are retired, or long-term sick or disabled.

This question was tested during the 2019 Rehearsal (2019:15) with no major issues found.

Availability to work

The data from this question are used, alongside data from other economic activity questions, to classify unemployed respondents as economically active or inactive.

2011 Census

If a job had been available last week, could you have started it within two weeks?

[ ] Yes

[ ] No

Previous censuses adopted the International Labour Organisation (ILO) definition of "availability for work" as a period of time no more than two weeks around a specific reference date. In the 2011 Census, we interpreted this as the week on either side of the census reference date. Therefore, respondents were required to work out if they could have started a job within two weeks, if that job became available one week before the census reference date.

The 2011 Census Quality Survey (PDF, 1.4MB) found that 14% of respondents gave different answers to those they had provided in the 2011 Census. This suggested that respondents interpreted the question differently when recalling the same reference period.

We reviewed the ILO definition and found there was scope to change the reference period, therefore simplifying the question. The census reference date was moved to the beginning of the "two week" reference period, so the question was reworded to "If a job became available now, could you start it within two weeks?"

Testing the revised question

An external research agency conducted multi-round testing (2018:14) to assess the question for comprehension, accuracy, ease of completion and applicability across groups with different education and employment circumstances. Cognitive interviews were held with participants with non-standard qualifications and work circumstances. Testing was conducted across three rounds. Feedback from each round was used to amend the question for the next round.

The results showed that respondents who were retired or had no plans to return to work interpreted "could you start" to be asking whether they could work in theory, even if they had no intention of doing so. The question stem was reworded for the third round of testing to "Are you available to start work in the next two weeks?" to make it clear the question is referring to actual availability to work.

Longer question for the paper questionnaire

On the paper questionnaire, the questions on "Looking for work", "Availability to work" and "Waiting to start a job already accepted" are asked to all respondents who are unemployed. On the electronic questionnaire, automatic routing will direct respondents around one or more of these questions.

In testing (2018:14) the longer question stem "If a job became available now, could you start it within two weeks?" was considered easier to understand on the paper questionnaire, where respondents would be asked all questions on looking for work and availability to work. The shorter question stem "Are you available to start work in the next two weeks?" was brought forward for testing on the electronic questionnaire only, as the online routing means that only people who are looking for work are asked this question.

Further testing identified no problems with respondents' understanding of the question on the paper or electronic questionnaire. Therefore, the amended question was included in the 2019 Rehearsal (2019:15), where no major issues were found. The paper questionnaire used the longer version ("If a job became available now, could you start it within two weeks?"), while the electronic questionnaire used the shorter version ("Are you available to start work in the next two weeks?"). The same designs are to be used for Census 2021.

Waiting to start a job already accepted

We collected views on two question variants in a qualitative test (2018:1). Informal interviews were conducted with ONS staff. Each participant was given one of three versions of the questionnaire and asked to answer on behalf of a member of their household, relative or friend. This provided a wide range of responses for different educational and employment backgrounds.

The first question variant was identical to the 2011 question, except the word "obtained" was changed to "accepted" to improve comprehension of the question.

Last week, were you waiting to start a job already accepted?

• Yes

• No

The second variant was not a standalone question, but a response option to the "Reasons for not working" question. However, this response option required a separate routing instruction to the other responses. As this is a multi-tick question, if a respondent ticks two or more response options with different routing, it would not be possible to follow both instructions. Therefore, the standalone question on waiting to start a job already accepted was brought forward for further testing.

Multi-round testing

An external research agency conducted a test (2018:14) to assess the question for comprehension, accuracy, ease of completion and applicability across groups with different education and employment circumstances. Cognitive interviews were held with participants with "non-standard" qualifications and work circumstances. Testing was conducted across three rounds, with feedback from each round being used to amend the question for the next round.

In the first round of testing, the question was carried over from the previous test with no changes.

In the second round of testing, the question stem was amended so "Last week" became "In the last seven days". Testing on other questions found that respondents differed in how they interpreted "Last week", so this question was updated for consistency with the other questions.

Respondents were able to answer the question. However, some respondents felt negatively towards being repeatedly asked to explain why they were not in paid work.

Feedback from respondents during the 2019 Rehearsal (2019:15) showed that those who were retired often questioned why they were asked about having or starting a job. Therefore, we added some guidance to the electronic questionnaire, in the form of an accordion, to explain why this question is being asked.

Ever worked

In the 2011 Census, the "Ever worked" question included a write-in response for "Year last worked". This was used to differentiate respondents who are short-term unemployed (less than 12 months) and long-term unemployed (more than 12 months). These data are used in the derivation of the National Statistics Socio-economic classification (NS-SEC).

The 2015 topic consultation identified a low user need for data on year last worked. Response to this question in 2011 was low, and the high respondent burden of recalling the year last worked led to poor data quality. Therefore, we will not include a write-in response for 2021 Census. However, there was still a need to capture data on the long-term unemployed for the NS-SEC derivations. Therefore, the following changes were made to the 2011 question:

• The "Year last worked" write-in response was removed and replaced by two responses: "Yes – less than a year ago" and "Yes – more than a year ago".

• The response "No, have never worked" has been rephrased "No – I have never worked" for consistency with the "Yes" responses.

Testing the new response options

The new question design was tested (2017:12) to see if respondents understood the new response options. It was tested alongside a new Armed Forces question, and was the only labour market question in this test. Participants were all members of the armed forces or had members of the armed forces in their household. In the 2011 Census, respondents in work were routed around this question. However, in this test, all respondents were asked this question, regardless of employment status.

Most participants understood the question stem and considered it acceptable. Most understood "work" to mean paid employment, however some were unsure whether they should count volunteering or caring roles.

A large number of participants answered incorrectly when choosing between the two "Yes" response options, and there was very little consistency in how the response options were interpreted. The "No" response option was correctly understood.

Testing three variants of the question

We conducted cognitive testing (2018:1) on three variants of the question across three questionnaire versions. Informal interviews were conducted with ONS staff, who were asked to respond on behalf of a household member, family member or friend. This provided insight into people with a range of educational and employment backgrounds.

All three variants used the timeframe "the last 12 months" rather than "less than a year ago" to reduce confusion about the expected timeframe.

• Variant 1: A one-stage question based on the previous test.

• Variant 2: A two-stage question. The first stage asked "Have you worked in the last 12 months?" and the second stage asked "Have you ever worked?". Both questions had "Yes/No" response options.

• Variant 3: A one-stage question "Have you worked in the last 12 months?" with "Yes", "No", and "No, I have never worked" response options.

In general, participants preferred the first question variant. Some respondents were unsure if they should include volunteering, casual work or temporary work.

Before the next round of testing, the question stem was rephrased to "Have you ever done any paid work?" to make it clear that the response should not include unpaid work or volunteering.

Have you ever done any paid work?

• Yes, in the last 12 months

• Yes, but not in the last 12 months

• No, I have never worked

This question was tested (2018:6) in informal interviews with members of the public at a jobs fair. As in the previous test, some respondents were unsure whether they should include casual or temporary work. However, respondents were able to answer the question correctly.

Multi-round testing

Cognitive interviews (2018:14) were held with participants with non-standard qualifications and work circumstances. Testing was conducted across three rounds, with feedback from each round being used to amend the question for the next round.

The question from the previous test (2018:6) was brought forward. Participants found the question clear and easy to answer. No changes were made to the question across all three rounds of testing.

Electronic questionnaire testing

The online version of the question was tested (2018:32) through informal interviews conducted with members of the public at a job centre. There were no differences between the online question and the paper question.

Some participants interpreted the response options incorrectly. However, most respondents were able to understand the question and provided accurate responses.

We commissioned an external research agency to conduct an online survey (2018:37) to test the question. No problems were encountered with the question during this test.

Therefore, the question was included in the 2019 Rehearsal (2019:15) where no major issues were found.

Hours worked

Two variants of the "Hours worked" question were tested (2018:1) across three questionnaire versions. Participants were recruited from ONS staff, who were asked to respond on behalf of a household member, family member or friend. This provided insight into people with a range of educational and employment backgrounds.

The question stem and response options for both variants were identical to the 2011 questionnaire:

In your main job, how many hours a week (including paid and unpaid overtime) do you usually work?

[ ] 15 or less

[ ] 16 – 30

[ ] 31 – 48

[ ] 49 or more

The only difference between the variants was how the response options were presented:

• Variant 1: the response options were presented vertically, as in the 2011 Census

• Variant 2: the response options were presented horizontally

Participants found the horizontal layout easy to read, and some commented that it was a better layout than the vertical list. They felt that the existing categories worked well.

Respondents whose work hours varied from week to week found this question difficult to answer. There was no instruction for how many options can be selected, nor was there a time reference period. Some respondents assumed the question referred to the last week. Most respondents took an average, selected the highest response option, or ticked multiple response options.

Several changes were made to the hours worked question to improve accessibility and reduce respondent burden:

• The phrase "in your main job" was removed to simplify the question stem.

• The instruction to include paid and unpaid overtime was separated from the question stem.

• Response options were displayed horizontally.

• Hyphens were changed to the word "to" (for example, "16-30" was changed to "16 to 30") to conform to Government Digital Service Standards.

• The response "15 or less" was changed to "0 to 15", as some respondents interpreted the expected timescale as starting from 15 hours and would not count time worked under 15 hours.

The question brought forward for further testing was:

How many hours a week do you usually work?

Include paid and unpaid overtime

• 0 to 15

• 16 to 30

• 31 to 48

• 49 or more

Multi-round testing

Cognitive interviews (2018:14) were held with participants with non-standard qualifications and work circumstances. Testing was conducted across three rounds, with feedback from each round being used to amend the question for the next round.

Respondents understood the question, but respondents who were self-employed or had multiple jobs were unclear whether they needed to answer this for "job last week" or "main job".

For Round 2 of testing, "In your main job" was reinstated to the question stem.

While this clarified which job the question applied to, respondents who were self-employed or held multiple jobs were still unlikely to factor unpaid overtime into their calculation. Respondents on zero-hour contracts or flexible hours also found the question difficult to answer.

Electronic questionnaire testing

We commissioned an external research agency to conduct a small-scale online survey (2018:37) on the online version of the question.

As the electronic questionnaire only presents responses vertically, we included the word "hours" in each response option to improve accessibility and align the question with the unpaid care question.

The testing and the 2019 Rehearsal (2019:15) identified no issues with this question.

Question design for online

In addition to the testing described earlier, the economic activity and hours worked questions have undergone significant user experience (UX) testing (2017:2, 2018:2, 2019:1, 2020:2). UX testing focuses on understanding user behaviours as people interact with online services. Through observation techniques, task analysis and other feedback methodologies, it aims to develop a deep knowledge of these interactions and what it means for the design of a service.

UX research has taken place on a rolling basis since 2017. Before the Census Rehearsal in October 2019, 458 interviews were conducted at 99 events. The UX testing programme will continue through 2020. All participants are purposively selected to include a wide range of ages and digital abilities.

UX testing included various iterations of the questions described elsewhere in this report. Feedback from this research informed decisions made on the design of these questions. For more information on UX testing, see the question and questionnaire development overview for Census 2021.

Welsh language question development

Between 2017 and 2018, an external agency with Welsh-speaking researchers was commissioned to undertake focus groups (2017:17) and a series of cognitive interviews (2017:18). In 2018, further cognitive interviews (2018:40) were undertaken by the same agency. The qualitative research tested public acceptability and comprehension of amended and newly designed census questions in Welsh. The questions were tested with people across Wales with varying dialects and Welsh language proficiencies.

To ensure questions adhere to Cymraeg Clir guidelines some changes to the text or questions across the census questionnaires were translated by our contracted specialist Welsh language translation service provider. These changes were quality assured by the Welsh Language Census Question Assurance Group. This group includes Welsh language and policy experts from the Welsh Language Commissioner and the Welsh Government and was convened to give advice on the accuracy, clarity and acceptability of the language as well as other policy issues pertaining to the Welsh language and bilingual design.

The suite of questions in the Welsh language has changed since the 2011 Census, using phrasings and terms that are consistent with other questions on the questionnaires, adhering to Cymraeg Clir guidelines. We have also added digits "7" rather than "seven" to keep the translation simple and avoid mutations in the electronic questionnaire.

Employment activity last week

We added a new translation defining freelance, the definition text is underneath the response options "Self-employed or freelance" which reads "Mae ar eich liwt eich hun yn golygu eich bod yn hunangyflogedig ac yn gweithio i wahanol gwmnïau neu bobl ar ddarnau penodol o waith. Gair arall am hyn yw llawrydd." meaning "Freelance means that you are self-employed and work for different companies or people on particular pieces of work."

Reasons for not working

The changed English language response option "Studying" is translated to "Astudio".

We have amended the 2011 Census response option to the translation "Anabl neu sâl am gyfond hir", meaning "Long-term sick or disabled". The new translation meaning is more accurate to the English response option for Census 2021.

Looking for work

We have changed the question stem in Welsh to reflect the addition of the word "actively" in English, incorporating 'chi wrthi'n' in Welsh. The question stem reads "Yn ystod y 4 wythnos diwethaf, oeddech chi wrthi'n chwilio am unrhyw fath o waith am dâl?".

Availability to work

The changed English language question stem "Are you available to start work in the next two weeks?" is translated to "Ydych chi ar gael i ddechrau gweithio yn y pythefnos nesaf?".

Waiting to start a job

In line with the change to the English question stem, the question is translated to use the term "derbyn" meaning "accepted". The question stem reads "Yn ystod y saith diwrnod diwethaf, oeddech chi'n aros i ddechrau swydd roeddech chi eisoes wedi'i chael?".

Ever worked and Hours worked

The question design was translated to match the amendments made to the Census 2021 English design. The Welsh ever worked question design tested well (2018:14), no changes were recommended.

The hours worked question design tested well (2018:14), the instruction used the 2011 Census amended to meet Cymraeg Clir guidance. The response options used "i" (meaning "to") to adhere to Government Digital Service's Service Standards.

The design of the questions recommended for Census 2021 was informed by the research and testing detailed in this report. The questions and response options for Census 2021 have now been finalised through the census secondary legislation.

The images that follow show the questions on the electronic questionnaire. We have also published the paper questionnaires for Census 2021.

Guidance text and instructions are not part of the legislation, but we consider these to be finalised as well. However, it is possible that guidance text or instructions may change if there is enough evidence to support doing so.

Like all labour market questions, these questions are only asked if the respondent is aged 16 years or older.

Employment activity last week

Figure 1: Online question on employment activity last week in English and Welsh

Source: Office for National Statistics

Download this image Figure 1: Online question on employment activity last week in English and Welsh

.png (233.6 kB)

Reasons for not working

Respondents are only asked this question if they indicated they are not in paid employment by responding "None of these apply" to the question on employment activity last week.

Figure 2: Online question on reasons for not working in English and Welsh

Source: Office for National Statistics

Download this image Figure 2: Online question on reasons for not working in English and Welsh

.png (117.8 kB)

Looking for work

Respondents are only asked this question if they indicated they are not in paid employment by responding "None of these apply" to the question on employment activity last week.

Figure 3: Online question on looking for work in English and Welsh

Source: Office for National Statistics

Download this image Figure 3: Online question on looking for work in English and Welsh

.png (115.1 kB)

The accordion guidance reads:

Why we ask this question is you are retired or long-term sick or disabled.

To get a true picture of the working population, we ask this question of everyone who is not currently working.

We ask people who are retired because the number of people continuing to work after retirement is increasing.

We ask people who are not working due to long-term illness or disability because some intend to go back to work.

Availability to work

This question is only asked if respondents have indicated that they are looking for work in the previous question.

Figure 4: Online question on availability to work in English and Welsh

Source: Office for National Statistics

Download this image Figure 4: Online question on availability to work in English and Welsh

.png (56.4 kB)

On the paper questionnaire, the question is “If a job became available now, could you start it within two weeks?”.

Waiting to start a job

Figure 5: Online question on waiting to start a job in English and Welsh

Source: Office for National Statistics

Download this image Figure 5: Online question on waiting to start a job in English and Welsh

.png (112.5 kB)

The accordion guidance is the same as for the question on looking for work.

Ever worked

Figure 6: Online question on ever worked in English and Welsh

Source: Office for National Statistics

Download this image Figure 6: Online question on ever worked in English and Welsh

.png (81.1 kB)

Hours worked

This question is only asked if respondents indicate that they have done paid work.

Figure 7: Online question on hours worked in English and Welsh

Source: Office for National Statistics

Download this image Figure 7: Online question on hours worked in English and Welsh

.png (98.7 kB)

Respondents aged 16 years and over can request an individual access code or paper form if they wish to respond separately to the rest of their household. This enables people to answer the census privately, without having to tell the person completing the household form they have done so. Individual answers will override any answers submitted on the household form. This is vital to protect people's privacy and ensure good quality data.

There are small differences to the wording of the online questions depending on context. For example, if a respondent is replying on behalf of someone else, their name, rather than "you", is included in the question.

Question order

Within the suite of the labour market questions, the questions on economic activity are followed by questions about occupation, industry, hours worked and travel to work. The labour market questions follow the questions about qualifications as this, generally, reflects the chronology of events in respondents' lives.

Reasons for not working

Testing (2018:14) showed that people who are not currently working, particularly those who are either retired or long-term sick or disabled, struggled to answer the questions about looking for work and availability to work because they may have no intention of returning to work. We received negative feedback about the number of questions they had to answer before being able to state that they are retired or long-term sick. Therefore, the reasons for not working question has been moved directly after the employment activity last week question. This allows respondents who are retired, long-term sick or disabled to give their reason for not working before answering the questions on looking for work and availability to work.

Hours worked

In the 2011 Census, the question on hours worked came after the question on travel to work and was the final question on the individual questionnaire. The workplace address question has been split into multiple questions and required new routing instructions on the electronic questionnaire. As a result of these changes, the hours worked question has been moved to before the re-ordered questions on travel to work, where you mainly work, and workplace address.

Question evaluation

The question designs put forward in this report are based on extensive research and assessment using evaluation criteria that were set out in the publication The 2021 Census – Assessment of initial user requirements on content for England and Wales: Response to consultation (PDF, 796KB).

The evaluation considered the potential impact that including a topic on the census would have on data quality, public acceptability, respondent burden, financial concerns, and questionnaire mode. The evaluations were used in conjunction with the user requirements criteria to steer the development of the census questions and questionnaire.

A topic that has been assessed as having a "High" potential for impact is closer to the threshold for exclusion from the census than a topic that has been assessed as having a "Low" potential for impact.

After completing the research and development phase, we evaluated the recommended questions against the same criteria using an updated tool that considers the type of evidence we have available and the Census 2021 context. A description of this updated evaluation tool is provided in the question and questionnaire development overview for Census 2021.

All questions meet our thresholds to ensure reliable information will be collected in Census 2021.

For each question, the evaluation of risk of impact has been reduced to "Low", except for some evaluations of questionnaire mode which have remained as "Medium". We present the evidence used to assess questions as having a "Medium" potential for impact on the evaluation criteria in Annex 3.

Question harmonisation

As in previous censuses, there will be separate censuses in Scotland and Northern Ireland. The questions for England and Wales have been developed through close collaboration with National Records of Scotland (NRS) and the Northern Ireland Statistics and Research Agency (NISRA), who are responsible for conducting the censuses in Scotland and Northern Ireland, respectively.

We recognise that each country has its own user and respondent needs; however, we aim for harmonisation of census questions and topics where possible to produce UK-wide statistics that are consistent and comparable. The questions on economic activity and hours worked are the same for the censuses in England, Wales, Scotland and Northern Ireland.

The economic activity and hours worked questions were developed for use in the context of Census 2021 in England and Wales, a mandatory household form. Therefore, it is possible that in different contexts, such as social surveys, a different approach may be more suitable.

In the Census 2021 topic consultation response, the Office for National Statistics (ONS) made clear commitments to the public. We committed to continuing to ask about economic activity and hours worked. We made three further commitments.

Our commitments were:

• to ensure that the questions align with changing international definitions such as the International Labour Organization's (ILO's) 2013 Resolution concerning statistics of work, employment and labour underutilization and ensure that the labour market concepts remain consistent with those collected in the Labour Force Survey (LFS)

• to use the responses to the ONS Consultation on revising the Standard Occupational Classification 2010 (SOC2010) to inform the detailed quality requirements for the National Statistics Socio-economic Classification (NS-SEC) and whether this identifies any further scope for reducing the sub-topics needed to derive the current version of NS-SEC used in census outputs

• to explore how the online questions can improve the collection of the data while still meeting user needs

Ensuring that the questions align with changing international definitions and the concepts remain consistent with those collected in the LFS

We have ensured that any changes to questions provide outputs consistent with the LFS.

Following a change in the ILO definition of unpaid training as a form of economic activity, we removed the response "On a government sponsored training scheme" from the employment activity last week question.

Investigating any potential to reduce the data collected via the Census that is used in the derivation of NS-SEC

In the 2011 Census, respondents were asked to write in the year they last worked. These data were used to derive the NS-SEC definitions of long-term unemployed as someone who has worked, but not in the last 12 months. However, respondents who had not worked for some time found it difficult to answer and non-response was relatively high, which led to poor data quality.

We have revised the ever worked question response options to tick-boxes, in line with NS-SEC definitions: "Yes, in the last 12 months", "Yes, but not in the last 12 months" and "No, have never worked". This provides the data necessary for NS-SEC derivations and reduces respondent burden.

Improving data collection online while meeting user needs

Our development process, as outlined in this report, has aimed to improve the quality of data by making it easier for respondents to answer the questions. We have followed best practices, including Government Digital Service Standards, when designing and developing the electronic questionnaire. We present further details of new functionality in the electronic questionnaire designed to improve the online experience in the question and questionnaire development overview for Census 2021.

References to tests take the form (Year: Test number). “Year” refers to the calendar year the test was undertaken in and the test number is the position of the test within the year considering all testing that took place in that year. For example, the fifth test conducted in 2017 would be referenced as (2017:5).

A full description of each of these items can be found in summary of testing for Census 2021.

Evaluation of reasons for not working, waiting to start work and hours of work questions

Potential for impact of these questions on data quality, public acceptability, respondent burden, financial concerns and questionnaire mode was assessed as "Low".

Evaluation of employment activity last week question

Potential for impact of this question on data quality, public acceptability, respondent burden and financial concerns was assessed as "Low".

Potential for impact on questionnaire mode: "Medium"

This question contains sensitive information and answers could be subject to social desirability bias. This question also has mutually exclusive response options, which are only enforced online.

Evaluation of looking for work question

Potential for impact of this question on data quality, public acceptability, respondent burden and financial concerns was assessed as "Low".

Potential for impact on questionnaire mode: "Medium"

The online question has guidance text in an accordion and enforces the mutually exclusive response options.

Evaluation of availability to work question

Potential for impact of this question on data quality, public acceptability, respondent burden and financial concerns was assessed as "Low".

Potential for impact on questionnaire mode: "Medium"

This question has a slightly different question stem online and uses radio buttons to enforce the mutually exclusive response options.

Evaluation of ever worked question

Potential for impact of this question on data quality, public acceptability, respondent burden and financial concerns was assessed as "Low".

Potential for impact on questionnaire mode: "Medium"

The online question uses radio buttons to ensure only one answer is given and respondents are also required to answer this question before proceeding to the next set of questions.

Employed or unemployed

A person is employed if they:

• do paid work (either as an employee or self-employed)

• have a job they are temporarily away from

• are placed with employers on government supported training or employment programmes

• do unpaid work for their own or their family's business

A person is unemployed if they either:

• do not have a job, have actively sought work in the past four weeks, and are available to start work in the next two weeks

• have found a job and are waiting to start in the next two weeks

The definitions of employed and unemployed are the same as those used by the Labour Force Survey (LFS). This is to ensure the data are comparable.

Economically active or inactive

A person is economically active if they are aged 16 years or over and either employed or unemployed.

A person is economically inactive if they are aged 16 years or over, do not have a job, have not actively sought work in the past four weeks, and/or are not available to start work in the next two weeks.

The definitions of economically active or inactive are the same as those used by the LFS. This is to ensure the data are comparable.

Long-term unemployed

The Census uses the National Statistics Socio-economic classification (NS-SEC) definition of long-term unemployment. A person is considered long-term unemployed if they:

• have been unemployed for a period of 12 months or more

• have never worked