Counting residents and visitors question development for Census 2021

In December 2018, the government presented to Parliament a White Paper Help Shape our Future: The 2021 Census of Population and Housing in England and Wales (PDF, 967KB). This outlined the Office for National Statistics' (ONS) proposal for the enumeration base to remain the same for Census 2021 as for the 2011 Census. This maintains continuity and means that we will continue to collect information on:

• usual residents of the UK

• census short-term UK residents

• overnight visitors

Definitions for these groups can be found in Outputs and enumeration bases: residential address and population definitions for Census 2021. The remainder of this report discusses usual residents of the UK and Census short-term UK visitors together as "residents" for brevity as this report focusses on the division between these two groups and visitors. Identification of census short-term UK residents takes place in the migration questions, as described in the report Second address, migration and citizenship question development for Census 2021.

Census 2021 will be an online-first census, with a target of 75% online returns. Implementation of the correct enumeration base on the electronic questionnaire is crucial for the 2021 Census to be successful.

On paper, we recommend using the 2011 Census questions on counting residents and visitors in households as it was a successful design and ensures consistency with prior censuses.

Online, we will collect resident and visitor data in a way that meets Government Digital Service (GDS) Standards for the online questionnaire for Census 2021. As such, the structure of how this information is collected has changed. The overall user journey for the collection of names of residents is:

• questions about if the respondent is resident at the address and their name

• a question on the types of resident at the address

• questions about the names of other residents, collecting one name at a time

• a check of the count of residents to confirm that those temporarily away, or temporarily staying have been remembered

• a question on the types of overnight visitor staying on Census night

• questions about the names of overnight visitors, collecting one name at a time

• a summary of the information provided

The paper and online forms collect the same information, the names of all residents and all visitors, using questions optimised for the mode of completion.

Residents are asked individual questions on housing, citizenship, migration, second addresses, socio-demographic topics, qualifications, labour market and employment. Reports on these question topics can be found on the Question development pages.

For residents, a write-in box to capture middle names has been added on both paper and online questionnaires.

Overnight visitors are asked for name, date of birth, sex, and usual address only. They are not asked for middle name.

Since the publication of the White Paper, the Office for National Statistics (ONS) has conducted and concluded the final phase of testing on counting residents and visitors.

This report provides links to previously published research and the findings of additional testing that led to the final recommended questions for Census 2021 in England and Wales. The questions and response options for Census 2021 have now been finalised through the census secondary legislation: the Census (England and Wales) Order 2020, and Census Regulations for England and for Wales.

The evidence base for the design used in the 2019 Rehearsal is included in the section Research that led to the 2019 Rehearsal question design. The evidence base for the finalisation of the questions for Census 2021 is discussed in the section Research that led to the recommended Census 2021 question design.

In June 2015, the Office for National Statistics (ONS) held a formal, 12-week consultation process asking census users for their views on the topics that were required in the questionnaire in England and Wales. The aim of the consultation was to promote discussion and encourage the development of strong cases for topics to be included in Census 2021.

In May 2016, the ONS published its response to the 2021 Census topic consultation. This set out our updated view on the topics to be included in Census 2021, including:

• a summary of proposals for new topics
• next steps
• an overview of our plans

The 2021 Census topic consultation revealed a clear requirement to count and record household residents and visitors using the same definitions as in 2011. A detailed summary of the consultation responses relating to this topic can be found in the Outputs and enumeration bases report (PDF, 661 KB).

You can find more information about how we define who is a resident or visitor in the People to be counted section of the 'Output and enumeration bases: residential address and population definitions for Census 2021' report.

Following this, we began a comprehensive programme of research and development. We provide a full list of the tests used in the development of these questions in Annex 1. Further details are provided in the summary of testing for Census 2021. References to tests take the form (Year:Test number). For example, the fifth test conducted in 2017 would be referenced as (2017:5).

The tests utilised a range of qualitative and quantitative research methods. A short description of the different research methods and sampling techniques is given in the question and questionnaire development overview for Census 2021. Testing included respondents from a wide range of backgrounds and household types.

In December 2018 the government presented to Parliament a White Paper Help Shape our Future: The 2021 Census of Population and Housing in England and Wales (PDF, 967KB). This outlined our proposal to ensure that online functionality is designed to meet GDS Standards, minimises respondent burden and improves data quality.

The questions for Census 2021 are now finalised. We have evaluated the questions for their potential impact on data quality, public acceptability, respondent burden, financial concerns and questionnaire mode. We present details of this evaluation in Annex 3.

As part of our preparations for Census 2021, we held a census rehearsal in autumn 2019. The rehearsal provided an important opportunity for us to test the processes, systems, and services we will need to use for Census 2021.

To enable this, the rehearsal questionnaire had to be agreed earlier in the year. As the enumeration section of the form is crucial to ensuring Census 2021 is a success it was agreed that development of these questions would continue in parallel with preparations for the rehearsal.

Therefore, in the White Paper Help Shape our Future: The 2021 Census of Population and Housing in England and Wales (PDF, 967KB), we stated that we were finalising the questions for Census 2021.

Collection of names

Names have been collected in the census since 1841. In earlier censuses respondents entered their full name in one box. For the 2011 Census respondents were asked for the first time to enter their first name and last name in separate boxes.

2011 Census

In 2011, the name of respondents was collected in three locations in the paper questionnaire.

• in the household listing,

• in the relationship questions, and

• at the start of the questions about each person in the household.

Names of visitors were also collected. In all cases first and last name were collected.

On the electronic questionnaire each name was entered once, and then automatically inserted into further questions throughout the questionnaire. For example, into the relationship questions and visitor section.

Middle names

To meet an ONS operational need to improve the matching of census records to the Census Coverage Survey, and other data sources, we investigated collecting middle names. This was also requested during the topic consultation by some genealogists.

We asked respondents' views of the middle name question during cognitive testing of the population base and visitor questions (2018:33). Respondents that lived with family members knew middle names and were happy to provide them. Some respondents in family households entered it into the first name write-in box before being asked. They saw it as important and they were surprised that middle name was not on the form.

Middle names of non-family household members were not well known and were more difficult to provide in households with different cultural naming conventions.

For Census 2021, we will collect middle names of all residents. On the paper questionnaire it is collected at the start of the questions about each individual in the household. In the household listing and the relationship matrix no change has been made due to space constraints and the need to minimise respondent burden.

On the electronic questionnaire, a middle name field has been added to the household listing as this is the only place name is collected.

We will not ask for middle names of visitors.

Paper questionnaire household listing design

Our aim in developing the questions used to count residents and visitors was to collect this data in a way that was consistent with previous censuses. For this reason, on the paper questionnaire these questions have not changed since the 2011 Census.

There are some changes to the structure of the form, household paper questionnaire will collect up to five persons in the household. After that, a continuation questionnaire is needed for larger households. This is discussed in Question and questionnaire development overview for Census 2021.

Additionally, some instructional material on who to count has been moved from the questionnaire itself, into a separate guidance leaflet. You can find the household paper questionnaire in this collection. More information on the guidance leaflet is provided in Annex 2.

Online questionnaire resident listing design

Census 2021 is online-first, so we need to ensure that we count residents and visitors on the electronic questionnaire in the best way possible. Since the last census in 2011 digital technology has changed greatly. We need to ensure our online functionality minimises respondent burden while ensuring data quality is maximised.

Prior to the 2019 Rehearsal our research and testing investigated three approaches for the online questionnaire. These were:

• Approach 1: 2011 Census design

• Approach 2: Collect names of residents split across 4 types of resident

• Approach 3: Collect names of residents split across 3 types of resident

The first approach was tested in the 2017 Census Test. The third approach was used in the 2019 Rehearsal.  Two further approaches were tested following the 2019 Rehearsal, these are discussed in the section Research that led to the recommended Census 2021 question design.

User experience (UX) testing

The questions have undergone significant user experience (UX) testing (2017:2, 2018:2, 2019:1, 2020:2). UX testing focuses on understanding user behaviours as people interact with online services. Through observation techniques, task analysis and other feedback methodologies, it aims to develop a deep knowledge of these interactions and what it means for the design of a service.

UX research on the online questionnaire has taken place on a rolling basis since 2017. Before the Census Rehearsal in October 2019, 458 interviews were conducted at 99 events. The UX testing programme continued in 2020 and has now ended. All participants were purposively selected to include a wide range of ages and digital abilities and understanding of the questions counting residents and visitors was a focus of these interviews.

UX testing included various iterations of the questions described elsewhere in this report. Feedback from this research informed decisions made on the design of these questions.

Approach 1: 2011 Census design

We began our testing programme by conducting usability testing (2016:1 and 2016:3) of the 2011 Census electronic questionnaire on mobile devices. Online completion is expected to be a frequent method of completion in Census 2021.

In this design the questions are presented in the same way as they are on the paper form, leading to a large volume of text on some pages. Additionally, all names were collected on a single page, with the respondent selecting a button to add another for each new name.

The findings showed that the questions aimed at counting household residents were capturing incorrect data. For example, some participants did not include themselves in the count. The response options caused confusion as participants felt that there was too much information to read and some were unhappy about the amount of scrolling. Several participants were not able to complete the questionnaire on their mobile device without interviewer intervention.

These findings were replicated in the 2017 Test  which used a design very similar to the 2011 Census. The names of everyone living in the household were collected on the same page by using the plus symbol button.

This increased the complexity of the task on small screens and caused unnecessary burden, particularly for people with low digital skills. For instance, adding all names on the same page led to long pages for large households. People also put the same person in more than once as they could not recall who they had already added.

When intending to move to the next screen, some people mistakenly pressed the plus symbol button again, they were then unable to close the name and move on.

In addition, people with more complex living arrangements, such as those with lodgers, or whose children lived with another parent part of the time, were not sure who to count.

The same issues were encountered during further user testing of the 2017 Test design (2018:2 and 2019:1).

Based on these findings, and in line with GDS guidelines, we suggested to improve the quality of data collected by:

• collecting one name per page only

• replaying the names entered so the household would "build" as the person progresses through the form

• include the definitions provided as supplementary information on the paper questionnaire's accompanying guidance leaflet

In addition, throughout this testing, there was low public acceptability for asking about visitors. Separately to the work on developing the questions, we conducted work to understand why the visitor questions were not acceptable.

Collect names of residents split across types of resident

Based on feedback from the 2017 Test and user experience testing, two further approaches for counting household residents and visitors were tested ahead of the 2019 Rehearsal. We cognitively tested (2018:33 and 2018:42) both designs in three phases.

Both versions removed the check box question on the types of resident and the question on number of residents as there is no data need for these questions. They also collected names over multiple pages to reduce the amount of information on each screen:

• Approach 2: A four-stage design with questions like "Do you usually live here?", "Who else lives here?", "Is anyone away?" and "Is anyone staying temporarily?".

• Approach 3: A three-stage design with questions like "Do you usually live here?", "Who else lives here?" and "Apart from those included is there anyone else who should be added?".

In these designs the intent was that the later stages act as reminders to prompt for household members that may have been forgotten.

In both approaches the initial questions were the same and were well understood. These questions remained largely the same throughout all iterations of testing, though the accompanying guidance was further developed to aid understanding of the concept "usually live"

As personal information should only be collected where needed, the first question asked is "Do you usually live at [the address]?" with the answer options "Yes, I usually live here" and "No, I don't usually live here". This allows those completing the form on behalf of someone else to be routed around the subsequent question "What is your name?".

Identifying that the respondent lived there and collecting their name ensured that the participant had invested in the survey before being asked questions about other household members.

Approach 2: Questions split across four types of resident

Testing (2018:33) found that in most cases the household was captured in the first two stages ("Do you usually live here" and "Who else lives here"), thus participants reported a lot of repetition.

The question on "Who else lives here" was also generally understood, however, there were some participants who had correctly added all household members at the first and second question, who found the third and fourth questions confusing. Some deleted household members after reading them, others added them again and caused overcount.

Based on feedback from this phase of testing this approach was rejected. It was also recommended that:

• respondents should be able to edit and remove household members more easily (in advance of a summary page at the end of the section).

• extra guidance should be made available regarding who should be included, and which address should be captured in an accordion.

The third stage, asking about anyone that was missed, was retained as this was needed to help those with more complex households to include all residents.

Approach 3: Questions split across three types of resident

The second and third phase of the cognitive testing (2018:42) focused on the three-stage design.

As in the previous round of cognitive testing (2018:33), participants felt that the questions were repetitive where they had successfully included all household members in the first two stages. This increased burden and caused confusion.

In addition, the stages of the question continued to be misinterpreted as trying to distinguish between permanent and temporary household members. This resulted in frustration and overcount.

The summary of names on each question page helped participants to ensure their responses were correct but did not eliminate the confusion caused by the questions.

Online questionnaire overnight visitor listing design

The purpose of the visitor questions is to ensure that everyone is counted in the census. All visitors should be included, even if they have been included on a census questionnaire at another address. We use the information on visitors solely to produce statistics on the size of the population.

As with the resident listing, initial UX testing (2017:2) removed the questions on type of and number of visitors as these are not needed from a data perspective. However, it was quickly shown that the type of visitor question was needed to separate the resident suite of questions from the visitor questions, so the question was reinstated as a check box question with the same options as on the paper form question H4.

Although this did help to distinguish between residents and visitors, and identify who might be a visitor, participants continued to find the distinction confusing. In addition, the purpose of these questions was questioned, with participants being reluctant to provide information on their guests. To understand the broader level of acceptability when asking residents about any visitors staying at their home on census night, we asked these questions in a small-scale nationally-representative quantitative survey in England and Wales (2018:34).

Two-thirds (66%) of respondents felt it was acceptable to ask about overnight visitors.

Approximately half of respondents who were uncomfortable with the visitor question felt that there was nothing that ONS could do that would make them comfortable answering this question.

However, approximately a fifth of those with concerns would find more information on why the question was asked helpful. Others were worried about including visitors, as they felt this would be creating "overcount" as they would also be included at their home address if this were in the UK.

We addressed these issues in the design for the online questionnaire by the inclusion of a guidance accordion which explained why this information was needed, what the data would be used for, and that double counting of people was not a risk.

This is not achievable on the paper questionnaire because of space constraints.

Finalising the 2019 Rehearsal design

The 2019 rehearsal design was a refinement of Approach 3. The overall user journey for the collection of names of residents in the 2019 Rehearsal was:

• Question 1: Do you usually live at [address]?

• Question 2: [If "Yes" at Q1] What is your name?

• Question 3: Does anyone else live at [address]?

• Question 4: [If "Yes" at Q3] Who do you need to add to [address]? [Returns to Q3]

• Question 5: [If "No" at Q3] Apart from the people already included, is there anyone else who is temporarily away or staying that you need to add to [address]?

• Question 6: [If "Yes" at Q5] Who do you need to add to [address]? [Returns to Q5]

Once the respondent selected "No" at Question 5 they moved onto the questions on visitors. Throughout these questions, as requested by respondents in testing, all the household members already added are listed at the top of each page. This means the respondent can see who they have added. Next to each listed name are options to change the name or to remove the name. This allows spelling errors, or mistakenly added entries to be corrected immediately.

The overall user journey for collection of visitor information in the 2019 Rehearsal was:

• Question 7: Are there any visitors staying overnight on 13 October 2019 at [address]?

• Question 8: [If not "No" at Q7) What is the name of the visitor staying overnight on 13 October 2019 at [address]?

• Question 9: Are there any other visitors staying overnight on 13 October 2019 at [address]? [If "Yes" at Q9 returns to Q8]

As with residents, the names already input are listed at the top of each page with options to change or remove. Once the respondent selects "No" at Question 9 they moved onto a summary page:

• stating how many household members they have added

• listing the names of the household members

• stating how many visitors they have added

• listing the names of the visitors

This approach overcame some of the key issues with the 2011 Census design:

• collection one name per page allowed the respondent to have a sense of progressing and works well on a mobile device.

• constantly replaying the names input in a list at the top of each page, with options to change or remove each entry, allows respondents to see who is already input and to immediately notice and correct any mistakes.

However, other key issues remained. Many participants found the third stage questions on residents confusing, and those with more complex living arrangements remained unsure who to count.

Therefore, it was agreed to continue development of these questions after the 2019 Rehearsal design was finalised. This allowed an additional year of development to take place before making a recommendation for Census 2021.

While the 2019 Rehearsal design worked well for those with simple living arrangements, there were continued issues for those with more complex living arrangements.

The first stage of the further work was a quantitative test (2019:9). This was a randomized split sample quantitative test of households that may be more likely to make an error in deciding who should be included as a member of their household. Participants completed one version of the online form, then participated in a telephone follow-up interview. The aim was to collect the household information twice, and therefore identify any errors made during the electronic form completion.

Purposive sampling was used to recruit 415 respondents from groups where there was a high risk of making an error in deciding who to include. These included:

• unrelated households with high churn (student house shares, or single professional house shares, but not halls of residence or care homes)

• related households with additional unrelated residents (for example a lodger, friend, au pair)

• households with people living away during term-time for study (for example a university student, or child living away at boarding school)

• households with people living away for work (for example living in London during the week, or living in an armed force base)

• households with residents that have more than one permanent or family home (for example, children with parents who live at different addresses)

The two designs tested were a further refinement of 2019 Rehearsal design and a fourth approach.

The first two questions of both designs were identical in information captured as these had performed well in earlier testing. These were:

• Question 1: Do you usually live at [address]?

• Question 2: [If yes at Q1] What is your name?

The next steps in the suite of questions differed between refined 2019 Rehearsal design and the approach four design.

Refined 2019 rehearsal design

In the refined 2019 Rehearsal design, guidance on screen at Question 3 "Does anyone else live at [address]?" was embedded into the answer "Yes, I need to add someone" as subtext on the response option. This guidance was taken from the 2011 Census H1 response options 2 to 5 and read "Include partners, children, babies born on or before [date of completion of this survey], housemates, tenants and lodgers, students and schoolchildren who live away from home during term time, where this is their permanent or family home".                                                         

Question 4 then asks, "Who do you need to add to [address]?" and provides space to enter the full name of the household member.

Question 5 is "Apart from the people already included, is there anyone else who is temporarily away or staying that you need to add to [address]?" Guidance on screen at this question was taken from response options 6 to 9 of the paper form H1 response options and was grouped into those temporarily staying and those temporarily away. The guidance was:

"Include people who are temporarily away

• People who work away from home within the UK if this is their permanent or family home

• Members of the armed forces if this is their permanent or family home

• People who are temporarily outside the UK for less than 12 months

• Other people who usually live here but are temporarily away

Include people who are temporarily staying

• People staying temporarily who usually live in the UK but do not have another UK address for example, relatives, friends

• People who usually live outside the UK who are staying in the UK for 3 months or more"

In the visitors' section, the questions were largely the same as implemented in 2019 Rehearsal.

Approach 4: Questions split across three types of resident with type of resident and count of resident questions

In Approach 4, the multi-tick paper form question H1 was reinstated after Question 2. However, it was split over two questions, firstly items 1 to 5 then items 6 to 9. Participants were then asked for the count of residents, reinstating the H2 count question on the paper form. Then they enter a loop base system to collect names on all the household members.

In the visitors' section, the questions were largely the same as implemented in 2019 Rehearsal except the paper form visitor count question H4 was reinstated.  

Quantitative test findings

The quantitative test (2019:9) investigated coverage error. Under-coverage error is people that should have been included but were not. Over-coverage error is people that were included but should not have been.

Results showed that the household level coverage error (any under- or over-coverage error, or both) was approximately 21% for both designs.

• there was evidence of higher household level under-coverage error for the Approach 4 design compared to the 2019 Rehearsal design, although this was not statistically significant.

• there was evidence of higher household level over-coverage error for the 2019 Rehearsal design compared to the Approach 4 design. Again, this was not statistically significant

The proportion of households with both under- and over-coverage errors in the test was approximately the same for both designs (3.3% to 3.4%).

The mean size of coverage error within a household (that is, mean number of people under- and over-covered) was also similar across designs at 0.4.

The median interview length was about five minutes for the 2019 Rehearsal design and about six minutes for the Approach 4 design.

The use of the edit functions to change a name was used 16 times (7.6%) in the 2019 Rehearsal design and eight times (3.9%) for the Approach 4 design.

We concluded that neither design was sufficiently developed to collect the high-quality data required for Census 2021. In both designs, there was not yet sufficient explanation of who should and should not be included, and the staged design still led to confusion over who to include.

Approach 5: Questions split across two types of resident with type of resident and check of count questions

We therefore developed a fifth approach which adopted the areas that worked best from both designs:

• identifying if the participant was resident, and if relevant their name, as the first stage

• collecting other household members names one at a time

• listing who has been added with options to edit or remove people throughout the name collection

• including an end summary page to name and count residents and separately visitors

Areas of development were:

• further guidance information was embedded into the questions where needed to aid respondents in deciding who to include.

• reinstating H1 from the paper form, but with some tick-boxes grouped to reduce scrolling.

• developing the name collection questions

• rephrasing the third stage question to be a confirmation of the count of people living there, rather than a collection of people that may have been missed at stage two

• Distinguishing visitors from residents

These changes were tested and further developed over two more waves of cognitive testing (2019:14, 2019:17). Purposive sampling was used to recruit 22 participants and 16 participants, respectively.

As for the quantitative testing, this testing focussed on households which are more likely to make errors in the "Who usually live here?" section. For example, large households and those with a second address.

Adding guidance information

The initial information page, that precedes the questions on who lives here and who is visiting, was developed to provide detailed definitional information on who to count as living in the household. For example, students, armed force members and people with a second address for work. This is crucial information for respondents to understand and is equivalent to the information provided in the information leaflet accompanying the paper form (shown in Annex 2).

In addition, we added question specific guidance in subsequent questions. For example, in the "Do you usually live at [address]?" question we added the instruction "If you spend exactly half your time at this address, and half at a second home address, include yourself at the address where you were staying overnight on 21 March 2020".

When the information page was read, it was understood (2019:14). However, participants did not retain this information well enough to apply it later in the form. Further, because they had already read the information page, participants did not read the question specific guidance. The perceived it as a duplication of what they had already read.

It was concluded that the information page should not be used to provide detailed definitional information, but only to note that information on residents, and separately information on visitors, would be collected in this section. The definitional information must be provided at the points where needed, throughout the question set. This is crucial to enable the respondents to access and apply the required information without being overloaded with information.

As such, this page only retains the definition of a household, as this is the first point the term is used, and the respondent needs this information to decide if they need to order a second household access code.

Reinstating the H1 tick-list

The paper form multi-tick question H1 provides a lot of definitional information on who to include. Including this information as instructions was not successful in previous approaches, so the question was reinstated but in a reduced form. This allowed the information to be provided, but without having to scroll through ten long response options.

To enable this, we changed the question "Does anyone usually live at [address]?" to "Do any of the following groups of people usually live at [address]?", and added definition text on who to include in this question. The five response options initially provided if the participant was resident were:

• family members

include partners, babies born before [census date], students, children

• housemates, tenants, or lodgers

• people currently away

for example, on holiday, working in the Armed Forces, abroad for up to a year, living in a care home for up to six months

• people currently staying who don't have another UK address

for example, UK residents with no home, people staying in the UK for three months or more"

or

• none of the above, no-one usually lives here

If the respondent was not resident at the address, the final response option was "None of the above, I am the only person who usually lives here". Testing (2019:14) showed that in some cases the text had been reduced too much, and some of the detail needed to be reinstated or moved.

In the response option "Family members"

• the subtext to include "partners" was not noticed, leading to confusion as to where to include this group. Therefore, we rephrased the first line of the response option to "Family members and partners"

• the subtext to include "students" was misunderstood to include any students here, even if not family, and was perceived as contradictory; therefore, we reinstated the full statement "students and schoolchildren who live away from home during term-time" to make it clear who was being referred to here

In the response option "People currently away" it was not clear to respondents if those away for work should be included. Therefore, the example "working away" was added to the subtext.

The response option "People staying temporarily who usually live in the UK but do not have another UK address" was not well understood and residents of this type were missed. As this response option covers two key groups who are not automatically considered to be resident by the respondent, we split this back into two separate response options. One for people who usually live outside the UK but who are staying for three months or more. And one for people staying temporarily who usually live in the UK but do not have another UK address.

The second round of cognitive testing (2019:17) led to the recommendation that the latter of the response options should have subtext to further describe who is included in this group, "for example, UK residents between addresses or currently without a home".

Developing the name collection questions

Due to the addition of the question on types of household member, the subsequent question "Who do you need to add?" was no-longer understood as intended (2019:14). Some respondents thought it was a continuation of the question on types of resident. We therefore changed the question to "Who [else] lives at [address]?".

In the final round of cognitive testing (2019:17) no blank household members were created and where duplicate household members were created in large households, the error was seen and the change and remove option was used to correct it.

Although adding one name per page is repetitive for larger households the design keeps the respondent moving, works on a mobile device, ensures names are replayed in a list at every additional page, and means respondents can edit names throughout the process. These were all consistently found to be respondent needs during UX testing (2017:2, 2018:2, 2019:1). No participants deliberately omitted household members to speed the process.

Confirming the count of residents

All previously tested staged designs caused confusion to respondents who interpreted the stages as intending to group residents, rather than to check if anyone had been missed. Therefore, we changed the final question into a check of the count of residents. We included detailed definitional information as instructions, reminding the respondent of who should be counted, rather than within the question stem.

The question stem was "You said [count of names] people live at [address]. Is this correct?". With response options of "Yes", and "No, I need to add someone".

This design was interpreted as intended by respondents (2019:14), as a check that they had input the correct information in the previous questions and considered logical.

However, in the first round of cognitive testing the response options were misinterpreted, as previous entries into the name collection loop had been on a "Yes" response. Respondents automatically followed the previous pattern, leading to confusion.

We therefore rephrased the question to allow a "Yes" response to mean that another person needs to be added and make it clear what the action on the page is. The new question was "You said [count of names] people live at [address]. Do you need to add anyone?"

In the next round of testing (2019:17), respondents in complicated households could complete this question accurately and questions were correctly interpreted as encompassing full and part-time members of the household.

Distinguishing visitors from residents

In Approach 4, as with previous iterations, there was confusion between who is a household member and who is a visitor when participants were asked "Are there any visitors staying overnight on 21 March 2021 at [address]?" with multi-tick response options denoting the types of visitor. This was most prevalent amongst residents that were:

• people who usually live outside the UK who are staying in the UK for three months or more

• people staying temporarily who usually live in the UK but do not have another UK address

In some cases (2019:14), people who had been correctly included as household members, were later removed and added as visitors instead; leading to undercount. Participants also struggled to link the date within the question stem, which was amended to a date relevant to the testing, to a specific day.

To address this, we:

• rephrased the question to explicitly exclude those already counted, and to use the term "visitors" in the instructions rather than the question stem

• continued to include the multi-tick response options listing the different types of visitor that should be included as this had been helpful in previous testing

• included the day of the week, Sunday, when referencing the date to help participants relate it to a specific day

The final question therefore read, "Apart from everyone already included, who else is staying overnight on Sunday, 20 March 2021 at [address]?" with the instruction "These people are counted as visitors".

Following the cognitive testing we agreed the final recommended questions for Census 2021, including question variants for when the person completing the form does not usually live at the address. Ongoing UX testing (2020:2) continued to focus on these questions, however no further major issues were encountered. These questions were also a specific focus of work looking at the impact of the coronavirus (COVID-19) pandemic on respondents' answers (2020:3, 2020:4), and the existing guidance was found to be sufficient.

Ordering the questions

The section on counting household residents and visitors is the first in the questionnaire, as it was in the 2011 Census. On paper the questions within the section remain in the same order as in 2011.

Online the structure of how this information is collected has changed. The overall user journey for the collection of names of residents is:

• Question 1: Do you usually live at [address]?

• Question 2: [If "Yes" at Q1] What is your name?

• Question 3: Do any of the following people also live at [address] on Sunday 21 March 2021?

• Question 4: [If not "None" at Q3] Who else lives at [address]?

• Question 5: Does anyone else live at [address]? [If "Yes" returns to Q4]

• Question 6: [If "No" at Q5] You said [count of names] people live at [address]. Do you need to add anyone? [If "Yes" returns to Q4]

Once the respondent selects "No" at Question 6 they move onto the questions on visitors. The overall user journey for collection of visitor information is:

• Question 7: Apart from everyone already included, who else is staying overnight on Sunday 21 March 2021 at [address]?

• Question 8: [If not "There are no visitors" at Q7] What is the name of the visitor staying overnight on Sunday 21 March 2021 at [address]?

• Question 9: Are there any other visitors staying overnight on Sunday 21 March 2021 at [address]? [If "Yes" at Q9 returns to Q8]

Within the questions about the visitors, the visitor's sex question is asked before visitor's date of birth question, to align the order of questions with the individual question ordering.

Welsh language question development

Between 2017 and 2018, an external agency with Welsh speaking researchers was commissioned to undertake focus groups (2017:17) and a series of cognitive interviews (2017:18). In 2018, further cognitive interviews (2018:41) were undertaken by the same agency. The qualitative research tested public acceptability and comprehension of amended and newly designed census questions in Welsh. The questions were tested with people across Wales with varying dialects and Welsh language proficiencies.

To ensure questions adhere to Cymraeg Clir guidelines some changes to the text or questions across the census questionnaires were translated by our contracted specialist Welsh language translation service provider. These changes were quality assured by the Welsh Language Census Question Assurance Group. This group includes Welsh language and policy experts from the Welsh Language Commissioner and the Welsh Government and was convened to give advice on the accuracy, clarity and acceptability of the language as well as other policy issues pertaining to the Welsh language and bilingual design.

Testing (2017:18 and 2018:40) revealed that most participants were unfamiliar with the term "cydletywyr" ("housemates"). Some guessed its meaning, but most assumed it meant "cohabitants". It was recommended to use an alternative word or phrase such as "people who share a house with you" or to keep the previous term and ensure it is fully explained in a guidance note.

There were a few other words in this sequence of questions unfamiliar to some, such as "Deyrnas Unedig" ("United Kingdom") and "Lluoedd Arfog" ("Armed forces") and "ychwanegu" ("add"). However, the context of the question provided enough information for participants to work out the meaning.

We also carried out end-to-end user testing of the Welsh online questionnaire (2019:11) to explore the impact of piping an individual's name into the question wording. We examined the questionnaire for consistent grammar, comprehension and respondent burden in Welsh compared to the English and to ensure participants were able to accurately complete the questionnaire with no errors or unnecessary navigation.

The end-to-end test findings were positive. The only issue raised was that the participants had uncertainty around whether to use a formal name or their nickname as it is common for the Welsh community to use nicknames only. It was suggested for there to be an instruction to use only formal name, however there was not a high enough need to include this instruction on the Welsh online questionnaire.

In the 2019 Rehearsal respondents in Wales could complete the questionnaire in English or Welsh. Since the 2019 Rehearsal the suite of questions have been re-translated and assured by the Assurance Group to reflect the changes in design for Census 2021.

The design of the questions recommended for Census 2021 was informed by the research and testing detailed in this report. The questions and response options for Census 2021 have now been finalised through the census secondary legislation.

The images that follow are from the electronic questionnaire. We have also published the paper questionnaires for Census 2021.

All questions on counting residents and overnight visitors are mandatory fields, meaning respondents cannot proceed through the online questionnaire or submit the questionnaire without entering data for this suite of questions. An error message will appear if no data is entered and the respondent selects to continue to the next question.

Mutually exclusive functionality is applied to the questions shown in Figure 4 and Figure 8, where respondents can select any of the first set of response options or the last response option only.

Interstitial page and respondents' details

Figure 1: Information page on residents and overnight visitors, in England and Wales, in English and Welsh

Source: Office for National Statistics

Download this image Figure 1: Information page on residents and overnight visitors, in England and Wales, in English and Welsh

.png (134.6 kB)

The text within the guidance accordion 'What we mean by "this household"' is"This is one person living alone or a group of people, not necessarily related, living at the same address, who share cooking facilities and share a living room or sitting room or dining area".

Any other household at this address will need to request a new household access code to start their own census.

Figure 2: Online question on if respondent is a resident at the address, in England and Wales, in English and Welsh

Source: Office for National Statistics

Download this image Figure 2: Online question on if respondent is a resident at the address, in England and Wales, in English and Welsh

.png (95.0 kB)

The text within the guidance accordion 'What we mean by "usually live"'is:

"This is often your permanent or family home.

"If you have more than one address, include yourself at the home address where you generally spend most of your time. If you split your time equally then use the home address where you are staying overnight on Sunday 21 March 2021.

"Students, include yourself at both your term-time and out of term-time addresses.

"People with a second address for work, include yourself at your home address.

"Armed forces members, include yourself at your home address if you have one."

If the respondent selects "No, I don't usually live here" they are routed past the question on name seen in Figure 3. In addition, subsequent questions are amended slightly to reflect the fact that the respondent does not live there.

Figure 3: Online question on respondents’ name, in England and Wales, in English and Welsh

Source: Office for National Statistics

Download this image Figure 3: Online question on respondents’ name, in England and Wales, in English and Welsh

.png (34.6 kB)

Types and names of other residents

Figure 4: Online question on types of resident living at the address, in England and Wales, in English and Welsh

Source: Office for National Statistics

Download this image Figure 4: Online question on types of resident living at the address, in England and Wales, in English and Welsh

.png (456.6 kB)

If the respondent does not live at the address, the word "also" is removed from the question stem and the final response option reads "None of these apply, no one usually lives here (for example, this is a second address or holiday home)".

Text for the guidance accordion is the same as for the question "Do you usually live at [address]?" except references to "your" are replaced with "their".

If respondents select any of the first five response options, they are routed to "Who else lives at [census address]?" (Figure 5), where they can enter names of the other household members.

If respondents select the response option "None of these apply, I am the only person who usually lives here", they are routed to the resident count confirmation question "You said one person lives at [census address]. Do you need to add anyone?" seen in Figure 7.

Figure 5: Online question on adding resident names, in England and Wales, in English and Welsh

Source: Office for National Statistics

Download this image Figure 5: Online question on adding resident names, in England and Wales, in English and Welsh

.png (63.5 kB)

If the respondent does not live at the address the word "else" is removed from the question stem the first time they loop through this question.

Figure 6: Online question on resident addition loop, in England and Wales, in English and Welsh

Source: Office for National Statistics

Download this image Figure 6: Online question on resident addition loop, in England and Wales, in English and Welsh

.png (78.1 kB)

If respondents select the response option "Yes, I need to add a [numeric position] person" they are routed back to the question "Who else lives at [census address]?" where they can enter the name of the second household member. After this, the respondent begins the loop again, and is asked "Does anyone else live at [census address]?".

For each name and household member added, their names will be replayed above the response options. Respondents can check the names entered and the number of household members entered. If any names are entered incorrectly, respondents can select "Change". The respondents are then routed to the page where they entered the individuals name, they can change the name and continue in the loop again.

Once all residents are added, the respondent selects the response option "No, I do not need to add anyone".

Resident count confirmation question

Figure 7: Online resident count confirmation question, in England and Wales, in English and Welsh

Source: Office for National Statistics

Download this image Figure 7: Online resident count confirmation question, in England and Wales, in English and Welsh

.png (288.3 kB)

The text within the guidance accordion "Why we ask this question" is "We ask this question to ensure that everyone is correctly counted in the census. This includes people who are staying temporarily or are away. The information is vital for planning services, whether it is for hospital beds, school places, GP and dental services or where to build houses and roads".

The question will show how many people have been entered through the household resident suite of questions. For example, if five people in total have been added through the user journey the check question will be "You said 5 people live at [census address]. Do you need to add anyone?", and the "No" response option will be "No, there are 5 people living here". Names of these five residents are displayed below the question, and respondents can amend these using the "Change" and "Remove" functions.

If respondents select the "Yes, I need to add someone" they are routed to "Who else lives at [census address]?" question (Figure 5).

If respondents select "No, there are [count] people living here", they are routed to the overnight visitor section (Figure 8).

Types and names of overnight visitors' questions

Figure 8: Online question on overnight visitors, in England and Wales, in English and Welsh

Source: Office for National Statistics

Download this image Figure 8: Online question on overnight visitors, in England and Wales, in English and Welsh

.png (384.9 kB)

If respondents select "There are no visitors staying overnight here on Sunday 21 March 2021", they are routed to the summary page (Figure 11).

The text within the Guidance accordion "Why we ask about visitors" is "This is to ensure that everyone is counted in the census. Add any visitors, even if they have been included on a census questionnaire at another address."

This guidance accordion is shown on all the visitor questions.

Figure 9: Online question on name of visitors, in England and Wales, in English and Welsh

Source: Office for National Statistics

Download this image Figure 9: Online question on name of visitors, in England and Wales, in English and Welsh

.png (113.1 kB)

Figure 10: Online question on any other visitors, in England and Wales, in English and Welsh

Source: Office for National Statistics

Download this image Figure 10: Online question on any other visitors, in England and Wales, in English and Welsh

.png (97.8 kB)

If respondents select the response option "Yes, I need to add a [2nd] visitor" they are routed to the question "What is the name of the [2nd] visitor staying overnight on Sunday 21 March 2021?" where they can enter the name of the second visitor. After this, the respondent begins the loop again, and is asked "Are there any other visitors staying overnight on Sunday 21 March 2021 at [census address]?".

Functionality, such as change and remove options, and piping of the visitor number is as per the question "Does anyone else live at [address]?"

Once all overnight visitors are added, the respondent selects the response option "No, I do not need to add anyone" where they exit the loop and continue to Figure 11, the summary page.

Summary page

Figure 11: Online question summary page for residents and overnight visitors, in England and Wales, in English and Welsh

Source: Office for National Statistics

Download this image Figure 11: Online question summary page for residents and overnight visitors, in England and Wales, in English and Welsh

.png (144.9 kB)

Names of the residents and overnight visitors are displayed in this screen. Respondents again, have the function to change or remove individuals.

If there are no residents entered the summary page is "You said 0 people are living here on Sunday 21 March 2021", underneath where names are replayed it reads instead "There are no householders". Respondents can also "Add someone to this household" on this screen.

If there are no visitors entered the summary page is "You said 0 visitors are staying overnight here on Sunday 21 March 2021", underneath where names are replayed it read instead "There are no visitors". Respondents can also "Add a visitor" at this screen.

Each resident or visitor name added creates a section of the form for that individual. Visitors are only asked:

• what is [visitor name]'s date of birth?

• what is [visitor name]'s sex?

• what is [visitor name]'s usual address?

The usual address question has two response options.

• If respondents select "An address in the UK" they are asked to enter the visitors usual address - "What is [visitor name]'s usual address in the UK?". This question has address look-up functionality.

• If respondents select "An address outside the UK", they are asked to enter the current name of the country they usually live in - "In which country outside the UK does [visitor name] usually live?". This question has search-as-you-type functionality.

On each visitor question there is a guidance accordion "Why we ask for visitor details". These state

"Your answer helps to ensure that everyone is counted in the census. Add visitor details, even if you think they have been included on a census questionnaire at another address".

Question evaluation

The question designs put forward in this report are based on extensive research and assessment using evaluation criteria that were set out in the publication The 2021 Census -- Assessment of initial user requirements on content for England and Wales: Response to consultation (PDF, 796KB).

Assessment of these questions was not published during the consultation process as they are collected for operational purposes. As the final recommended questions for Census 2021 are significantly different to those used in the 2019 Rehearsal and have not been quantitatively tested it is not possible to conduct a full evaluation using the evaluation tool described in the question and questionnaire development overview for Census 2021.

Instead the 2019 Rehearsal questions have been evaluated. The evaluation considers the potential impact that the questions have on data quality, public acceptability, respondent burden, financial concerns, and questionnaire mode.

A topic that has been assessed as having a "High" potential for impact is closer to the threshold for exclusion from the census than a topic that has been assessed as having a "Low" potential for impact. All questions meet our thresholds to ensure reliable information will be collected in Census 2021.

We present the evidence used to assess questions as having a "Medium" or "High" potential for impact on the evaluation criteria in Annex 3.

Question harmonisation

As in previous censuses, there will be separate censuses in Scotland and Northern Ireland. The questions for England and Wales have been developed through close collaboration with National Records of Scotland (NRS) and the Northern Ireland Statistics and Research Agency (NISRA), who are responsible for conducting the censuses in Scotland and Northern Ireland, respectively.

We recognize that each country has its own user and respondent needs; however, we aim for harmonization of census questions and topics where possible to produce UK-wide statistics that are consistent and comparable.

Northern Ireland has an identical question design to England and Wales. Scotland has a different approach to Northern Ireland, England, and Wales. The routing for Scotland's suite of questions is similar to ONS' Approach 4 design.

Scotland's design captures firstly the name of the respondent, then asks "Do you usually live at [census address]?" with the following response options, "Yes, this is my permanent or family home", or "No, I do not usually live here".

The question is different depending on whether the respondent lives at the address or not.

If the respondent does live at the address, the question and response options are:

Does anyone else usually live at [address]?

Yes, I need to add someone else

No, there is no-one else living here

If the answer to the question is yes and they need to add someone (else), the respondent is asked to supply that person’s name(s) and they move onto the next screen on people living at address who are temporarily away.

If the householder does not live at the address, the question and response options are:

Does anyone usually live at [address]?

Yes

No, no one counts this address as their permanent or family home

Respondents are then asked if anybody else lives at the address, instructing the respondent to include

• family members, including partners, children and babies born before 20 March 2022

• students and/or schoolchildren who live away from home during term-time

• housemates, flatmates or lodgers

• people staying temporarily who usually live in the UK but do not have another UK address

• people who usually live outside the UK who are staying in the UK for six months or more

The response options are "Yes, I need to add someone" or "No, there is no-one living here".

The next screen asks about those who are temporarily away "Does anyone else usually live at [census address] who is temporarily away?", and respondents are instructed to include:

• people who work away from home within the UK, or are members of the Armed Forces, if this is their permanent or family home

• people who are temporarily away from home on the night of 20 March 2022

Again, the response options are "Yes, I need to add someone else" or "No, there is no-one else living here".

Respondents are then asked to supply names of those who are temporarily away only in the loop-based system. Scotland’s response options in collecting names are “First name(s)” and “Last name”. Respondents are not instructed to include middle names but if they want to include a middle name, they can write this under “First name(s)”.

Additional functionality in Scotland occurs when respondents enter two people with the same name. Respondents are asked to confirm this is correct, and if so to include a reminder to tell the people apart as they progress through the questionnaire. This could be a middle name, or position in the family, for example, "father" or "son".

For overnight visitors, Scotland has the following question design, "Is there anyone staying at [census address] on the night of 20 March 2022 whose permanent or family home is elsewhere?" with respondents instructed to include:

• people staying here because it is their second address, for example, for work or a holiday home. Their permanent or family home is elsewhere

• people who usually live somewhere else in the UK, for example, boy/girlfriends, friends, relatives

• people who usually live outside the UK who are staying in the UK for less than 6 months

• people here on holiday

The response options are "Yes, I need to add someone else" or "No, there is no-one else staying here on the night of 20 March 2022". If applicable, respondents go on to give the person’s name, which is a loop-based system to add all visitors.

A summary page is seen next by respondents before they continue to a screen asking them to confirm the number of residents and of overnight visitors staying at the address. Respondents can add or remove household members or visitors before confirming this is everyone.

Overnight visitor demographic questions for are the same for all UK countries.

The count of residents and overnight visitor questions were developed for use in the context of Census 2021 in England and Wales, a mandatory household form. Therefore, it is possible that in different contexts, such as social surveys, a different approach may be more suitable.

References to tests take the form (Year: Test number). "Year" refers to the calendar year the test was undertaken in and the test number is the position of the test within the year considering all testing that took place in that year. For example, the fifth test conducted in 2017 would be referenced as (2017:5).

A full description of each of these items can be found in summary of testing for Census 2021.

The information leaflet includes important information the respondent may need to complete the census. The sections of the leaflet are:

• What the census is and why you should take part

• Who to include in your census questionnaire

• Which questions to answer

• How we use your information and protect your privacy

• How to get help

The section "Who to include in your census questionnaire" was included as part of the paper form in 2011. The text in the leaflet is as follows.

Who to include in your census questionnaire

People who usually live here

Include everyone who usually lives here. This might include family members, housemates, tenants, or lodgers. This is the address where they spend most of their time and might be their permanent or family home.

People who are temporarily away

Include everyone who usually lives here, but who is temporarily away on Sunday 21 March 2021. This includes:

• people staying (or expecting to stay) somewhere else for less than 6 months, such as a hospital, care home or hostel

• members of the Armed Forces

• people staying at another address for work, holiday, travel or staying with friends or relatives (unless they are outside the UK for a year or more)

• people in prison with a sentence of less than a year, or on remand

People temporarily staying here overnight on Sunday 21 March 2021

Anyone who does not have another UK address should be included on your census form as a member of your household. This includes:

• people from the UK who do not have another address

• people from outside the UK who plan to stay in the UK for three months or more

Anyone who usually lives at another address should be included on your census form as a visitor. This includes:

• people who usually live at another UK address, such as boyfriends, girlfriends, friends, or relatives

• people who are staying here for work, whose permanent or family home is elsewhere

• people who usually live outside the UK and are staying in the UK for less than three months

• people here on holiday

People who stay at more than one address

People with more than one UK address should be included on the census form at their permanent or family home. If they do not have a permanent or family home, they should be included on the census form at the address where they spend most of their time.

Children with parents who live apart

Children should be included on the census form at the address where they spend most of their time. If they spend time equally between two addresses, include them on the census form at the address where they are staying overnight on Sunday 21 March 2021.

Boarding school children and students away from home during term time

Include boarders and students staying away from home in the questions about who lives in this household. They will also be included on the census form at their term time address.

Evaluation of name, type, and number of residents and visitors' questions used in 2019 Rehearsal

Potential for impact of these questions on public acceptability and financial concerns were assessed as "Low".

Potential for impact on data quality: "Over threshold -- further development needed"

The definitions of residence are very complex. The online question suite used in the 2019 Rehearsal was not sufficiently developed for use in Census 2021. Respondents were not clear on who to include when providing the number and type of residents in household.

Additionally, those in shared accommodation may not know the full names of all those they live with.

Qualitative testing of the Census 2021 final design suggests that this suite of questions will have a medium potential for impact in 2021. Issues with the respondent not knowing all members of the household cannot be addressed through questionnaire design.

Potential for impact on respondent burden: "Over threshold -- further development needed"

In the 2019 Rehearsal (2019:15) respondents took 20 seconds on each resident and visitors' question, longer than the average (15 seconds) for similar type of questions.

In addition, 2.11% of participants used the previous button during the visitors' question suite, over the threshold (2%) of concern.

Qualitative testing of the Census 2021 final design (2019:17) suggests that this suite of questions will have a medium to low potential for impact in 2021. Although the questions still take time to navigate, they are clearly understood and considered logical by respondents.

Potential for impact on questionnaire mode: "High"

As described in this report, the online question suite for 2019 Rehearsal was completely different to the paper questionnaire collection of name, type, and number of residents, because of using a multi-page sequential approach. The difference in collection was made to meet GDS standards and provided minimal guidance information. On paper substantial definitional information is contained either in the questionnaire, or the accompanying guidance booklet (shown in Annex 2).

Inclusion of the definitional information in the Census 2021 final design suggests that although the paper and online designs remain very different this suite of questions will have a medium to low potential for impact in 2021 as they will both collect the count and names of household residents and visitors effectively.

Evaluation of visitor's details questions

These questions have not substantially changed between 2019 Rehearsal and the Census 2021 final design. Therefore, the 2019 Rehearsal question evaluation is expected to apply to the Census 2021 questions.

Potential for impact of these questions on public acceptability, financial concerns and questionnaire mode was assessed as "Low".

Potential for impact on data quality: "High"

These questions have a chance for recall bias and captures information that cannot be observed. Respondents may not know the date of birth or full address of their overnight visitors.

In addition, the visitor sex question had a non-response rate that was above the quantitative acceptance threshold questions (7%). The dropout rate was above the acceptance threshold of 1%. 4.7% of total page views used the previous button, above the 2% threshold.

Potential for impact on public acceptability: "Medium"

This question had negative feedback from the 2019 Rehearsal on issues relating to public acceptability. Testing (2018:34) found only 66% of respondents thought it was acceptable to ask about overnight visitors in general. We have added guidance accordions to help address this.

Potential for impact on respondent burden: "High"

Analysis of the 2019 Rehearsal found that for each of these questions:

• item non-response rate was above 7% threshold  

• the dropout rate was above the 1% threshold

• use of the previous button was above the 2% threshold

In addition, the questions include write-in fields, increasing the burden for respondents to answer the question. However, in Census 2021 on the electronic questionnaire respondents will be able to use standard postcode look-up functionality on the visitors address question.