Health and unpaid care question development for Census 2021

In December 2018, the government presented to Parliament a White Paper Help Shape our Future: The 2021 Census of Population and Housing in England and Wales (PDF, 967KB). This outlined the Office for National Statistics's (ONS's) proposal to collect information on health and unpaid care.

For Census 2021, data are collected on:

-   general health

-   long-lasting health conditions and illnesses

-   activity restriction

-   provision of unpaid care

Data collected on health and unpaid care contribute to the funding, development and planning of health care and carers' services and to helping reduce inequalities experienced by those with disabilities.

The questions on health, long-lasting health conditions or illnesses, and activity restriction are:

-   How is your general health?

-   Do you have any long-term physical or mental health conditions or illnesses lasting or expected to last 12 months or more?

-   Do any of your health conditions or illnesses reduce your ability to carry out day-to-day activities?

The question on provision of unpaid care is:

-   Do you look after, or give any help or support to, anyone because they have any long-term physical or mental health conditions or illnesses, or problems related to old age?

Census 2021 will be an online-first census, with a target of 75% online returns. We have reviewed and tested each question online and have recommended additional design and functionality amends specific to the electronic questionnaire. We have also recommended additional changes to the question designs following our stakeholder engagement, evaluation and testing programmes. These changes will be detailed in this report.

The main recommended changes to question designs are:

-   the long-term health or disability question should change terminology from "health problems or disability" in the question stem to "physical or mental health conditions or illnesses" to clarify that the question asks about mental and physical health

-   the term "disability" should be removed from both health and unpaid care question designs

-   the 2011 Census long-term health or disability one-stage design should be changed to a two-stage design measuring long-lasting health conditions and illnesses and then activity restriction, which is based on the Government Statistical Service's (GSS's) harmonised principle questions

-   the time ranges in the question asking about unpaid care should be broken down into more response options, narrowing the time ranges to capture more detailed information

The Census 2021 variables will be long-lasting health conditions or illnesses and activity restriction for Census 2021 outputs.s.

Since the publication of the White Paper, the Office for National Statistics (ONS) has conducted and concluded the final phase of testing on health and unpaid care. This report provides links to previously published research and the findings of additional testing that led to the final recommended questions for Census 2021 in England and Wales.

The questions and response options for Census 2021 have now been finalised through the census secondary legislation: The Census (England and Wales) Order 2020 and Census Regulations for England and for Wales.

The evidence base for the recommendations made in the White Paper is discussed in Section 4: Research that led to the 2018 White Paper recommendations. The evidence base for the finalisation of the questions for Census 2021 is discussed in Section 5: Research that led to the recommended Census 2021 question designs.

In June 2015, the Office for National Statistics (ONS) held a formal, 12-week consultation process asking census users for their views on the topics that were required in the questionnaire in England and Wales. The aim of the consultation was to promote discussion and encourage the development of strong cases for topics to be included in Census 2021.

In May 2016, the ONS published its response to the Census 2021 topic consultation. This set out our updated view on the topics to be included in Census 2021, including:

• a summary of proposals for new topics
• next steps
• an overview of our plans

The Census 2021 topic consultation revealed a clear requirement for information on general health and long-term health and disability to guide public health spending, resource allocation and policy development. There was also a clear requirement for information on unpaid care to plan, develop and deliver of a wide range of care services, including those required under the Care Act 2014 and the Social Services and Well-being (Wales) Act 2014.

A detailed summary of the consultation responses relating to the health topic can be found in the health topic report (PDF, 1.32MB). The report includes questions on general health and long-term health or disability, now long-lasting health conditions and illnesses, and activity restriction. A detailed summary of the provision of unpaid care topic can be found in the carers topic report (PDF, 606KB).

In these reports, the ONS made clear commitments to the public: we do not intend to change the questions on the health and unpaid care topics, and we will review the health and unpaid care questions' guidance, formats and wording to improve data quality or reduce burden placed on respondents.

However, preliminary work on reviewing the guidance, format and wording showed that the two commitments were not fully compatible. The 2011 Census health and unpaid care questions did not adhere to current Government Digital Service's Service Standard for the electronic questionnaire.

Also, the 2011 long-term health or disability question did not follow what is now understood to be best practice on disability measurement. The ONS therefore investigated whether to base the question for Census 2021 on the Government Statistical Service's (GSS's) service standard. This was developed between 2009 and 2011 and was published in 2011, after the last census, following broad consultation with government departments and disability groups.

Following this, we began a comprehensive programme of research and development. We provide a full list of the tests used in the development of the topics of health and unpaid care in Annex 2. Further details are provided in the Summary of testing for Census 2021.

The tests used a range of qualitative and quantitative research methods. A short description of the different research methods and sampling techniques is given in the Question and questionnaire development overview for Census 2021. Testing included respondents from a wide range of backgrounds. Respondents included people with a range of long-term health conditions and impairments and people who provide unpaid care.

In 2018, a stakeholder consultation was held considering whether to use the same long-term health or disability question as we used for the 2011 Census or whether it would be beneficial to data users to change to a question based on the GSS's harmonised principle on classification of disability in terms of the Equality Act 2010.

In December 2018, the government presented to Parliament a White Paper Help Shape our Future: The 2021 Census of Population and Housing in England and Wales (PDF, 967KB). This outlined our proposals for the topic of health and unpaid care.

This response included commitments related to Census 2021 questions. These were to:

• align the questions on long-term health conditions and disability with the GSS's harmonised principle questions, subject to testing
• amend the response "Yes, 20-49 hours a week" so that the census can gather information on Carer's Allowance as requested by data users, subject to space constraints on the paper questionnaire
• adapt all health and carer questions to follow the online Government Digital Service's Service Standard for the electronic questionnaire
• explore improving question guidance on the health and unpaid care topics
• investigate linking 2011 Census data to administrative health data

We have provided an update on how we met these commitments in Annex 1.

The questions for Census 2021 are now finalised. We have evaluated the question for its potential impact on data quality, public acceptability, respondent burden, financial concerns and questionnaire mode. We present details of this evaluation in Annex 3.

Annex 4 provides details of the definitions and terms used in this report.

As set out in the White Paper Help Shape our Future: The 2021 Census of Population and Housing in England and Wales (PDF, 967KB), we proposed aligning the questions on long-term health conditions and disability with the Government Statistical Service’s (GSS’s) harmonised principle questions, subject to testing, and updating the question on unpaid care to include smaller time ranges so that information on Carer’s Allowance can be collected.

We discuss details of the testing related to the finalisation of the questions for Census 2021 in Section 5: Research that led to the recommended Census 2021 question designs.

References to tests take the form (Year: Test number). “Year” refers to the calendar year the test was undertaken in and the test number is the position of the test within the year considering all testing that took place in that year.

General health

A question on general health was asked in the 2001 Census and again in the 2011 Census. The general health question measures the current health of the population of England and Wales and helps to inform the allocation of vital resources both at the national and regional level.

The question and response options for the question on general health are unchanged from the 2011 Census. This ensures data continuity and allows comparison of data across the censuses. The general health question was tested as part of the development of the long-term illness and disability and unpaid care questions (2018:28). The question was found to be subjective, but no other concerns were raised about the question or response options.

We recommend using the 2011 Census general health question design for Census 2021.

2011 Census

How is your health in general?

• Very good
• Good
• Fair
• Bad
• Very bad

Long-lasting health conditions or illnesses

A question on long-term health or disability was first introduced in the 1991 Census. In the 2015 topic consultation, there was a clear user need for this information to meet and monitor disability under the Equality Act 2010, statutory duties under the Health and Social Care Act 2012, developing and implementing policy, and assessing health inequalities at small area level.

2011 Census question

Are your day-to-day activities limited because of a health problem or disability which has lasted, or is expected to last, at least 12 months?

Include problems related to old age

• Yes, limited a lot
• Yes, limited a little
• No

We stated in the Census 2021 topic consultation that we recommend no change to the long-term health or disability question.

However, the 2011 Census Quality Survey (PDF, 1.39MB) showed that the long-term health or disability question had a relatively low agreement rate of 88.4%. The question also created a high respondent burden by asking about health conditions and activity restriction in one question. We therefore needed to develop the question to ensure that these issues were addressed.

In addition, we began to consider using the GSS harmonised principle. The GSS harmonised principle was published in 2011, after the 2011 Census, following broad consultation with government departments and disability groups and testing. This interview-led question is used by many other social surveys, such as the Labour Force Survey (LFS). The development of the GSS’s question is discussed in further detail in Annex 5.

The GSS two-stage question was designed for interview-led purposes. Taking this question design forward, we would need to adapt it for self-completion.

Whether a two-stage design should be used

We undertook cognitive testing (2018:28) to investigate the 2011 Census design and potentially two-stage question designs for comprehension, levels of respondent burden and acceptability. A total of 31 participants took part in the cognitive interviews. Participants had a range of physical and mental health conditions, including varying age-related issues, progressive health conditions, and disability that was registered and not registered.

Three different versions of the long-term health condition or disability questions were tested:

Version 1 – the 2011 Census design:

• Are your day-to-day activities limited because of a health problem or disability which has lasted, or is expected to last, at least 12 months? (With response options: “Yes” or “No”)

Version 2 – a two-stage design based on the 2011 Census wording:

• Do you have a health problem or disability which has lasted, or is expected to last, at least 12 months? (With response options: “Yes” or “No”)
• Does your health problem or disability limit your day-to-day activities? (With response options: “Yes, limited a lot”, “Yes, limited a little” or “No”)

Version 3 – the two-stage GSS’s harmonised principle questions design:

• Do you have any physical or mental health conditions or illnesses lasting or expected to last 12 months or more? (With response options: “Yes” or “No”)
• Do any of your conditions or illnesses reduce your ability to carry out day-to-day activities? (With response options: “Yes, a lot”, “Yes, a little” or “Not at all”)

Each participant answered a set of census questions including Version 1 of the long-term health conditions question and was then interviewed about their experience and views on the question. During the interview process, the other versions were shown to allow the participants to compare and comment on each version.

Results from testing (2018:28) indicated that, overall, participants preferred the two-stage questions to the one-stage question used in 2011 (Version 1) as this reduced respondent burden and allowed for a more accurate response.

Participants felt that the one-stage question was asking two questions instead of one and that it could not distinguish between those with no health conditions and those with a long-term condition that did not impact on day-to-day activities.

The recommendations from this test were that a two-stage question should be used and the design should be based on the GSS harmonised principle health question.

Whether the GSS harmonised principle should be used

In parallel to the initial testing, we conducted a consultation survey to understand the impact of using a two-stage question design. A survey was sent out and 316 users of the health data produced by the census responded. The aims of the survey were to understand users’ needs in more detail and investigate whether the two-stage questions met users’ needs.

The survey contained two-stage versions of the long-term health questions (Versions 2 and 3 in the initial testing phase) and highlighted the similarities and differences between them. Respondents were asked to indicate which question best met their data requirements.

Results from the stakeholder survey showed that changing to the two-stage GSS harmonised principle questions would have a positive impact for the majority of users. More than three times as many stakeholders preferred the GSS’s questions (70.30%) to the two-stage 2011 Census question (18.32%). Some users stated neither of the question designs would meet their needs, with concerns for comparability and continuity with previous censuses, but they stated there would be no impact from using the GSS two-stage design. Stakeholders also stated that using the GSS’s harmonised principle questions would mean the data can be compared against other survey data, making it more useful to them.

The recommendation from this engagement was that the design should be based on the GSS harmonised principle health question.

We undertook further evaluations on terminology to refine the recommended design for Census 2021.

Whether “mental health” and “disability” terminology should be used

Participants (2018:28) responded positively to the inclusion of mental health conditions and preferred the use of “conditions” over “problems” when talking about mental health, with “problems” thought to have a negative connotation.

Including the term “disability” was welcomed by participants with recognised disabilities and their carers. However, some thought that the term “disability” was not necessary, as it was covered by “conditions”. Some considered that the “disability” term might be less inclusive to those with milder long-term health conditions and illnesses. “Disability” was also considered to be more linked to physical than mental health conditions.

In the 2018 survey results, stakeholders reacted positively to the inclusion of mental health conditions in the GSS’s questions and reported that they were happy that the questions were now more fully aligned with the classification of disability used in the Equality Act 2010.

Some stakeholders noted that the removal of the term “disability” was in alignment with the social model of disability.

The recommendations were that the question should ask about mental as well as physical health and use the term “conditions” instead of problems and that more testing on the necessity of using the term “disability” was needed.

We investigated the implications of using the GSS’s questions further to understand if we should make this change in Census 2021.

Whether “reduce” or “limited” should be used to describe day-to-day activities

The first round of testing (2018:28) found that participants preferred the term “limited” to “reduce” when considering impact of health conditions on day-to-day activities, as it was considered to be a more understandable and clearly defined term. However, the term “reduce” is used in the GSS’s harmonised principle question, and this term is more aligned with the social model of disability than the term “limited”. To investigate which term was appropriate to use in the census, we:

• conducted a review of the evidence from the development of the 2011 Census and the GSS’s harmonised principle question
• considered the results of previous testing (2018:28)
• engaged with stakeholders

Extensive testing was done on the 2011 Census question on health (PDF, 360KB), including the terms used within the question. There were no tests involving the use of “reduce” but some involving the use of “limited”. The testing found that the final question was easily understood by respondents. Additionally, participants preferred that the term “limited” was included in the response options. For example, instead of “Yes, a lot” the response option should be “Yes, limited a lot”.

Evidence from the development of the GSS’s question on activity restriction showed that using “limited” or “reduce” resulted in very similar responses. However, during the consultation into developing harmonised standards for disability data (PDF, 456KB), some stakeholders reported that the word “limited” was unacceptable as it was too negative and does not follow the social model of disability.

As a result of the review of evidence and consultation with internal Office for National Statistics (ONS) stakeholders, we recommend using “reduce” instead of “limit” in the question. The main reasons behind this decision were:

• the GSS’s harmonised principle question uses “reduce” and the census question should be in alignment with this question
• the term “limited” could be considered as being negative
• the term “limited” does not follow the social model of disability

Whether the instruction to include problems related to old age be should removed

In the previous census in 2011, the long-term health conditions question had an instruction to include problems related to old age. However, the GSS’s question does not mention old age. We undertook some further analysis to understand the implications of removing the explicit reference to age-related problems.

The Annual Population Survey (APS) has used the GSS’s harmonised principle questions since 2013. We compared the disability data collected in the 2011 Census to data collected by the APS between 2014 and 2016. Results showed that a higher percentage of people aged 75 years old and older were classed as disabled using data from the 2011 Census, compared to those classed as having a disability using data from the APS. A similar proportion of people aged between 65 and 74 were classed as disabled regardless of whether using data from the APS or 2011 Census. However, a higher percentage of people in each age group under 65 years old were classed as having a disability using data from the APS compared to data from the 2011 Census.

On further analysis, the wording of the question, guidance given in answering the question and the collection methodology (the APS is collected via interviews rather than by a self-completed questionnaire) have contributed to the discrepancies between the outputs.

As a result of this analysis and previous discussions with stakeholders, and considering that the GSS’s question has had extensive testing and usage since the 2011 Census, it was decided to remove the guidance. This means that there will be no reference to old age in the questions. This also aligns with the definition of disability in the Equality Act 2010, which does not reference age.

The final recommended designs for health and disability questions for Census 2021 are shown in Section 6: Questions recommended for Census 2021.

Unpaid care

The unpaid care question was first introduced in the 2001 Census and then again in 2011, enabling users to look at the increases in the “carer” population and its implications for specific groups such as young carers aged 5 to 17 years.

2011 Census question

Do you look after or give any help or support to family members, friends, neighbours or others because of either:

• long-term physical or mental ill-health or disability?
• problems related to old age?

Do not count anything you do as part of your paid employment.

• No
• Yes, 1 – 19 hours a week
• Yes, 20 – 49 hours a week
• Yes, 50 or more hours a week

After the Census 2021 topic consultation, we made commitments to not change the question on “amount of care provided”. However, there was a user need to collect information on carers who were eligible for Carer’s Allowance, which would require changing the response options.

In addition, the 2011 Census question design did not meet the Government Digital Service’s Service Standard for online questions.

So, we also took the opportunity to investigate if we could make any other improvements to reduce respondent burden, improve data quality and collect more detailed information for data users. As part of this, we considered how the question compared to the GSS harmonised principle question on unpaid care (2008 to 2020).

To ensure we aligned the health and unpaid care questions, we designed and tested these questions simultaneously and changed to identical terminology.

Whether the 2011 Census question is well understood

The first round of testing (2016:5) carried out was to investigate how a possible question on volunteer work interacted with the 2011 Census question on unpaid care. Further detail on the outcome of this testing for volunteering is found in Volunteering topic research for Census 2021.

Each of the 29 participants took part in a short, online questionnaire that included a question on unpaid care and volunteering, using their own mobile device, while in the presence of an interviewer. Following completion of the questionnaire, the interviewer asked them about their experience and opinion of the unpaid care and volunteering questions.

When asked about their thoughts on the 2011 Census unpaid care question, many participants felt it was too long and complicated, as it contained text, bullet points and instructions. Some participants also felt the time ranges (for example, 1 to 19 hours) were too wide and that there should be more options with smaller time ranges. The interviews also showed that many participants confused unpaid care with other activities such as general childcare, volunteering and paid care.

The recommendations from this testing were to investigate if time ranges should be smaller by consulting with stakeholders to understand their need and to shorten the question and ensure respondents understand what should be recorded as unpaid care.

Alternative phrasings of the question to be considered

Based on the first test findings and guidance from the Government Digital Service’s Service Standard, we tested including the 2011 Census bullet points in the question stem and various shortened versions of the question.

Different versions of the question were peer reviewed (2018:15). The questions incorporated various modifications of the 2011 Census question, including:

• “family members, friends, neighbours or others” changed to “anyone”
• “give help” changed to “give unpaid help”
• “look after” removed from question
• “do not count anything you do as part of your paid employment” instruction removed

Participants were shown each version of the question on paper and asked for their views on each question and then probed further on their understanding and ease of answering for each modification. They were also asked about the acceptability of each modification.

The results showed that participants favoured the use of “anyone” over “family members, friends, neighbours or others” in the question stem. The term “anyone” was considered clearer and easier to read and understand. Participants struggled with the use of “others”; it confused participants and made participants think about caring for someone as part of a volunteering activity, reducing the quality of the data.

Participants considered the use of “unpaid” in the question to be ambiguous. For example, some participants were confused whether receiving gifts or Carer’s Allowance counted as being paid. Removing “look after” did not test well. Unpaid carers identified with “look after” rather than “help or support”, particularly those who care for family members. Participants thought “look after” along with “help or support” encompassed all caring responsibilities.

Participants gave positive feedback on the inclusion of the instruction not to count anything done as part of paid employment. They reported that it was clear and distinct from the question stem, meaning it was less likely to be missed than the term “unpaid”.

Participants also suggested shortening the question even further to include a statement at the start of the question stem regarding not including paid work, such as “Excluding paid work, do you look after...”.

The recommendations from this round of testing were:

• not to include “unpaid” in the question stem but to have a separate instruction about not including paid employment similar to “Do not count anything you do as part of your paid employment”
• to keep “look after” in the question stem as per the 2011 Census
• to use “anyone” instead of “family members, friends, neighbours or others”
• to consider a question stem that starts with “Excluding paid work”
• that further testing should include participants who are unpaid and paid carers

Response options that stakeholders require

We consulted with stakeholders with the aim of understanding if changing the time ranges would be beneficial to them. This was in response to some requests for data on those receiving Carer’s Allowance in the topic consultation. In addition, some participants in the previous round of testing (2016:5) had suggested making the time ranges in the response options smaller, creating more response options.

We asked 43 core stakeholders to respond to an online survey asking about their data needs. The results showed a medium to high need to increase the number of response options and reduce the time ranges used in the response options. By reducing the time ranges in the question, the data collected can be better used for service provision, policymaking and resource allocation.

One main user need for increasing the number of response options and reducing the time ranges was to understand the number of carers eligible for Carer’s Allowance; the eligibility for Carer’s Allowance is 35 hours a week and over. We split the “Yes, 20 – 49 hours a week” response option into “20 – 34 hours a week” and “35 – 49 hours a week”.

Another main user need was to understand the number of people at risk of poor health and employment outcomes because of caring responsibilities; the risk increases at 10 hours a week and over. We split the “Yes, 1 – 19 hours a week” response option into “1 – 9 hours a week” and “10 – 19 hours a week”.

The recommendation from the stakeholder consultation was to use the following response options:

• Yes, 1 – 9 hours a week
• Yes, 10 – 19 hours a week
• Yes, 20 – 34 hours a week
• Yes, 35 – 49 hours a week
• Yes, 50 or more hours a week

We also consulted with stakeholders on the redesign of the question, including:

• using “anyone” instead of listing out the groups of people that someone could care for
• including the bullet points of the reasons people could be cared for in the question stem
• including the instruction “Do not count anything you do as part of your paid employment”

After evaluating the impact of these changes, stakeholders agreed with the changes.

In finalising the Census 2021 paper questionnaire designs, we found that there was sufficient space to implement all of the recommended changes on both the paper and electronic questionnaires.

Whether the question wording should align with the health questions

Based on the findings and recommendations of the previous internal peer review (2018:15), we undertook further cognitive testing on different versions of the unpaid care question. A total of 31 participants took part in this testing (2018:28), including 15 unpaid carers, 2 paid carers and 14 people with long-term health conditions or disabilities. The unpaid care question was tested alongside other questions, including the long-term health questions.

The purpose of the cognitive interview was to get further feedback on the phrasing recommended in the previous research. In addition, we investigated using “long-term physical or mental ill-health or disability”, as in the 2011 Census, or “physical or mental health conditions or illness” in line with the questions on health.

In addition, participants were asked to comment on their understanding of each version and how easy they found it to answer.

Three versions of the question were tested:

Version 1 question:

Excluding paid work, do you look after, or give any help or support to anyone?

Include

• long-term physical or mental health conditions or illness
• problems related to old age

Version 2 question:

Do you look after, or give any help or support to anyone because they have long-term physical or mental health conditions or illnesses, or problems related to old age?

Exclude anything you do as part of your paid employment.

Version 3 question:

Do you look after, or give any help or support to family members, friends, neighbours or others because they have long-term physical or mental ill-health or disability, or problems related to old age?

Exclude anything you do as part of your paid employment.

The response options were the same for each version of the question:

• No
• Yes, 1 – 9 hours a week
• Yes, 10 – 19 hours a week
• Yes, 20 – 34 hours a week
• Yes, 35 – 49 hours a week
• Yes, 50 or more hours a week

Each participant answered a set of census questions including Version 1 of the unpaid care question and was then interviewed about their experience and views on the question. During the interview process, the other versions were shown to allow the participants to compare and comment on each version.

The results showed that participants disliked Version 1 of the question because the instructions were too long. Participants preferred Versions 2 and 3 because they contained more details in the question stem and had shorter instructions.

Some participants reported that Version 3 was too long and burdensome to answer, and some questioned the use of “ill-health” in the question, which they considered an old-fashioned phrase.

Overall, participants favoured Version 2 because they thought the question:

• started with the relevant information (“Do you look after…?”)
• used clear and precise language in the question text
• used appropriate descriptions (“long-term physical or mental health conditions”)
• contained short but clear instruction text

The recommendations from this testing were:

• include a list of conditions in the question stem, for example, “physical and mental health conditions and illnesses”
• the shorter description of “anyone” can be used instead of the longer and more burdensome “family members, friends, neighbours or others”
• ensure it is clear to respondents that they should only include care that is unpaid

Following this research, to align with Government Digital Service’s Service Standard, the dash in the time ranges within the response options were replaced with the word “to”. For example, the response option “Yes, 10 – 19 hours a week” was changed to “Yes, 10 to 19 hours a week”.

Whether the question should align more closely with the GSS harmonised principle

The GSS harmonised principle question on unpaid care is designed to be asked by an interviewer (not as part of a self-completed survey like the census) and is a two-stage question, first asking if unpaid care is provided by the respondent and then asking how much time is spent providing care.

There were several differences between the question developed through testing and the GSS harmonised question. We undertook research to determine if the question used in the census should more closely match the GSS question. We also needed to assess the question to ensure it met Government Digital Service’ Service Standard.

Whether the census should use a two-stage question

We reviewed other survey questions on unpaid care and found that interviewer-led questions most commonly use two-stage questions, similar to the question used by the GSS. We found that surveys that are self-complete most commonly use one-stage questions.

The mode of a questionnaire, for example, interviewer-led or self-complete and paper or online, can impact the data collected. This is particularly true of subjective questions, such as the unpaid care question. This highlighted the need to use the same questions on the paper and electronic questionnaires.

We compared the data collected in the one-stage self-complete 2011 Census on unpaid care to the data collected by the two-stage interview-led Family Resources Survey in 2016 to 2017. We found that the overall distribution of unpaid carers was similar for both surveys. The distribution of carers by sex was also similar, there was a slight difference in carer estimates by age, but it was not significant. There was no clear evidence that data quality would be impacted by asking a one-stage question instead of a two-stage question.

We recommended to continue to ask a one-stage question.

Whether the census question on unpaid care should capture data on travel time

The GSS harmonised principle question in 2018 asked respondents to include travel time in their answer. However, there is no specific mention of travel time in the Care Act 2014 and travel time is not included when calculating Carer’s Allowance. Stakeholders told us that they use the census data to allocate provisions for carers and therefore it is important to provide them with data that align with current legislation. During our extensive cognitive testing of the question designs, only one person mentioned including travel time.

The recommendation was that the question should not include any instructions regarding travel time and that only time spent providing unpaid care should be recorded. The GSS harmonised principle question for unpaid care has since been revised and no longer included travel time.

Whether the census question should capture unpaid care of less than one hour

We recommended to change the second response options to capture those doing less than one hour of unpaid care a week. This aligns with question design in other social surveys such as the Health and Lifestyle Survey and the GSS harmonised principle question. The change was supported by stakeholders, including the Equality and Human Rights Commission and the Department for Work and Pensions (DWP).

The response option of “Yes, 9 hours or less” was included in an omnibus test (2018:41). An analysis of the distribution of answers from this omnibus test showed that the data collected were similar to that of the 2011 Census. This suggested that including this response option would not have any impact on the quality of data collected.

In the White Paper Help Shape our Future: The 2021 Census of Population and Housing in England and Wales (PDF, 967KB), we stated that we were finalising the Census 2021 questions. We also stated that we would not be changing the question on general health.

Question design for online

In addition to the testing described earlier, the questions have undergone significant user experience (UX) testing (2017:2, 2018:2, 2019:1, 2020:2). UX testing focuses on understanding user behaviours as people interact with online services. Through observation techniques, task analysis and other feedback methodologies, it aims to develop a deep knowledge of these interactions and what it means for the design of a service.

UX research has taken place on a rolling basis since 2017. Before the Census Rehearsal in October 2019, 458 interviews were conducted at 99 events. The UX testing programme will continue throughout 2020. All participants are purposively selected to include a wide range of ages and digital abilities.

UX testing included various iterations of the questions described elsewhere in this report. Feedback from this research informed decisions made on the design of these questions. For more information on UX testing, see the Question and questionnaire development overview for Census 2021.

Findings from UX testing suggested adding guidance accordions to the long-lasting health conditions or illnesses and activity restriction questions to further explain the terminology used. This is to improve public acceptability and data quality collected from these questions.

Welsh language question development

Between 2017 and 2018, an external agency with Welsh-speaking researchers was commissioned to undertake focus groups (2017:17) and a series of cognitive interviews (2017:18). In 2018, further cognitive interviews (2018:41) were undertaken by the same agency. The qualitative research tested public acceptability and comprehension of amended and newly designed census questions in Welsh. The questions were tested with people across Wales with varying dialects and Welsh language proficiencies.

To ensure questions adhere to Cymraeg Clir guidelines, some changes to the text or questions across the census questionnaires were translated by our contracted specialist Welsh language translation service provider. These changes were quality assured by the Welsh Language Census Question Assurance Group. This group includes Welsh language and policy experts from the Welsh Language Commissioner and the Welsh Government and was convened to give advice on the accuracy, clarity and acceptability of the language as well as other policy issues pertaining to the Welsh language and bilingual design.

Findings from testing (2018:40) suggested a strong recommendation that the term “anabledd” (disability) could be added to the Welsh language questions for clarity. However, investigating the term further, we decided to keep the question identical to the English question because of the sensitive nature of the topic and omit using “disability”, making the question similar to the GSS harmonised principle and aligning the question to the social model of disability in both languages.

The design of the questions recommended for Census 2021 was informed by the research and testing detailed in this report. The questions and response options for Census 2021 have now been finalised through the census secondary legislation.

The paper questionnaires used in Census 2021 are now available.

Guidance text and instructions are not part of the legislation, but we consider these to be finalised as well. However, it is possible that guidance text or instructions may change if there is enough evidence to support doing so.

General health question

Figure 1: Online question on general health, in English and Welsh

Source: Office for National Statistics – Census 2021

Download this image Figure 1: Online question on general health, in English and Welsh

.png (62.5 kB)

Long-lasting health conditions or illnesses question

Figure 2: Online question on long-term physical or mental health conditions or illnesses, in English and Welsh

Source: Office for National Statistics – Census 2021

Download this image Figure 2: Online question on long-term physical or mental health conditions or illnesses, in English and Welsh

.png (142.6 kB)

Guidance accordion

What we mean by “physical and mental health conditions or illnesses”

This is about health conditions, illnesses or impairments you may have.

Consider conditions that always affect you and those that flare up from time to time. These may include, for example, sensory conditions, developmental conditions or learning impairments.

Activity restriction question

If a respondent answers “Yes” to the long-term health conditions question, they are asked this question.

Figure 3: Online question on ability to carry out day-to-day activities, in English and Welsh

Source: Office for National Statistics – Census 2021

Download this image Figure 3: Online question on ability to carry out day-to-day activities, in English and Welsh

.png (137.4 kB)

Guidance accordion

What we mean by “reduce your ability”

This is about whether your health condition or illness currently affects your ability to carry out day-to-day activities.

Consider whether you are still affected while receiving any treatment, medication or using any devices for your condition or illness. For example, if you use a hearing aid and are not restricted in carrying out your day-to-day activities when doing so, select “Not at all”.

You should select “Yes, a lot” if you usually need some level of support from family members, friends or personal social services for most normal daily activities.

Unpaid care question

Respondents aged under 5 years old do not have to answer this question.

Figure 4: Online question on unpaid care, in English and Welsh

Source: Office for National Statistics – Census 2021

Download this image Figure 4: Online question on unpaid care, in English and Welsh

.png (285.6 kB)

Question order

In the 2011 Census, the questions on general health and unpaid care followed the questions on demography and migration.

The long-term health problems and illness question was separate from these questions, placed later in the form after identity questions such as ethnic group, national identity, religion and passports.

For Census 2021, we are recommending that the health questions are placed together as our investigations shows that questions on the same topic should be placed together where possible. Further research into other social surveys suggests that if health questions appear in the same section as the unpaid care question, the unpaid care question typically follows the health questions. In addition, our testing (2018:28) showed that placing the unpaid care question after the health questions did not cause confusion when answering the questions.

We therefore recommended that the topics should be ordered together as follows:

• general health
• long-lasting health conditions or illnesses
• activity restriction
• provision of unpaid care

Question evaluation

The question designs put forward in this report are based on extensive research and assessment using evaluation criteria that were set out in the publication The 2021 Census – Assessment of initial user requirements on content for England and Wales: Response to consultation (PDF, 796KB).

The evaluation considered the potential impact that including a topic on the census would have on data quality, public acceptability, respondent burden, financial concerns and questionnaire mode. The evaluations were used in conjunction with the user requirements criteria to steer the development of the census questions and questionnaire.

A topic that has been assessed as having a “High” potential for impact is closer to the threshold for exclusion from the census than a topic that has been assessed as having a “Low” potential for impact.

After completing the research and development phase, we evaluated the recommended questions against the same criteria using an updated tool that considers the type of evidence we have available and the Census 2021 context. A description of this updated evaluation tool is provided in the Question and questionnaire development overview for Census 2021.

All questions meet our thresholds to ensure reliable information will be collected in Census 2021.

We present the evidence used to assess questions as having a “Medium” or “High” potential for impact on the evaluation criteria in Annex 3. Because we split the 2011 Census long-term health problems or disability into two separate questions, long-lasting health conditions or illnesses and activity restriction, we evaluated each question separately for Census 2021.

Question harmonisation

As in previous censuses, there will be separate censuses in Scotland and Northern Ireland. The questions for England and Wales have been developed through close collaboration with National Records of Scotland (NRS) and the Northern Ireland Statistics and Research Agency (NISRA), which are responsible for conducting the censuses in Scotland and Northern Ireland, respectively.

We recognise that each country has its own user and respondent needs; however, we aim for harmonisation of census questions and topics where possible to produce UK-wide statistics that are consistent and comparable.

The NRS and NISRA intend to use the same general health question as the Office for National Statistics (ONS) in their next census.

The NRS and NISRA use the same question about health problems and disability as used in the England and Wales 2011 Census. However, both the NRS and NISRA censuses also include questions asking respondents if they have specific long-term health conditions; the conditions are listed as response options. The list of conditions on the NRS and NISRA censuses are different. These differences between statistical offices for these questions are because of differing stakeholders and uses of these data.

The unpaid care question used by NISRA is similar to the England and Wales Census 2021 question, except the first “Yes” response option is “Yes, 1 to 19 hours a week” on the census to align with 2011 Census. The NRS are recommending their 2011 Census unpaid care question to use for their next census.

The health and unpaid care questions were developed for use in the context of Census 2021 in England and Wales, a mandatory household form. Therefore, it is possible that in different contexts, such as social surveys, a different approach may be more suitable.

Data users will need to be cautious when comparing disability figures between the censuses in 2011 and 2021. However, using the Government Statistical Service’s (GSS’s) harmonised principle questions means that data can now be compared with other social surveys and align with current legislation, allowing data users to accurately assess how well legal requirements to people with disabilities are being met and plan and deliver services effectively.

There is also ongoing work to adapt the Census 2021 unpaid care question to be used in social surveys, and an updated GSS harmonised principle will be published shortly.

In the Census 2021 topic consultation response, the Office for National Statistics (ONS) made clear commitments to the public. We committed to:

• not change the question on “amount of care provided” or “health topics
• continue to measure disability in line with national guidelines currently set by the Equality Act 2010 (Great Britain)
• review the question guidance to both improve data quality and reduce the burden placed on respondents for all health and unpaid care topic questions

However, research showed that the first of these commitments was not compatible with the others. In addition, data user needs progressed since the 2015 topic consultation. As such, we later committed in the 2018 White Paper to align the questions on long-term health conditions and disability with the Government Statistical Service’s (GSS’s) harmonised principle questions, subject to testing, and to update the question on unpaid care to include smaller time ranges.

Aligning the question on health conditions and disability with the GSS’s harmonised principle questions

We tested the GSS’s harmonised principle questions and consulted with stakeholders to ask them their views on using the GSS’s questions. We concluded that using the GSS’s questions would be beneficial to both data quality and our stakeholders’ needs, so we will use these questions on Census 2021.

Updating the question on unpaid care to include smaller time ranges

We have increased the number of response options to this question to allow more detailed information on respondents’ time spent providing unpaid care. The new time ranges allow us to capture data on those eligible for Carer’s Allowance.

References to tests take the form (Year: Test number). “Year” refers to the calendar year the test was undertaken in and the test number is the position of the test within the year considering all testing that took place in that year. For example, the fifth test conducted in 2017 would be referenced as (2017:5).

A full description of each of these items can be found in Summary of testing for Census 2021.

Evaluation of general health question

Potential for impact on data quality, public acceptability, respondent burden and financial concerns were all evaluated as “Low”.

Potential for impact on questionnaire mode: “Medium”

The general health question differs between online and paper. The question is classified as a sensitive topic and has a potential impact to be answered differently between the two modes.

Evaluation of long-lasting health conditions or illnesses question

Potential for impact on financial concerns was evaluated as “Low”.

Potential for impact on data quality: “Medium”

Feedback from the 2019 Rehearsal (2019:15) showed that respondents found this question difficult to answer on behalf of another person and that respondents used the context-sensitive help more than was expected.

Potential for impact on public acceptability: “Medium”

The long-lasting health conditions or illnesses question is a sensitive topic and as such, respondents may answer in the way they think they should be perceived in society. During the 2019 Rehearsal (2019:1), and user experience (UX) testing (2019:15), there was some negative acceptability feedback for this question. As such, for Census 2021 we have added a guidance accordion for the question on the electronic questionnaire to mitigate and provide clarity on what we are asking.

Potential for impact on respondent burden: “Medium”

Although the question has been updated to include mental as well as physical health, increasing the length of the question, testing (2018:28) showed that respondents reacted positively to the inclusion of reference to mental health conditions and illnesses. However, feedback from the 2019 Rehearsal (2019:15) showed that respondents were spending more time (13 seconds) than expected on this question and it was above the expected threshold of 10 seconds.

Potential for impact on questionnaire mode: “Medium”

To improve public acceptability and data quality, for Census 2021 we have added a guidance accordion for this question on the electronic questionnaire only. Guidance text cannot be added to the paper question because of space constraints. This difference in mode means there is a potential for impact on this question.

Evaluation of question on activity restriction

Potential for impact on public acceptability, respondent burden and financial concerns were assessed as having a “Low” potential for impact.

Potential for impact on data quality: “Medium”

The activity restriction question is a sensitive topic and is personal to individuals. The sensitive nature and the information collected from respondents may not be observed, particularly when another person is answering on proxy for another person. In the 2019 Rehearsal (2019:15), respondents on average took 13 seconds to answer this question, which is above the average of all questions.

Potential for impact on questionnaire mode: “Medium”

To improve public acceptability and data quality, for Census 2021 we have added a guidance accordion for this question on the electronic questionnaire only. Guidance text cannot be added to the paper question because of space constraints. This difference in mode means there is a potential for impact on this question.

Evaluation of unpaid care question

Potential for impact on public acceptability, financial concerns and questionnaire mode were assessed as “Low”.

Potential for impact on respondent burden: “High”

The question length has been increased to include more instruction text within the question instead of underneath it, as testing showed that respondents tended to skip over instruction text (2016:15). In addition, the time ranges used have been narrowed, meaning there are more response options. Although respondent burden has been increased by moving the instructions into the question stem, the question has also been simplified by removing “family members, friends, neighbours or others” and replacing it with “anyone”. Testing (2018:41) showed that the new question design was well understood by respondents.

Potential for impact on data quality: “Medium”

The number of response options has increased, allowing for more accurate reporting of time spent caring for others, including the ability to gather information on Carer’s Allowance, which was a stakeholder requirement. We have also changed the wording of the question to include instructional text, so it is clearer to respondents what kind of unpaid care should be counted. Despite these improvements, the question remains as having a “Medium” potential for impact on data quality.

There are two main models of disability: the medical model, which considers a person to be disabled based solely on the impairment they have, and the social model, which considers a person to be disabled based on environmental or social factors, such as lack of accessible toilets or prejudice within society and not the solely result of a person’s impairment. The UK government encourages the use of the social model of disability.

It is the Equality Act 2010 that gives a disabled person rights in law.

The guidelines for the Equality Act 2010 (PDF, 693KB) state that, in general, a person is considered to be disabled if they have an impairment that is physical or mental and the impairment has substantial adverse effects that are long-term and affect day-to-day activities.

This definition of a disabled person recognises that a person may have an impairment that is long term but may not affect day-to-day-activities or an impairment that can affect day-to-day activities but is not long term. It is usually the combination of both long-term impairment and adverse effects on the ability to carry out day-to-day activities that means a person is recognised by law as being disabled.

This report uses the definition of disability from the Equality Act 2010. The questions used in the census, which are based on the Government Statistical Service (GSS) harmonised principles, were developed primarily using this definition. Using the same definition that gives disabled rights in law is vital to the users of the data provided by the census.

In line with the Government Statistical Service’s (GSS’s) harmonisation principles, the GSS developed a question designed to capture data on long-term illnesses and conditions. The question asks, “Do you have any physical or mental health conditions or illnesses lasting or expected to last for 12 months or more?”.

The GSS also developed a follow-up question designed to capture data on activity restriction caused by long-term health conditions for respondents who answer “Yes” to the previous question. This question asks, “Do any of your conditions or illnesses reduce your ability to carry-out day-to-day activities?” with response options of “Yes, a lot”, “Yes, a little” and “Not at all”.

The harmonised principle questions were developed by the GSS using guidance from sources including:

• the Equality Act 2010
• the EU Statistics on Income and Living Conditions project
• respondents of the Life Opportunity Survey
• other core stakeholders, such as voluntary organisations and charities

Evidence from these sources supported the separation of the concepts of health conditions, impairment and disability.

The questions were specifically designed to reflect the definition of disability in the Equality Act 2010, so the questions do not use the term “disability” but instead ask about physical or mental health conditions or illnesses. Aligning the questions with the Equality Act 2010 meant the data collected from these questions could be used to monitor legal obligations to people with disabilities.