Armed forces community (veterans) question development for Census 2021

In December 2018, the government presented to Parliament a White Paper, Help Shape our Future: The 2021 Census of Population and Housing in England and Wales, (PDF, 967KB). This outlined the Office for National Statistics’s (ONS’s) proposal to ask a new question on the armed forces community to meet the needs for service providers and others who support veterans in line with the Armed Forces Covenant.

For Census 2021, the recommended question on armed forces community (veterans) is:

Have you previously served in the UK Armed Forces?
Current serving members should only tick ‘No’
[ ] Yes, previously served in Regular Armed Forces
[ ] Yes, previously served in Reserve Armed Forces
Or [ ] No

The armed forces community (veterans) question will be asked to all respondents aged 16 years and over.

Ongoing work around this topic will focus on understanding and meeting user requirements for outputs.

Since the publication of the White Paper, the Office for National Statistics (ONS) has conducted and concluded the final phase of testing on the armed forces community (veterans) topic. This report provides links to previously published research and the findings of additional testing that led to the final recommended questions for Census 2021 for England and Wales. These recommendations will be subject to parliamentary approval through the census secondary legislation.

Additional information on the overall design of the Census 2021 questionnaire can be found in the question and questionnaire development overview for Census 2021. The questions and response options for Census 2021 have now been finalised through the census secondary legislation: The Census (England and Wales) Order 2020 as well as Census Regulations for England and for Wales.

The evidence base for the recommendations made in the White Paper is discussed in the section, Research that led to the 2018 White Paper recommendations. The evidence base for the finalisation of the questions for Census 2021 is discussed in the section, Research that led to the recommended Census 2021 question design.

In June 2015, the Office for National Statistics (ONS) held a formal, 12-week consultation process asking census users for their views on the topics that were required in the questionnaire in England and Wales. The aim of the consultation was to promote discussion and encourage the development of strong cases for topics to be included in Census 2021.

In May 2016, the ONS published its response to the 2021 Census topic consultation. This set out our updated view on the topics to be included in Census 2021, including:

• a summary of proposals for new topics
• next steps
• an overview of our plans

The topic consultation revealed a clear requirement for information on the armed forces community in the UK. However, we identified that these requirements were partially met by existing administrative data sources. In our response, we included a commitment to investigate the feasibility of linking administrative sources on armed forces veterans to data from Census 2021 to produce the outputs required by users.

A detailed summary of the consultation responses relating to the armed forces community topic can be found in the armed forces community topic report (PDF, 615.7KB). Within this report, the ONS made clear commitments to the public. We have provided an update on how we met these commitments in Annex 1.

Following this, we began a comprehensive programme of research and development. We provide a full list of the tests used in the development of the topic of the armed forces community in Annex 2. Further details are provided in the summary of testing for Census 2021.

The tests utilised a range of qualitative and quantitative research methods. A short description of the different research methods and sampling techniques is given in the question and questionnaire development overview for Census 2021. Testing included respondents from a wide range of backgrounds. This included people who had never served in the armed forces, those who were actively serving or had previously served in the Regular or Reserve Armed Forces, as well as their family members.

In December 2017, we published a further census topic research update, which highlighted work undertaken on the investigation of using administrative data to collect information on the armed forces community. The research had concluded that the administrative data available did not provide full coverage of those who previously served in the armed forces. Therefore, we had started investigating question designs collecting information on personnel who had previously served in the armed forces leading to the inclusion of a new question in Census 2021.

In December 2018, the government presented to Parliament a White Paper, Help Shape our Future: The 2021 Census of Population and Housing in England and Wales, (PDF, 967KB). This outlined our proposals to include the topic of the armed forces community (veterans) in Census 2021. Service providers would use these data to support veterans in line with the Armed Forces Covenant.

The question recommendations for Census 2021 are now finalised. We have evaluated the question for its potential impact on data quality, public acceptability, respondent burden, financial concerns and questionnaire mode. We present details of this evaluation in Annex 3.

We provide details of the definitions and terms used in this report in Annex 4.

As set out in the White Paper, Help Shape our Future: The 2021 Census of Population and Housing in England and Wales, (PDF, 967KB), we made a recommendation to collect information on veterans of the UK Armed Forces. We discuss the details of the testing related to the finalisation of the questions for Census 2021 in the section, Research that led to the recommended Census 2021 question design.

References to tests take the form (year: test number). "Year" refers to the calendar year the test was undertaken in and the test number is the position of the test within all testing that took place in that year. For example, the fifth test conducted in 2017 would be referenced as (2017:5).

It was agreed that the question scope would be limited to collecting information on armed forces leavers and their families who live at the same address. Because the census is a household form, the question will not identify family members who live separately to those who have previously served. In addition, where an individual has served and since died, their surviving family members will not be identified.

Developing an initial question for testing

We began our research by reviewing existing questions on other social surveys as potential designs for a census question. We looked at:

• Annual Population Survey (2015)
• British Social Attitudes Survey (2011)
• National Survey for Wales (2014 to 2015)

Of the three question designs, the Annual Population Survey’s (APS’s) first question was assessed as closest to meeting census user needs. This question was:

Have you ever served in the UK Armed Forces or in the UK Reserve Forces?

[ ] Yes - as a Regular in the Royal Navy or Royal Marines
[ ] Yes - as a Regular in the Army
[ ] Yes - as a Regular in the Royal Air Force
[ ] Yes - as a Reservist
[ ] No

We adapted the question to ensure it would work in a self-completion household questionnaire:

• we changed the question stem to “Have you previously served in the UK Armed Forces?” to remove repetition and focus the question on those who have previously served
• we added an instruction that specified what service should be included
• we reduced the number of response options, as there was no requirement to identify the particular branch of the armed forces the respondent had served in
• we added an instruction to “Please select all that apply” to ensure that a respondent’s full service history was captured

The question used in our initial testing was:

Have you previously served in the UK Armed Forces?

Please select all that apply

Include:

• UK regular Armed Forces
  
• UK reserve Armed Forces
  
• UK national service
  [ ] Yes – regular Armed Forces
  [ ] Yes – reserve Armed Forces
  [ ] No
  

This initial question design was cognitively tested (2017:5) with previous servers and current servers. This included people who had served in the UK Armed Forces and people who had served in the armed forces of other countries.

Testing identified that current servers interpreted “previously served” to include their current service and selected one or more “Yes” response options. When they were informed that they should have selected “No”, many reacted negatively, as they were proud of their service and wanted to know they were recognised and could be distinguished from a civilian.

We concluded that the distinction between current servers and those who had previously served would need to be clearer. Current servers must either be counted directly or given clear guidance to select “No”.

Following this test, we also made the following changes to the question design:

• we removed the detailed instructions, as the testing revealed that they did not help participants to respond
• we moved the “No” response option to the top, as it is standard practice to have the most commonly selected response option first
• we changed the dashes within the response options to commas, to align with Government Digital Service (GDS) Standards for improving text for screen readers and other assistive technologies

Ensuring an accurate response by current servers

We conducted two cognitive tests (2017:12 and 2018:14) focused on developing a question that currently serving members of the UK Armed Forces would understand as intended and answer correctly.

Question option 1: adding a response option for those currently serving

This question design changed the phrase “previously served” to “ever served” so it was clear it included current service. A response option for “Yes, currently serving” was added, and the “previously served” responses were amended. This question design, without any additional instruction on who to include, was: Have you ever served in the UK Armed Forces? Select all that apply
[ ] No
[ ] Yes, currently serving
[ ] Yes, previously served in Regular Armed Forces
[ ] Yes, previously served in Reserve Armed Forces

Testing (2017:12) found that current servers welcomed the new response option, as it enabled them to answer that they were serving quickly and accurately.

This finding was reinforced by a public acceptability survey (2018:27). The survey was targeted at participants who were currently serving, had previously served, or lived in a household where someone was currently serving or had previously served.

Overall, 88% of the 1,030 participants found the question acceptable. Those participants who did not consider the question to be acceptable mostly stated that was because they were unsure why they were being asked the question.

Question option 2: asking two questions, one on current service followed by one on previous service

A two-stage question was positively received by currently serving members of the armed forces, as it allowed them to answer that they were serving. However, this question design was considered more burdensome than the single question design owing to the need to read and answer a second question. Therefore, we did not continue to develop a two-stage question design.

Question option 3: clarifying the meaning of “previously served” within the question

We tested (2017:12) a question design that expanded the phrase “previously served” to “previously served and now left”, to make it clear that current servers should answer “No”.

To use this expanded phrasing within a single question design would result in both “Yes” response options containing three components: “Yes, previously served”, “in Regular/Reserve Armed Forces”, and “and now left”.

This question design increased respondent burden when answering the question. Therefore, we did not continue to include “and now left” in the response options.

Question option 4: explicitly including current servers in the “No” response option

We tested (2018:29) using a “No, or currently serving” response option as a way of directing current servers in how to respond. The test found that this was not clear enough to current servers, as they did not continue reading the response option after “No”, missing the second part of the response option that related to them. When participants were shown the response option, they reacted negatively to being grouped with those who had never served.

This design was also tested (2018:14) with those who had never served. While participants were able to answer the question correctly, they did not find the grouping intuitive.

Outcome of testing

The most successful question design was Option 1. The White Paper, published in December 2018, outlined our recommendation to ask a new question that captured those currently serving and those who previously served. This would meet the user need for data on those who have previously served in the UK Armed Forces and would be acceptable to those who are currently serving.

In the White Paper, Help Shape our Future: The 2021 Census of Population and Housing in England and Wales, (PDF, 967KB), we proposed introducing a new question collecting information on the armed forces community, including current and previously serving members and their families.

The data need has always been focussed on those who have previously served and now left, but it was suggested that collecting information on current and past serving personnel would be the best option to capture people that had previously served in the armed forces. In parallel with the testing conducted prior to the publication of the White Paper, we liaised with the Ministry of Defence (MoD). The MoD reviewed the inclusion of a “Yes, currently serving” response option and concluded it posed a potential security risk, as current servers would be identifying themselves. It was agreed that the MoD would provide the Office for National Statistics (ONS) with details of those currently serving in place of the response option. This risk concerned the data on the form before they were received by the ONS, not the storage of the data on our secure data systems.

Therefore, despite positive results from testing, we have recommended that the question does not collect data on those currently serving. Instead, we have recommended a question design that includes the instruction: “Current serving members should only tick ‘No’”.

Question used in the 2019 Rehearsal

The question design presented in the 2019 Rehearsal (2019:15) was:

Have you previously served in the UK Armed Forces?

Current serving members should only tick ‘No’
[ ] No
[ ] Yes, previously served in Regular Armed Forces
[ ] Yes, previously served in Reserve Armed Forces

The word “previously” in both response options was presented in bold text on the paper questionnaire. This reinforces the intention that the “yes” response option should only be selected by those who have previously served.

Changes to the question following the 2019 Rehearsal

Following the 2019 Rehearsal, we identified that some previously and currently serving personnel struggled to distinguish between the response options. We have recommended the following changes:

• present the instructional text “Current serving members” in bold text on the electronic and paper questionnaires
• present the “regular” and “reserve” part of the two “yes” response options in bold text on the paper questionnaire; this was not possible to replicate on the electronic questionnaire design template
• move to mutually exclusive question design by using an “or” between the “yes” and “no” response options

These actions are to prevent “no” being mistakenly selected alongside one of the “yes” options and to clarify the distinction between the Regular and Reserve “yes” options. Online, this includes considering adding mutually exclusive rules around valid responses. The mutually exclusive rules apply where two or more responses to a single question cannot both be true at the same time. The electronic questionnaire includes a functionality that de-selects previously ticked responses. For example, if a respondent selects the “no” response option in the armed forces (veterans) question, then any previously selected responses will be automatically de-selected. This functionality removes the possibility of a multi-tick error when processing the online returns.

Question design for online

User experience (UX) testing focuses on understanding user behaviours as people interact with online services. Through observation techniques, task analysis and other feedback methodologies, it aims to develop a deep knowledge of these interactions and what it means for the design of a service.

UX research has taken place on a rolling basis since 2017. Before the Census Rehearsal in October 2019, 458 interviews were conducted at 99 events. The UX testing programme will continue through 2020. All participants are purposively selected to include a wide range of ages and digital abilities.

UX testing testing included various iterations of the armed forces community question described elsewhere in this report. Feedback from this research informed decisions made on the design of this question. For more information on UX, see the question and questionnaire development overview for Census 2021.

Guidance accordion

During public acceptability testing (2018:27), we identified a need to provide context for why this question was being asked. On the electronic questionnaire, we have included a guidance accordion “Why your answer is important”. We developed a guidance accordion with stakeholders prior to a round of cognitive testing (2019:10) with participants who were previously or currently serving and proxy respondents for these groups. Participants found the guidance helpful but suggested that the wording should be more respondent focussed.

Following this suggestion, we refined the guidance accordion with stakeholder input. When the user selects the “Why your answer is important” text, the following guidance appears:

“Your answer will help your local community by providing information needed to support people who used to serve in the UK Armed Forces but have now left.

Councils and government will use this information to carry out the commitments they made under the Armed Forces Covenant. This is a promise by the nation to ensure that those who serve or have served in the UK Armed Forces, and their families, are not disadvantaged.”

This guidance accordion was included in the 2019 Rehearsal (2019:15). We found that no respondents had difficulty using or understanding the guidance accordion, and therefore we are recommending its inclusion in Census 2021. However, the text of the guidance accordion could change if there is enough evidence to support changing it.

Welsh language question development

Between 2017 and 2018, an external agency with Welsh-speaking researchers was commissioned to undertake focus groups (2017:17) and a series of cognitive interviews (2017:18). In 2018, further cognitive interviews (2018:40) were undertaken by the same agency. The qualitative research tested public acceptability and comprehension of amended and newly designed census questions in Welsh. The questions were tested with people across Wales with varying dialects and Welsh language proficiencies.

To ensure questions adhere to Cymraeg Clir guidelines, some changes to the text or questions across the census questionnaires were translated by our contracted specialist Welsh language translation service provider. These changes were quality assured by the Welsh Language Census Question Assurance Group. This group includes Welsh language and policy experts from the Welsh Language Commissioner and the Welsh Government and was convened to give advice on the accuracy, clarity and acceptability of the language as well as other policy issues pertaining to the Welsh language and bilingual design.

These tests did not include any current servers, and findings revealed very few respondents understood the meaning of the phrase “Lluoedd Arfog” (Armed Forces). Therefore, the recommendation was to include “er enghraifft, y fyddin” (for example, Army) within the question stem in the Welsh language version only. This was more commonly used and understood by Welsh speakers and directed them to the meaning of “Lluoedd Arfog”.

Another recommendation was that the abbreviation “DU” (UK) should be written out in full as “Deyrnas Unedig” (United Kingdom) to ensure better understanding.

Subsequent changes made to the final recommended question have been translated by our contracted specialist Welsh language translation service provider. The recommended changes adhered to Cymraeg Clir guidelines and were subsequently quality assured by Welsh language experts from the Welsh Language Commissioner and the Welsh Government.

The design of the questions recommended for Census 2021 was informed by the research and testing detailed in this report. The questions and response options for Census 2021 have now been finalised through the census secondary legislation. The paper questionnaires used in Census 2021 are available. Guidance text and instructions are not part of the legislation, but we consider these to be finalised as well. However, it is possible that guidance text or instructions may change if there is enough evidence to support doing so.

Armed forces community (veterans)

Figure 1: Online question on armed forces community (veterans) in English and Welsh

Source: The Office for National Statistics

Download this image Figure 1: Online question on armed forces community (veterans) in English and Welsh

.png (162.0 kB)

Question order

Census 2021 will be the first census to ask a question on the armed forces veteran community, so its position within the form needed to be considered. The final recommended order of all the questions on the household questionnaire can be seen in the question and questionnaire development overview for Census 2021.

To serve in the UK Armed Forces, a person needs to be aged 16 years or over. The existing set of questions, which were only asked of respondents aged 16 years or over, form the second half of the individual questionnaire. We considered three positions:

• after the gender identity question and before the qualification questions
• after the qualification questions and before the labour market questions
• after the labour market questions at the end of the questionnaire

In early testing (2017:5), the question was placed after the suite of labour market questions. This testing showed that participants who were currently serving were more likely to select a “Yes” response when the question was in this location because they are proud of the service and their current service is their main job. Owing to this influence, this position was discounted.

In public acceptability testing (2018:27), gay, lesbian and bisexual members of the armed forces community were less likely to view the armed forces community question as acceptable than members of the armed forces community who were heterosexual. Therefore, we recommended separating the armed forces leaver question from the questions on sexual orientation and gender identity. By separating these questions, we expect a minimal negative effect on the response rate of the armed forces community (veterans) question and the overall completion rate of the questionnaire.

We have therefore recommended positioning the armed forces community (veterans) question between the qualifications questions and the suite of labour market questions. This positioning also provides a logical progression through a respondent’s life: qualifications, armed forces, labour market.

Question evaluation

The question design put forward in this report is based on extensive research and assessment using evaluation criteria that were set out in the publication, The 2021 Census – Assessment of initial user requirements on content for England and Wales: Response to consultation (PDF, 796KB).

The evaluation considered the potential impact that including an armed forces community topic on the census would have on data quality, public acceptability, respondent burden, financial concerns and questionnaire mode. The evaluations were used in conjunction with the user requirements criteria to steer the development of the census questions and questionnaire.

At the time the consultation was conducted, we had not yet proposed asking a question on the armed forces community, as we were committed to investigating alternative administrative data sources. Therefore, the armed forces community topic was not evaluated alongside other topics.

After completing the research and development phase, we evaluated our recommended questions against the same criteria using an updated tool that considers the types of evidence we have available and the Census 2021 context. A description of this updated evaluation tool is provided in the question and questionnaire development overview for Census 2021.

All questions meet our thresholds to ensure reliable information will be collected in Census 2021. The “potential for impact” categorisations are provided within the context of other census questions. Therefore, a question that was categorised as “high” is closer to the threshold for exclusion than a question that was categorised as “low”, but both questions will capture reliable information.

Table 1 provides the evaluation scores for the question on armed forces community (veterans). We present the evidence used to assess the question in Annex 3.

Question harmonisation

As in previous censuses, there will be separate censuses in Scotland and Northern Ireland. The questions for England and Wales have been developed through close collaboration with National Records of Scotland (NRS) and the Northern Ireland Statistics and Research Agency (NISRA), who are responsible for conducting the censuses in Scotland and Northern Ireland, respectively.

We recognise that each country has its own user and respondent needs. However, we aim for harmonisation of census questions and topics where possible in order to produce UK-wide statistics that are consistent and comparable.

The same question will be asked in Scotland's Census 2021, but the response options are in a different order. The Census 2021 question for Northern Ireland is still under review, and updates will be published on NISRA's news and events.

The armed forces community (veterans) questions were developed for use in the context of Census 2021 in England and Wales, a mandatory household form. Therefore, it is possible that in different contexts, such as social surveys, a different approach may be more suitable.

There is currently no Government Statistical Service (GSS) harmonised principle for collecting data on the armed forces community.

In our response to the 2021 Census topic consultation, the Office for National Statistics (ONS) had no plans to undertake further work for a census question on this topic. However, we committed to two strands of work on the armed forces community topic.

Develop our understanding of user needs for information on armed forces service leavers

We held a stakeholder workshop with the Ministry of Defence (MoD), Royal British Legion, Department of Health and Social Care (DHSC), Local Government Association, and a number of local authorities and county councils with experience in delivering services to armed forces leavers and the armed forces community.

This workshop concluded that users required information on:

• those who are, or were, regulars or reservists, in any role, to ensure that the commitments of the armed forces covenant are met
• the dependents of those persons who are serving, or have served, to provide education and health services
• persons who left the armed forces and are of working age, so that their health and employment outcomes can be reviewed

This clearer understanding of the user need being focused on service leavers fed into the second strand of research into the use of administrative data as a way of meeting this user need.

Explore alternative sources that could meet the user need

We worked with the MoD to explore whether the Veteran Leavers Database would meet the user need. While there is value in the dataset on service leavers, it only partially meets the user need. We provided an update on this work in October 2017.

Following the publication of this update, we committed to including a question on armed forces leavers in Census 2021.

References to tests take the form (Year: Test number). “Year” refers to the calendar year the test was undertaken in, and the test number is the position of the test within the year considering all testing that took place in that year. For example, the fifth test conducted in 2017 would be referenced as (2017:5).

A full description of each of these items can be found in summary of testing for Census 2021.

All questions meet our thresholds to ensure reliable information will be collected in Census 2021.

A question that has been assessed as having a “High” potential for impact is closer to the threshold for exclusion from the census than a topic that has been assessed as having a “Low” potential for impact.

Potential for impact on data quality, public acceptability and respondent burden were all rated as “Low”.

Potential for impact on financial concerns: “Medium”

This question contains potentially sensitive information and could cause a respondent to request an individual form.

Potential for impact on questionnaire mode: “Medium”

This question asks for sensitive information, is visually different online and on paper, and contains guidance information in an accordion on the electronic questionnaire.

The armed forces community describes:

• current servers (including regular and reservist personnel)
• veterans
• those who have died
• their families

All members of the armed forces community are covered by the Armed Forces Covenant. This is a promise by the nation that ensures those who serve or have served in the armed forces, and their families, are not disadvantaged.

Although the census topic is the armed forces community, the final recommended question has been developed so it focuses on those who have previously served in the UK Armed Forces for at least one day and have now left. This group is referred to as “veterans” or “armed forces leavers”. This service can be as regular or reservist within any of the following institutions:

Regular Armed Forces

• Royal Navy Regular
• Royal Marines Regular
• British Army Regular
• Royal Air Force Regular
• National Service
• Gurkhas
• Ulster Defence Regiment
• Royal Irish Rangers
• Royal Irish Regiment

Reserve Armed Forces

• Full Time Reserve Service
• Part Time Reserve Service
• Royal Naval Reserve
• Women's Royal Naval Service
• Royal Marines Reserve
• Royal Fleet Reserve
• Royal Fleet Auxiliary
• Royal Navy Sponsored Reserve
• Army Reserve
• Territorial Army
• Auxiliary Territorial Service
• Army Sponsored Reserve
• Ulster Defence Regiment
• Royal Irish Rangers
• Royal Irish Regiment
• Royal Air Force Reserve
• Royal Air Force Sponsored Reserve
• RAF Reserve of Officers
• RAF Reserve of Ranks
• Royal Auxiliary Air Force
• Women's Auxiliary Air Force
• Royal Air Force Volunteer Reserve
• Royal Air Force Sponsored Reserve
• Called-out Reservists
• Home Guard

Families of armed forces members

The Armed Forces Covenant defines “family” as comprising of:

“not only those we recognise in terms of entitlement, but those we need to engage and communicate with […] Accordingly, the definition of family will be different for different purposes – all-encompassing when it comes to communications and rightly narrower in definition when it comes to defined benefits.”