Developing methods to produce Census 2021 income data, England and Wales: June 2023

This article provides an update on our research to develop "census-type income data that can be integrated with the data collected on the 2021 Census", as set out in the Census 2021 White Paper (PDF, 967KB).

This commitment is in response to the clear need of our users for income data produced for small geographies or populations sub-groups. This was not delivered through adding an income question to the Census 2021 data collection, as it was found in our 2007 Census Test (PDF, 449KB) that including an income question on the census had an unacceptable impact on response rates and data accuracy.

The Office for National Statistics (ONS) already produce National Statistics for Income estimates for small areas, using a combination of aggregate survey and administrative data through modelling. However, by meeting the White Paper commitment we aim to produce individual estimates linked to Census 2021, that will allow more granular breakdowns by subgroups of the population and their characteristics.

Our ongoing development of the Admin-based income statistics (ABIS) has also demonstrated the potential of using administrative data to estimate income for individuals and occupied addresses. This forms part of our ambition to put administrative data at the core of our future population and migration statistics system. This ambition underpins our consultation on the future of population and migration statistics in England and Wales, launching on 29 June 2023. The Consultation informs the National Statistician's forthcoming recommendation on the future of population and migration statistics in England and Wales.

Our Working paper series no 21 methodology, published in January 2021 presented results from a proof of concept study using administrative data linked to 2011 Census data. This study explored the feasibility of using imputation to account for missingness in these administrative data sources.

This article provides information on new exploratory research to investigate an alternative approach to developing income data linked to Census 2021. It explores whether administrative and survey data can be used in combination to produce an income estimate for every individual in the census population of interest.

This research contributes to our goal of meeting our Census White Paper commitment to provide users with the small area income data they need for the Census 2021 population, using the best combination of methods and data sources. It also demonstrates the potential for using linked administrative and survey data to produce timely small area estimates of income in future.

The latest admin-based income statistics (ABIS) (tax year ending 2018) demonstrated that our current admin-based methods can identify income information from at least one ABIS data source for the majority of individuals aged 16 years and over in England and Wales on our population base. This research shows potential for producing small area ABIS within a similar timeframe to the current system (two years after the reference date) subject to data availability.

We have also demonstrated the potential for producing these more frequently (annually), and more granularly (Lower layer Super Output Area, LSOA) than the Income estimates for small areas, England and Wales that are published every two years for Middle layer Super Output Area (MSOA), subject to data availability. 

While these findings show potential for this administrative data approach, we are still not able to identify any income information using these methods for some individuals in our population base. Our latest ABIS release presented several possible causes for this, including income information missing because administrative data sources are unlikely to be able to capture certain income components, or because missing or unavailable pseudonymised identifiers mean we have failed to link individuals in the population base to their income record(s) in the administrative data.

Ongoing development of our ABIS research will explore expanding the ABIS to capture additional income components currently missing from our measures.

Our previous proof of concept research explored the feasibility of using a nearest neighbour donor-based approach to impute an administrative-based income variable linked to the 2011 Census data to account for those cases that failed to link to income record(s). This research demonstrated that the approach was able to successfully impute the person-level dataset, but only where the source of missingness was linkage error.

Our current research explores an alternative approach and aims to use linked administrative, survey and census data sources to explore whether they can be used in combination to create a modelled income estimate for everyone in the census population.

As a preliminary step towards this, a linked administrative, survey and census analytical dataset was created to assess the viability of using these data sources in combination and to understand further data requirements. We linked the following pseudonymised data sources at an individual level:

• Census 2021 population data, providing the full census population base

• admin-based income data, providing benefits and Pay As You Earn data for the tax year ending 2021

• Living Costs and Food Survey (LCF) respondent data, providing data for January to March 2021

This linkage produced an individual-level dataset with census, administrative and survey income information for individuals, where available. The data sources used at this stage were selected to simplify the linkage process. They have a number of limitations relative to our ultimate aim of creating an income estimate but allow us to develop a better understanding of the data requirements for future stages of the work.

Self Assessment administrative data were only available for the tax year ending 2018 and have therefore been excluded from the analyses. Previously published linked data comparisons of self-employment income using Self Assessment and LCF data sources complement the research presented here.

The linked administrative, survey and census analytical dataset was used to produce an initial assessment of how the capture of income may differ between these sources at the individual level. This will feed into the design of our next research steps towards delivering against the Census 2021 White Paper commitment.

This exploratory research suggests that:

• current admin-based methods identify income information from at least one ABIS data source for the majority (86%) of the census population of interest

• the number and type of income sources captured for individuals in the admin-based and survey-based data sources were broadly similar, with further investigations needed to explain differences observed for some income sources

• there is some variation in the income amounts captured for each income source in the admin-based and survey-based data sources, although comparisons of income percentiles show the two data sources follow relatively similar trends

These initial insights must be considered alongside the limitations of this initial research, including the small survey sample and non-aligned time periods used, meaning any conclusions should be drawn with caution.

The linked admin-based and survey-based comparisons have focused on the income sources available across both data sources (see the Glossary section for a full list of income sources included). However, these results demonstrate progress towards the development of "census-type income data that can be integrated with the data collected on the 2021 Census".

Building on this preliminary stage, the next phase of the research will aim to develop an income variable linked to the census population, assessing methodological options for using administrative data in combination with survey data. This will also include building on this preliminary stage to widen and update data sources used and explore the most appropriate linkage methods.

This exploratory research will form part of our wider work towards delivering the Census 2021 White Paper commitment by April 2024. It will inform our development of income estimates as part of our ambition to put administrative data at the core of our future population and migration statistics system.

Income source

Income source refers to the different sources from which an individual can receive income. The income sources included in this exploratory research are those that were available across both the administrative and survey data sources specified in Section 2: New exploratory research to develop Census 2021 income data.

These sources include:

• Pay As You Earn (capturing employed and occupational and private pensions income)

• Attendance Allowance

• Bereavement Benefit and Widow's Benefit

• Carer's Allowance

• Disability Living Allowance

• Employment and Support Allowance (including contribution-based)

• Incapacity Benefit

• Income Support

• Jobseeker's Allowance

• Pension Credit

• Severe Disablement Allowance

• State Pension

• Housing Benefit

• Personal Independence Payment

• Universal Credit

• Child Benefit

• Working Tax Credit

• Child Tax Credit