1. Output information
National statistic: No
Data collection: Administrative data
Frequency: Ad hoc
How compiled: Administrative data from HM Revenue and Customs and the Department for Work and Pensions
Geographic coverage: National, regional, local authority and Lower layer Super Output Area (LSOA) in England and Wales
2. About this Quality and Methodology Information report
This Quality and Methodology Information report contains information on the quality characteristics of the data (including the European Statistical System's five dimensions of quality) as well as the methods used to create it.
The information in this report will help you to:
understand the strengths and limitations of the data
learn about potential uses and users of the data
understand the methods used to create the data
reduce the risk of misusing data
3. Important points
The admin-based income statistics (ABIS) provide experimental measures of individual and occupied address income and have been developed to investigate the potential of administrative data to measure income for small areas.
This research forms part of our population and social statistics transformation programme.
The ABIS are produced using a variety of administrative data from sources such as the Department for Work and Pensions and HM Revenue and Customs, as well as those feeding into the ABIS' population base.
Development of the ABIS is guided by the Canberra Group Handbook on Household Income Statistics, which provides guidance on defining and measuring components of income.
Their incomplete nature, as per the Canberra Group Handbook, means the ABIS should not be used to infer current financial trends of individuals and occupied addresses; the income estimates for small areas are currently the recommended estimates for small area household income.
4. Quality summary
Overview
Admin-based income statistics (ABIS) are being developed to investigate the feasibility of using administrative data sources to measure income for small areas in England and Wales.
The 2021 Census topic consultation identified a clear user need for small area income data produced for small geographies or populations.
However, previous census testing showed an unacceptable impact of capturing income on response rates and data accuracy, therefore, income information was not collected in Census 2021.
Instead, we have been working with tax and benefits data from the Department for Work and Pensions (DWP) and HM Revenue and Customs (HMRC) to investigate the potential for administrative data to meet this user need.
The ABIS provide experimental measures of individual and occupied address income. Those identified with some source of income in the administrative data sources, while also appearing in our population base (Statistical Population Dataset, V3.0, 2018), are included in the ABIS.
This research forms part of our population and social statistics transformation programme, which aims to provide the best insights on population, migration and society using a range of data sources. These findings will contribute to the evidence base for the National Statistician's Recommendation in 2023 on the future of population, migration and social statistics in England and Wales.
Uses and users
Income information is used by a range of users including central and local government. It is used in many ways including service planning, policy development and strategic planning to tackle issues such as debt and poverty.
The ABIS demonstrate the potential for administrative data to meet the identified user need for income data for small geographies or populations, as identified in the 2021 Census topic consultation. Ongoing development of these Experimental Statistics is shared with users for information through ad hoc publications, with users invited to assess these developments against their needs and provide feedback to guide future developments.
The ABIS currently have limited use for decision making, with the income measures and coverage incomplete at present.
Strengths and limitations
Strengths
The ABIS demonstrate the potential of administrative data to better meet user needs for small area income statistics. The administrative data are provided at record level, allowing the ABIS to provide income distributions at lower levels of geography (for example Lower layer Super Output Area level) than is typically possible using survey-based methods.
These record-level data allow for the production of multivariate statistics, through linking the ABIS to other record-level admin-based datasets, such as ethnicity statistics.
There are known issues in existing survey data collection methods regarding the underreporting of benefit receipts, resulting in underestimating gross income for low-income families. For example, further information can be found in the Resolution Foundation's Improving our understanding of UK poverty will require better data report (PDF, 448KB). The use of administrative data can help to overcome this issue through the accurate capture of benefit income, sourced from the relevant operational systems, without the reliance on self-reported methods.
Limitations
The ABIS are currently badged as Experimental Statistics because of their incomplete nature. We need to develop further to fully achieve our ambition to put administrative data at the centre of our official income statistics, supported by data from our social surveys. This includes the development of quality measures to accompany these developing statistics.
The method to produce the ABIS is reliant on the ability to bring administrative data sources together, at multiple stages of the production process, to create a combined measure of income per person. This is achieved through linkage of these data sources using pseudonymised identifiers. The use of data linkage introduces a risk of missed links, where we are unable to link an individual's data across sources because of missing or incorrect pseudonymised identifiers. This could result in reduced coverage from the administrative sources, where an individual's income is not being captured despite being present in the data. As our linkage methods are further refined, the quality of the linkage should improve.
The ABIS impute two benefit types (Christmas Bonus and Winter Fuel Payment) which are not currently captured in the available administrative data sources. The imputation methods aim to align with the eligibility criteria for each benefit, with available administrative sources used to improve this alignment. However, there will be a risk of under or overestimation when imputing measures which are currently not captured directly from a data source.
The ABIS produce income statistics for individuals and occupied addresses. The use of occupied address to capture household income differs from the concept of a household used in the production of survey and census statistics. The concept of a household is challenging to meet when using administrative data as these data are usually collected at an individual level. We can identify when individuals are living at the same address by linking administrative data, however it is much harder to identify whether those individuals share cooking facilities or social space (as per the household definition). Future research can explore the extent to which one occupied address is equivalent to one household, alongside investigation into the extent to which this definition meets user needs.
Recent improvements
Recent improvements include:
incorporating Personal Independent Payment and Universal Credit benefit income to improve our coverage of benefits income
updating the reference period of the ABIS from the tax year ending 2016 to tax year ending 2018
5. Quality characteristics of the admin-based income statistics data
This report provides information that describes the quality characteristics of the data and identifies issues that should be noted when interpreting the outputs.
Relevance
(Relevance is the degree to which the statistical product meets user needs in terms of coverage and content.)
The admin-based income statistics (ABIS) are being developed to investigate the potential of administrative data to measure small area income, for which a strong user need was identified in the 2021 Census topic consultation.
We have regularly shared our progress with users including highlighting gaps in coverage and content to ensure the statistics are developed in line with user needs and remain relevant. Feedback received following previous releases of the ABIS have directly fed into the ongoing development of these statistics, with ABIS publications highlighting where specific feedback has been addressed.
The recently opened Household Financial Statistics Consultation is also seeking to understand views on our wider plans to transform our household financial statistics. Responses to this consultation will feed into the ongoing development of the ABIS.
Accuracy and reliability
(Accuracy is the degree of closeness between an estimate and the true value.)
The ABIS are derived from administrative data collected and processed by the Department for Work and Pensions (DWP) and HM Revenue and Customs (HMRC). These data are produced for operational rather than statistical purposes, which can impact on the accuracy of the administrative data produced. More information on the accuracy of the data sources used in the ABIS is provided in the main data sources and accuracy section.
Quality checks are completed at multiple stages of the data journeys, as well as during the ABIS production and publication stages.
Coherence and comparability
(Coherence is the degree to which data derived from different sources or methods, but refer to the same topic, are similar. Comparability is the degree to which data can be compared over time and domain, for example, geographic level.)
The ABIS are being developed to investigate the feasibility of using administrative data to produce small area income statistics in place of capturing income through the census. As such, a census measure of small area income is not available to assess coherence.
The closest measure to assess coherence against the ABIS is the income estimates for small areas. These estimates are produced at the Middle layer Super Output Area (MSOA) level for England and Wales. They are produced using a model-based approach, combining Family Resources Survey, census and administrative data sources.
Direct comparison between these model-based estimates and the ABIS is not currently possible. The two measures cover different components of income (given the ABIS are still under development) and are produced at different geographical levels (with the ABIS produced for Lower layer Super Output Areas (LSOA)). An important goal within the Household Financial Statistics Transformation is to improve the coherence of our measures of income, which will include investigating the coherence between these two income releases for local areas.
In the absence of a direct comparator, the ABIS have been compared with survey-based gross household income data sourced from the effects of taxes and benefits on UK household income (ETB) and the households below average income (HBAI) statistical releases. The type of statistics, income components, methods of analysis and data sources for these statistics differ, however we have previously compared regional ABIS aggregates with equivalent ETB and HBAI aggregates to help evaluate the ABIS.
The ABIS are not currently comparable over time, given the changing nature of the underlying methodology as we develop these statistics. As the methodology becomes more complete, there is potential for future research into the comparability of the ABIS over time.
Accessibility and clarity
(Accessibility is the ease with which users can access the data, also reflecting the format in which the data and supporting information are available. Clarity refers to the quality and sufficiency of the metadata, illustrations, and accompanying advice.)
Our recommended format for accessible content is a combination of HTML web pages for narrative, charts and graphs, with data being provided in usable formats such as Excel spreadsheets and comma-separated values (CSV) files. Users can also download the narrative in PDF format. In some instances, other software may be used, or may be available on request. Available formats for content published on our website, but not produced by us, or referenced on our website but stored elsewhere, may vary. For further information, contact Admin.Based.Characteristics@ons.gov.uk.
For information regarding conditions of access to data, please refer to the following links:
Timeliness and punctuality
(Timeliness refers to the lapse of time between publication and the period to which the data refer. Punctuality refers to the gap between planned and actual publication dates.)
Progress in developing these experimental statistics is shared on an ad hoc basis to seek user feedback, for example, following the inclusion of new sources of administrative income data. They are not published to provide users with statistics reflecting the latest income trends for the population, therefore the timeliness of these statistics is of less relevance at this stage.
Our ability to produce timely statistics is also heavily reliant on the timeliness of the administrative data sources, which will be affected by the design of the underlying operational system. For example, HMRC's current Self Assessment system requires individuals to submit their tax return after the end of the tax year (5 April), with deadlines for submitting this information ranging between 31 October and 31 January depending on mode of submission. This will affect the timeliness of outputs, such as the ABIS, which rely on Self Assessment data for their production.
Concepts and definitions
(Concepts and definitions describe the legislation governing the output and a description of the classifications used in the output.)
We are developing the ABIS to align with the Canberra Group Handbook on Household Income Statistics, which is widely agreed as the international standard for household income statistics. It provides detailed guidance on how components of income should be defined and measured.
The ABIS remain under development, therefore the concepts as defined here will continue to change and may not fully align with the concepts as defined in the Canberra Group Handbook.
Gross income
Gross income derived in the ABIS is the income received before Income Tax and National Insurance contributions are deducted.
Gross income as captured in the ABIS consists of:
- income from employment, including employee income captured by HMRC's Pay As You Earn (PAYE) system and income from self-employment as reported by HMRC's Self Assessment system
- current transfers received, including occupation and private pensions captured by HMRC's PAYE system, social assistance benefits, pensions or schemes as listed in Section 6: Methods used to produce the admin-based income statistics data, and imputed measures of DWP's Christmas Bonus and Winter Fuel Payment
Net (disposable) income
Net income is calculated by deducting Income Tax and National Insurance contributions (derived in accordance with HMRC guidelines for the particular tax year) from gross income.
Individual income
Individual income refers to the income a single individual receives, and contributes to the income of their occupied address.
Occupied address income
The ABIS use the concept of occupied address, grouping individuals who are recorded as living at the same address, to capture household income. A unique property reference number (UPRN) is used to group individuals at the same address, with UPRN assigned to all individuals resident on the population base, where one can be identified. For our Admin-based income statistics, England and Wales: tax year ending 2018 article, 97% of Statistical Population Dataset V3.0, 2018 records were assigned a UPRN.
When using measures of occupied address income, we assume that the income is shared evenly between all occupied address members. This assumption might not be true for all occupied addresses.
Equivalisation
To increase the comparability between different types of occupied addresses, the admin-based occupied address income statistics are adjusted to account for the different resource needs of occupied addresses of different sizes and compositions. For example, occupied addresses with many members are likely to need a higher income to achieve the same standard of living as occupied addresses with fewer members. This process is referred to as equivalisation.
The Organisation for Economic Cooperation and Development-modified (OECD) equivalence scale is used to equivalise our gross and net occupied address income measures. This has been rescaled to use an adult couple occupied address with no children as the reference group, rather than the single adult reference group used in the original OECD-modified scale. This aligns our methodology with those used by ONS' Effects of taxes and benefits on household income and DWP's Household below average income official household income statistics.
During equivalisation, each occupied address member is allocated a standard weighting, as outlined in Table 1. The weights of all occupied address members are summed to create the total occupied address weight. The total occupied address income is then divided by the occupied address weight to give an equivalised occupied address income.
Type of occupied address member | OECD-modified scale – rescaled to couple without children |
---|---|
First adult | 0.67 |
Additional adult | 0.33 |
Child aged 14 and over | 0.33 |
Child aged 0-13 | 0.2 |
Download this table Table 1: Organisation for Economic Cooperation and Development-modified equivalence scale weights
.xls .csvThis method would result in income increasing for occupied addresses consisting of a single adult, remaining the same for occupied addresses with two adults, and decreasing for occupied addresses with more than two adults.
After equivalisation, the occupied address income statistics such as percentiles and median are produced using the individual as the unit of analysis.
Usually resident population
Usually resident population refers to people who reside in the area for a period of at least 12 months, whatever their nationality, in line with the standard UN definition for population estimates.
Further information on defining the usually resident population is available in our Update on our population and migration statistics transformation journey: a research engagement report.
Geography
The ABIS are produced at a national, regional, local authority, and LSOA level for England and Wales, with geographical information sourced from the Statistical Population Dataset, V3.0, 2018.
Back to table of contents6. Methods used to produce the admin-based income statistics data
Main data sources and their accuracy
The admin-based income statistics (ABIS) are produced using a combination of Department for Work and Pensions (DWP) and HM Revenue and Customs (HMRC) administrative data sources, and Office for National Statistics (ONS) statistical datasets. The ONS datasets are produced using health, education and life events administrative data sources. These administrative data are sourced from operational systems and delivered to us for use in statistics production and statistical research. As such, these administrative data are reliant on the accuracy of information recorded and processed in these operational systems.
DWP National Benefits Database
The National Benefits Database (NBD) contains information on 13 different benefits distributed by the DWP:
- Attendance Allowance
- Bereavement Benefit
- Carer's Allowance
- Disability Living Allowance
- Employment and Support Allowance
- Incapacity Benefit
- Income Support
- Jobseeker's Allowance
- Pension Credit
- Passported Incapacity Benefit
- Severe Disablement Allowance
- State Pension
Widow's Benefit
The NBD data are created on a monthly basis, capturing claim-level information about the benefits an individual has claimed past and present.
The data in the NBD are processed for statistical purposes, producing a snapshot of the benefits system at a certain point in time. Data from multiple sources are combined using well-defined rules to ensure all claimants are correctly classified, with appropriate spells data showing when their claims begin and end.
The NBD data used in our ABIS methodology provide record-level NBD information for each individual, with claim start and end dates and weekly amounts recorded. From this we produce an annual NBD income amount for each benefit. The annual amount derived from these data is an approximation as the weekly benefit amount specified in the data is a snapshot and the amount paid could vary over the claim duration.
Further information regarding DWP NBD data can be found in DWP's Methodology statement: DWP benefits statistical summary.
DWP Single Housing Benefit Extract
Housing Benefit supports individuals who are unemployed, on a low income or claiming other qualifying income-related benefits to pay their rent. It is classed as a "legacy" benefit and is being replaced by Universal Credit for working age claimants (see our Universal Credit section for more details).
Housing Benefit data returns are collated from local authorities to produce a single data file called the Single Housing Benefit Extract (SHBE). The SHBE dataset contains data about all Housing Benefit and Council Tax Benefit claims and includes a range of characteristics about the status of each claim, the personal characteristics of claimants, payable amounts and any deductions that may have been made. Data are collected using well-defined specifications to ensure consistency and quality.
The SHBE data can be prone to known data quality issues as the information supplied by local authorities may be incomplete or missing for a given month. To account for this, the DWP apply threshold-based quality checks to determine where issues may be present in the data. Those data identified as falling outside of these acceptable thresholds will be replaced by the DWP with data from the previous month. The DWP record how often this occurs for each local authority to identify emerging or persistent quality issues, which can then be fed back to the local authority.
The SHBE data used in our ABIS methodology provide record-level SHBE information for each individual, with claim start and end dates and weekly rates recorded. From this we produce an annual SHBE income amount for each individual.
Further information regarding DWP SHBE data can be found in DWP's Housing Benefit caseload and flows statistics: methodology statement.
DWP Personal Independence Payment
Personal Independence Payment (PIP) is a benefit that can be claimed to help with extra living costs if you have both a long-term physical or mental health condition or disability, and difficulty doing certain everyday tasks or getting around because of your condition. It can be claimed alongside work, savings, or most other benefits. It is replacing the Disability Living Allowance for most adults.
The PIP data are transferred from the live operational system to the PIP data store daily, with a lag of two to three days from when the live case management system records any changes. DWP staff follow detailed guidance and undergo training to minimise errors during data entry.
The PIP data used in our ABIS methodology provide payment-level PIP information for each individual, with payment due dates and benefit amounts recorded. From this we produce an annual PIP income amount for each individual.
Further information regarding DWP PIP data can be found in DWP's PIP statistics: background quality and methodology report.
DWP Universal Credit
Universal Credit (UC) aims to help with living costs for those who are on a low income, out of work, or cannot work. Support for housing costs, children and childcare costs are integrated into UC. It also provides support for people with a disability, health condition or caring responsibilities which may prevent them from working. UC has been created to replace the following legacy benefits and tax credits:
- Child Tax Credit
- Housing Benefit
- Income Support
- Income-based Jobseeker's Allowance
- Income-based Employment and Support Allowance
- Working Tax Credit
The Government first launched UC in 2013, rolling it out in stages, and expects all households claiming legacy benefits and tax credits to have moved across to UC by the end of 2024. While this migration takes place it is likely that individuals will appear in both the legacy benefit and UC systems, as individuals who are still eligible for the legacy benefit will keep receiving payment for two weeks after applying for UC.
UC data are obtained from the UC Full Service administrative system, which consists of information entered and submitted by claimants, agents and work coaches and through automated procedures.
The UC data used in our ABIS methodology provide payment-level UC information for each individual, with payment dates and amounts recorded. From this we produce an annual UC income amount for each individual.
Further information regarding DWP UC data can be found in DWP's Universal Credit statistics: background information and methodology.
HMRC Child Benefit
Child Benefit (CB) is paid to those responsible for bringing up a child aged under 16 years or a qualifying young person (under 20 if they stay in approved education or training), with only one individual receiving CB for a child. CB claimants are liable to repay some or all of their CB if they or their partner have an individual income of more than £50,000 per year. CB data are sourced from HMRC's Child Benefit System, which holds information on the number of CB claimants and their children.
The CB data used in our ABIS methodology provide individual-level CB information for each tax year, with an annual CB income amount recorded for each individual.
Further information regarding HMRC CB data can be found in HMRC's Child Benefit Statistics: Quality report, data as at August 2021.
HMRC Tax Credits
Tax Credits (TC) include Working Tax Credit (WTC) and Child Tax Credit (CTC).
WTC provides in-work support for people on low incomes, with or without children, for individuals who are aged 16 years or over and who meet certain criteria.
CTC provides income-related support for children and qualifying young people aged 16 to 19 years who meet certain criteria, payable to the main carer. Families can claim CTC regardless of whether or not the adults are in work.
Details of all TC claims and claimants are held on the National TC system. Information is held on the number of WTC and CTC claimants and whether they are overpaid, underpaid or paid the entitled amount within a specific tax year.
TC data are based upon the information provided by the claimant in their initial application. Claimants are sent an annual renewal pack to confirm that the information held by HMRC remain accurate. Claimants are required to inform HMRC if there are changes to their circumstances in the intervening period.
The TC data used in our ABIS methodology provide claim-level TC information for each individual, with an annual TC income amount recorded for each claim.
Further information regarding HMRC TC data can be found in HMRC's Quality Report — Child and Working Tax Credits statistics: Provisional awards — April 2022.
HMRC Pay as You Earn
Pay as You Earn (PAYE) is the system through which HMRC collects information about earnings, Income Tax, National Insurance Contributions (NICs), and workplace pensions from employers. Employers risk fines if they fail to file this information or make errors in the information reported.
In April 2012, HMRC started the phased introduction of Real Time Information (RTI) for PAYE. This system requires employers registered on the PAYE system to submit real-time payment information to HMRC. Employers are not required to use the PAYE system if none of their employees are paid above the weekly National Insurance threshold (the Lower Earnings Limit), get expenses and benefits, have another job, or receive a pension. Employers submit a Full Payment Submission (FPS) every time employees are paid, on or before the date each salary or pension payment is made. An FPS contains the employees' personal details, earnings they have been paid, deductions of tax and NICs made and their year-to-date amounts of pay and deductions. It also provides details of any statutory payments (for example, maternity pay) the employer has paid out to employees. These submissions are fed into the National Insurance and PAYE Service system, which calculates individuals' overall tax and other payments.
Before the introduction of the PAYE RTI system, employers were required to submit a P14 form for each employee for whom they had been required to complete a P11 deductions working sheet during the year. Employers were required to submit the P14 form at the end of the tax year, which captured the relevant earnings, Income Tax, NICs and workplace pensions information for the preceding tax year as an annual amount. P11 deduction working sheets were not required to be completed for employees who were paid less than the Lower Earnings Limit.
Our ABIS methodology currently sources PAYE data from HMRC's P14 records. Research is being conducted to understand the feasibility of using RTI data as a replacement in future iterations of our ABIS methodology.
The PAYE P14 data used in our ABIS methodology provide individual-level PAYE P14 information for each tax year, with an annual PAYE P14 income amount recorded for each individual.
Further information regarding HMRC PAYE RTI data can be found in the ONS and HMRC's New methods for monthly earnings and employment estimates from Pay As You Earn Real Time Information (PAYE RTI) data: December 2019.
HMRC Self Assessment
Self Assessment (SA) is a system used by HMRC to collect Income Tax. Tax is usually deducted automatically from wages and pensions through the PAYE system. Tax on bank and building society interest is also collected from most individuals through the PAYE system. However, individuals and businesses with other types of income must report their income via the Self Assessment system.
The SA system is used to capture incomes such as:
self-employment
partnership
UK property
foreign income or gains
trusts
capital gains
We currently use a subsection of SA data in our ABIS methodology to capture self-employment income. We have plans for future research to investigate the potential to capture additional SA income sources in line with the Canberra Group Handbook definition of household income.
Individuals must complete a tax return if they were self-employed as a sole trader and earned more than £1,000 or if they were a partner in a business partnership in the last tax year. A single tax return can include multiple SA forms, with the form(s) submitted dependent on the individual's circumstances.
Individuals meeting the requirements for a self-employed sole trader must complete form SA103, while partners in a business partnership complete form SA104. A third form — SA200 — is completed by a subset of self-employed individuals and partners who are deemed by HMRC to have simpler tax affairs. These individuals complete form SA200 in place of SA103 or SA104.
Sole traders with a turnover below the mandatory £1,000 threshold can still choose to submit a SA tax return, for example to gain access to benefits or to provide proof of income.
Those returning SA forms must do so by a particular deadline or risk having to pay interest or a penalty. This is likely to improve the accuracy of the information provided by individuals. The self-reported nature of SA tax returns does however introduce the risk of misreporting incomes, for example, through human error or tax evasion.
The SA data used in our ABIS methodology provide return-level SA information for each individual per tax year, with various variables recorded, such as turnover and expenses. From this, we produce an annual SA self-employment income amount for each individual. Our current measure of self-employment income incorporates SA data from forms SA103 and SA104, with work ongoing to incorporate form SA200 in future versions of the ABIS methodology.
Further information regarding HMRC SA data can be found in HMRC's Personal Incomes Statistics 2019 to 2020: Supporting Documentation. SA data feed into these Personal Income Statistics through HMRC's Survey of Personal Incomes, which samples individuals from HMRC's PAYE, SA and Claims operational systems.
ONS Statistical Population Dataset
The Statistical Population Dataset (SPD) is an administrative data-based "usually resident" stock population developed by the ONS. The SPD is under development and forms part of our population and social statistics transformation programme.
The version used in our Admin-based income statistics, England and Wales: tax year ending 2018 article was the latest iteration available at the time of production (V3.0).
The SPD aims to produce population stocks using administrative data by bringing a range of administrative data sources together through the linkage of education, health, income and birth records. It uses data to derive rules of registrations and "signs of activity" to identify a "usually resident" population.
Further information regarding the latest ONS SPD data and methodology can be found in our Developing admin-based population estimates, England and Wales: 2016 to 2020 article.
How we process the data
The income administrative data sources are first individually processed as follows:
NBD — data are processed into a single row per individual with a value for each benefit component contained within NBD; the value of the benefit income is set as the claim end amount, where this is not available the claim start amount is used
SHBE — data are processed into a single row per individual; where a claim amount for the required tax year is not present, the claim amount from the previous year (if available) is rolled forward
PIP — data are processed into a single row per individual; PIP amounts are summed if a claimant has more than one claim in the tax year
UC —data are processed into a single row per individual to produce total UC income
CB — data are already processed into a single row per individual; no initial processing is required
TC — data are processed into a single row per individual for the tax year
PAYE P14 — data are processed into a single row per individual
SA — data from the relevant sole trader and partnership tax returns are extracted and combined as explained in Measuring self-employment income using administrative data: tax year ending 2016
Once processed, the income administrative data sources are linked together using pseudonymised identifiers in the DWP and HMRC data sources to create a combined income dataset. Individuals in the administrative data sources without these pseudonymised identifiers cannot be included in subsequent analyses. A Christmas Bonus amount is then imputed for those identified as eligible in our administrative data sources. We impute for this benefit type as our administrative data sources do not currently capture it.
The combined income dataset is then linked to our chosen population base (Statistical Population Dataset, SPD, V3.0, 2018) using a pseudonymised identifier. Individuals in the combined income dataset who do not link to the SPD are removed from the dataset. The resultant dataset provides combined income amounts for individuals in the usually resident population.
A Unique Property Reference Number (UPRN) is then assigned to each individual in our dataset, where one can be identified. This UPRN is used in the production of occupied address income.
A Winter Fuel Payment amount is then imputed. We impute for this benefit type as our administrative data sources do not currently capture it. While the eligibility criteria for this benefit covers all individuals born before a certain date, we aim to only impute for individuals identified as receiving State Pension payments, as we can be confident these individuals did receive this benefit. This methodology would not capture individuals who would need to make a claim to receive the benefit, as this would risk overestimating the number of recipients.
An error has been identified within the methodology used to impute Winter Fuel Payment, which is underestimating the number of recipients for which imputation is needed. This will be corrected as part of our ongoing development of the ABIS.
The final linked dataset is used to produce our measures of gross and net income for individuals aged 16 years and over and total occupied address income (gross and net). Equivalisation is applied to the total occupied address incomes, with the final occupied address outputs produced at the individual level such that an income value represents an individual's equivalised occupied address income.
In producing statistics using linked administrative data, particularly for small populations, we apply the same rigour in data security and privacy as with all official statistics. For further information about the security of these linked data, see the Population and social statistics transformation: 2019 progress update.
How we analyse and interpret the data
The ABIS outputs typically produced are:
gross and net income deciles for individuals
gross and net income for occupied addresses
percentage of individuals with income information
percentage of occupied addresses with income information
These outputs are available by geography (region, local authority, Lower layer Super Output Area), with individual income outputs also available by age and sex.
How we quality assure and validate the data
Quality assurance is carried out at all stages of the ABIS production. Specific methods include:
examining the input data to investigate the accuracy of the data and identify any unexpected values or trends
communicating with data suppliers to improve and confirm our understanding of the data
building in quality assurance checks at regular intervals in the analytical code and implementing these checks to ensure the code is working as intended
checking output tables produced for publication to ensure no errors or inaccuracies occurred during creation
checking outputs against previous ABIS releases to ensure general trends are as expected; we aim to develop these checks into a more robust method as the development of the ABIS continue
How we disseminate the data
The ABIS are published online on an ad hoc basis to demonstrate progress in investigating the feasibility of using administrative data to measure small area income. This also presents users with the opportunity to give feedback at each stage of development.
For enquires or feedback regarding the ABIS, email Admin.Based.Characteristics@ons.gov.uk. We take user feedback on board as part of ongoing development of the ABIS.
How we review and maintain the data processes
We are continually improving the ABIS throughout our feasibility research. As such, we regularly review and update our data processes.
Back to table of contents7. Other information
Useful links
Admin-based income statistics, England and Wales: tax year ending 2018
Article | Released 20 December 2022
Experimental small area gross and net, individual and occupied address income statistics using administrative data from Pay As You Earn, Self Assessment and benefits systems.
Measuring self-employment income using administrative data: tax year ending 2016
Article | Released 23 June 2021
Research using HMRC's Self Assessment data to produce estimates of self-employment income.
Income and earnings statistics guide
Methodology | Last revised 2 March 2022
Explains the relationship between income and earnings data and outlines the statistics produced by the ONS, DWP and HMRC.
Future of population and social statistics
Corporate information | Released 9 May 2016
Our progress towards more regular and responsive statistics, building the richest picture of our population.
8. Cite this QMI
Office for National Statistics (ONS), released 20 December 2022, ONS website, Quality and Methodology Information report, Admin-based income statistics QMI