Levelling up subnational data project, monitoring update, UK: April 2022 to June 2023

The Levelling Up Subnational Data (LUSD) project is a significant cross cutting project being delivered through a collaboration between the Office for National Statistics (ONS) and the Department for Levelling Up, Housing and Communities (DLUHC) for the period 1 April 2022 to 31 March 2025.

During its first year, the ONS has delivered a step change in making more granular data available to enable flexible, user defined geographies. In addition, the project is using new data sources to uncover insights about different parts of the UK and collaborating and building capability across government.

This has also been seen on our website, where local statistics are now available in a much easier way for users to navigate. The establishment of ONS Local, our analytical advisory service based right across the UK, has ensured that this progress is proactively used in local areas. It has also given us a much better understanding of user needs to help us decide how to prioritise our development activity.

This monitoring report compares year one of the project with our baseline and sets out recommendations to support the acceleration of the project through year 2 and year 3. It also makes recommendations for future monitoring and evaluation activity.

Highlights from the report include the development of measures of productivity at areas the size of 400 to 1,200 households, which can be built up flexibly for user defined areas, such as around a railway line or catchment areas. This data means that government can use synthetic control approaches to enable much more robust evaluation of locally targeted policies than ever before.

We have also developed an area hub on the ONS website, significantly improving the journey for users who want to find statistics for their area. The ONS Local service has been co-designed through engagement with local users, setting up a service that will have a significant impact in future now that it is established.

This progress report provides an update, from an internal evaluation perspective, on the LUSD project outcomes so far, covering the period April 2022 to June 2023.

The LUSD project aims to improve the government's subnational data capabilities, while also providing policy makers and the public with evidence for the missions and metrics in Levelling Up White Paper, published on GOV.UK.

The term "subnational" refers to all data that are produced for the 12 international territorial level 1 (ITL1) areas in the UK and smaller geographical areas.

The LUSD project has six main objectives:

1. Produce and disseminate more timely and accurate subnational data that will enable the UK Government to track the delivery of levelling up missions up to 2030, and the impact of local growth programmes, including the performance of specific places.

2. Increase the granularity of subnational data.

3. Empower local leaders through access to reliable spatial data, statistics, and analysis to inform decision making and increase accountability.

4. Improve the consistency of and ability to compare subnational data across all parts of the UK.

5. Broaden the range of subnational data that the government collects to reflect modern economies, emerging priorities, and the levelling up agenda - for example, data linked to research and development, climate change and wellbeing.

6. Improve data science, standards and sharing capabilities within the Office for National Statistics, the Department for Levelling Up, Housing and Communities, and across government.

Much of the ONS' previous and planned subnational work is outlined on our Subnational statistics and analysis, current and upcoming work: October 2023 article.

The LUSD project is aligned with the Government Statistical Service (GSS) subnational data strategy ambitions, which are to:

• produce more timely, granular, and harmonised subnational statistics

• build capability and capacity for subnational statistics and analysis

• improve the dissemination of subnational statistics

The Office for National Statistics's (ONS's) Evaluation Strategy sets out the ONS's vision for embedding evaluation best practice across all work. To align with this, the levelling up subnational data (LUSD) project aims to publish evaluation updates throughout its lifecycle, increasing the transparency of the work conducted by the ONS.

The purpose of monitoring and evaluating the LUSD project is to provide learning points throughout the project delivery and ensure continued accountability. The purpose of publishing this update is to provide a transparent overview of:

• progress of the LUSD project since its commencement, as measured against the baseline

• early stakeholder feedback from the first year of the project delivery

• recommendations for the LUSD project in year 2

• next steps for project evaluation

The data for the baseline was collected in autumn 2022 by combining desk research, meetings with stakeholders and holding a benefits workshop to better understand the current state and gain a better understanding of the problem areas that the project will seek to address. Meetings were held with stakeholders from across the ONS and Department for Levelling Up, Housing and Communities (DLUHC), as well as local stakeholders across UK regions.

A theory of change (ToC) for LUSD was created in summer 2022. To develop this, subnational statistical needs of local decision makers and the public were defined and translated into intended outputs, outcomes, and impacts, mapped against the inputs and activities required to achieve them.

To collect evidence on the impact of the LUSD outputs, a range of monitoring and evaluation methods are being considered, in line with HM Treasury evaluation guidance, known as The Magenta Book.

The LUSD ToC sets out how the LUSD project can achieve several outcomes, with the overall aim to:

• enhance the availability and improve access to subnational data

• develop capabilities of local government and decision makers through boosting engagement with subnational data

The overarching impact of the LUSD ToC outcomes is that policies and local decision making is better informed by subnational data. This aligns with the ONS organisational theory of change model (PDF, 69.7KB) outcome ("users have improved access to analytical tools and critical data impacts") and impact ("increased proportion of decision making and debate" that "is informed by ONS Statistics").

Evaluation questions were defined and aligned with outcomes in the LUSD ToC, with the main questions for each workstream commented on in the following workstream sections.

Increasing the granularity of subnational statistics is a common ask from local decision makers and is the focus of this workstream, helping to meet project objectives 1, 2, 3, and 4.

Baseline (March 2022)

Key economic statistics and levelling up metrics were targeted for exploration and potential development to lower levels of geography. At baseline:

• gross value added (GVA) was available at middle layer super output areas (MSOAs) and intermediate zone (IZ) levels

• gross domestic product (GDP) was available at local authority district, metropolitan district, London borough and Scottish council area (LAD) level

• gross disposable household income (GDHI) was available at LAD level

• household final consumption expenditure (HHFCE) was available at international territorial level (ITL) 2 regions

• government research and development (R&D) expenditure was available at national figure level

• no national statistics on interregional trade in the UK are currently published

• public sector expenditure was available at ITL 1 regions

Year 1

Gross value added (GVA) estimates are now the most detailed the UK has ever had, being produced at lower-layer super output area (LSOA, England and Wales), data zone (DZ, Scotland) and super output area (SOA, Northern Ireland) levels. You can read more about this in the Disaggregating UK annual subnational gross value added (GVA) to lower levels of geography: 1998 to 2020 article. We have also developed a proof of concept to show how this data can be used alongside synthetic control methods to significantly improve the robustness of evaluation of locally targeted policies. Future evaluation work will look to seek case studies and determine if the potential benefits are being realised.

Development work into GDHI at lower levels has also now begun.

Work was completed to publish an article, regional data and methods on government R&D expenditure at ITL1 level, alongside methodology outlining research into compiling regional business R&D at ITL1 level.

Work was conducted to develop the Experimental methodology for producing UK interregional trade estimates. This approach is now being used in the early development of future planned interregional trade statistical releases.

Going forward

In June 2023, the Office for National Statistics (ONS) published details of their investigation into Using local authority financial data to improve the granularity of public sector expenditure, and a one-off experimental investigation into the feasibility of using administrative data to help improve the granularity of public sector finance statistics.

Work is ongoing to produce GDHI estimates at LSOA, DZ, and SOA levels. Additionally, there are plans to publish regional capital estimates (GFCF) at lower levels and investigate the feasibility of producing HHFCE at local authority level.

The GVA data has been produced at a granular level to help inform decisions, as these can be used to create bespoke area statistics. The impacts of this will be evaluated, along with other granular local statistics in years 2 and 3 of the LUSD project.

The expected outcomes of producing these low-level geographical publications are that they will be used by central and local government to better evaluate policies. Further information around the use of such granular statistics will be sought to evaluate this.

Innovative research and expanded use of new and administrative data sources, data science techniques and methodology is being carried out to improve timeliness, granularity, and insight into local areas. This will support the implementation, monitoring and evaluation of the levelling up missions. Additionally, the workstream aims to fulfil project objectives 5 and 6, as well as contributing to objectives 1 and 4.

Baseline (March 2022)

There was no specific resource committed to investigating, scoping and acquiring new data sources that would help provide insight into local areas.

Year 1

In January 2023, the Data Science Campus published an interactive display of UK-wide data on feasible public transport travel distance.

At the end of 2022, we completed a review of the measures of national well-being. This enabled the production of an improved subnational UK Measures of National Well-being Dashboard, published at the start of year 2.

Going forward

A range of experimental analysis and publications is planned to use new sources of data. For example, investigation into the use of new credit card and financial transactions data is ongoing after being initiated in year 1.

Like the other workstreams, an aim of evaluation is to assess whether data are a factor in local and national decision making. Future evaluation will seek to gather evidence of policies being informed by new data and data products developed.

ONS Local is an analytical advisory service for local leaders. It has dedicated analysts based across the UK, ensuring those locally have access to data, statistics, and analysis to support decision making. Through engaging with local decision makers, ONS Local strives to build local analytical capabilities across the UK to drive evidence-based policymaking. To this end, this workstream aims to contribute towards the levelling up subnational data (LUSD) project objectives 1, 3, 4 and 6.

This service was officially launched across all English regions in March 2023.

Because of the infancy of the service, evaluation activities include stakeholders within England only. Future evaluations and reports will contain feedback from stakeholders across the UK, including the devolved administrations of Wales, Scotland, and Northern Ireland.

Baseline (March 2022)

In March 2022, there was no Office for National Statistics (ONS) service available to all local users across England. The combined authorities and the Greater London Authority had small, dedicated ONS teams that could help with user-defined data and analytical requests and queries. These staff were based in the ONS's offices in Newport, Titchfield, and London. No central funding was available to support every local area. There was limited local outreach from the ONS, with few regional events and webinars being held. A newsletter was distributed to 150 contacts.

Year 1

From July 2022 to March 2023, ONS Local focused on engagement, co-design of the service, and building presence in locations across the country. Regional roundtable discussions were held, with feedback captured from local stakeholders on their analytical, statistical, and data needs and requirements to help them take informed policy and operational decisions. On average, 30 to 40 representatives from local authorities attended these roundtables, either in person or virtually, ensuring regional and local stakeholders participated in the design of ONS Local.

Over this period, as regional leads were recruited, ONS Local provided a less-than-comprehensive service that enhanced and developed the "ONS-to-many" offer. This included revamping the newsletter and providing virtual forums for peer-to-peer learning, delivering over 130 events, averaging around 70 attendees at each. During this start-up phase, the service supported the completion of over 100 stakeholder requests, resulting in the ONS sharing information, knowledge, and signposting. The ONS Local monthly newsletter is now sent to approximately 430 people (an increase of 186%). The most popular topics were labour market, population, business, productivity, tourism, and health.

ONS Local launched in all English regions in March 2023.

Going forward

ONS Local continues to co-design its bespoke service with the devolved administrations, with the eventual aim to launch UK-wide by March 2024. The service continues to respond to user requests, has developed regional work plans, and is starting to deliver more substantial analytical projects.

Evaluation activities are planned to fully assess whether ONS Local has assisted in:

• local government decision makers finding it easier to find subnational data

• improvements to local analysis capabilities

• local decisions being more commonly informed by data

These evaluation questions will be addressed as the ONS Local service matures.

ESS is a public-facing digital dissemination service that allows users to find out more about local areas across the UK. This work focuses on meeting the project objectives 1, 3, and 6.

The Subnational indicators explorer was the first output and minimum viable product. It was published in February 2022 in anticipation of future ESS work and levelling up subnational data (LUSD) funding. This dashboard makes it easier for users to access and visualise subnational indicators in one place and is published quarterly.

Baseline (March 2022)

Before the subnational indicators explorer was published, subnational data was not in one single place and was not consistently published at geographical levels, which are useful for comparing single areas across multiple topics.

There was a lack of platforms with dashboarding or visualisation tools to enable shared product development for the public.

Year 1

The Subnational indicators explorer has been developed further with each (now quarterly) update. Since the first addition, additional datasets and more levels of geography have been provided. The first iteration of this, published on the same day as the Levelling Up White Paper, received over 4,500 unique page views in the four weeks after publication. Subsequent iterations have received over 700 views in their first 4 weeks on average.

Moving towards the Beta phase for ESS, the Find facts and figures about areas in the United Kingdom tool has been developed so users can easily find links to relevant local content, data, and tools. An element of this includes what structured datasets are available on the Integrated Data Service's data explorer, allowing users to filter and download the underlying data, as well as query the data.

New small area estimates, such as the disaggregated gross value added (GVA) estimates have been made available through an application programming interface (API) on Nomis, and then incorporated into the ESS custom profile prototype.

The prototypes and code are shared through GitHub, which has supported others to use the code and visualisations for their own products.

Going forward

Early external feedback on ESS has evidenced good engagement and use of the subnational indicators explorer by local users. Roundtable discussions and feedback received from ONS Local presents seminars have provided further evidence of positive engagement. Future evaluation of this and other ESS products intend to collect more evidence on whether it has become easier to access subnational data, and whether outputs allow local decision making to be more commonly informed by data.

An improved geography focused user journey and data exploration functionality is currently in development. This will allow users to explore and interrogate a wider variety of local level data. The data explorer is also set to include a REST API to allow users to retrieve datasets and metadata through programmatic ways, allowing for automation.

A cross-government collaboration platform with a full suite of dashboards is being created that will support decision making and analysis for central government in a coherent and consistent way, powered by the Integrated Data Service (IDS). In doing so, this workstream aims to meet project objectives 1 and 6.

Baseline (March 2022)

Subnational data was published within several different statistical releases. There was no single place for cross-government levelling up analysts to access, analyse and visualise relevant data sources in collaboration with each other.

Year 1

A broad range of levelling up data are now available on the X-Gov levelling up platform. Functionality has been developed for semi-automated data pipelines to process and update data feeds to the platform. Ingest is manual but can be responsive to user needs.

The platform has been limited to hosting open data, and functionality for exporting data out of the platform was implemented in January 2023. The aim has always been to transition to the IDS, which will allow the addition of non-publicly available data to be shared with analysts across national and local government.

Investigation found that, while the IDS is exploring additional tooling for its users, the current roadmap does not provide acceptable dashboard solutions against user requirements. The project is now exploring suitable alternatives but ensuring user requirements for open source analytical dashboarding tools are captured by the IDS.

Data for our Clustering Local Authorities against subnational indicators article was developed within the X-Gov levelling up platform. The production code was shared through GitHub and is used by stakeholders.

The platform was also used for an exploration of an emerging policy evaluation method. Using a framework developed by the Alan Turing Institute known as policy priority inference (PPI), this project applied PPI to the levelling up indicators available in the X-Gov levelling up platform.

Going forward

We aim to fully integrate the X-Gov levelling up platform onto the IDS and increase data availability, including access to record-level data for more in depth research. Supporting a dashboarding solution also remains the main priority for the upcoming year.

Future evaluation activities will aim to capture the impacts (if any) of information sharing, using the platform across government departments, and whether this project has allowed for collaborations to take place that otherwise would not have been made possible.

Early feedback on the project up to June 2023 was received from stakeholders. Two separate feedback exercises were completed: an anonymous survey sent to a select list of engaged stakeholders, and feedback gathered from the Office for National Statistics (ONS) workstream leads.

Evaluation activity commenced in March 2023. Therefore, much of the data collection is continuing.

Early feedback suggests stakeholders are making positive use of new tools and services, including, but not limited to:

• the subnational indicators explorer

• the experimental local gross value added (GVA) estimates

• interactions with ONS Local, such as attending webinars or engaging with regional leads

A common point of feedback is that stakeholders are benefitting from the availability of more granular subnational data.

However, feedback also highlights that not all stakeholder needs are being met yet, often because they would further benefit from data with smaller geographic splits. For example, respondents in local authorities mention that while it has been useful to have access to the performance of levelling up metrics in a time series, having more subnational indicators at local authority and combined authority geographical levels would be more beneficial. Another suggestion on functionality is to expand the combination of data across different topics, such as, "travel, economic, population, and education".

All feedback is being captured and fed into user need considerations for future development. Further gathering of feedback and evaluation work is planned to identify the needs and barriers of stakeholders using subnational data more widely.

The ONS Local service draws positive feedback around stakeholders being able to regularly communicate and develop contacts at the ONS who have data expertise at a local government level. Events such as the ONS Local presents webinars and in person regional roundtables have enabled both local and wider reaching collaboration. Stakeholder feedback suggests these are filling some gaps in their analytical capabilities, as local authorities can use ONS Local to carry out in depth analysis and research, which they would otherwise not have the resources to do.

For future reports, we plan to use theory based evaluation methods, such as contribution analysis. We also aim to use more participatory approaches, such as conducting focus groups and more detailed interviews with external stakeholders to assess the impact more fully. We aim to broaden our reach to ensure feedback is more representative.

Project recommendations for year 2

As the levelling up subnational data (LUSD) project continues in its second year (April 2023 to March 2024), the following activities are recommended, in line with feedback collected from stakeholders:

1. Continue to develop platforms and products to improve data accessibility, including the Find facts and figures about areas in the United Kingdom tool and the Custom profiles tool, alongside the Subnational indicators explorer.

2. Migrate the X-Gov levelling up platform onto the Integrated Data Service (IDS) to enable analysis of non-publicly available data across government and give local government colleagues the ability to collaborate on analysis with central government colleagues.

3. Establish dashboarding functionality as part of the X-Gov levelling up workstream, allowing collaboration on dashboard building and sharing across government.

4. Continue to champion subnational granular data, sharing the findings of evaluation activities and other evidence with internal workstreams, informing user needs; continue to increase the range of statistics and data available at subnational levels, prioritising statistics that will provide most impact for local and national users.

5. Commit to continued ONS Local engagement with local, regional, and national stakeholders and being their local expert points of contact, as well as completing the launch of the service in the devolved administrations.

6. Expand the offering of ONS Local beyond engagement and into regional workplans and analytical projects of local, regional and national benefit.

7. Increase public facing content to help explain the remit of ONS Local.

8. Improve the mechanisms for collection and monitoring of feedback from requests raised to ONS Local to enable better service monitoring.

9. Ensure intelligence gathered by ONS Local through stakeholder feedback is used to improve statistics delivered across the rest of the programme and more broadly throughout ONS's subnational work.

10. Broaden the list of stakeholders for future evaluation activities, especially targeting those who are less engaged with current feedback mechanisms and the devolved administrations.

11. Conduct qualitative analysis, exploring levelling up subnational data (LUSD) user needs to enable continuous strengthening of all LUSD workstreams to better meet user needs.

These recommendations will be incorporated into an update to our workplan, Subnational statistics and analysis, current and upcoming work: October 2023.

Evaluation Framework

Following this year 1 review, the evaluation priority will be to develop an evaluation framework for the remainder of year 2 and for all of year 3 of the LUSD project, across all the workstreams.

The evaluation framework will include plans for focus groups with stakeholders and quarterly monitoring of key performance indictors (KPIs). It will set out a plan for more timely and consistent collection of evaluation evidence, including case studies, ahead of the year 2 evaluation report in summer 2024.

An early priority will be to evaluate the outcomes and impacts of the ONS Local service, six months after its launch in March 2023.