Travel and tourism review methodological research progress update

The Office for National Statistics (ONS) is currently conducting a review of travel and tourism statistics. The review, due to conclude in spring 2022, aims to better understand the needs of users of these statistics and to investigate alternative data sources and methods that could be used to compile these statistics in the future. The review will produce a series of recommendations to transform the statistics to deliver improvements to the extent to which users' needs are met and to maximise the use of alternative data sources.

We provide updates on progress in October 2021, as we are launching the public consultation on our proposed changes to how travel and tourism statistics are produced. This paper also summarises the methodological research work undertaken to date as part of the review.

A review of the travel and tourism statistics was started in response to recommendations from the Office for Statistics Regulation (OSR) made in October 2019, highlighting that our international statistics did not sufficiently meet users’ needs. Key issues reported included timeliness, accuracy and level of disaggregation as well as poor levels of user engagement.

In early 2020, the coronavirus (COVID-19) pandemic and associated travel restrictions led to most surveys used to estimate different components of travel and tourism industries being stopped. As a result, alternative sources of data that could be used to produce those estimates were sought, and international travel and tourism statistics were published based solely on administrative data and modelling (Overseas travel and tourism, provisional: April to June 2020). This approach builds on a transformation mindset across the Office for National Statistics (ONS) to improve our statistics by:

• using new sources of data including administrative data

• closer integration with the work from other statistics producers

• making best use of new methodological advances in the fields of statistics and data science

The scope of the review includes visits made by foreign residents to the UK and domestic and international visits made by UK residents. It does not include any aspect of tourism supply such as hotel occupancy rates. As such, the key statistics being considered are number of visits (broken down by nationality and residency of visitors), and their characteristics, for example, the purpose of the visit, spending, duration of stay, location or destination, respectively for foreign residents and UK residents.

Up to now travel and tourisms statistics within the scope of the review have been derived from a range of sources. These are:

• the Office for National Statistics' (ONS) International Passenger Survey (IPS)

• the Northern Ireland Passenger Survey (NIPS)

• the Civil Aviation Authority's (CAA) Departing Passenger Survey

• the Great Britain Tourism Survey (GBTS) and Great Britain Day Visits Survey (GBDVS) from VisitEngland, VisitScotland and Visit Wales

• Northern Ireland (NI) Continuous Household Survey (CHS) run by Northern Ireland Statistics and Research Agency (NISRA).

The IPS is a passenger survey of both international residents travelling to the UK and UK residents travelling overseas running on departures and arrivals at airports, seaports and rail ports, namely Eurostar and Eurotunnel. The estimates obtained from this survey are then calibrated to ensure that the total estimated numbers of travellers align with passenger counts obtained from Eurostar, Eurotunnel, the Department for Transport and the CAA. The CAA also runs its own passenger survey on departures only; this provides a range of information to the industry, including for route planning and optimisation.

For domestic travel and tourism, the main data source used is the GBTS and GBDVS, which measure the volume, value and profile of overnight and day trips taken by British residents to destinations in England, Scotland, Wales and Northern Ireland. These two surveys are currently undergoing transformation change with the aim to meet users' needs better. Northern Ireland additionally collects and produces statistics on domestic overnight and day visits by NI residents domestically, to the Republic of Ireland (RoI) and Great Britain.

The suspension of the IPS and other surveys because of the COVID-19 pandemic and associated travel restrictions has accelerated the transformation towards using alternative approaches and sources of data to improve on the current ways of delivering those statistics focusing on the users' needs through extensive stakeholder engagement.

Considering how we could compile travel and tourism statistics if we started afresh, we looked at three options. These were:

• sole use of alternative and administrative data

• sole use of surveys

• a hybrid approach 

The use of alternative data enabled the production of estimates at short notice during the International Passenger Survey (IPS) suspension in 2020. Relying on administrative data or automatically collected data to produce statistics could be thought as more reliable than using surveys as it avoids some of the common pitfalls of surveys such as non-response bias or recall bias. However, these alternative data sources are not without their own limitations and sources of errors, making them not suitable as the only inputs for the production of travel and tourism statistics at present. The main quality issues with such data stem from having been collected for a different purpose than the production of statistics, such as:

• imperfect coverage - not everyone of interest for the statistics is represented in these data, some people are counted multiple times, some people are included who should not be

• bias - certain groups/demographics are over- or under-represented

• lack of control over collection process - changes in data definition, processes related to the data gathering and processing, cost or confidence in securing long-term access to the data at the required frequency

On the other hand, initial findings from the review concluded that surveys alone could not achieve the level of accuracy and precisions required by the users without considerable cost implications. As such, a survey-only approach would not be a viable, cost-effective solution to address the key issues highlighted by the Office for Statistics Regulation (OSR) in their report.

We therefore propose using a hybrid method. This combines surveys with alternative data sources to compile travel and tourism statistics, with the option of becoming even less focused on survey data over time, as more alternative data become available. The proposed approach sets out a high-level vision for the future of travel and tourism statistics with the aims to:

• deliver statistical outputs that better meet the needs of users

• maximise the use of alternative data sources

• use surveys only where necessary

• deliver improved coherence across all travel and tourism statistics

From a methodological point of view, this hybrid method will maximise the use of alternative sources of data to derive estimates that meet the needs of users for more granular, more frequent and timely statistics. It will also build on the strengths of surveys, which provide estimates of defined quality at national scales. 

Following those principles, the methodology strand of the review has completed comprehensive audits of existing and potential alternative data sources, assessing the quality of the data and evaluating the availability, reliability and cost of their procurements. Work was also undertaken to review existing passenger surveys comparing questions, designs and the auxiliary data used to calibrate the derived estimates, as well as looking further afield to approaches used in different countries.

As a result of this initial research, we considered different permutations of surveys and alternative data sources. Through discussions with stakeholders, we narrowed down to what we considered the most suitable all-round approach to deliver reliable and robust statistics of appropriate quality and value to users. At this stage of the review, we are not able to define the methods in detail nor the full range of statistical outputs that will be available. We, however, present in the next section, the different workstreams included in our proposed approach.

The proposed approach will make use of alternative data sources to improve granularity, frequency and timeliness, with all baseline estimates being derived from surveys:

• a harmonised at departure-only passenger survey, for estimates related to inbound tourism (foreign residents visiting the UK)

• a household survey, for estimates related to international and domestic visits of UK residents

Surveys are designed and optimised to provide reliable estimates for a specific reference period and at specific levels of geography. The choice of design for the survey is crucial as it will determine the quality of the estimates, but it is also about balancing the quality dimensions with the operational costs of running the survey. In the case of the International Passenger Survey (IPS) and the Northern Ireland Passenger Survey (NIPS), the chosen design aims to provide reliable annual and quarterly estimates of number of visits and spend for the whole of the UK and each of the four nations. Monthly estimates are also derived but with wider uncertainty around them because of limited sample sizes. Alternative sources of data, such as mobile connection, financial transactions or bookings data, may be used in conjunction with surveys to produce more detailed breakdowns of estimates, and to provide early estimates, which will, however, be subject to later revisions.

We detail below the different strands supporting this approach at a high level, as further work remains to be done to define the exact methods and any operational implications.

Two harmonised surveys: a departure-only passenger survey and a household survey

With our aim for better coherence and efficiency, it is envisaged that estimates of the number of visits and their characteristics for foreign visitors will be measured using a harmonised passenger survey conducted at departures only.

The team evaluated the similarities and differences between the three main passenger surveys, comparing questionnaires and designs. We identified a set of core questions and considered ways to combine estimates obtained from different surveys. While discussions are still ongoing with the relevant parties regarding operational details, our research concluded that harmonisation of the three passenger surveys would be possible, bringing consistency for the users and ultimately producing better survey estimates. Within the harmonisation work, we will also consider additional questions to the survey to enable better use of alternative data sources. The next steps of the harmonisation work will consist of finalising the questionnaire, optimising sample collections across ports, shift allocation, sample sizes and mode of collection.

The quality of estimates obtained from a survey can also be improved via the calibration by including additional poststratification variables to reduce variance in the sample estimates. In parallel to the harmonisation work, exploratory work conducted by the review team showed that estimates of spend obtained from the IPS could potentially be improved by accounting for country of destination/origin.

Using breakdowns of passenger numbers by nationality, residency and destination from new sources, such as the Home Office, may improve the quality of the estimates further by providing a single consistent source of passenger counts regardless of the mode of transport (air, rail or sea). This may also address part of the timeliness issues caused by relying on multiple sources of passenger counts for the different modes of transport.

Further work is needed to understand the coverage and quality of such data sources and therefore assess their suitability for this purpose.

Similar to the approach taken by Northern Ireland, it is proposed that statistics related to UK residents' travel abroad and UK domestic tourism could be obtained from a household survey. As part of evaluating the feasibility of using a household survey to derive travel and tourism statistics for UK residents, we have considered a range of options from adding travel and tourism related questions to existing household surveys (with or without changing their design) to the creation of a completely new household survey. We will continue to work closely with Northern Ireland Statistics and Research Agency (NISRA), VisitEngland, VisitScotland and Visit Wales to identify potential for harmonisation of questionnaire, and collaboration on the design and operational details of such a household survey. The Office for National Statistics' experience of running large scale continuous household surveys across Great Britain is helping to inform the feasibility of a household survey for travel and tourism statistics.

In addition to the workstream looking at the surveys described above, the methodological strand of the review has a further two parallel workstreams focusing on maximising the use of alternative data sources.

Improving spatial granularity

When considering whether alternative sources of data can be used to break down survey estimates, the first step is to evaluate the strength of the relationships between the survey estimates and estimates obtained from the alternative sources.

The use of anonymised, aggregated mobile connection data as a proxy for international travel statistics has been investigated jointly by the review team and the Data Science Campus (DSC). Preliminary results show good correlation in the observed trends of number of international residents travelling to the UK broken down by regions derived from mobile data and the IPS. Similar findings were also obtained when comparing volume of UK residents travelling overseas obtained from the two sources (IPS versus mobile connection data).

Preliminary research undertaken by the Office for National Statistics (ONS) prior to the travel and tourism review also found correlation between estimates of expenditure from international travel and estimates of spend derived from financial transactions data. Their results indicated that financial transactions data, such as credit and debit card data, could be used in models to provide estimates by more detailed expenditure categories.

These initial results indicate that mobile connection and financial transactions data have the potential to improve the spatial granularity of estimates of number of visits and associated spend. Importantly, as all alternative data sources are received in an anonymised and aggregated form to address any privacy concerns, work is also being undertaken to assess the different quality dimension of these data.

Notwithstanding the additional work to fully understand these data sources before they are incorporated into official statistics, the next steps are to confirm those results by building statistical models using these data and evaluate the gains made in the precision and accuracy of the travel statistics of interest. A statement on data sources, which includes these sources has been published.

More timely estimates

The final methodological strand of the review has been looking at improving the timeliness of the statistics produced by leveraging alternative data sources such as financial transactions or booking data. This work was inspired by the work from the ONS UK trade statistics team, who, in collaboration with the DSC, developed models based on alternative data sources to publish early indicators to address the need for more timely estimates during the coronavirus (COVID-19) pandemic.

So far, our review has chosen a complementary approach based on state-space models. These models, being related to a time series model, have some advantages in terms consideration of seasonality, the ability to extrapolate future values based on past trends and fluctuations. The team has investigated the use of this method to model the underlying fluctuations over time both in the number of international visits and the level of associated spending. The model considered to date focuses on the level of spend and uses primarily financial transactions and can be used to nowcast the level of spend expected, based on alternative data and historical trends, before survey estimates are available. It is planned that a similar model can be developed using bookings data to provide faster indicators of the number of visits (international and/or domestic).

While these models offer the potential to generate early indicators, they require good quality past data so that they can be trained, as well as reliable and accurate estimates of the statistics of interest. An ongoing independent source of accurate estimates would be required as input to the models, and the models would need periodic review and possibly retraining to protect against model drift. In addition, the publication of early/faster indicators for any statistics is also more likely to require revisions of the statistics to be considered.

The methodological work to date has focused on researching and evaluating different potential options for the production of travel and tourism statistics, supporting the proposal of a hybrid approach to meet our users' needs. Evaluation of the feasibility of the proposed approach has been started, but more work remains to be done covering all the operational implications of the proposal.

Specifically, from the methodological strand, the next steps of the review will therefore focus heavily on:

• the harmonisation work for a departures-only passenger survey, including the optimisation of the sample design, core questions, and shift and routes allocations

• defining the specifications for a household survey to measure UK residents domestic and international tourism; this will include all aspects of sample design, as well as considering the potential to harmonise across existing Great Britain Tourism Surveys (GBTS), Great Britain Day Visits Survey (GBDVS) and Northern Ireland Continuous Household Surveys (NICHS)

• investigating whether the survey estimates can be improved further by calibration using additional or new sources of auxiliary data on travelers, such as advance passenger information or exit checks data from the Home Office

• using additional data sources in statistical models to produce estimates with enhanced granularity and timeliness

Parallel to these specific methodological research pieces, the team will also continue evaluating the feasibility of the proposal from an operational point of view and engaging with our stakeholders throughout, starting with a public consultation running from today (5 October 2021) to Tuesday 21 December 2021.