Business Enterprise Research and Development Survey QMI

This quality and methodology report contains information on the quality characteristics of the data (including the European Statistical System five dimensions of quality) as well as the methods used to create it.

The information in this report will help you to:

• understand the strengths and limitations of the data
• learn about existing uses and users of the data
• understand the methods used to create the data
• decide suitable uses for the data
• reduce the risk of misusing data

• The Business Enterprise Research and Development (BERD) Survey measures expenditure on research and development (R&D) performed by UK businesses, the source of funding for this R&D work and the employment of people working on R&D in UK businesses.

• Results from the survey are published annually in the Business enterprise research and development statistical bulletin.

• The survey uses the Organisation for Economic Co-operation and Development (OECD) definition of R&D, as set out in the Frascati Manual 2015. The Frascati Manual defines R&D as “creative and systematic work undertaken in order to increase the stock of knowledge – including knowledge of humankind, culture and society – and to devise new applications of available knowledge”. R&D must contain an appreciable amount of novelty.

• R&D performed by businesses in Northern Ireland is measured in a separate survey conducted by the Northern Ireland Statistics and Research Agency (NISRA); the results from this are aggregated with those collected from the survey of Great Britain to form UK estimates.

The statistics on research and development (R&D) expenditure by UK businesses consist primarily of:

• expenditure on in-house R&D performed by businesses in the UK (intramural R&D)
• the source of funding for R&D performed
• employment of people working on R&D

Estimates are provided in current or constant prices. Current prices (also known as nominal or cash) are expressed in terms of the prices of the reference period, while constant prices (also known as real terms) are adjusted using the gross domestic product (GDP) deflator. This removes the effects of inflation by fixing the prices of goods and services in one period (the base year), so that only the volumes change, allowing changes in government expenditure on R&D to be examined on a comparable basis over time.

Business Enterprise Research and Development (BERD) data

What it measures

The BERD Survey collects information on expenditure on R&D undertaken by UK businesses, R&D employment, and sources of funds for the R&D performed.

Frequency

The survey is conducted on an annual basis (results are published in the November after the reference year).

Sample size

The UK sample size is approximately 5,400 businesses (that is, 4,000 in Great Britain and 1,400 in Northern Ireland).

Periods available

The BERD survey estimates have been produced annually since 1993.

Sample frame

The sampling frame is a reference list of all known R&D performers in Great Britain. The reference list is updated annually, from a range of other sources, such as the Annual Business Survey. It should be noted there is no definitive list of businesses that perform R&D.

Sample design

For Great Britain the top 400 businesses by size of previously reported R&D expenditure are automatically selected and receive the long form (referred to as “large R&D performers”).

The remaining 3,600 businesses are selected from the pre-determined list of R&D performers by stratified random sampling where the strata are defined using employment and industry product group (ie type of R&D).

For Northern Ireland, a census survey is carried out of approximately 1,400 known R&D performers.

Imputation, weighting and estimation

Long form non-responders are imputed based on their previous returns.

Short form non-responders and non-sampled businesses are estimated using ratio estimation, with employment as the auxiliary variable.

There are businesses performing R&D which are listed on the Inter-Departmental Business Register (IDBR) but have not been selected for the BERD reference list. We estimate these by dividing the number of R&D businesses estimated from the IDBR by the number estimated from the reference list, and use their ratio to uplift BERD estimates.

Outliers

For the 3,600 smaller responders that receive the short questionnaire, an outlier ratio is calculated and the top and bottom 5% are trimmed. The automatic outlier ratio is calculated by dividing R&D employment by total business employment.

Manual outliers are identified during results investigations by expert judgement from staff.

Strengths and limitations

Strengths of the survey include:

• the BERD response rate has historically been above 80% but has been lower since the coronavirus (COVID- 19) pandemic began, the 2021 survey response rate was lower at 53%

• short forms, which are sent to most responders apart from the largest R&D performers, to minimise respondent burden

• the high level of detail in BERD allows the user a thorough insight into the R&D of businesses within the UK

Limitations of the survey include:

• with a sample size of 4,000, detailed breakdowns of results beyond those published in the statistical bulletin are limited

• it is assumed short form responders only perform one type of R&D and that this relates to their main business classification, this may not always be the case

• the data release is published 11 months after the period to which the data relate

The Business Enterprise Research and Development (BERD) Survey is the largest contributor to measuring the UK's gross domestic expenditure on research and development (R&D). The other components are the UK government, the higher education sector and the private non-profit sector. Gross expenditure on R&D in the UK performed by all sectors of the economy is reported separately, in the UK gross domestic expenditure on research and development (GERD) statistical bulletin. 

The UK government's Industrial Strategy published in 2017 includes a target to "raise UK investment on R&D to 2.4% of GDP by 2027". As the largest contributor to total UK R&D expenditure, the business sector is integral to achieving this objective. BERD statistics help monitor the business sector's progress towards reaching this target.

Information is used by government departments and other organisations for planning, policy and monitoring purposes. There are many other users within and outside government who use these data to produce various analyses and to inform policy decisions. These include Eurostat and the Organisation for Economic Co-operation and Development (OECD).

The UK used to provide statistics measuring R&D activity to the EU's Statistical Office (Eurostat) in accordance with the European Commission Regulation Number 995/2012 of the European Parliament and the Council. The UK now provides R&D statistics to the Organisation for Economic Co-operation and Development (OECD). The estimates in the BERD publication are comparable with other EU member states because of the use of the Frascati Manual definition and collection guidelines on R&D statistics.

The OECD uses UK BERD data in preparing internationally comparable data tables and producing regular statistical publications such as the Main Science and Technology Indicators (MSTI) and the Analytical Business Enterprise Research and Development statistics (ANBERD).

Other BERD data users include:

• the Department for Business, Energy and Industrial Strategy (BEIS) uses BERD data to assess policy impact and inform debate; R&D data underpin their assessments of UK R&D performance as well as international work in the field
• the Welsh Government (WG), the Scottish Government (SG) and the Northern Ireland Executive use BERD data as an important indicator for measuring the performance of their respective economies within the UK, as well as to monitor and develop R&D policies that seek to increase R&D investment

• HM Revenue and Customs (HMRC) uses BERD data to support analysis and advice on policy development

How we collect the data

The Business Enterprise Research and Development (BERD) Survey measures "business enterprises" as defined in the Frascati Manual. This excludes government organisations, higher education establishments and private non-profit organisations.

The survey uses a paper questionnaire to collect respondent data. The statutory basis of the BERD Survey in Great Britain is the Statistics of Trade Act 1947 and in Northern Ireland, it is the Statistics of Trade and Employment (NI) Order 1988.

Sample design

The BERD Survey uses a reference list of known research and development (R&D) performers to form the sampling frame from which the sample is drawn. The reference list is updated annually, which minimises the selection of wrongly classified businesses.

For the 2021 survey, this list contained approximately 39,900 UK businesses (38,500 in Great Britain and 1,400 in Northern Ireland) and is updated annually before the survey selection begins.

Once a business has been identified as an R&D performer, it is added to the reference list. Upon selection for survey, the business must return a completed questionnaire. If the returned questionnaire states the business does not perform R&D, the business will be removed from the reference list. If a business ceases trading, this information will feed through to the Inter-Departmental Business Register (IDBR), which is used to create the reference list. This means that there is also the potential for some businesses to remain in the universe after they have ceased trading until the following year's selection, resulting in potential over-coverage.

The reference list is updated from many sources including R&D-related questions on other surveys, such as the Annual Business Survey (ABS), the International Trade in Services Survey and the Innovation Survey. We have also received information on new businesses undertaking R&D from both the Welsh and Scottish Governments and also the Department for Business, Energy and Industrial Strategy (BEIS).

All businesses in Great Britain previously reporting R&D in-house expenditure of above a particular threshold, are included in the sample and receive the long questionnaire. This threshold is set to identify the top 400 businesses with the highest R&D expenditure from the most recent survey and the value of the threshold is updated from year to year. It was approximately £7.0 million for the 2021 survey.

The rest of the sample is drawn from the list of R&D performers, using sampling fractions utilising employment size band and industry product group of the businesses. These 3,600 selected businesses are sent a shorter version of the R&D questionnaire, which requests R&D expenditure and employment totals. The three employment size bands used in the sampling process are 0 to 99, 100 to 399, and 400 and over.

All R&D performers in Northern Ireland are sent a version of the long questionnaire, which includes additional questions required by the Northern Ireland Statistics and Research Agency (NISRA).

Survey questionnaires

There are two types of questionnaire sent out to businesses each year; a long and a short questionnaire. Both of these request a calendar year report, from January to December. If this is not available, businesses are asked to provide the dates of the 12-month period they are reporting for. Estimates are acceptable if actual figures are unavailable.

The long questionnaire, which is sent to the 400 businesses in Great Britain that have previously returned the highest values of performed R&D, asks for:

• a breakdown of capital and non-capital expenditure on in-house R&D
• description of the type of R&D performed (product group) and the type of research undertaken (basic, applied or experimental)
• whether R&D relates to civil and/or defence
• how in-house R&D expenditure for the relevant year was funded
• how much R&D was purchased
• number of employees working on R&D and the full-time equivalent (FTE)
• type of employees (researchers, technicians and support staff)
• postcodes that relate to the workplaces where R&D is carried out

The short questionnaire, which is sent to the 3,600 businesses in Great Britain, asks for:

• expenditure on in-house R&D

• whether R&D relates to civil and/or defence

• how much R&D was purchased

• the number of employees working on R&D

Questionnaires are dispatched annually in mid-February with a return date of early April. To achieve optimum response, two written reminders are sent to businesses, the first in mid-April and the second in mid-May. All businesses who have a significant impact on the survey (known as "key responders"), who have not replied, are also telephoned before the first set of results are produced.

How we process the data

Returned questionnaires are scanned by our Survey Processing Centre with Optical Character Recognition (OCR) used to create an image. Images are stored electronically, reducing paper handling, retrieval and storage. For confidentiality purposes, the paper version is then shredded and recycled. Once the data are recorded on the database, a series of credibility checks are applied to aid data validation and dubious data are investigated.

Results are processed for the three employment size bands within each industry product group.

The research and development (R&D) expenditure of businesses who return the long form is allocated to the product group or groups according to the categories selected by responders that best describe the subject type of R&D activity they undertake. The expenditure of short form responders is allocated to a product group that relates to their Standard Industrial Classification (SIC) 2007 (which describes the main activity undertaken by the business, regardless of its R&D).

Estimation for non-responding and non-sampled businesses

Long forms

Estimates for non-responding long form businesses are calculated using growth factors that are derived from businesses in receipt of the long form type that have returns in both the current and previous reference period, who are in the same cell as the non-responder. If there are insufficient long form responders in the cell to calculate a growth factor, then information from the previous year is used. Where this is not available, historical data are used.

Short forms

For those businesses on the BERD reference list receiving the short form who do not respond, as well as non-sampled businesses, R&D expenditure is estimated separately in each strata using business employment as the auxiliary variable.

The calculation is conducted at cell level, where a cell is defined by product group, employment and the particular variable in question. Outliers are removed before estimation and added back in after estimation has been conducted. An outlier is an observation that appears to deviate markedly from other members of the sample in which it occurs. The top and bottom 5% in each cell are trimmed as outliers as long as the cell contains 20 observations or more.

A per head ratio is calculated for each of the long questionnaire cells and applied to the corresponding short questionnaire cell totals. For the value questions not collected on the short form, ratio estimation is used to estimate data, based on data received from long form R&D performers in the same product group.

To estimate for short questionnaire totals, total business employment for all businesses in the sampling frame is calculated for each cell. All businesses not trimmed as outliers have their R&D total (for example, employment, in-house expenditure or whichever total is being calculated) aggregated. This is divided by the aggregate total business employment to give a per head value to use as a base for the cell to be weighted by. The sample frame employment for each non-responding or unsampled business is then multiplied by the per head R&D variable value to give an estimated value for that cell. The outliers then have their question values aggregated together and added to the estimated figures.

Methodological developments for 2021 survey estimates using an interim uplift factor approach

The BERD reference list does not include all R&D performers because of the design of the surveys that are used to update it. These surveys are themselves random samples of the IDBR, which is a list of all UK businesses, and they only sample a small fraction of IDBR businesses from which they estimate UK-wide figures. All the IDBR businesses which are not sampled have no means of being added to the BERD reference list, so need to be estimated for. This is done by estimating the number of R&D businesses direct from the IDBR using the ABS – the largest of the surveys that update the BERD reference list. This “IDBR” estimate is divided by the estimated number of R&D businesses as estimated by the BERD survey from its reference list. T he “BERD” estimate, and the ratio of the two is used to uplift the estimates from the BERD survey. The uplift is carried out within each Published Product Group, for large and small businesses, separately. The BERD estimates for the 400 largest R&D performers, which receive the long questionnaire, do not have any uplift applied.

This uplift approach was used to compile estimates from 2018 to 2021 in the 2021 BERD statistical bulletin.

Note the final published estimates were also constrained so that the uplift method preserved year-on-year survey data growth at the UK and four nations of the UK levels as much as possible. Other year-on-year growths should be treated with caution as they were not constrained to growth in the survey data, and are therefore subject to greater uncertainty.

Output quality

Sampling error

Sampling error arises when estimates are based on a sample rather than a full census of the population. The difference between the estimates derived from the sample and value that would be obtained from a census is referred to as the sampling error. Sampling errors are not currently provided in the Business Enterprise Research and Development (BERD) publication.

Non-sampling error

Non-sampling errors are not easy to quantify and include errors of coverage, measurement, processing and non-response. There is some difficulty in identifying the population of actual or likely research and development (R&D) performers and also problems in ensuring that businesses adhere to the Frascati Manual R&D definitions.

Response rates provide an indication of the likely impact of non-response error on estimates. The response target for Great Britain for 2019 was 80% for short questionnaire responders and 93% for long questionnaire responders.

Coherence and comparability

Comparability between BERD and HMRC statistics

HM Revenue and Customs (HMRC) publishes an annual report Research and Development Tax Credits Statistics (PDF, 573KB). This contains detailed information about the number and value of research and development (R&D) tax credit claims made by businesses. The report also includes statistics on R&D expenditure that relates to claims for tax credits. In addition, HMRC's report includes comparisons of their R&D estimates, with those published by the Office for National Statistics (ONS).

More information on the comparability between BERD and HMRC statistics is available in our article Comparison of ONS business enterprise research and development statistics with HMRC research and development tax credit statistics published on 29 September 2022.

OECD

The UK BERD statistics are supplied to the Organisation for Economic Co-operation and Development (OECD). The UK R&D results are published by the OECD to allow comparisons to be made between countries.

Concepts and definitions (including list of changes to definitions)

Research and development (R&D) and related concepts follow internationally agreed standards defined by the Organisation for Economic Co-operation and Development (OECD), and published in the Frascati Manual. An updated version of the manual was published in 2015 and this included the five new criteria to determine whether an activity is R&D:

• novel

• creative

• uncertain

• systematic

• transferable and/or reproducible

R&D activity is distinguished by the presence of an appreciable element of novelty. If the activity follows an established pattern it is excluded; if it departs from routine and breaks new ground it is included. For example, activities such as routine testing, market research, patent applications, trial production runs and artistic work are excluded. Overheads of R&D projects are included. Value Added Tax (VAT) is excluded.

The reclassification of R&D as an intangible asset in the European System of Accounts 2010 (ESA 2010) resulted in us carrying out the work to implement the change in the national accounts from treating R&D as intermediate consumption, to gross fixed capital formation.

Geography

Results for Great Britain are added to results for Northern Ireland to produce UK totals. These are broken down by UK country and region for the November publication. Some users have expressed a need for more detailed regional breakdowns; the size of the survey means that it is difficult to produce robust estimates at detailed geographical areas. Therefore, although estimates used to be provided to Eurostat at the detailed Nomenclature of Territorial Units for Statistics 2 (NUTS 2) regional level, results at this level have not been included in the BERD statistical bulletin.

Accessibility and clarity

The ONS's recommended format for accessible content is a combination of HTML web pages for narrative, charts and graphs, with data being provided in usable formats such as CSV and Excel. Our website also offers users the option to download the narrative in PDF format. In some instances, other software may be used or may be available on request. Available formats for content published on the ONS website but not produced by the ONS, or referenced on the ONS website but stored elsewhere, may vary. For further information, please refer to our accessibility statement.

Timeliness and punctuality

The time between the end of the reference year and the publication date is approximately 11 months for the BERD Survey, as the results are published in November each year. In the unlikely event of a change to the release dates, public attention will be drawn to the change and the reason fully explained, as set out in the Code of Practice for Statistics.

For more details on related releases, the UK National Statistics release calendar is available online and provides 12 months' advance notice of release dates. If there are any changes to the pre-announced release schedule, public attention will be drawn to the change and the reasons for the change will be explained fully at the same time, as set out in the Code of Practice for Statistics.

Why you can trust our data

The Office for National Statistics (ONS) is the UK's largest independent producer of statistics and is its national statistical institute (NSI). The data policies detail how data are collected, secured and used in the publication of statistics. We treat the data that we hold with respect, keeping it secure and confidential, and we use statistical methods that are professional, ethical and transparent.

How we disseminate the data

Our primary method of disseminating the data is through our UK business enterprise research and development statistical bulletin.

Results are also supplied to OECD. Bespoke special analyses can also be commissioned on request, and these will also be published on our website.

While an EU member state and for the duration of the Brexit transition period, the UK has an obligation to collect and provide R&D data to Eurostat under European Commission (EC) Regulation Number 995/2012.

How we review the data

Revisions because of updated survey data are usually made to the previous two data periods, but the previous three data periods, back to 2018, were revised in the 2021 published results.

For information regarding conditions of access to data, please refer to the following links:

• terms and conditions (for data on the website)
• copyright and licensing
• accessibility