Living Costs and Food Survey QMI

• National Statistic: yes.

• Survey name: Living Costs and Food Survey (LCF).

• Data collection: face-to-face field interviewing.

• Frequency: annual.

• How compiled: cross sectional sample survey.

• Geographic coverage: UK.

This QMI contains information on the quality characteristics of the data (including the European Statistical System five dimensions of quality) as well as the methods used to create it.

The information in this report will help you to:

• understand the strengths and limitations of the data

• learn about existing uses and users of the data

• understand the methods used to create the data

• help you to decide suitable uses for the data

• reduce the risk of misusing data

• The Living Costs and Food Survey (LCF) became part of the Household Finances Survey (HFS) in financial year ending (FYE) 2020 alongside the Survey for Living Conditions (SLC); for more information on the HFS, please see our Household Finances Survey Quality and Methodology Information (QMI).

• The Northern Ireland LCF, the companion survey to the Great Britain LCF, is conducted by the Central Survey Unit of the Northern Ireland Statistics and Research Agency (NISRA).

• Results from the LCF are published in our Family spending bulletins and GOV.UK's Family food statistics; our bulletin provides a comprehensive overview of household expenditure and income and includes background information on the survey's history and methodology, whereas the family food statistics, produced by the Department for Environment, Food and Rural Affairs (DEFRA), provides detailed statistical information on the purchase and consumption of food and drink.

• Microdata for the LCF are available from the UK Data Service (UKDS), Secure Research Service (SRS) or the Integrated Data Service (IDS).

• All reasonable attempts have been made to ensure that the data are as accurate as possible; however, there are two potential sources of error that may affect the reliability of estimates and for which no adequate adjustments can be made, known as sampling and non-sampling errors (more information can be found in our Uncertainty and how we measure it for our surveys methodology).

Overview

The Living Costs and Food Survey (LCF) is an annual survey, designed primarily to measure household expenditure on goods and services. It also gathers information about the income of household members. Respondents, including children, keep a detailed diary of expenditure for two weeks. Respondents also record the weights and measures of food and drink items bought.

The LCF is a voluntary sample survey of private households. The basic unit of the survey is the household. The LCF (in line with other government social surveys) uses the harmonised definition of a household: a group of people (not necessarily related) living at the same address who share cooking facilities and share a living room or sitting room or dining area. The LCF omits communal establishments such as student halls of residence and armed forces, however, both students and armed forces are included if they live in private accommodation.

Information about regular expenditure, such as rent and mortgage payments, is obtained from a household interview along with retrospective information on certain large, infrequent expenditures such as those on vehicles. Each individual aged 16 years or over in the household visited is asked to keep diary records of daily expenditure for two weeks. Children aged between 7 and 15 years keep a simplified diary.

Detailed questions are asked about the income of each adult member of the household. In addition, personal information such as age, sex and marital status is recorded for each household member.

For the financial year ending (FYE) 2021, during the coronavirus (COVID-19) pandemic, respondents did not complete a face-to-face interview with questions on regular items of household expenditure and income details. Instead, respondents participated in telephone interviews. The knock to nudge intervention was also introduced, as discussed in our Impact of COVID-19 on Office for National Statistics (ONS) social survey data collection methodology. This involves interviewers visiting households to "nudge" those selected to participate in ONS surveys (following stringent health and safety protocols).

Once interviewed, LCF respondents were asked to provide copies of receipts (electronic or paper) for the two-week diary period, and interviewers recorded non-receipt-based expenditure via regular telephone calls during the two-week diary period. As a result of these changes and changes to data processing, not all the collected data could be processed, this is detailed in our Living Costs and Food Survey technical report: FYE March 2021. In October 2022, the LCF survey returned to face-to-face interview mode.

Uses and users

The LCF is the primary source of official information on household expenditure on goods and services and is also an important and detailed source of income data which can shed light on the impact of real-world events on expenditure patterns for different groups of people. LCF data feeds into four sets of national statistics: our Family spending, Average household income, Household income inequality and Effects of taxes and benefits on UK household income bulletins. The survey design and outputs are tailored to user needs.

The results of LCF are used by:

• HM Revenue and Customs (HMRC)

• the Department for Environment, Food and Rural Affairs (DEFRA)

• HM Treasury (HMT)

• Department for Work and Pensions (DWP)

• devolved governments of the UK

Many other government departments use LCF data as a basis for policymaking, for example, in areas of housing and transport. Within the ONS, LCF feeds into our Consumer Prices Index (CPI) and Retail Price Index (RPI), National Accounts and regional accounts of Consumer trends, which are important macroeconomic measures of the UK economy. Users outside government include independent research institutes, academic researchers and business and market researchers.

The Household Finances Survey (HFS) expenditure steering group is used to engage with users and comprises of a group of important customers in the ONS and other government departments. Meetings are usually held biennially. The steering group is also consulted between meetings on proposals and developments, for example, on variable usage and questionnaire changes.

Strengths and limitations

Limitations

Experience of household surveys in the UK and in other countries indicates that reported expenditure on a few items (notably tobacco, alcohol and restaurants and catering) is below the levels that might be expected by comparison with other sources of information. National Lottery spending has also been under-recorded in the LCF, particularly for scratch cards.

Granularity

One of the key strengths of the LCF dataset is the level of detail within the collected information; the disaggregated levels of expenditure and person level information within the data can provide low level insights into expenditure trends over time. However, the survey sample is small, which can cause some volatility within the results depending on the level of detail the analysis is conducted.

In July 2021, the Office for Statistics Regulation carried out an Assessment of the LCF. The ONS initiated a programme of work to meet the OSR recommendations, and in October 2022, OSR confirmed that LCF was to retain its national statistics status for our Family Spending in the UK bulletins.

Recent improvements

LCF has recently undertaken an uplift of the systems used to process the data from legacy statistical processing tools such as SPSS and SAS to R and Python. This has led to an improvement in the time taken to process the data to allow for more thorough quality assurance and scrutiny in the survey results.

Accuracy and reliability

Multiple quality assurance methods ensure that the LCF data are as reliable as possible. These methods are applied during the interview and after collection through outlier detection and comparisons of the data between years. All data that are identified as possible errors are investigated and, where appropriate, adjusted.

Coherence and comparability

Comparability

A household expenditure survey has been conducted each year in the UK since 1957. From 1957 to March 2001, the Family Expenditure Survey (FES) and National Food Survey (NFS) provided information on household expenditure patterns and food consumption. Both surveys were well-established, important sources of information for government and the wider community, charting changes and patterns in the UK's spending and food consumption since the 1950s. In April 2001, these surveys were combined to form the Expenditure and Food Survey (EFS). The EFS was renamed to the Living Costs and Food (LCF) survey in January 2008.

A methodological discontinuity was introduced when the FES and NFS were combined to form the EFS in April 2001. At the same time, the survey adopted the Classification of Individual Consumption by Purpose (COICOP). This meant a significant change to the categorisation of expenditure.

Until 2005 to 2006, annual data were collected and published on a financial year basis. From 2006, data were collected on a calendar year basis.

In 2014 to 2015, the survey reverted to a financial year basis.

Other more minor changes to definitions used in the survey have been introduced. Changes made since 1991 are documented in the LCF User Guides, which are available through the UK Data Service (UKDS).

Coherence

The main comparator for LCF estimates of expenditure data are the figures on final household consumption expenditure (HHFCE), published in our Consumer trends bulletin and used in UK National Accounts. LCF data feed into some of the estimates published in consumer trends, but other sources are also used. While differences occur in the estimates published, the differences are credible. Research is ongoing into the different estimates produced and their causes.

Accessibility and clarity

The UKDS at the University of Essex provides researchers with access to LCF microdata under an End User License.

Our recommended format for accessible content is a combination of HTML webpages for narrative, charts and graphs, with data being provided in usable formats such as CSV and Excel. Our website also offers users the option to download the narrative in PDF format. For further information, please refer to the contact details on our Accessibility statement. 

Timeliness and punctuality

Quarterly datasets are provided to internal Office for National Statistics (ONS) customers and important government departments approximately four months after the end of each quarter.

Annual datasets are made available approximately one year after the end of the survey cycle.

For more details on related releases, the GOV.UK release calendar provides advance notice of release dates. If there are any changes to the pre-announced release schedule, public attention will be drawn to the change and the reasons for the change will be explained fully at the same time, as set out in the Code of Practice for Official Statistics.

Concepts and definitions (including list of changes to definitions)

The LCF utilises a suite of standard concepts and definitions. Some of the main terms are defined in this section, while a list of definitions can be found in our Family spending bulletin.

Classification of Individual Consumption by Purpose (COICOP)

The COICOP coding frame for expenditure items was introduced in 2001 to 2002. COICOP has been adapted to the needs of the Household Budget Survey (HBS) across the EU and is therefore compatible with similar national accounts and consumer prices indices. This allows the production of indicators which are comparable Europe-wide, such as the Harmonised Indices of Consumer Prices (CPI).

National Statistics Socio-economic classification (NS-SEC)

In 2001 to 2002, our NS-SEC was adopted for all official surveys, replacing the social class based on occupation and socio-economic group. The long-term unemployed, which fall into a separate category, are defined as those unemployed and seeking work for 12 months or more. This group is derived from our UK Standard Occupation Classification (SOC2020) coding frame.

Output quality

Response rates

Response rates are available in our Living Costs and Food Survey technical report: financial year ending (FYE) March 2022. In 2021 to 2022, the response rate was 27% in Great Britain (and 28% for the UK). This is lower than previously achieved because of the change to mode (from face-to-face to telephone mode) necessitated by the coronavirus (COVID-19) pandemic. A long-term decline in response has been observed for the LCF, in common with other social surveys. Non-response weighting is applied to help mitigate non-response bias.

The LCF is conducted with people who volunteer their time to answer questions about themselves and keep a diary for two weeks. The voluntary nature of the survey means that people who do not wish to take part in the survey can refuse to do so. Reasons for not participating in the survey include people who "cannot be bothered" and "refusal to headquarters after interviewer's visit". The sample is designed to ensure that the results of the survey represent the population of the UK. The risk of the survey not being representative may increase with every refusal or non-contact with a sampled household (survey non-response). One measure of the quality of survey results is therefore the response rate.

Sources of error 

Survey results are subject to various sources of error. The total error in a survey estimate is the difference between the estimate derived from the data collected and the true value for the population. It is helpful to distinguish between systematic and random error. 

Systematic error 

Systematic error, or bias, covers those sources of error that will not be expected to average to zero over repeats of the survey. Bias may occur, for example, if a certain section of the population is excluded from the sampling frame, because non-respondents to the survey have different characteristics to respondents, or if interviewers systematically influence responses in one way or another. 

Substantial efforts are made to avoid systematic errors, and these include: 

• processes to ensure that households are selected in accordance with the sample design 
• extensive measures to minimise non-response 
• training to ensure that interviewers ask questions in such a way as to avoid biasing response 

Additionally, the data are weighted to compensate for non-response (see Section 6: Methods used to produce the LCF data); each respondent is given a weight so that they represent the non-respondents that are like them in terms of the survey characteristics. The sample distribution is weighted so that it matches the population distribution in terms of region, age group and sex and tenure. 

Random error

Random error is the difference from the estimates derived using the sample data from the true values for the population that occur through chance occurrences. Random error may result from sources such as variation in a respondent's interpretation of the survey questions, or interviewer inconsistencies in asking questions. Efforts are made to minimise these effects through pilot work to ensure questions are understood by respondents and through interviewer training emphasising the need for consistency in dealing with selected sample members. 

Sampling error

An important component of random error is sampling error, which arises because the estimate is based on a sample rather than a full census of the population. The results obtained for any single sample may, by chance, vary from the true values for the population. Precision is usually estimated through the calculation of standard errors (for more information, see our Uncertainty and how we measure it for our surveys methodology). Standard errors are estimated for some of the main variables on LCF. These are published in our Living Costs and Food Survey: technical report data tables.   

The LCF uses a multi-stage stratified sample design. Consequently, it is inappropriate to estimate standard errors based on a simple random design, as this will not reflect the true sampling variation because of the complex sample design. The two-stage sample of addresses can lead to a substantial increase in standard error if the households or individuals within Primary Sampling Units (PSUs) are relatively homogeneous, but the PSUs differ from one another. Stratification tends to reduce sampling error and is most advantageous where the stratification factor is strongly related to characteristics of interest in the survey. 

Sampling

Approximately 5,400 responding households in the UK per year are in the LCF survey sample.

Sampling frame

The sampling frame used for the LCF is the Royal Mail's Postcode Address File (PAF) of small users. The PAF is the most comprehensive address database in the UK. It is updated daily and contains approximately 30 million addresses.

Sample design

The LCF sample for Great Britain is a multi-stage stratified random sample. Addresses on the PAF with "small user" postcodes are used as the sample frame. Postal sectors are used as the Primary Sampling Units (PSUs), with 20 addresses selected from each PSU to form the monthly interviewer quota. Approximately 700 PSUs are selected annually after being arranged in 41 strata defined by Nomenclature of Territorial Units for Statistics Level 2 (NUTS2) areas and two 2011 Census variables: socio-economic group of the head of household or household reference person and ownership of cars.

During the coronavirus (COVID-19) pandemic, the sample sizes were increased to combat the impact of the change to interviewer mode from face-to-face to telephone. This fluctuated between 40 and 28 addresses per quota starting from June 2020 until it reduced to 25 addresses in January 2023 and returned to "normal" (20 addresses) in March 2023.

The Northern Ireland LCF, the companion survey to the Great Britain LCF, is conducted by the Central Survey Unit of the Northern Ireland Statistics and Research Agency (NISRA). A systematic random sample of private addresses is drawn from the NISRA Address Register (NAR). Addresses are sorted by district council and ward, so the sample is effectively stratified geographically.

Fieldwork

The fieldwork is conducted by the Office for National Statistics (ONS) in Great Britain and by NISRA for the Department of Finance and Personnel in Northern Ireland using largely identical questionnaires. Differences between the two questionnaires reflect the country-specific harmonised standards for ethnicity, nationality and national identity, and the different systems of local taxation used in Great Britain and Northern Ireland.

Households at the selected addresses receive an advance letter before being visited by an interviewer. They are asked to co-operate in the survey. The calling strategy which achieves the highest contact rate at the lowest cost is to vary calling times. Many households will be easily contacted within the first couple of calls, but for those which are not it is important to make sure that successive visits are at different times of the day (including evenings) and on different days of the week.

Interviews are conducted by Computer Assisted Personal Interviewing (CAPI) using laptop computers. Respondents complete a face-to-face interview and each individual aged 16 years or over in the visited household is asked to keep a record of daily expenditure for two weeks. Children aged between 7 and 15 years keep a simplified diary.

Response outcome categories

A full response denotes a household in which:

• all adults aged 16 years and over co-operated with the interview
• no income questions in the questionnaire were refused
• all adults kept a two-week record of their expenditure
• the information given was complete and usable

There are three types of partial responses on the LCF:

• one or more adults, that are not the main diary keeper (MDK), the main shopper in the household, refuse to keep the diary, but all adults complete the full income section of the interview
• one or more adults refuse the full income section but all adults in the household keep the diary and main income information is collected for all adults
• one or more adults (that are not the MDK) refuse to keep the diary and one or more adults refuse the full income section but main income information is collected for all adults

All partial responses must contain a diary from the MDK. If the MDK refuses to complete the diary, the household is classified as a refusal.

An outright refusal is a household that refuses to respond to the survey and the interviewer feels that there is no chance of an interview at the given time. In addition to outright refusals, there are also refusals when some of the information has not been collected. Refusals on the LCF are defined as:

• all adults complete the full income section, but the MDK of the household refuses to keep the diary
• one adult refuses to give the main income information in the questionnaire

A non-contact arises when an address is occupied but where it has not been possible to contact any member of the household in the field period.

Proxy interview

Ideally, all adult members of the household should be present during the interview, so that the income section can be asked personally. However, where a member of the household is not present during the interview, another member of the household (such as a spouse) may be able to provide documentary information about the absent person via a proxy interview.

How we process the data

Weighting

The weighting procedure comprises two stages. First, a non-response adjustment is made using weighting classes derived from an analysis of respondent and non-respondent LCF households using addresses linking LCF response status to the 2011 Census data for LCF addresses selected around the time of the census. Second, the non-response weights are calibrated to population totals. In response to the effects of the pandemic, tenure was introduced from financial year ending (FYE) 2021 as a further calibration control analogous to the changes for LFS with owning outright, owning through mortgage and renting as calibration groups, constructed from the numbers of males and females in different age groups and, separately, for regions. 

Outliers

Extreme values in the data are identified and treated during the quarterly production process. For the annual file creation, to identify and treat outliers in household and person level datasets for income and expenditure, a draft run of the family spending tables is produced. From this, outliers are identified from a decimal representation of the expenditure influence, referred to as "most influence". Influence caps are set, and if the spending for the household exceeds the cap, then the expenditure can be classed as an outlier and different from the normal trend. The weight for that household is then treated accordingly to bring the weighted spending below the cap whilst attempting to minimise any additional impact to other spending categories. All the annual household weights are then scaled to account for the impact of the adjusted figures. This technique effectively reduces the impact of any outlying values without deleting them altogether.

How we quality assure and validate the data

Prior to publication LCF data are subjected to a rigorous process of quality assurance. An initial series of automatic checks are applied to raw household and income data as they are collected from respondents and entered onto the CAPI version of the questionnaire. These data are further checked by a team of editors within the ONS who also impute for missing values. Missing data are imputed using a combination of the following methods:

• by reference to tables based on external (non-LCF) data produced elsewhere
• by reference to tables based on LCF data from previous years showing average amounts according to household income
• by using information collected elsewhere in the questionnaire or by referring to the interviewers

Respondents' diaries are checked to ensure the process of recording daily expenditure has been understood and before they are returned to the ONS to ensure recorded information is complete. In manually entering diary data to the ONS systems, further checks are made to these data.

Once the data has been processed, detailed quality assurance is conducted on each of the stakeholder outputs during the processing and outputs stage. The LCF team also carries out a series of checks on the time series data to identify odd movements and extreme values or outliers in household and person level datasets.

How we disseminate the data

Our Statistical disclosure control methodology is also applied to LCF data. This ensures that information attributable to an individual is not disclosed in any publication. The Code of Practice for Official Statistics and specifically, Principle 5: Confidentiality, sets out practices for how we protect data from being disclosed. The principle includes a guarantee to survey respondents to "ensure that official statistics do not reveal the identity of an individual or organisation, or any private information relating to them".