Opinions and Lifestyle Survey methodology

The scope of the Opinions and Lifestyle Survey, like other Office for National Statistics (ONS) social surveys, has the potential to cover the widest range of perspectives drawn from all adults (those aged 16 years and over) living in private residences in Great Britain.

The new survey design allows for mixed mode collection for the Opinions and Lifestyle Survey, meaning respondents are invited to take part through an Online self-completion questionnaire alternatively if required one of our highly skilled interviewers will conduct the interview by telephone.

As part of development work to change the method of collection, the socio-demographic questions were reviewed. In some instances, they were redesigned to suit the new mode of collection, as were some of the modules that are regularly carried on the survey. Several large-scale quantitative tests of the redesigned questions were also conducted to understand any likely effect. For more information on the pilots and findings, please see the Opinions and Lifestyle Survey: mixed mode pilot analysis article.

Since April 2018 the sample for the Opinions and Lifestyles Survey is drawn from the Annual Population Survey (APS), which consists collectively of those who successfully completed the last wave of the Labour Force Survey (LFS) or the local LFS boost. Around 38,000 households respond to the LFS each quarter and it is the largest regular household survey in the UK.

The sampling frame used for the LFS is the Royal Mail’s Postal Address File (PAF) of small users. The PAF is the most comprehensive address database in the UK. It is updated every three months and contains approximately 26 million addresses. Our sample for the LFS covers the whole of Great Britain, excluding the Isles of Scilly, and Scottish Highlands and Islands.

The OPN sampling frame contains those who have consented to be contacted for future research after completing their final APS interview. This includes members of households who have completed via proxy. Of these respondents, only those who have provided a valid telephone number are included in the frame so they can be contacted to take part.

The sampling frame for the Opinions and Lifestyle Survey specifically includes all individuals who have not objected to recontact from the ONS following successful completion of the LFS and the local LFS boost. As those respondents have taken part in previous surveys, it is possible to use their responses to increase precision in OPN estimates and reduce any potential bias.

A two-stage approach is applied to sampling – in the first stage a sample of households from the last wave of APS (and who have consented to follow-up) is drawn and in the second stage one individual from each sampled household is selected.

Each month 2,010 individuals are randomly selected to take part in the Opinions and Lifestyle Survey. The selected respondent is the only household member who is eligible to participate in the survey. Proxy interviews or responses are not permitted.

The first phase of transformation was completed in April 2018 when the OPN moved to telephone data collection. This phase of transformation included a change to the sampling frame (from the Postcode Address File (PAF) to the Annual Population Survey (APS)) and questionnaire redesign.

The second phase in transformation was completed in November 2019 when the OPN moved to mixed mode collection in November 2019. OPN now invites selected individuals to take part through an Online self-completion questionnaire. Alternatively if required one of our highly skilled interviewers can conduct the interview by telephone.

Specialist research, design and testing has been conducted to inform the transformation of the survey. Three pilot tests were conducted to understand the impact of changing mode and sample frame on data quality, sample representativeness and bias. Findings demonstrated minimal impact on most of the data. Where some differences were found, these were likely due to the redesigned question wording which was optimised for telephone collection. For more information on the pilots and findings, please see the Opinions and Lifestyle Survey: mixed mode pilot analysis article.

All respondents in the Opinions and Lifestyle Survey are individuals who have previously participated in an ONS social survey. This substantially reduces the number of ineligible addresses in the sample.

The survey response rate is calculated as a proportion of the eligible addresses only. Given the sampling method used we expect around 55% to 60% response for the Opinions and Lifestyle Survey.

The Opinions and Lifestyle Survey collects information on a sample of the population. To enable us to make inferences from this sample to the entire eligible population, we must weight the data. This involves assigning each responding case a weight, which can be thought of as the number of people in the population which that case represents. These weights are calculated in such a way that they sum to a set of known population totals.

Population weighting serves several purposes. It ensures that estimates reflect the sample design so that cases with a lower probability of selection will receive a higher weight to compensate. It also compensates for non-response among different sub-groups in the population and as such, should help guard against potential non-response bias. The use of weights also allows totals, as well as means and proportions, to be estimated easily. It is therefore important to use the weights when doing most types of analysis on our datasets. Failing to do so may introduce bias because the sample design will not be taken into account. Not using the weights will also result in estimates that are subject to more non-response bias and will make it difficult to estimate totals. The information in the APS sampling frame allows us to use existing APS data to account for non-response when creating the weights

Two weights are supplied with OPN datasets: wta and INDWGT.

wta

As a result of interviewing one person from each household, the likelihood that an individual is selected differs for households of different sizes. Also, individuals in the younger age groups, which are under-represented in the sampling frame, are assigned higher selection probabilities. The individual design weight (wta) accounts for these unequal selection probabilities.

INDWGT

The individual analysis weight (INDWGT) is calculated for each case. Despite the considerable efforts made to maximise response rates, a proportion of the individuals selected either refuse to take part or cannot be contacted. The individual analysis weight compensates for some of the potential non-response bias. A model-based approach (logistic regression) is applied for non-response adjustment using information from APS data (based on characteristics such as region, age, sex, tenure and economic status). The calibration factors ensure that the cases gross up to ONS population totals of age group by sex and Government Office Region, as well as APS estimates for tenure, National Statistics Socio-economic Classification, economic activity and smoking.

HHWGHT (upon request)

The household weight (HHWGT) can be used when analysing household-level variables rather than individual-level variables. The household calibration factor accounts for the characteristics of all household members of the responding individuals.

The Opinions and Lifestyle Survey is a sample survey and so estimates are subject to sampling variability. The size of the sampling variability is dependent upon several factors, including:

• sample size

• effects of the sampling method

• effects of weighting

The impact of these varies for different estimates. Standard errors give an indication of the amount that a given estimate deviates from a true population value. We supply standard errors that account for each of these factors, known as complex standard errors.

The design factor (DEFT) measures the effect of the sample design and weighting on the standard error. This is the ratio of the complex standard error to the standard error that would be associated with a simple random sample design without these design features.