National Bereavement Survey (VOICES) QMI

• first commissioned by the Department of Health in 2011 as part of the End of Life Care Strategy
• focuses on the last 3 months of a persons life
• roughly 49,000 adults that have died in England are selected from our deaths registration database
• 2011 and 2012 surveys were based on Primary Care Trust clusters and surveys from 2013 onwards are based on NHS Local Area Team

The National Bereavement Survey (VOICES – Views of Informal Carers – Evaluation of Services) is an annual survey designed to look at the quality of end of life care. The questionnaire is sent by post to the person that registered the death of the deceased (usually a relative or friend of the deceased) between 4 and 11 months after the death. If no response is received it is followed up with 2 additional reminders.

To make sure it is a representative sample of deaths in England and that it covers the main areas of interest, the sample is divided by cause of death, place of death and geographical spread. Deaths due to an accident, suicide of homicide or if the death occurred somewhere other than home, care home, hospital or hospice, or where the address details of the informant are not given are not included in the sample frame.

The Department for Health and the NHS use this information to inform policy decisions and evaluate the quality of end of life care. Other users of the statistics include a range of organisations and people involved in end of life care, including the National End of Life Care Network, the National End of Life Care Programme, academics, health researchers and charities.

The National Survey of Bereaved People (VOICES – Views of Informal Carers – Evaluation of Services) is an annual survey designed to measure the quality of end of life care. The Department of Health (DH) first commissioned this survey in 2011 to follow up on a commitment made in the End of Life Care Strategy. Previously, very little systematic information was available about the quality of care delivered to people approaching the end of life, despite reports from the Healthcare Commission and the Neuberger review highlighting deficiencies in care. The commissioning responsibility for the survey has moved from DH to NHS England, following the restructuring of the Health and Care systems in England in April 2013.

The VOICES survey particularly focuses on the last 3 months of life. Results are used to inform policy decisions and enable evaluation of the quality of end of life care by age group, sex, in different settings (home, hospital, care homes and hospices) and by different causes of death. Quality of end of life care is also included as an indicator in the NHS Outcomes Framework and the VOICES survey will be used to monitor progress against this.

Each year a sample of approximately 49,000 adults who died in England is selected from the deaths registration database held by Office for National Statistics (ONS). To ensure the sample represents the deaths in England for the given period and covers the main domains of interest, the sample is stratified according to the cause of death, place of death and geography. For the 2011 and 2012 surveys, geography was based on Primary Care Trust (PCT) clusters. For the 2013 survey onwards, this is based on NHS Area Teams.

The VOICES survey uses the VOICES Short Form (SF) questionnaire, which is a modified version of the standard VOICES questionnaire developed by Professor Julia Addington-Hall at University of Southampton. The pilot report explains changes that were made to the survey to adapt it for national use. The VOICES questionnaire is sent by post to the person who registered the death of the deceased; this is usually a relative or friend of the deceased. Questionnaires are sent out between 4 and 11 months after the patient has died. As is standard in most postal surveys, if no response is received, this first questionnaire is then followed up with two reminders.

Once fieldwork, data capture, cleaning and processing are complete, findings are disseminated at both the national and sub-national level.

* Quality and Methodology Information (QMI) replaced Summary Quality Reports (SQR) from April 2011.

This report contains the following sections:

• Output quality
• About the output
• How the output is created
• Validation and quality assurance
• Concepts and definitions
• Other information, relating to quality trade-offs and user needs
• Sources for further information or advice

VOICES consultation and future publications

To ensure the survey remains fit for purpose, NHS England, the survey commissioners, ran a public consultation between 27 March and 23 June 2015. The consultation was open to anyone with an interest in feedback on the quality of end of life care.

A report summarising the feedback from the consultation is available on the NHS England website. In light of the consultation findings, NHS England has commissioned an options appraisal. This will explore how capturing feedback on end of life care can be improved by seeking to act upon the consultation responses.

If you would like any further information on the future of the survey please contact us via email at englandvoices@nhs.net.

This report provides a range of information that describes the quality of the output and details any points that should be noted when using the output.

We have developed Guidelines for Measuring Statistical Quality; these are based upon the five European Statistical System (ESS) quality dimensions. This report addresses these quality dimensions and other important quality characteristics, which are:

• relevance
• timeliness and punctuality
• coherence and comparability
• accuracy
• output quality trade-offs
• assessment of user needs and perceptions
• accessibility and clarity

More information is provided about these quality dimensions in the following sections.

Relevance

(The degree to which the statistical outputs meet users’ needs.)

The National Survey of Bereaved People (VOICES – Views of Informal Carers – Evaluation of Services) is an annual survey designed to look at the quality of care in the last 3 months of life. Each year a sample of approximately 49,000 adults who died in England is selected from the deaths registration database held by Office for National Statistics (ONS). The VOICES questionnaire is sent by post to the person who registered the death of the deceased; this is usually a relative or friend of the deceased. Questionnaires are sent out between 4 and 11 months after the patient has died. For the 2012 survey and onwards, the sample is representative of deaths that occurred between January and April each year.

The results from the VOICES survey are used by a range of users, including Department of Health (DH) and NHS England, to inform policy decisions and to enable evaluation of the quality of end of life care in different settings, across different age groups, different causes of death, sex and place of death.

Main users of the statistics comprise the range of organisations and people involved in end of life work. These include the National End of Life Care Network, the National End of Life Care Programme, academics, health researchers and charities. It is anticipated that members of the public, including survey respondents, will also have a particular interest in the results of this survey. A full list of uses of official statistics can be seen in the UK Statistics Authority’s Monitoring brief.

Strengths and limitations

When surveying end of life care, it is not possible to ask the patients involved for their feedback. The survey is aimed at asking a relative or friend of their opinion of the end of life care provided to the deceased instead. Therefore, responses on the experience of end of life care for the deceased are a proxy measure of a person who knew about the care, rather than the person experiencing it. Responses may well differ from the experience of the deceased. However, with this limitation, the survey does provide a suitable proxy measure to allow researchers and policy-makers to gain an insight into end of life care in England.

The questionnaire is sent to respondents between 4 and 6 months after the death has occurred. Research undertaken by the University of Southampton suggests this is the best time, balancing intruding into a person’s grief with ensuring that the memory of the last few months of care of the deceased remains salient. However, this could lead to recall biases where stronger or more emotional memories influence reports of care provision.

The survey uses a postal data collection method, which has both strengths and limitations. This was deemed the best approach to gather the data following research undertaken by the University of Southampton (the designers of the VOICES survey). They found evidence to suggest that people prefer to answer questions about end of life care and experiences in their own time, rather than in an interview setting. This also allows for the questionnaire to be passed to a more appropriate person, or for different family members to complete the survey together. It also allows the respondent to spend time completing sensitive information at a pace that suits them, rather than discussing emotional experiences with an interviewer. The VOICES survey was therefore designed as a self-completion postal questionnaire.

Undertaking surveys by post has a number of drawbacks compared with face-to-face interviewer-led surveys. The limitations of a postal survey are well known and widely reported, but the main limitations are:

• the questions have to be as simple and concise as possible; however, questions can still be misinterpreted and with no interviewer present to assist interpretation, it is possible for answers to not accurately reflect the question asked
• although the questionnaire is addressed to a specific individual, we do not know who has actually completed the questionnaire and how accurate the response is
• postal surveys require a level of English literacy to enable the respondent to understand and reply to the survey

Despite being a postal survey, the number of returned questionnaires is approximately 22,000 (a response rate of around 45%), which is high for a survey using this methodology. In the pilot of the VOICES survey in 2010, evidence was provided as to why some opt out of responding to the questionnaire. Reasons can be categorised as:

• the questionnaire was too distressing
• the informant registered the death in a professional capacity and did not know the decedent
• limited knowledge of the deceased’s care or inability to recall the events leading up to the death
• the deceased died suddenly
• the deceased received no care
• wrong recoding of the death registration
• the questionnaire was undelivered, the addressee had moved, or questionnaires are lost in the post

A number of measures have been put in place to try to reduce non-response. These include rewording the introduction to be more sensitive, excluding officials from the sampling frame from 2013, clarifying that the survey can be passed to anyone better placed to complete it and ensuring that addresses appear correct, with complete post codes.

Efforts are made to remove obvious sudden deaths from the sample, but it is not always possible to identify them, for example, where someone dies suddenly of a heart attack, rather than after a period of care. Respondents may also interpret “sudden death” differently, for example, not feeling the questionnaire is relevant to someone who dies of a fall resulting from dementia. To avoid this, routing instructions from the option “sudden death” will be made clearer in the 2014 survey, so that respondents are directed to a section of relevant questions.

The 45% response rate provides an adequate sample size for analysis at the national level, but limits the robustness of sub-national analysis. For 2011, analysis was published for the 51 Primary Care Trust (PCT) clusters. For 2012, sub-national analysis was published for the 25 NHS area teams (which replaced PCT clusters in April 2013). Two datasets (for example, data collected in autumn 2013 and autumn 2012) are combined for the purpose of increasing the sample size to improve the robustness of data. This does, however, reduce the timeliness of the data as combined 2012 and 2013 information is published in 2015. The combined data do not support extensive analysis at geographic areas smaller than NHS local area teams.

The survey design allows comparisons to be made between the quality of care given to people dying at different ages (for example, aged under 65 years, aged 65 to79 years, or aged 80 years and over) and different causes of death (such as cardiovascular, cancer or other) and for people receiving care in different settings during the last 3 months of life (for example, home, hospital, care home and/or hospice). Despite this, the survey is often criticised by respondents for being too restrictive and that their experiences cannot accurately be captured by a tick-box exercise.

The design of the survey does aim to capture the breadth of experiences and the final section of the VOICES questionnaire provides space for people to write freely about their experiences and views. Although this information is not used to produce the results for the main annual statistical bulletin, it is used to look for recurrent issues reported by respondents as well as strengthening messages in reports on end of life care. In addition, the free text collected from respondents is currently being anonymised so Approved Researchers can apply to analyse this wealth of data.

Timeliness and punctuality

(Timeliness refers to the lapse of time between publication and the period to which the data refer. Punctuality refers to the gap between planned and actual publication dates.)

The VOICES survey is carried out annually with the field period running for approximately 15 weeks. The sample is selected from deaths that occurred between January and April each year. Informants are then contacted between 4 and 11 months following the death (the recommended time for such surveys). Following the data collection period, the data are cleaned, weighted, analysed and tabulated. Previously, national data reports were created and published 7 months after the end of the field period, however, the 2015 survey data was published in April 2016.

For more details on related releases, the release calendar provides 12 months’ advance notice of release dates. If there are any changes to the pre-announced release schedule, public attention will be drawn to the change and the reasons for the change will be explained fully at the same time, as set out in the Code of Practice for Official Statistics.

Questionnaire development

The National Survey of Bereaved People (VOICES) questionnaire was developed over many years following extensive research asking bereaved relatives about their perceptions of the care given to their deceased relative or friend. The research was carried out by Professor Julia Addington-Hall and colleagues from the University of Southampton.

Following publication of the End of Life Care Strategy, Professor Addington-Hall was commissioned by Department of Health (DH) to modify the standard VOICES questionnaire (containing 144 items) to measure important aspects of the quality of care identified in the strategy. The pilot study of the short form of the VOICES questionnaire demonstrated the feasibility of the approach and the acceptability of the new survey tool. The findings from the pilot also showed that the questionnaire could detect differences in the quality of care provided in different settings (for example, hospitals versus hospices). Minor modifications were made to the questionnaire following the pilots so that it could be used in the National Survey of Bereaved People. In response to user feedback, an item regarding overall quality of care in the last 3 months of life was added.

The 2013 Neuberger review raised issues with aspects of end of life care, which were not measured by the VOICES survey. As a result, new questions were developed through requests from the steering group, development by researchers and cognitive testing with bereaved people. These questions cover provision of fluids and nutrition, and communication with family and carers. During cognitive testing, the whole survey was reviewed and numerous changes were made to wording. These changes were implemented in the 2014 questionnaire.

Sampling

Each year a sample of approximately 49,000 adults in England is selected from the deaths registration database held by Office for National Statistics. Deaths are selected from those which occurred between January and April from 2012 onwards and November 2010 to June 2011 in 2011. To ensure a representative sample of deaths in England and that it covers the main domains of interest, the sample is stratified by cause of death, place of death and geographic spread. Respondents are excluded from the sampling frame if the death was due to accident, suicide or homicide or where the death had occurred “elsewhere” than the designated locations (home, care home, hospital or hospice) or where address details of the informant were missing.

Exploratory analysis has been undertaken in 2014 to look at the seasonality of the sampling frame. A number of tests were run comparing the monthly death registrations from the annual deaths registration database, from 2001 until 2013. The investigation analysed numbers of deaths by age, cause of death and place of death. The analysis showed that as a whole, there is seasonality in the sampling frame, with, for instance, a greater number of deaths registered in the winter months and a greater number of deaths registered among older age groups.

From 2012, to limit any seasonal change in the annual death register used for the VOICES survey sample, the time frame for sampling has remained constant (January to April). The estimates produced are then weighted to be representative of the population for that period. This ensures that the estimates produced year-on-year are comparable and that no seasonal adjustment would need to be applied.

Similarly, there may be seasonal effects in the provision of care and the rating of quality of care by respondents at different times of year. The VOICES survey is always conducted using the same methodology, ensuring comparability between years. However, VOICES data only reflects views on the quality of end of life care for deaths that occurred between January and April and rated by respondents between September and December the same year.

Data collection

The VOICES questionnaire is sent by post to the person who registered the death of the deceased; this is usually a relative or friend of the deceased. If the informant feels unable to take part, they are asked to pass the questionnaire on to another family member or friend of the deceased if appropriate.

Informants are contacted between 4 and 11 months following the death with the data collection period running for approximately 15 weeks. Due to the sensitive nature of the questionnaire, it is necessary to allow respondents adequate time for completion and the opportunity to return to the questionnaire if they do not wish to complete it all in one session.

As is standard in most postal surveys, there is a maximum of three mail-outs to respondents. The first comprises a mail-merged, personalised questionnaire, a pre-paid return envelope and an information leaflet. For the 2012 VOICES survey onwards, the reply slip was removed and a box was added to the back of the questionnaire that respondents could tick if they did not wish to take part. A reminder letter is sent 3 weeks after the first mail-out to non-responders only, followed by a final invitation to take part a month later, if no response has been returned.

The information leaflet is produced in a question and answer format, with contact details of the ONS Survey Enquiry Line (SEL) team and the postal address of the ONS office in Wales. The information leaflet and letters provide the SEL details in large print and informs respondents that large print versions of the questionnaire and leaflet are available on request. In addition, the information leaflet encourages respondents to call the SEL if English is not their first language. Calls of a particularly sensitive nature or expressing a serious complaint are referred through to the survey manager who deals with them personally. Contact details are also provided for the national charity Cruse Bereavement Care, who provide help and support to those who have lost loved ones.

The mailing system, including the pre-paid return envelope, is suitable for use abroad and for returning items from abroad. This permits the inclusion of informants whose usual residence is outside of the UK (about 500 a year).

Completed questionnaires returned to ONS are scanned and processed through a bespoke data capture system, excluding the front page of the questionnaire, which contains the name and address of the informant and the name of the deceased. All data are stored under study ID number only on a secure server with restricted access to those ONS staff working on the survey.

Analysis and weighting

Following data collection, a weight is created for each respondent as the combination of the sampling weight and the non-response weight. The sampling weights are constructed as the inverse of the selection probability. The non-response weights are created to adjust for bias in response. The sampling weight and non-response weight are then combined by taking the product of the two.

Sample weight

This weight is created for each sampled respondent to correct for their unequal probabilities of selection. The sample weight of a respondent is the inverse of its probability of selection into the sample. In mathematical notation, if a respondent is included in the sample with probability Pi, then its sample weight, denoted by wi, is given by wi equals 1 divided by pi.

For example, a sampled unit selected with probability 1 divided by 25 represents 25 units in the population from which the sample was drawn. The sample weight is an inflation factor designed to represent the number of respondents in the survey population that are accounted for by the respondents to which the weight is assigned.

Non-response weighting

Non-response to the survey can lead to bias in estimates based solely on the responses of returned questionnaires. Systematic differences between the respondents and non-respondents can lead to this bias. To reduce this it is important to keep non-response as low as possible. To ensure estimates created take account of non-response bias a non-response weight is created

Non-response weighting was established in the pilot of the VOICES survey in 2011. The pilot work investigated the level of non-response bias in the sample for sex, age, area deprivation, cause of death and place of death, to see if there was a significant difference between the level of response and non-response (section 9.3 in the Pilot report provides details of this work). Statistically significant differences were found for respondent’s sex and the deceased’s cause of death and place of death. No statistically significant difference was found for area deprivation score, deceased sex or age between responders and non-responders.

The weighting method of the survey run by ONS in 2011 was designed to test for non-response using logical regression for age, Primary Care Trust (now Clinical Commissioning Group) geography, cause of death, place of death, and sex. If there is significant non-response then that variable is included in the non-response weighting.

The non-response weight is created through the use of logistic regressions with “response” as the outcome variable and each characteristic as the predictor. Significant predictors are then entered into the logistic regression together to check the association with response. The predicted values indicating the probability of response are saved and the non-response weight is created by taking the inverse of this value.

In 2014, a review was undertaken of the weighting methodology for VOICES and area deprivation was found to have significant non-response, and has now been included, alongside other variables outlined previously, as part of the non-response weight. This was implemented on the 2013 dataset onwards.

Analysis

Analysis is undertaken in various statistical packages using both the weights and the sampling stratification. It is important for data users to note that the weighting is not calibrated to represent all deaths within the year. The estimates produced are only representative of the deaths that occurred in the sampling period January to April.

From 2012, sub-national estimates are created using combined data from 2 consecutive years (for example, 2011 and 2012). This increases the robustness of the sub-national estimates. For the combined dataset, the weights are adjusted in relation to the sampling frame size for each of the 2 years. No adjustments are made for missing data. The datasets from 2011 and 2012 were combined for analysis at NHS area team level despite the differing sampling periods. There may be some seasonality effects between these datasets and this has not been controlled. Since 2011 the sample period has remained the same each year, controlling for any seasonality effects.

Disclosure

Statistical disclosure control methodology is applied to the data before publication. This ensures that information attributable to an individual organisation or person is not disclosed in any publication. The Code of Practice for Official Statistics, and specifically Principle 5: Confidentiality, sets out practices for how we protect data from being disclosed. The principle includes a guarantee to survey respondents to “ensure that official statistics do not reveal the identity of an individual or organisation, or any private information relating to them”. More information can be found in the National Statistician’s Guidnce: Confidentiality of Official Statistics and also on the ONS statistical disclosure control methodology page.

Accuracy

(The degree of closeness between an estimate and the true value.)

The main threats to the accuracy of the data are sampling error and non-sampling error, where non- sampling error includes: coverage error, non-response error, measurement error and processing error.

Sampling error

The National Survey of Bereaved People (VOICES) is a sample survey, so estimates are subject to sampling variability. Sampling variability is dependent on several factors, including the size of the sample and the effect of weighting on the variable of interest. To measure the amount of sampling variability, confidence intervals are calculated around the estimated value, which gives a range in which the true value for the population is likely to fall. For example, with a 95% confidence interval, it is expected that in 95% of the survey samples, the resulting confidence interval will contain the true value that would be obtained by surveying the whole population.

Non-sampling error

The VOICES survey is subject to non-sampling error including non-response. Within non-sampling error there is systematic and random error. Systematic error occurs when data are biased in a certain direction. Random error is the variation in sample data from the true values of the population which occurs by chance. Substantial efforts have been made in the design of the survey to avoid errors. There are a number of procedures that can be used to compensate for non-response bias. These can be summarised as:

• create non-response weights to take account of bias
• imputation of missing data
• oversampling targeting specific non-response characteristics or creating a reserve sample from which replacements are selected in case of non-response

For the VOICES survey design, non-response weights have been created to minimise the effects of non-response bias by accounting for differential non-response (see “weighting and estimation” section).

No imputation is currently applied to the VOICES data. We will be investigating if imputation is required for the survey. There are two types of imputation:

• unit non-response – this applies when a respondent does not respond to the survey and all data are missing; imputation can be used to impute all data for that respondent
• item non-response imputation – this applies when a respondent has provided a response to the survey but the response is incomplete for all questions; imputation is applied to remove missing data for each question

Currently, estimates produced from the data exclude missing data, which could introduce non-sampling bias. In September 2015, it is planned to undertake feasibility work on item non-response. This will investigate if there is non-response bias being caused for important questions in the questionnaire. If there is evidence of non-response bias then the use of imputation will be considered. There are no plans to undertake unit imputation for missing respondents. The non-response weighting accounts for this.

Oversampling relates to drawing a greater sample from characteristics where non-response is greater. For example, only 3% of responses to the survey are from respondents of black ethnic minority groups. It is not possible to over sample them because ethnicity is not known from the sample information. Non-response from other characteristics, such as areas of greater deprivation and geography, are adjusted for in the weighting method, but oversampling may be considered if the survey is redesigned for analysis at Clinical Commissioning Group (CCG) level. This would enable analysis by CCG, even where population size is small, as well as analysis at national level.

Coherence and comparability

(Coherence is the degree to which data that are derived from different sources or methods, but refer to the same topic, are similar. Comparability is the degree to which data can be compared over time and domain, for example, geographic level.)

The Views of Informal Carers – Evaluation Of Services (VOICES-SF) questionnaire used in the national survey is a modified version of the standard VOICES questionnaire, developed by Professor Julia Addington-Hall and colleagues from the University of Southampton, which has been used to assess the quality of care given to patients dying from a range of conditions and in different locations. Studies that have used the standard VOICES questionnaire may therefore be used to compare data from the national survey.

To test the feasibility of the VOICES-SF questionnaire in monitoring the quality of end of life care, the questionnaire was piloted in two Primary Care Trust (PCT) clusters, East Berkshire and the Isle of Wight. Results from the pilot study provide some local-level data, which may be used as a source of comparison with the national survey, although the small sample size may be restrictive.

The VOICES-SF questionnaire used in the national survey is freely available on the Department of Health (DH) website. We provide support and assistance to other government departments and academics who wish to carry out local studies using the same questionnaire. Results from these local studies may also be used as a source of comparison with the national survey.

The survey methodology has not changed significantly since the survey commenced, therefore results from each subsequent year will be comparable with previous year’s results. This will allow for the monitoring of progress on improving end of life care both at national and local levels.

A literature review of PubMed, SocINDEX, as well as searches of end of life care research in numerous countries revealed no national studies of the quality of end of life care. The most comparable tool for assessing end of life care is the Patient Outcome Scale (POS). This is a patient-reported outcome measure, which has 12 items relating to pain management and management of other symptoms, mood, impact of health on family members, and main problems affecting the patient. In contrast to VOICES, it relates more to feelings, rather than rating aspects of care provided.

In 2011, the Positive diversities of European priorities for research and measurement in end of life care (PRISMA) project collected data on engagement with patient outcome measures from 9,000 healthcare professionals across Europe and Africa. This study found that professionals require more support for the use of patient outcome measures to improve patient care. The ONS VOICES team has received increasing numbers of queries relating to running small-scale VOICES surveys to evaluate care in their areas, indicating the increasing importance of the quality of end of life care provision on the policy and practice agenda.

Numerous national surveys are conducted within the NHS on patient experience, most notably the Cancer Patient Experience survey. The methodology for this survey differs from VOICES in that it contacts surviving patients on their experience of cancer care. Some items are comparable to questions asked on VOICES, such as dignity and respect shown by doctors and nurses and it provides large-scale annual results on specific types of cancer care. This survey may offer an interesting national comparison for cancer patient care and palliative care for cancer patients.

(Concepts and definitions describe the legislation governing the output, and a description of the classifications used in the output.)

Glossary

Bias

Bias is an effect that prevents a statistical result from being representative of a sample by distorting it. For instance, bias can be introduced through the sampling method when one case is more likely to be selected than another, or through responses, for instance, when women are more likely to respond to the survey than men. There are ways to mitigate against bias, such as weighting.

Cancer

A disease resulting from malignant growth or tumour, caused by abnormal and uncontrolled cell growth. In the survey sample International Statistical Classification of Diseases (ICD-10) codes are used to define deaths caused by cancer. These include ICD codes C00 to D48.9 (this includes benign tumours).

Cardiovascular disease (CVD)

CVD is a class of diseases that include the heart and/or circulatory systems, such as blood vessels. In the survey sample, International ICD-10 codes are used to define deaths caused by cardiovascular disease. These are ICD codes I00 to I99.

Care home

The survey definition of care homes also includes residential homes and nursing homes. It describes a residential setting where people, often the elderly, live together (either long-term or short-term) and are given some form of on-site care.

Carer

A person who looks after someone who is ill or old; this may be an unpaid family member or a paid professional.

Cause of death

The relevant cause of death on the death certificate are coded ICD-10. Deaths were excluded from the sampling frame where the underlying cause of death was accident, suicide or homicide (ICD-10 codes V01 to Y98 and U50.9). The following deaths were included where they were recorded as the underlying cause:

• cardiovascular disease (CVD: ICD-10 codes I00 to I99)
• cancer: ICD-10 codes C00 to D48.9 (this includes benign neoplasms)
• other: ICD-10 codes A00 to R99 (excluding CVD and Cancer)

Clinical Commissioning Group (CCG)

CCGs are the lowest level of health geography in England, with 211 CCGs. CCGs have responsibility for commissioning most of the hospital and community NHS services that are needed in their area and ensuring they are provided.

Confidence interval

Confidence intervals are commonly used as indicators of the extent to which the estimate, based on a sample, may differ from the true population value; the larger the confidence interval, the less precise the estimate is. The end points of the confidence interval are called confidence limits with an upper limit and a lower limit. The probability of the confidence interval containing the true mean of the estimate being calculated is called the confidence level. For example, we may say there is a 95% chance that the mean will be within the confidence limit. The confidence level would therefore be 95%.

The width of the confidence interval provides an indication of the level of precision the estimate has. The wider the confidence interval the less precision there is around the estimate.

District or community nurses

A nurse who works for the general practitioner (GP) surgery and comes to a person’s home to give medications, medical advice and change dressings. They normally wear a uniform.

General practitioners (GPs)

GPs are usually the first point of contact for most patients. They usually see patients through a consultation in their surgery and during home visits.

Home

A person’s usual place of residence, excluding those living in a care home. This may include the individual’s home, or the home of family or friends where the person is cared for.

Hospice

A facility that provides specialist care for people with advanced illness at the end of their lives. They will normally provide both inpatient and outpatient services and may have a day centre.

Hospital

An institution that provides medical, surgical, palliative or psychiatric care and treatment for the sick or the injured.

Hospital staff

Hospital personnel who support care for patients in a hospital setting. This includes all staff such as cleaners, food servers, therapists, nurses, doctors, surgeons and consultants.

NHS England area team

NHS area teams are the middle layer of health geography for England, with four NHS regions being the highest and 211 CCGs being the lowest. CCGs nest into each of the 25 area teams.

Other cause of death (excluding cancer and cardiovascular)

Where “other cause of death” is mentioned in the bulletin, this includes all International Classification of Diseases codes A00 to R99, with the exception of C00 to D48.9 (cancer) and I00 to I99 (cardiovascular disease). Codes beyond R99 are excluded as these represent deaths from external causes, which would not normally require end of life care.

Place of death

This is the location where the deceased died as recorded on the death certificate. The National Survey of Bereaved People (VOICES) includes registrations where the place of death falls within the following groups:

• home: the home of the deceased
• hospital: NHS and private
• care homes (including residential or nursing homes)
• hospices

Deaths were excluded where the place of death was recorded as “elsewhere”, which includes external sites (such as roads or parks), public venues (such as shops or restaurants), work places and any other place that could not be identified to a specific location type.

Sudden death (died suddenly)

Sampling for the survey aims to exclude sudden deaths, for example, deaths from external causes such as accidents. Question one of the survey has a response option “they were not ill, they died suddenly” and in this instance it is the respondent’s interpretation of “died suddenly” that is reflected. For instance, these may relate to cases where the person died suddenly from a heart attack or where a person was elderly but died suddenly from a fall. In these cases the respondent is routed away from quality of care questions to the final questions on circumstances surrounding a death.

Respondent

The respondent is the person who responds to the VOICES survey. This will usually be the person who registered the death, as this is the person who the questionnaire is sent to. However, the questionnaire can be passed to anyone that they feel is more suited to complete it. The respondent’s review of the quality of care provided to the deceased gives a proxy measure of the quality of care experienced.

Reliability

Reliability describes a measure’s ability to gain similar results under consistent conditions. Where there is high reliability, sampling variability will be low and changes in results can be attributed to changes in conditions, such as changes in end of life care practice. Reliability tends to be low where a sample size is small, such as for lower-level geographies and questions that few people respond to.

Sample

The VOICES sample is drawn from the deaths that occurred between January and April each year. Ineligible cases are excluded to create the sampling frame. Deaths are then randomly selected from the remaining cases. The VOICES sample is designed to be representative of the people who died in the selection period each year.

Sampling frame

The sampling frame is the sample drawn from all deaths that occurred between January and April each year, once ineligible cases have been removed. The sampling frame is usually about 150,000 people, from which the final VOICES sample draws 49,000 cases.

Sampling variability

Sampling variability describes the variance of how much a statistic differs from sample to sample and is measured by its standard error; the smaller the standard error, the less the sampling variability. Standard errors are used within the calculation of confidence intervals.

Significance

When comparing two results, it is important to note if a difference is significant or not. Results may differ due to small natural differences, due to sampling variability, due to chance or due to an actual difference. A difference is usually only significant if it is due to an actual difference (although up to 5% of significant findings will be due to chance). Significance is indicated when the confidence intervals surrounding two estimates do not overlap.

Weight and weighting

The VOICES survey calculates a weight for each response to the survey, to account for biases within the sample design, adjust for non-response bias and calibrate weights to the population of the sampling frame. These weights are used to calculate weighted percentages.

Weighted percentage

This is a percentage adjusted by the weighting variable. This means that the percentage reported has been adjusted to account for biases in the sampling method and responses, and calculated to amount to the population size in the sampling frame.

Assessment of user needs and perceptions

(The processes for finding out about uses and users, and their views on the statistical products.)

All Social and Analysis Division (S&A) statistical bulletins seek feedback from users.

S&A maintains a list of known users, including which statistical outputs they use and the use made of them. All known users will be invited to participate in any future consultation.

Feedback is also received through regular attendance at user group meetings by Office for National Statistics (ONS) staff, such as National End of Life Care Intelligence Network meetings, and conferences.

We maintained regular contact with the Department of Health (DH) throughout the development and running of the first two annual surveys. Responsibility for commissioning and overseeing the survey moved to NHS England in April 2013.

A steering group, consisting of representatives from various government departments and end of life care networks, provide feedback and advice on the implementation of the survey as well as the scope and content of the analysis and dissemination of the results.

Accessibility and clarity

(Accessibility is the ease with which users are able to access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the release details, illustrations and accompanying advice.)

Our recommended format for accessible content is a combination of HTML web pages for narrative, charts and graphs, with data being provided in usable formats such as CSV and Excel. We also offer users the option to download the narrative in PDF format. In some instances other software may be used, or may be available on request. Available formats for content published on our website but not produced by us, or referenced on our website but stored elsewhere, may vary. For further information please contact us via email at EOLC@ONS.gsi.gov.uk.

More information regarding conditions of access to data is available:

• Terms and conditions (for data on the website)
• Accessibility

Useful links

The Department of Health website contains important findings and information from the pilot of the national survey including, in Appendix B, the 2011 to 2013 version of the VOICES questionnaire.