The impact of updating cancer survival methodologies for sub-national estimates; geographic patterns of cancer survival in England

From February 2018, the Office for National Statistics (ONS) and Public Health England (PHE) partnership will publish sub-national cancer survival estimates in the Geographic patterns of cancer survival in England National Statistics bulletin. This follows on from working in partnership to collaboratively produce the national (England) cancer survival estimates published in June 2017 and the Index of cancer survival for Clinical Commissioning Groups in England published in November 2017.

This report compares the methods that will be applied from February 2018 in producing these sub-national survival statistics to methods used previously. The approach improves comparability with international cancer statistics and is consistent with the methodology applied to the national cancer survival estimates since June 2017.

Correction, 13th April 2018

Following publication on Monday 19th February 2018, a correction to the 2007 to 2011 dataset published alongside this methodology paper was identified. This was due to a small error in the patient population affecting both the patient counts and cancer survival estimates. We have re-published and corrected this error in Tables 1, 2, 5, 8, 10 and 11 (labelled Tables 1 to 6 in the previous version of this data) in the dataset. You can see the original version of this data on the previous versions page. 

Table 3, Figure 3 and some accompanying text have also been updated in Section 5 to reflect these changes. We apologise for any inconvenience.

This publication is produced in partnership with Public Health England Cancer Survival Team, part of the National Cancer Registration and Analysis Service (NCRAS).

The official statistics on cancer survival form an evidence base to inform cancer policy and programmes that aim to improve cancer outcomes. The statistics are commissioned by the Department of Health and are used to:

• help inform government policy on cancer

• provide non-government bodies with accurate and timely data on the disease

• provide citizens with accessible data on the disease to help inform debate

To ensure the data are relevant, the Office for National Statistics (ONS) and Public Health England (PHE) strive to produce estimates that are timely, accurate and accessible.

Given that a significant gap remains in survival compared with the European average, the Department of Health identified cancer as a specific improvement area for preventing people dying prematurely in the National Strategy (announced in 2011). In 2015, a new five year cancer strategy for England (PDF, 4.90MB) was developed by the Independent Cancer Taskforce, which sets out recommendations for how the NHS can improve cancer outcomes for patients. The new strategy is being reviewed by government bodies.

Cancer survival estimates also feed into outcomes strategies that set out how the NHS, public health and social care services will contribute to the progress agreed with the Secretary of State, in each of the high-level outcomes frameworks. The indicators set for the National Health Service (NHS) Outcomes Framework include 1-year and 5-year survival for all cancers combined and separately for colorectal, breast and lung cancer at Clinical Commissioning Group (CCG) level. Specifically, these sub-national cancer survival estimates feed into the Compendium of Population Health Indicators, which is useful for comparing the health profiles of local areas.

The following elements of cancer survival methodology are compared in Table 1:

• the cancer registrations that are included in the analysis (exclusion and inclusion criteria)

• the type of survival estimator

• the weights used to age-standardise the estimates

• the version of International Coding Definition (ICD) used to define cancer sites

• the tumour sites included in the bulletin

• the non-standardised survival estimates used in the accompanying trend analysis

Table 1: Summary of methods used in the Geographic patterns of cancer survival statistical bulletin

In summary, the main changes to the method for the upcoming Geographic patterns of cancer survival statistical bulletin are:

• the adoption of the International Classification of Survival Standard (ICSS) international cancer patient population for age standardising survival ratios

• using the complete approach on five years of aggregated data for age-standardised survival estimates

• expanding the analysis to include a wider range of cancer sites

The reasons for adopting the ICSS patient population are:

• the ICSS weights are publicly and readily available

• these weights are widely used, for example, in the UK by Northern Ireland and Scotland and internationally by the United States National Cancer Institute and by the International Cancer Benchmark Partnership

• the ICSS weights continue to vary by tumour type reflecting age distributions of the different cancers

• to bring the geographic patterns of cancer survival methods in line with the national methodology, by including sub-national estimates for additional cancer sites and by providing more up-to-date estimates for patients diagnosed most recently by moving to the complete approach

• it will enable sub-national and national comparisons within England and be a step forward towards national comparisons within the UK and international comparisons of survival estimates

Summary of the International Cancer Survival Standard for age-standardisation

When studying diseases in a population, many of these diseases and their effects are closely related to age. Although comparing survival rates for specific age-groups can be informative, it is often useful to be able to have a summary of survival for every patient in the population diagnosed with a type of cancer. Comparisons using a summary estimate made over time or between geographies can be misleading if the age profiles of the underlying populations are different.

To overcome these potential drawbacks, a weighted average of the age-specific rates is calculated; this process is called age-standardisation. The weights used are independent of geography and time, so they allow for direct comparison of survival estimates over time and between different geographically-defined populations.

Many types of cancer, like lung cancer, are more commonly diagnosed as age increases; others, like Hodgkin’s lymphoma, are more commonly found in younger people. There are also cancers, like brain cancer, that have two peaks in the age distribution where they are commonly diagnosed in the younger and older age-groups but are less commonly diagnosed in middle age. A fourth profile, similar to the first group, is observed for prostate cancer but is far less likely to occur in younger men than other cancers fitting the first profile.

The ICSS system of weights (Table A2) results from analysing which cancer types best fit one of the population profiles described above. An optimal set of weights is then determined that provides the largest number of estimates with good agreement between the standardised and raw survival estimates. There are four sets of weights, reflecting the four commonly observed age profiles of diagnosing different cancers and the weights are set out in Table A2.

Summary of the change from cohort to complete approach

The complete approach allows more reliable estimates than the cohort approach, particularly for smaller geographic regions and less prevalent cancer sites, over the same time period. This is because the complete approach uses all patients diagnosed from a particular year onwards up to the most recent year with follow-up. For example, 5-year survival from 2007 using the cohort approach only uses patients diagnosed in 2007. Whereas, the complete approach would include patients diagnosed between 2007 and 2011.

The survival approach diagrams (Figure 1 and Figure 2) highlight the diagnosis year for the demonstrated approach and the patient years of follow-up included in that approach. A patient pathway begins with diagnosis in year 0 when there are no years of follow-up, this continues right across the diagram increasing for each year of follow-up. For example, patients diagnosed in 2008 with follow-up until 2015 have at least seven years of follow-up. These diagrams focus on 5-year survival.

Cohort approach

Figure 1 highlights 5-year survival using the cohort approach. This approach requires at least five complete years of potential follow-up for each patient. It’s the simplest approach as all patients could be diagnosed in the same year and potentially followed up for the same length of time, as shown in Figure 1.

However, it could also be used for patients diagnosed in different years, for example, to calculate survival for patients diagnosed in 2008 to 2010; as long as all patients have full follow-up available for the latest year. The restriction with this approach is that it has implicit latency and cannot be calculated until at least five years have passed.

Figure 1: An overview of the cohort approach for survival analysis, follow-up to 2015

Source: Office for National Statistics

Download this image Figure 1: An overview of the cohort approach for survival analysis, follow-up to 2015

.PNG (9.9 kB)

Complete approach

The complete approach to survival analysis, a variant of the classical cohort approach, is used when some patients may have been followed up for less than the full period. For example, it is viable to use the complete approach to produce estimates for patients diagnosed during 2010 to 2014 with follow-up until 31 December 2015, even though not every patient has had the opportunity to be followed up for the full five years. In this example, the potential follow-up time varies between a single year and five years, depending on the year of diagnosis.

Figure 2 highlights the complete approach for 5-year survival. This approach uses all potential years of follow-up for patients diagnosed in a five-year period. The advantage to this approach is it combines timeliness and efficiency using all the available follow-up. However, this approach cannot be used to give an estimate for a single diagnosis year.

Figure 2: An overview of the complete approach for survival analysis, follow-up to 2015

Source: Office for National Statistics

Download this image Figure 2: An overview of the complete approach for survival analysis, follow-up to 2015

.PNG (10.9 kB)

Assessing the impact of methodology changes

This impact paper will be accompanied by the publication of the 2007 to 2011 datasets for the eight cancer sites included in the previous release. Since the previous release covered 2008 to 2010 under the three diagnosis years cohort approach, presenting 2007 to 2011 represents the most consistent match, under the new five diagnosis years approach, as both datasets are centred on 2009. As there is now five complete years of follow-up for these data, both datasets are measured under the cohort approach.

The inclusion of two additional diagnosis years in the age-standardised survival estimates results in the patient counts, presented in the datasets, increasing by approximately two-thirds.

The differences between the two sets of results are due to:

1) the move to the International cancer patient population for age standardising survival ratios (ICSS) weights

2) the aggregation being based on five diagnosis years rather than three

Looking ahead to the Geographic patterns of cancer survival statistical bulletin, the most visible change is in the presented data. The complete approach will enable the publication of 2011 to 2015 data whereas, under the cohort approach, the latest available data would be 2011 due to the requirement for five years of complete follow-up. The statistical bulletin will be accompanied by the publication of analysis for the three interim years (between 2008 to 2012 and 2010 to 2014).

These changes to the methodology will allow the geographic patterns of cancer survival data to be more easily compared with that of the national (England) data. Information about the complete approach to calculating cancer survival can be found in the Cancer survival Quality and Methodology Information report.

There has been no change in methodology used to produce the non-standardised estimates by single year of diagnosis. However, the move from the cohort to the complete approach means that the period covered by the trend analysis will vary for 1-year and 5-year survival.

For both the 1-year and 5-year survival, the trend analysis will be based on the latest available eight non-standardised years (with follow-up), which is consistent with previous releases. However, the complete approach means that for 1-year survival the trend analysis will be based on 2008 to 2015, while the 5-year survival trend will be based on 2004 to 2011. For previous releases, the 1-year and 5-year survival trends were based on the same eight-year period.

England-level comparison

To assess the impact of the change in the age-standardised weighting, we have run the new method on a comparable cohort of cancer registrations that was used to produce the survival estimates published in 2017 (Difference column, Table 2). The use of estimates centring around 2009 is considered the best proxy for both methods although the estimates produced by the new methodology are based on five years, rather than three years, so contain additional data from patients diagnosed in 2007 and 2011.

Some differences will follow from the extra data included in the estimates. Using three diagnosis years produces estimates that do not pass current data quality tests, so a direct comparison of changing weights between the published estimates and the new methodology is not possible. The age-weights associated with the analysis can be found in Tables A1 and A2 of Appendix 1.

Table 2 shows that the impact from changing age-weights combined with expanding the diagnosis data used.

The absolute differences of 1-year survival, in Difference A, range from 0.7% to 2.0% in males (bladder and prostate cancer respectively) and from 0.6% to 2.9% in females (stomach and cervical cancer respectively).

Whilst the absolute differences of 5-year survival, in Difference A, range from 0.2% (lung and oesophageal cancer) to 2.9% (prostate cancer) in males and from 0.1% to 7.5% in females (lung and cervical cancer respectively). The impact on cancer of the cervix is discussed further in section 6.

Overall, these differences are similar to those published in Tables A3 and A4 of the report for national survival estimates (June 2017). This indicates that changes to the underlying data, as a result of including extra years of cancer diagnoses, do not substantially impact the survival estimates but the choice of age-weights does substantially impact the survival estimates.

Sub-national comparison

Although wide differences in 1-year cancer survival exist across the 44 Sustainability and Transformation Partnerships (STPs), Table 3 shows that overall there is little difference in the variation of survival estimates as a result of applying the updated methodology changes.

Overall, the replicated results have a smaller range compared with the published estimates, indicating there is less variation at this lower geographical level.

Focussing on the three most frequently diagnosed cancers (breast, prostate and lung), the differences between the applied methods for 1-year survival of these three cancer sites are presented in Figure 3, which provides a visual representation of the distribution of 1-year survival cancer estimates. Further information on interpreting a box plot can be found in Appendix 4.

Figure 3: Comparison of the distribution of 1-year survival by Sustainability Transformation Partnerships for breast cancer in women, lung cancer in persons and prostate cancer in men

England, for 2008 to 2010, as published in 2016, and for 2007 to 2011 for the ICSS-based weights

Source: National Cancer Registration and Analysis Service within Public Health England; Office for National Statistics

Download this image Figure 3: Comparison of the distribution of 1-year survival by Sustainability Transformation Partnerships for breast cancer in women, lung cancer in persons and prostate cancer in men

.png (51.2 kB) .xls (25.1 kB)

Figure 3 shows that, for breast cancer (in women), the distribution of estimates for 2007 to 2011 was the same as the range of estimates for 2008 to 2010. Table 2 showed that 1-year survival estimates for England decreased by 0.9 percentage points.

The distribution of 1-year survival estimates for prostate cancer (in men) has changed between the two methods as shown in Figure 3. The latest methodology has resulted in a narrower range of 1-year survival estimates for prostate cancer by STPs. Unlike for breast cancer in women (where the new national estimate was lower than the previously published estimates), the latest method has shifted the distribution of 1-year survival estimates to be approximately 2.0 percentage points higher than the previous methodology.

For lung cancer (in persons), the latest methodology has resulted in an increase to the national estimate and to the range of estimates by STP, as shown in Figure 3. The reason for each cancer site being affected differently is because survival estimates are standardised differently depending on the cancer site. Details of the different International Classification of Survival Standard (ICSS) age-standardisation profiles can be found in Table A2.

However, in all three cancers sites, as the distribution of previously published STP estimates and the distribution for the new ICSS-based estimates overlap, we would conclude that the difference between the methodologies is not statistically significant.

Correction 13 April 2018

Table 3, Figure 3 and some text below figure 3 has changed due to a small error in the patient population and updates to the survival estimates used to produce the 2007 to 2011 reference tables. We apologise for any inconvenience.

As a result of adopting the International Classification of Survival Standard (ICSS) weighting, when we consider the combinations of cancer site and sex that form the eight cancers included in this bulletin, for 1-year age-standardised net survival (Table 1, Difference column):

• 15 cancer sites had an absolute difference of less than 2%

• three cancer sites (cervix, prostate, oesophagus) had an absolute difference between 2% and 5%

• no cancer sites had an absolute difference greater than 5%

Similarly, for the 5-year age-standardised net survival (Table 1, Difference column):

• 15 cancer sites had an absolute difference of less than 2%

• two cancer sites (breast, prostate) had an absolute difference between 2% and 5%

• one cancer site had an absolute difference greater than 5%

The only cancer site that had an absolute difference greater than 5% (in 5-year age-standardised net survival) was cancer of the cervix (females). This is a cancer site with an age profile that is substantially different to the common profile of increasing incidence by age.

The above findings are consistent with the differences reported in Table 2 of the report measuring the impact of updating cancer survival methodologies for national (England) estimates (June 2017). Cancer of the cervix was the only cancer site (reported in both survival bulletins) with an absolute difference greater than 2%. At a national level, the change to ICSS-based age standardisation resulted in a decrease of 3.7 percentage points for 1-year survival and a decrease of 6.8 percentage points for 5-year survival from cervical cancer.

The observed differences may partially be explained by the categorisation of age-groups. The new analysis uses five age groups and weights set out by the ICSS, whereas the previous analyses used six age groups and weights provided in Table A1 (Coleman M 1999). Consequently, the age-group boundaries differ, as does the relative amount of weighting given to younger and older patients.

The changes to using five diagnosis years and International Classification of Survival Standard (ICSS)-based weights results in relatively small changes to the 1-year and 5-year survival estimates at the national level. The observed differences are dependent on the cancer type with some sites being greater affected by the move to the ICSS weights.

At Sustainability and Transformation Partnership (STP) level, there are some larger changes, due to the small number of cases diagnosed each year in each age, sex, cancer type and sub-national geography. However, all differences were smaller than 6% for individual cancer types and are consistent with those reported at the national level.

Overall, the changes represent an improvement in both the consistency with the national survival statistical bulletin and the comparability with international survival estimates.

To date, Office for National Statistics has published a suite of national and sub-national cancer survival publications.

The method changes outlined in this article will only impact the Geographic patterns of cancer survival in England publication.

The adoption of the International Classification of Survival Standard (ICSS) international cancer patient population for age standardising survival ratios has also been applied to other cancer survival estimates published in partnership with Public Health England, which includes:

• Cancer survival in England: adults
• Cancer survival by stage at diagnosis for England
• Childhood cancer survival
• Index of cancer survival for Clinical Commissioning Groups in England

1. Coleman MP, Babb P, Damiecki P, Grosclaude PC, Honjo S, Jones J, Knerer G, Pitard A, Quinn MJ, Sloggett A, De Stavola BL (1999), ‘Cancer survival trends in England and Wales 1971 to 1995: deprivation and NHS region’, Studies on Medical and Population Subjects, Number 61, London: The Stationery Office

2. Corazziari I, Quinn M, Capocaccia R (2004), ‘Standard cancer patient population for age standardising survival ratios’, European Journal of Cancer, Volume 40, Issue 15, pages 2,301 to 2,316

3. Li R and others (2014), ‘Control of data quality for population-based cancer survival analysis’ Cancer Epidemiology, Volume 38, pages 314 to 320

4. Martos C and others (2014) ‘A proposal on cancer data quality checks: one common procedure for European cancer registraies’, European Commission, Joint Research Centre

5. Pohar PM, Stare J, Estève J (2012), ‘On estimation in relative survival’, Biometrics, Volume 68, pages 113 to 120

6. World Health Organisation (1994), ‘International statistical classification of diseases and related health problems’, Tenth revision, Geneva: WHO

Figure A1: Example of a box plot

Download this image Figure A1: Example of a box plot

.png (13.0 kB)

If interquartile range (IQR) equals upper quartile subtract lower quartile, then upper adjacent value, lower adjacent value and outside values are defined as follows:

• upper adjacent value: the maximum observed value that is less than upper quartile plus 3 multiplied by IQR

• lower adjacent value: the minimum observed value that is greater than lower quartile subtract 3 multiplied by IQR

• outside values: any observed value either greater than upper quartile plus 3 multiplied by IQR, or less than lower quartile subtract 3 multiplied by IQR