Ninth meeting, 6 October 2005
The following work-in-progress papers were considered:
Paper 1: Testing for significance in revisions in ONS data
The Office for National Statistics (ONS) issues regular information about the revisions performance for a range of key indicators. This information includes an assessment of whether the average revision for a given indicator is significantly different from zero. This paper discusses the significance test used to assess whether revisions to time series are different from zero.
Committee conclusions:
- The ONS test and an alternative test were compared. The alternative test was not an improvement for the short series
-
To consider whether to retain the test and the opportunity to include more information in the annual revisions article
Paper 2: Weighting the National Travel Survey
In 2000, the National Statistics Quality Assurance Review of the National Travel Survey recommended it should be weighted to adjust for non-response bias. The National Centre for Social Research was commissioned to develop a methodology to weight the National Travel Survey. This paper describes the weighting strategy, results and estimation of standard errors for the National Travel Survey weighted data.
Committee conclusions:
-
Extra variables were suggested for modelling
-
The three types of global variance estimation possible were replication, generalised variance functions, and Taylorisation
-
However, the use of the jacknife was complicated by the fact that after dropping each Primary Sampling Unit, weights would have to be recalculated. A more direct choice was Taylorisation
-
Comparison of estimates with other sources was suggested
-
Intensive follow-up studies were suggested to capture more non-responders
-
The risk that outliers pose in weighting was noted
Paper 3: Some Methodological Challenges Relating to Implementing SIC (2007)
The Standard Industrial Classification is used to classify businesses by the type of economic activity in which they are engaged. ONS is planning to introduce a new Standard Industrial Classification from 2008. This paper describes some of the issues involved, and describes the sources of discontinuity this will bring to published economic series, along with the work that will be done within ONS to measure and overcome the discontinuities.
Committee conclusions:
-
Comparisons with related work were suggested: the last time the Standard Industrial Classification changed; how other National Statistical Institutes dealt with this problem
-
To calculate the standard deviation, treat 2007 as a domain and use modelling to produce a standard estimate (or synthetic estimate if samples are small)
-
Alternative options for weighting were discussed. Calculating calibration factors using both classification systems was the best choice. However, small samples and large weights might cause problem – possible solutions were a boost, or weighting down changes/errors in classification
-
The correlation matrix is a weighting matrix, which is a challenge because of the conversion factors changing owing to changes in the economy
-
Revisions were suggested at one digit level back to the start of the constant prices index (1948) as it was fairly stable, and back to 1987 for the current prices index
-
Different approaches for creating revised back series were appropriate at different levels/categories of data – the macro at the top level and the micro at the low level
Paper 4: 2007 Census Test Design: sample design
A major field test and evaluation of the proposed 2011 census design will take place in 2007, which will involve delivery of census questionnaires to a sample of households. This paper presents a summary of the current plans for the statistical design of the 2007 Census Test sample in England and Wales.
Committee conclusions:
-
Investigate whether treating the test as a survey experiment or a classical experimental design is more appropriate
-
Additional factors in stratification (or modelling) would lead to a reduction in variance and improvement in the properties of the test
Paper 5: Methodological issues arising from the Disclosure Review for the Health Statistics
In using and releasing health statistics there is a risk, generally associated with small numbers, of identifying individuals. To address this, a project led by ONS has been established to undertake a review and provide guidelines for interpreting the National Statistics Code of Practice and associated protocols. The aim is to balance data confidentiality risks with the public interest in the use of the figures, in the handling of health statistics across the health community. This paper presents some methodological issues that have arisen from the review.
Committee conclusions:
-
The framework for decisions on confidentiality protection was entirely reasonable
-
The disclosure by differencing detection algorithm seemed sensible. It was noted that overlaps, not only differencing, can lead to disclosure
-
For analysts wanting sparse data, safe access arrangements for microdata would be the natural solution. For sparse tables, consider providing access to disclosive sparse tables in a safe setting.
