Table of contents
- Executive summary
- Introduction
- Quality considerations
- Approach to assessment
- Assurance level assessment
- Action plan
- Annex A: Assessment of data sources
- Annex B: ONS data checking and validation
- Annex C: Quality assurance processes and checks – UK Consumer Research
- Annex D: Flow diagrams of quality assurance processes
1. Executive summary
There are a range of consumer price inflation measures in use in the UK; notably the Consumer Prices Index including owner occupiers’ housing costs (CPIH), and the Consumer Prices Index (CPI), which omits these housing costs.
CPIH is the first measure of inflation in our Consumer Price statistics bulletin. It was launched in 2013, but was subsequently de-designated as a National Statistic following the identification of required improvements to the methodology. We have now implemented all of the improvements, and are seeking re-designation for CPIH as a National Statistic.
The construction of CPIH and CPI is complex. Price and expenditure data are required for each of the approximately 700 items in the “basket” of goods and services. A variety of different data sources are used for this purpose.
The data used in the compilation of CPIH and CPI can be categorised as follows:
- Price collection from shops in various locations around the country (commonly referred to as the “local” collection), which is contracted to an external company called TNS
- Individual prices collected through a website, phone call to the supplier, or from a brochure.
- Expenditure weights or prices calculated from survey data, which are sourced from within ONS, or from another government department.
- Expenditure weights or prices calculated from administrative data, which taken from or compiled within ONS, other government departments, or commercial companies.
The owner occupiers’ housing costs (OOH) component of CPIH uses 4 administrative data sources to calculate the cost of owning, maintaining and living in one’s home; these are sourced from the Valuation Office Agency (VOA) in England, and from the Welsh and Scottish governments (Northern Ireland data are currently used from the TNS collection). Data from a range of sources are used to weight price data to reflect the owner occupied housing market.
Incorporating so many different data sources into any statistic, but particularly one used as a key economic measure, involves a certain degree of risk. Administrative data in particular may be collected and compiled by third parties, outside the Code of Practice for official statistics.
Our production processes are certified under an external quality management system: ISO9001: 2015. However, to further assure ourselves and users of the quality of our statistics, we have undertaken a thorough quality assessment of these data sources. This assessment is a continuous process, and we will publish updates periodically.
We have followed the Quality Assurance of Administrative Data (QAAD) toolkit, as described by the Office for Statistics Regulation (OSR). Using the toolkit, we established the level of assurance we are seeking (or “benchmark”) for each source. The assurance levels are set as either “basic”, “enhanced” or “comprehensive”, depending on:
- the risk of quality concerns for that source, based on various factors, such as the source’s weight in the headline index, the complexity of the data source, contractual and communication arrangements currently in place, and other important considerations
- the public interest profile of the item which is being measured, and its contribution to the headline index
The majority of items in the consumer prices basket of goods and services are constructed from just three key sources of data: the local price collection from TNS, expenditure data from Household Final Consumption Expenditure in the national accounts, and further expenditure data from the Living costs and Food Survey. This means that there are a few sources which will need a higher level of assurance, and many sources which are only used for one component of the index and so do not require a particularly high level of assurance.
Through engagement with our suppliers, we have assessed the assurance level that we have currently achieved by considering:
- the operational context of the data; why and how it is collected
- the communication and agreements in place between ourselves and the supplier
- the quality assurance procedures undertaken by the supplier
- the quality assurance procedures undertaken by us
The details below summarise the quality assurance benchmarks that were set, and the assurance levels that we have assessed each source at during this assessment.
For used cars Autotrader, the:
- risk was low
- profile was high
- benchmark QA level was enhanced
- achieved assessment is still in progress
For RDG LENNON, the:
risk was low
profile was high
benchmark QA level was enhanced
achieved assessment is still in progress
For Valuation Office Agency rental price data, the:
risk was medium
profile was high
benchmark QA level was comprehensive
achieved assessment was comprehensive
For Welsh Government rental price data, the:
risk was low
profile was medium
benchmark QA level was enhanced
achieved assessment was comprehensive
For Scottish Government rental price data, the:
risk was low
profile was medium
benchmark QA level was enhanced
achieved assessment was comprehensive
For Mintel, the:
risk was medium
profile was medium
benchmark QA level was enhanced
achieved assessment was comprehensive
For Glasses, the:
risk was low to medium
profile was low
benchmark QA level was basic
achieved assessment was basic (incomplete)
For Moneyfacts, the:
risk was low
profile was low
benchmark QA level was basic
achieved assessment was basic
For HESA, the:
risk was low
profile was high
benchmark QA level was basic
achieved assessment was basic
For Consumer Intelligence, the:
risk was low
profile was low
benchmark QA level was basic
achieved assessment was basic
For Kantar, the:
risk was low
profile was low
benchmark QA level was basic
achieved assessment was basic
For IDBR, the:
risk was low
profile was low
benchmark QA level was basic
achieved assessment was basic
For Website, the:
risk was low
profile was low
benchmark QA level was basic
achieved assessment was basic
For Direct Contact, the:
risk was low
profile was low
benchmark QA level was basic
achieved assessment was basic
For Brochures, the:
risk was low
profile was low
benchmark QA level was basic
achieved assessment was basic
For HHFCE, the:
risk was high
profile was high
benchmark QA level was comprehensive
achieved assessment was enhanced
For LCF, the:
risk was low
profile was high
benchmark QA level was enhanced
achieved assessment was comprehensive
For TNS, the:
risk was medium
profile was high
benchmark QA level was comprehensive
achieved assessment was comprehensive
For BEIS, the:
risk was low
profile was low
benchmark QA level was basic
achieved assessment was basic to enhanced
For IPS, the:
risk was low
profile was low
benchmark QA level was basic
achieved assessment was basic to enhanced
For Home and Communities agency, the:
risk was low
profile was low
benchmark QA level was basic
achieved assessment is still in progress
For the Department of Transport, the:
risk was low
profile was low
benchmark QA level was basic
achieved assessment is still in progress
As a result of this assessment, we have put in place an action plan to improve our quality assurance in some areas:
Household Final Consumption Expenditure (HHFCE) data also require a comprehensive level of assurance; however, we would like more information on the complex array of data sources used to compile the statistics. HHFCE QAAD assessment, still in progress
finally, there are a number of data sources which require basic assurance, for which we have not received all the requested quality assurance information; we will work with these suppliers to gain the level of assurance we require
We will continue to engage with our data suppliers to better understand any quality concerns that may arise, and to raise their understanding of how their data are used in the construction of consumer price inflation measures. The QAAD will be updated as new alternative data sources are introduced into live production.
Back to table of contents2. Introduction
There are currently two key consumer price inflation measures in the UK. The Consumer Prices Index including owner occupiers’ housing costs (CPIH) is the first measure of consumer price inflation in our statistical bulletin, and is currently the most comprehensive measure of inflation. This addresses some of the shortcomings of the Consumer Prices Index (CPI), which is an internationally comparable measure of inflation, but does not include a measure of owner occupiers’ housing costs (OOH): a major component of household budgets1. Both of these measures are based on the same data sources (with the exception of OOH and Council Tax, which are in CPIH but not CPI). These data sources are numerous and often complex. We therefore seek to assess the quality of each of these sources.
Our assessment of data sources is carried out in accordance with the Office for Statistics Regulation's Quality Assurance of Administrative Data (QAAD) toolkit. We are striving for a proportionate approach in assessing the required level of quality assurance for the many and varied data sources used in the compilation of CPI and CPIH. We seek to highlight and address the shortcomings that we have identified, and reassure users that the quality of the source data is monitored and fit for purpose.
In this paper, we set out the steps we have taken to quality assure our data, and our assessment of each source. In section 3 we discuss important quality considerations for CPIH and CPI. In section 4 we outline our approach to assessing our data sources. In section 5 we discuss the assurance levels we are seeking for each data source, and the resulting assessment and, in section 7, we detail our next steps towards achieving full assurance. Our detailed quality assurance information for each source is provided in Annex A.
This publication is part of an ongoing process of dialogue with our suppliers, to increase our understanding of any quality concerns in the source data, and to raise awareness of how it is utilised. Through this document, we aim to provide information and assurance to users that the sources used to construct our consumer price inflation measures are sufficient for the purposes for which they are used. We will therefore review this document every 2 years. We do not address the construction of, or rationale for, our OOH measure in CPIH here. This is discussed in detail in the CPIH Compendium. For more information on our consumer price inflation measures, please refer to our Quality and Methodology Information page.
Notes for: Introduction
- The Retail Prices Index (RPI) is a legacy measure, only to be used for the continuing indexation of index linked gilts and bonds. It is not a National Statistic.
3. Quality considerations
When considering the quality of UK consumer price inflation measures, there are some broader considerations that users should bear in mind. The first is the de-designation of CPIH as a National Statistic in 2014. The second is external accreditation under ISO9001:2015 for consumer price statistics processes. These are described in more detail in this section. Detail on the quality assurance procedures applied to our statistics is reproduced in Annex B.
3.1 Loss of National Statistics status
CPIH was introduced in early 2013, following a lengthy development process overseen by the Consumer Prices Advisory Committee (CPAC) between 2009 and 2012. CPIH became a National Statistic in mid-2013, but was later de-designated in 2014 after required improvements to the OOH methodology were identified. These were:
- improvements to the process for determining comparable replacement properties when a price update for a sampled property becomes unavailable, leading to more viable matches
- bringing the process for replacing properties for which there is no comparable replacement into line with that used for other goods and services in consumer price statistics
- optimising the sample of properties used at the start of the year, to increase the pool of properties from which comparable replacements can be selected
- reassessing the length of time for which a rent price can be considered valid before a replacement property is found
The required methodological improvements were implemented in 2015, and the series was fully revised to accommodate these changes. On 3 March 2016, the Office for Statistics Regulation (OSR) released their assessment report on CPIH, reviewing the statistic against all areas of the Code of Practice for Official Statistics.
We have subsequently undertaken an assessment of all data sources used in the production of CPIH using the OSR’s Quality Assurance of Administrative Data toolkit (QAAD). We have aimed to demonstrate that we have investigated, managed and communicated appropriate and sufficient quality assurance of all our data sources. Additionally, we have published a range of supporting information, such as the CPIH Compendium, which sets out the rationale for our choice of OOH measure, and the methodology behind it, the Comparing measures of private rental growth in the UK article, and the Understanding the different approaches of measuring owner occupiers’ housing costs article. The CPIH was re-designated as a National Statistic on 31 July 2017. More details on CPIH can be found via the CPIH compendium.
3.2 ISO9001 Accreditation
Prices Production areas are externally accredited under the quality standard ISO9001:2015. This is an international standard based on a set of quality management principles:
- customer focus
- leadership
- engagement of people
- process approach
- improvement
- evidence-based decision making
- relationship management
It promotes the adoption of a process approach, which will enable understanding and consistency in meeting requirements, considering processes in terms of added value, effective process performance and improvements to processes based on evidence and information. In other words, the main purpose of this standard is to ensure the quality of our production processes, to ensure that we fully evaluate risks and to ensure that we strive for continuous improvement.
The standard is applied to all areas of production involved in the compilation of the whole range of consumer price inflation statistics. Prices documentation is reviewed by trained internal auditors, based on an annual cycle planned by the quality manager. The depth of the audit is based on how frequently the processes change. A review by an external auditor is also conducted on an annual basis, and a 3-year strategic review is also conducted to assess suitability for re-certification.
Back to table of contents4. Approach to assessment
We have conducted our assessment of data sources used in Consumer Prices Index including owner occupiers’ housing costs (CPIH) using the Office for Statistics Regulation’s QAAD toolkit. We took the following steps for each data source:
- establish the risk of quality concerns with the data
- establish the level of public interest in the item that the data are being used to measure
- determine benchmark quality assurance levels, based on the risk and public interest.
- contact the suppliers of administrative data to understand their own practices and approach to quality assurance; generally, this consists of the following steps:
- send out questionnaires to our data suppliers requesting information on their QA procedures
- conduct follow up meetings with our data suppliers to request further information and clarification
- maintain ongoing dialogues with data suppliers to develop a better understanding of any quality issues in the data, and raise awareness of how the source data are used
- review our own quality assurance and validation procedures and processes
- conduct an assessment of each data source using the four practice areas of the Quality Assurance of Administrative Data (QAAD) toolkit:
- operational context and data collection
- communication with data suppliers
- quality assurance procedures of the data supplier
- quality assurance procedures of producer
- determine an overall quality assurance level based on our assessment
- if this assurance level does not match the benchmark assurance level, then put steps in place to work towards meeting the required assurance level
- review the quality assurance on an ongoing basis; we will publish a QAAD update every 2 years
4.1 Setting the benchmarks
In accordance with the QAAD toolkit, we have sought assurance for each data source based on the risk of quality concerns associated with that data source, and the public interest in the particular item being measured by that data source.
We considered a high, medium or low risk of data quality concerns based on:
- the weight that the item being measured by a particular data source carries in headline CPIH or Consumer Price Index (CPI); we consider items with a weight less than 1.5% to be very small, items with a weight between 1.5% and 5% to be small, items with a weight between 5% and 10% to be medium, and items with a weight higher than 10% to be large.
- the complexity of the data source; for example, whether it is compiled from a number of different sources, or based on survey data, which we would consider to be lower risk due to the fact that data are collected for statistical purposes and have a holistic, well designed collection strategy, their reliability is better understood, and quality assurance and validation procedures are typically robust
- the existing contractual and communication arrangements currently in place
- how much the measurement of a particular item depends on that data source (in other words, what would we do if we did not have this data?)
- other considerations, such as any existing published information on data collection, methodology or quality assurance, or mitigation of high risk factors with the data
We considered a high, medium or low public interest profile based on:
- the level of media or user interest in the particular item being measured
- the economic or political importance of the particular item being measured
- the contribution of the item being measured to the headline index, since we would consider both CPIH and CPI to be economically and politically important
- any additional scrutiny from commentators, based on particular concerns about the data
Together the risk of quality concerns and public interest profile are combined to set an overall assurance level that is required for a particular source. For more information on how we assessed the overall assurance level, please see the UK Statistics Authority Quality assurance matrix.
4.2 QAAD practice areas
We have aimed to assess the quality of each data source based on four broad practice areas. These relate to the quality assurance of official statistics and the administrative data used to produce them: our knowledge of the operational context in which the data are recorded, building good communication links with our data suppliers, an understanding of our suppliers’ quality processes and standards, and the quality processes and standards that we apply. This is in line with the Office for Statistics Regulations expectations for quality assurance of data sources. The full assessments for each data source can be found in Annex A.
Breakdown of the four practice areas associated with data quality
The operational context and admin data collection practice area covers:
- the environment and processes for compiling the administrative data
- factors which affect data quality and cause bias
- safeguards which minimise the risks
- the role of performance measurements and targets; the potential for distortive effects
The communication with data partners practice area covers:
- collaborative relationships with data collectors, suppliers, IT specialists, policy and operational officials
- formal agreements detailing arrangements
- regular engagement with collectors, suppliers and users
The Quality Assurance (QA) principles, standards and checks by data suppliers practice area covers:
- data assurance arrangements in data collection and supply
- quality information about the data from suppliers
- role of operational inspection and internal/external audit in data assurance processes
The producers’ QA investigations and documentations practice area covers:
- QA checks carried out by the statistics producer
- quality indicators for input data and output statistics
- strengths and limitations of the data in relation to use
- explanation for users about the data quality and impact on the statistics
5. Assurance level assessment
5.1 Setting the benchmarks
In this section we describe each of our data sources, and consider the assurance level that we are seeking (or “benchmark”) for these. We also summarise our current assessment of the data and outline any further steps that may be required to reach the benchmark assurance level. We will also use this process to build engagement with our suppliers to better understand the data source, as well as raising awareness of how the data are used in consumer price inflation statistics.
In the section that follows, the weights provided are for Consumer Prices Index including owner occupiers’ housing costs (CPIH) (the first measure of consumer price inflation in our bulletin) in February 2017 (expect for rail fares which has been updated and provides weights information in February 2024).
It is a feature of consumer price statistics that we require a data source for each of the approximately 700 items in the basket of goods and services. The majority of the index is constructed from just three data sources – the local price collection, conducted by an external company called TNS, and expenditure data from the Living Costs and Food Survey (LCF) and the Household Final Consumption Expenditure (HHFCE) branch of the national accounts.
Remaining items tend to be constructed from data sources which are quite specific to the item being measured. A consequence of this is that the distribution of assurance levels required for assessment is very heavily weighted towards basic assurance. This is because we have a few data sources which are used for the vast majority of items, and relatively few items which all require a bespoke data source.
Benchmark assurance levels are summarised below. The assurance levels required for this QAAD assessment are set out in detail below, with explanations provided accordingly. The assurance levels are based on an assessment of the risk of quality concerns, and the public interest profile, as described in section 3.2. These are used to set the overall assurance level.
Benchmark assurance levels and assessment
For Autotrader, used cars, the benchmark risk assessment:
for risk was low
for profile was high
overall was enhanced.
The justifications for this are:
used cars have low weight contribution.
using alternative data sources for used cars is methodological step change and there have been high levels of public engagement to this release.
a data sharing agreement, a stable automated data feed, and regular meetings with the supplier are in place.
Overall, actual risk assessment is still progress.
For RDG LENNON, the benchmark risk assessment:
for risk was low
for profile was high
overall was enhanced
The justifications for this are:
rail fares have a low weight contribution
high levels of public engagement have taken place prior to the release and rail fares will be the first alternative data source to go into live production
the contract, service level agreement, and regular meetings with the supplier are in place
Overall, the actual risk assessment is still in progress.
For HHFCE, the benchmark risk assessment:
for risk was high
for profile was high
overall was comprehensive
The justifications for this are:
a complex data source is compiled from numerous data sources
HHFCE are extensively used in CPIH, with no alternative data available
regular communication with the supplier and some information on methodology is published
Overall, the actual risk assessment was enhanced: not achieved.
For TNS, the benchmark risk assessment:
for risk was medium
for profile was high
overall was comprehensive
The justifications for this are:
data accounts for high proportion of prices in CPIH and CPI
a dedicated contract management branch assesses TNS's performance against contract
sampling frame and design are set by prices, with quality checks carried out by both parties
Overall, the actual risk assessment was comprehensive: achieved.
For Value Office Agency (VOA) rental price data, the benchmark risk assessment:
for risk was medium
for profile was high
overall was comprehensive
The justifications for this are:
a relatively high weight in CPIH
microdata is provided, allowing thorough quality assurance
OOH costs are economically important, with high user interest in methodology
Overall, the actual risk assessment was comprehensive: achieved.
For LCF, the benchmark risk assessment:
for risk was low
for profile was high
overall was enhanced
The justifications for this are:
survey data which represents most items at lower-level aggregation
collection, design and methodology are produced by the Office for National Statistics (ONS) and are well documented
data are used widely in construction of economically important CPIH and CPI
Overall, the actual risk assessment was enhanced: achieved.
For Mintel, the benchmark risk assessment:
for risk was medium
for profile was medium
overall was enhanced
The justifications for this are:
although data collection is complex, the process and procedures are well documented
a contract is in place with a designated contact
data do not represent as broad a cross-section of the basket as other data sources
Overall, the actual risk assessment was comprehensive: achieved.
For Scottish Government rental data, the benchmark risk assessment:
for risk was low
for profile was medium
overall was enhanced
The justifications for this are:
a very low weight component of CPIH
microdata is provided, allowing thorough quality assurance
less user and media interest on devolved regions
Overall, the actual risk assessment was comprehensive: achieved
For Welsh Government, the benchmark risk assessment:
for risk was low
for profile was medium
overall was enhanced
The justifications for this are:
a very low weight component of CPIH
microdata is provided, allowing thorough quality assurance
less user and media interest on devolved regions
Overall, the actual risk assessment was comprehensive: achieved
For BEIS, the benchmark risk assessment:
for risk was low
for profile was low
overall was basic
The justifications for this are:
data sources are collected through a survey, with relatively low weight contribution
methodology and quality assurance procedures are considered sufficient, and there is a dedicated BEIS contact
limited, niche media interest in items
Overall, the actual risk assessment was basic to enhanced: achieved.
For Brochures, the benchmark risk assessment:
for risk was low
for profile was low
overall was basic
The justifications for this are:
a small to medium weight, with a number of mitigating factors that reduce the risk, for instance being collected in-house
manually entered into prices system with robust quality assurance and validation
little media interest in items
Overall, the actual risk assessment was basic: achieved.
For Consumer Intelligence, the benchmark risk assessment:
for risk was low
for profile was low
overall was basic
The justifications for this are:
a very low weight in CPIH, with clear contingency should data become unavailable
a contract is in place with regular supplier meetings
little media interest in item index
Overall, the actual risk assessment was basic: achieved.
For Department of Transport, the benchmark risk assessment:
for risk was low
for profile was low
overall was high
The justifications for this are:
it contributes a small to medium weight, with suitable alternatives if data unavailable
data are sourced through email contact and imported directly into prices system
the series is of limited media and user interest
Overall, the actual risk assessment is still in progress.
For Direct Contact, the benchmark risk assessment:
for risk was low
for profile was low
overall was basic
The justifications for this are:
a small to medium weight, with a number of mitigating factors that reduce the risk being collected in-house
manually entered into prices system with robust quality assurance and validation
little media interest in items
Overall, the actual risk assessment was basic: achieved.
For Glasses, the benchmark risk assessment:
for risk was low to medium
for profile was low
overall was basic
The justifications for this are:
complex data source is compiled from several different sources
a relatively small weight contribution and alternative data sources being available
the detail of quality assurance is not yet provided
Overall, the actual risk assessment was basic: incomplete.
For HESA, the benchmark risk assessment:
for risk was low
for profile was low
overall was basic
The justifications for this are:
a very low weight in CPIH, with clear contingency should data become unavailable
data are sourced through email with no contractual agreement in place
little media interest in item index
Overall, the actual risk assessment was basic: achieved.
For Home and Communities agency, the benchmark risk assessment:
for risk was low
for profile was low
overall was basic
The justifications for this are:
a very low weight in CPIH, with clear contingency should data become unavailable
data are sourced through email with no contractual agreement in place
little media interest in item index
Overall, the actual risk assessment was basic: achieved.
For IDBR, the benchmark risk assessment:
for risk was low
for profile was low
overall was basic
The justifications for this are:
a very low weight in CPIH, with clear contingency should data become unavailable
the IDBR team is based within the ONS, so no contract is in place
little media interest in item index
Overall, the actual risk assessment was basic: achieved.
For IPS, the benchmark risk assessment:
for risk was low
for profile was low
overall was basic
The justifications for this are:
a low but not insignificant weight in headline CPIH
straightforward collection; primarily survey, supplemented by administrative
methodology and quality assurance are well documented
Overall, the actual risk assessment was basic to enhanced: achieved.
For Kantar, the benchmark risk assessment:
for risk was low
for profile was low
overall was basic
The justifications for this are:
a very low weight in CPIH, with clear contingency should data become unavailable
data are purchased annually with a dedicated contact provided
little media interest in item index
Overall, the actual risk assessment was basic: achieved.
For Moneyfacts, the benchmark risk assessment:
for risk was low
for profile was low
overall was basic
The justifications for this are:
a very low weight in CPIH, with clear contingency should data become unavailable
data are acquired through an annual magazine subscription
little media interest in item index
Overall, the actual risk assessment was basic: achieved.
For Website, the benchmark risk assessment:
for risk was low
for profile was low
overall was basic
The justifications for this are:
a small to medium weight, with a number of mitigating factors that reduce the risk, for instance being collected in-house
manually entered into prices system with robust quality assurance and validation
little media interest in items
Overall, the actual risk assessment was basic: achieved.
5.2 Assurance level: Comprehensive
We have assessed four of our data sources as requiring a comprehensive level of assurance. This means that we require a detailed understanding of the operational context in which data are collected, including sources of bias, error and mis-measurement. We also require strong collaborative working relationships with these suppliers, supported by firm agreements for data supply, and a detailed understanding of the supplier’s quality assurance principles and checks. Our own quality assurance and validation checks should be comprehensive and transparent, and we will communicate any risks that arise from the data.
More detail is provided for each of these four suppliers.
Household Final Consumption Expenditure (HHFCE)
Data usage
CPIH and CPI follow the Classification Of Individual Consumption According to Purpose (COICOP). Expenditure for COICOP categories are used to aggregate lower level indices together. Expenditure weights are based entirely on HHFCE data, produced by the national accounts. Data are taken from the Quarter 3 Consumer Trends publication, which is consistent with the latest Blue Book. Expenditure data are price updated to the relevant period, before being rescaled to parts per thousand for use as expenditure weights. For this reason we could consider HHFCE data to have an almost 100% weight in both CPIH and CPI.
Risk: High
HHFCE is a complex data source, compiled from the Living Costs and Food Survey (LCF), and numerous other administrative sources. Adjustments are also applied to the data; for example, for under- reporting and national accounts balancing. HHFCE data are produced within the ONS.
These data have a very high weight in CPIH and CPI, and there is no real alternative to this source. HICP regulations state that these data must be used as the source of weights for CPI. HHFCE data, however, are also required under European legislation and, as a key component of the national accounts, it is unlikely that they would be discontinued. Data are provided to Prices Division in spreadsheet form, which are fed into Prices systems, and Prices staff will comprehensively quality assure the data.
Some information on methodology, and quality assurance processes is published, and Prices Division have a regular communication mechanism with national accounts staff through a quarterly internal stakeholder board.
Considering the complexity of the data source, and the importance of the data to production of CPIH and CPI, we feel that a high-risk profile is appropriate.
Profile: High
Given the extremely wide coverage of HHFCE data, we have expenditure weights for COICOP categories of varying user interest. Moreover, given that CPIH and CPI are economically and politically important, and HHFCE data are used for all classes, it would be inappropriate to consider anything other than a high public interest profile.
Assessment: Enhanced (A2)
Status: Not achieved
HHFCE expenditure data are compiled from a complex range of administrative and survey data. HHFCE have detailed all of the sources used; however, there is not necessarily detailed quality assurance information provided for each of these data sources. HHFCE have provided detailed information on their quality assurance and validation procedures compilation process, coverage, and forecasting and imputation procedures, which we consider to be fit for purpose. These are reproduced in detail in Annex A.
Prices Division communicates regularly with HHFCE staff through the Prices Stakeholder Board, and there is a good awareness of how HHFCE data are used within consumer price statistics. HHFCE follow international standards; in particular, the European System of National Accounts 2010.
Remedial actions:
- HHFCE are in the process of completing a QAAD assessment of their data sources. They aim to complete their QAAD assessment by autumn 2017.
TNS
Data usage
Prices for approximately 520 of the items in the consumer prices basket of goods and services are collected from stores and venues across the country by a team of “local” price collectors. The collection is currently carried out by TNS. The total weight of items in the basket collected under local price collection could be as much as 40%.
Risk: Medium
Quality assurance for the local price collection is already well established. A contract is already in place to ensure ongoing price collection, and to ensure that the collection meets the required standard, including what data will be provided, when they will be provided by and in what form TNS will provide them. Prices Division has a dedicated contract management branch that assess TNS’s performance against the contract, using pre-established key performance indicators. Performance is reviewed with the supplier on a monthly basis.
The sampling frame and sample design are specified by Prices Division, and quality checks are carried out on the data by both Prices staff and TNS staff. The quality checks are transparent and clear on both sides, and the process for compiling the data is well established, well documented, and accredited under ISO9001: 2015 by an external body.
TNS data account for a very high proportion of prices in CPIH and CPI; however, there are many mitigating factors in place that reduce the level of risk. Therefore we feel that a Medium level of risk is appropriate.
Profile: High
Given the extremely wide coverage of the local price collection, there are likely to be prices collected for items which are of varying user interest. Moreover, given the very high weight of TNS data in CPIH and CPI, which are economically and politically important, it would be inappropriate to consider anything other than a high public interest profile.
Assessment: Comprehensive (A3)
Status: Achieved
Data collection is managed by TNS; however, Prices requirements are tightly specified under a comprehensive contract, which is periodically retendered. In the event of the contract being awarded to a new supplier, a dual collection would be necessary for one year to understand the impact on the quality and consistency of the data being provided. Prices Division are responsible for drawing up the sample frame and specifying the sampling methodology, whereas TNS manage the data collection. TNS’s performance against pre-specified key performance indicators is evaluated by a dedicated team within Prices Division. This is discussed with TNS at monthly operations meetings.
Quality assurance and validation procedures are applied by both TNS and Prices staff. These routines are fit for purpose, transparent and well understood.
Considering the evidence summarised above, and provided in detail in Annex A, we believe that TNS data meet the comprehensive level of quality assurance required for the production of CPIH. More detail on price collection arrangements, and quality assurance and validation procedures is provided in Annex A.
Valuation Office Agency (VOA)
Data usage
Valuation Office Agency (VOA) rental prices cover England and are used to construct indices for both private rents in CPIH and CPI, and owner occupiers’ housing costs (OOH) in CPIH. In particular, the OOH index is a very large component of CPIH, and data for England account for approximately 14% of the weight in the headline index. The private rental index accounts for 4% of the weight in the headline index, the majority of which will be due to England data.
Details on the quality assurance of administrative data can be found in our Quality assurance of administrative data used in Private Rental Housing statistics.
5.3 Assurance level: Enhanced
We have assessed a further four of our data sources as requiring an enhanced level of assurance.
This means that we require a relatively complete understanding of the operational context in which data are collected, with an overview of sources of bias, error and mis-measurement. We also require an effective mode of communication with these suppliers and agreement for ongoing data supply. We require a relatively complete understanding of the supplier’s quality assurance principles and checks. Our own quality assurance and validation checks should be proportionate and transparent, and we will communicate any risks that arise from the data.
More detail is provided for each of these four suppliers below.
Autotrader, used car data
Data usage
The data we receive from Autotrader is a point in time, snapshot of all live adverts across all vehicle types, taken daily. The dataset is rich with detailed attribute information, making it easier to define a unique product. As this is not an ecommerce platform but rather an online marketplace, we can only see listed prices in the data. The data are used to compile the second-hand cars index, thus elementary aggregates of the various car make indices, aggregated over the different fuel types (petrol, diesel, hybrid and electric), then into different age groups to construct the second-hand cars index. For more information, please see our Using Auto Trader car listings data to transform consumer price statistics, UK methodology.
Risk: low
Used cars carries a low weight in the Consumer Prices Index including owner occupiers’ housing costs (CPIH) and the Consumer Prices Index (CPI). Receiving daily data has enabled us to build contingency into the process, as we are able to iteratively run the indices as more data isare received. This allows us to identify and resolve issues early in the production round and effectively manage the risks. We have established a good relationship with the Autotrader data team and our terms of engagement are also governed by a data sharing agreement.
Profile: high
Considerable public engagement and interest has been generated with regards to the transformation of consumer price statistics. Used cars data will be the second alternative data source to go into live production as part of a continuous programme of improvement. Using an alternative data source for our used cars index marks a significant improvement in our methodology.
Assessment: enhanced
Status: in progress
As a remedial action
- We will seek further information from Autotrader on data collection, methodology, and quality assurance procedures to allow us to make an assessment of the data source.
Living Costs and Food Survey (LCF)
Data usage
LCF data are used to produce item level weights in CPIH and CPI. COICOP5 is the level of aggregation above item, and so LCF expenditure totals are rescaled to match the HHFCE expenditure totals at the COICOP5 level. LCF account for most of the weight at item level in CPIH and CPI (for example, the OOH item weight is taken from HHFCE data). LCF data are also one of the tools used in the annual basket update, to determine new items for inclusion and old items for removal. The data are delivered to Prices Division on an annual basis.
Risk: Low
LCF data represent all of the items in the basket of goods and services at the item level, but are not used for higher level aggregation. The data source is a survey and, although the survey design itself is complex, no other administrative data sources are used in its construction. We therefore consider it as a non-complex source. The data are collected by ONS field staff, and the survey is managed within our Social Surveys Division. If LCF data were unavailable we could consider using national accounts data instead; however, this risk is unlikely to occur.
Sample design, survey methodology and quality assurance procedures are well documented in LCF publications and, as the data are from a survey, we also have standard errors which help us to understand the accuracy of the data. One drawback of the data is that falling response rates reduce the LCF sample size and representativeness.
LCF supply the data in spreadsheet form, which can be automatically read into Prices spreadsheets. The data supply process is well established, and annual meetings are held with the LCF team.
Therefore, we consider LCF data to be low risk in the production of consumer price inflation measures.
Profile: High
Given the extremely wide coverage of LCF data, we have expenditure weights for items of varying interest amongst users. Moreover, CPIH and CPI are economically and politically important, and LCF data are used for nearly all items. The annual basket updates also tend to receive wide media interest, although LCF data are not the primary source of information for this. Therefore, it would be inappropriate to consider anything other than a high public interest profile.
Assessment: Comprehensive (A3)
Status: Achieved
The LCF team have provided detailed information on their data collection, processing, and quality assurance and validation procedures. The survey is managed by the LCF team and no other data sources are used; therefore, the information provided gives a comprehensive understanding of LCF data. Moreover, good communication mechanisms are in place with LCF, with supplier meetings held on a twice yearly basis (a planning meeting before delivery, and a review meeting after). Deliveries for CPI and CPIH are based on finalized data. There is a risk that falling response rates will introduce bias into the results; however, LCF have adopted a number of strategies to counteract this.
The LCF recently underwent a National Statistics Quality Review (NSQR), the recommendations of which are currently being delivered. The Prices delivery system was reviewed and rewritten, which has reduced the risk of manual errors.
Considering the evidence detailed in Annex A we believe that our level of quality assurance for LCF exceeds the standard required for the production of CPIH and CPI.
Mintel
Data usage
Prices Division purchases market research data from Mintel, for use in the production of some weights at and below the item level, for quality assuring unusual movements, and also for establishing new items for inclusion in the annual basket update, as well as new shops. It is hard to precisely specify the weight that Mintel data have in CPIH and CPI.
Risk: Medium
Mintel data do have quite wide coverage in the basket; however, they are used below the item level as strata weights and at item level to refine LCF weights. As with LCF data, they are subsequently constrained to COICOP5 totals. The data available from the Mintel website are drawn from a variety of sources, usually from surveys run by Mintel themselves. Their methodology, processes and quality assurance procedures are consistent and well documented. Data are generally copied into Prices spreadsheets from source.
The data are purchased on contract and, as part of this contract, Prices are allocated a designated contact. If we could not access Mintel, then it would be a straightforward matter to retender the contract and source similar data from an alternative market research company.
We assess Mintel data as being a medium risk of quality concerns. This reflects the variety of surveys used, and their relatively wide coverage in the basket.
Profile: Medium
Mintel data do not represent as broad a cross-section of the basket as HHFCE or TNS data do. This is, in part, due to the lower levels at which the data are employed, and partly the coverage. As with LCF data, they are used in the annual basket updates, and the coverage is wide enough that some items are likely to gain a wider media or user interest. For that reason we feel that a public interest profile of medium is appropriate for Mintel data.
Assessment: Comprehensive (A3)
Status: Achieved
Mintel are a well established and reputable market research company, who provide a variety of different reports drawn from various surveys and contracted agencies. Mintel have provided detailed information on questionnaire design, sampling procedure, quality assurance and validation checks, and audits. The detail provided is substantial, and Mintel’s procedures are comprehensive. We are therefore satisfied that the level of quality assurance for Mintel data is appropriate for the purposes for which the data are required.
Mintel data are provided to Prices under a contract, which is renewed every 2 years. Prices have a dedicated contact who will respond to queries and concerns.
Detailed information on the operational context, communications, and Prices and Mintel data checks are provided in Annex A.
Rail Delivery Group
Data usage
The Rail Delivery Group (RDG) produces data from the Latest Earnings Networked Nationally Over Night (LENNON) dataset for the Office for National Statistics (ONS) that include transaction level rail fares data, including expenditure and quantities, for rail journeys in Great Britain. These data are used to compile the rail fares indices, which are ranked by region and fare group (such as peak, off-peak, advance).
Risk: Low
Rail fares have a weight in headline the Consumer Prices Index including owner occupiers' housing costs (CPIH) and the Consumer Prices Index (CPI) of 0.9% and 1.1% respectively in 2023. Our relationship with RDG is governed by a legal contract with important performance indicators linked to quality assurance. Receiving daily data has enabled us to build contingency into the process, as we are able to repeatedly run the indices as more data are received. This allows us to identify and resolve issues early in the production round and effectively manage the risks.
Profile: High
The transformation of consumer price statistics has generated a lot of public engagement and interest. Rail fares will be the first alternative data source to go into live production as part of a continuous programme of improvement.
Assessment: In progress
Status: In progress
As RDG provides us with transaction level data, there are a limited number of transformations to be carried out on the data before it is sent to us, reducing the risk of error and bias within the data. Furthermore, the data were acquired based on detailed technical specification provided by the ONS. We are satisfied that our engagement with RDG has resulted in the acquisition of high-quality data. We are currently working with RDG to get the additional information required to complete this assessment.
Remedial actions:
- We will seek further information from RDG on data collection, methodology, and quality assurance procedures to assess the data source.
Scottish government rental data
Scottish government data are used to produce the rental price series for Scotland in the private rental price index, and the OOH component of CPIH. Scottish government data are also used to produce strata weights for the Scotland stratum of the OOH component in CPIH. This stratum has a weight of around 1% in CPIH. Scottish government data are likely to represent a small proportion of the approximately 4% weight for the private rents index.
Details on the quality assurance of administrative data can be found in our Quality assurance of administrative data used in Private Rental Housing statistics.
Welsh government
Welsh government data are used to produce the rental price series for Wales in the private rental price index, and the OOH component of CPIH. Welsh government data are also used to produce strata weights for the Wales stratum of the OOH component in CPIH. This stratum has a weight of around 0.6% in CPIH. Welsh government data are likely to represent a small proportion of the approximately 4% weight for the private rents index.
Details on the quality assurance of administrative data can be found in our Quality assurance of administrative data used in Private Rental Housing statistics.
5.4 Assurance level: Basic
We have assessed the remaining data sources as requiring a basic level of assurance. This means that we require an overview of the operational context in which data are collected, and any actions taken to minimise risks. We also need to provide the supplier with a clear understanding of our requirements, and have contacts in place to report queries to. We require an overview of the suppliers’ quality assurance principles and checks, and should have our own quality assurance checks I place on the data.
More detail is provided for each of these suppliers below.
Department for Business, Energy and Industrial Strategy (BEIS)
Data usage
BEIS data are used to construct weights for a number of energy items in the consumer prices basket of goods and services. In total the motor fuels items contribute a total weight of 2.58% to headline CPIH through:
- Prices for petrol (1.64%)
- Prices for diesel (0.94%)
Risk: Low
Data for motor fuels (petrol and diesel) are collected through a survey, administered by BEIS staff. The weight for motor fuels in CPIH is small (but not negligible) at 2.58%. If the data were unavailable to us, we would investigate alternative sources and, if no such sources exist, we would have to equally weight stratum level indices.
We have a dedicated contact to respond to data queries. Figures are provided by BEIS in spreadsheet form and transferred into Prices spreadsheets. Some methodology and quality assurance information is provided.
Given the low weight of BEIS data in headline CPIH and the relative simplicity of the source data, we consider BEIS data to have a low risk of quality concerns.
Profile: Low
Whilst there may be some media interest in price changes for motor fuels, this tends to be limited as regards consumer price inflation. The contribution of BEIS data to headline CPIH is not large enough to consider the economic importance of headline inflation here.
Assessment: Basic (A1) to Enhanced (A2)
Status: Achieved
BEIS data are derived from a survey conducted within the department. They have provided us with detailed information on the data collection, methodology and quality assurance procedures, which we consider to be fit for the purpose for which they are used within CPIH and CPI. These are provided in more detail in Annex A. We also have a dedicated contact for any data-related queries.
Department for Transport (DfT)
Data usage
Department for Transport (DfT) data are used in the calculation of a number of expenditure weights. In total these weights make up 5.21% of CPIH. Specifically they are used for:
- below item strata weights for used cars (1.40%), in conjunction with Glasses data
- below item strata weights for new cars (2.10%), in conjunction with Glasses data
- below item strata weights for vehicle excise duty (0.55%)
- below item strata weights for motorcycles (0.07%)
- item weights for London transport (0.25%) are constrained to COICOP5 totals
- item weights for underground fares (0.03%) are constrained to COICOP5 totals
- item weights for Euro Tunnel fares (0.04%) are constrained to COICOP5 totals
- item weights for rail fares (0.77%) are constrained to COICOP5 totals
Risk: Low
Together, DfT data constitute a small to medium weight in headline CPIH and CPI. However, there are a number of mitigating factors to consider:
- of this 5.21%, only 1.09 percentage points are used directly for item weights
- of the remaining 4.12 percentage points, 3.50 percentage points are used in conjunction with Glasses data to construct below item level strata weights
- the remaining 0.62 percentage points are used to calculate below item weights without reference to other data sources
- whilst all the data are sourced from DfT, each comes from a different DfT survey or output, we therefore seek quality assurance information for each of the components separately; however, for the sake of brevity, we consider setting assurance levels at the supplier level, taken separately, each of the components make a small to very small contribution to the total weight in CPIH and CPI
In each case, if the data were not available, we would seek alternative data sources and, in the absence of a suitable alternative, equally weight each item within COICOP5 (or item) totals. Much of the data are sourced through email contact with DfT, and either copied into Prices spreadsheet systems, or read in directly. Where item weights are being constructed, data are copied directly from tables in the latest release, and used to create a weight distribution constrained to COICOP5 totals.
Considering the various factors described above, and in particular that we are seeking a separate assurance for each series, we feel that the risk of quality concerns is low.
Profile: Low
All of the series above are of limited media and user interest. (Whilst rail fare increases are often covered by the media, this tends to be at the point when increases are announced; there is limited interest in the item index). Taken together, the series make a small to medium contribution to headline CPIH and CPI. We therefore suggest that a low public interest profile is appropriate.
Assessment: In progress
Status: In progress
Some detail on the data collection, methodology and quality assurance procedures for DfT data is available online, and they have provided us with comprehensive detail on their quality assurance, data collection, and general process for some items (Eurostar fares, rail fares, and London Transport). We are satisfied that our communications with DfT and the information provided give us a basic to enhanced level of assurance for these items. We are currently working with DfT to get the additional information required to allow us to complete this assessment.
Remedial actions:
- We will seek further information from DfT on data collection, methodology, and quality assurance procedures to allow us to make an assessment of these data sources.
Glasses
Data usage
Glasses provide valuation data for used cars around the country. They provide these valuations for various customers (notably, car dealers, who can set their price strategy appropriately). They are a well established and reliable producer of car valuation data. The data contribute to 4.25 percentage points of headline CPIH through the following item indices:
- Glasses data are combined with Department for Transport (DfT) data to produce below item strata weights for used cars (1.40%)
- Glasses data are combined with Department for Transport (DfT) data to produce below item strata weights for new cars (2.10%)
- price data for motorbikes (0.07%)
- price data for caravans (0.68%)
Risk: Low to medium
Taken together, the contribution of Glasses data to headline CPIH is not insignificant, but it is also not large. Of this, only 0.75 percentage points is used directly at the item level, the remaining 3.50 percentage points are used below the item level in conjunction with DfT data to produce strata weights. The data source is compiled from several different sources, and so is reasonably complex. If Glasses data were unavailable, we would switch to other sources, such as used car websites, or directly from company websites.
Data are purchased via annual subscription, and queries are dealt with through regular email contact. Price data are extracted manually from the website, whereas expenditure data are received in spreadsheet form, which can be read directly into Prices spreadsheet systems. There is also a great deal of information on their methodology and processes available online; however, detail of their quality assurance procedures is not provided.
Considering the small to medium weight, how the data are used, and the existing arrangements, we feel that Glasses data merit a low to medium risk profile.
Profile: Low
Indices for used and new cars, and for motorbikes and caravans are of little user and media interest, and their overall contribution to CPIH is not large enough to consider their contribution to the headline index relevant. Therefore we make an assessment of low public interest profile for Glasses data.
Assessment: Basic (A1)
Status: Incomplete
Glasses data are compiled from a variety of sources. The data are purchased through a yearly subscription, and a help desk number is provided for queries. There are some concerns over communication, as Glasses have not yet shared their quality assurance and validation procedures with us. However, there is a great deal of information available publically through their website. There was also a lack of communication from the supplier when data transfer moved from CD to online. Checks carried out by members of staff within Prices Division are comprehensive, and queries are raised through the help desk. Further detail is provided in Annex A.
Remedial actions:
- Establish better lines of communication with Glasses, by seeking a dedicated point of contact within the company
- Continue to request information on Glasses’ quality assurance procedures
International Passenger Survey (IPS)
Data usage
IPS data are used to construct strata weights below the item level for foreign holidays. They are used in conjunction with Mintel data. Foreign holidays make up 2.55% of the weight in headline CPIH.
Risk: Low
The data have a low, but not insignificant weight in headline CPIH. IPS data are collected through a survey, supplemented with some administrative data. Nonetheless, the data structure is relatively straightforward compared to some other sources. Moreover, as the basis of IPS data is a survey, their properties are better understood than data which are compiled from many administrative sources. The data are collected, processed and compiled by our staff within Social Surveys Division. If we did not have IPS data, we would instead use our market research data and, failing that, below item level indices would be given equal weight. The methodology, and quality assurance and validation procedures are well documented.
Profile: Low
Foreign holidays are of limited media and user interest. They also have a relatively low weight in CPIH, which is of greater economic importance. Therefore we will consider IPS data to have a low public interest profile.
Assessment: Basic (A1) to Enhanced (A2)
Status: Achieved
IPS data are largely produced via survey, which is run by the IPS team; however, some auxiliary administrative sources are also used. IPS have provided detailed information on the quality assurance procedures applied to their source data and their outputs, as well as methodology and processing. We are satisfied that the procedures described are fit for the purposes for which they are used in CPIH and CPI. Further details are provided in Annex A.
Consumer Intelligence
Higher Education Statistics Authority (HESA)
Home and Communities Agency (HCA)
Inter-Departmental Business Register (IDBR)
Kantar
Moneyfacts
Data usage
Consumer Intelligence data are used to get prices for house contents insurance and car insurance. The combined weight for these items is 0.43%.
HESA data are used to calculate strata weights (below the item level) for University tuition fees for UK and international students. The combined weight for this item is 1.05%.
HCA data are the source of rental price data for registered social landlords. The weight for this item is 1.34%.
IDBR data are used to derive below item strata weights for boats. The weight for this item is 0.29%.
Kantar data are used to calculate below item strata weights for a number of digital media items: internet bought video games, DVDs, Blu-Rays and CDs, and downloaded video games, music and e-books. The combined weight for these items is 0.36%.
Finally, Moneyfacts data are used as the source of price information for mortgage fees. The weight for this item is 0.12%.
Risk: Low
All of the data sources listed above feed into items with a very low weight in CPIH, generally less than 1.5%. As such their impact on headline CPIH or CPI will be minimal. Should any of these sources of data become unavailable, there is a clear contingency for each:
- Consumer Intelligence: Create a smaller sample, based on price quotes from comparison websites
- HESA: Equally weight courses and institutions below the item level
- HCA: Investigate the use of alternative sources of price data
- Kantar: If finances are not available to purchase the data, Mintel data can be used instead
- Moneyfacts: Collect prices from individual company’s websites
Kantar data are collected through the use of a survey, and Consumer Intelligence data are scraped from supplier websites. IDBR data are more complex, being compiled from 5 different data sources, and HESA data are compiled from all Higher Education institutes across the UK. We are not aware of the sources for Moneyfacts data. All of the data are manually fed into spreadsheets, which use formulae to derive the subsequent price index.
A contract is in place to receive Consumer Intelligence data, and regular supplier meetings are in place. MoneyFact data are acquired through an annual magazine subscription, and Kantar data are purchased annually on an ad-hoc basis. Kantar also provide a dedicated contact. There is no contractual agreement in place for either HESA or HCA; data are instead sourced through direct email contact with the supplier. The IDBR team is based within ONS, so no contract is in place. None of these arrangements are out of keeping with the weight accorded to these items in the basket.
There are some risks associated with these data; however, given their negligible impact on headline CPIH or CPI, we do not feel that the risks associated with use of these data sources merit anything higher than a low level of risk.
Profile: Low
The above data sources are used in the construction of very specific low-level item indices. They may be used to capture the price element of the index, or they may be used for below item-level strata weights. They will generally be combined with prices or strata weights to create the particular index.
With the possible exception of tuition fees, none of the item indices are considered to be of wider user or media interest, and are certainly not politically or economically sensitive. They are generally of niche interest and are politically neutral. Tuition fees can be of interest following a major change; however, such changes are rare and HESA data are only used below the item level. As described under risk, their contribution to CPIH and CPI, which are considered to be economically important and market sensitive, is very small (less than 1.5%) and, as such, their impact on the headline figures is negligible.
Assessment: Basic (A1)
Status: Achieved
Consumer Intelligence is a well established and reputable market research company, who send us a sample of insurance quotes. We have a dedicated contact; however, at present we have been unable to obtain further quality assurance information as the contact has not responded.
HESA data are sent to Prices Division in an Excel spreadsheet. There is a data sharing agreement in place to access the data, and a dedicated contact. Quality assurance procedures are well documented by HESA, and all input data sources are listed.
HCA rental prices for registered social landlords are obtained through direct email contact with the supplier. We have engaged with HCA, who have provided an overview of their data collection process, and quality assurance and validation procedures, which we consider to be fit for the purpose for which they are used in CPIH and CPI.
IDBR have provided us with information on their data collection, methodology and quality assurance procedures. Data are compiled from a number of sources and IDBR’s procedures for validating these sources are clear.
Kantar is a well-established and reputable market research company. Data collection is administered through a longitudinal survey, and the survey methodology and quality assurance procedures have been communicated to us. We consider these to be fit for the purpose for which they are used in CPIH and CPI.
Moneyfacts are a price comparison company, who collect data from websites. We collect the data through a monthly magazine subscription. There is no dedicated contact, so contact details must be sought from the Moneyfacts website. We have some information on the coverage and data collection; however, quality assurance information is not readily available for Moneyfacts. Prices collection and quality assurance procedures are robust. For example, we have often checked extreme movements against company websites, and found the data to be correct.
More detailed information on all of the above is available in Annex A.
Remedial actions:
- Clarify contact details for Consumer Intelligence
- Seek further quality assurance information from Consumer Intelligence
- A dedicated contact for Moneyfact should be established and kept current
- Further detail of Moneyfacts’ quality assurance procedures should be sought
Websites
Direct contact
Brochures, reports and bulletins
Data usage
Price collection from websites is used to collect prices for many of the items which are not sourced through the local price collection (currently conducted by TNS). Website collections account for approximately 5% to 10% of the weight in CPIH.
Price collection through direct contact (typically by phone or email) accounts for approximately 5% to 10% of the weight in CPIH, and is used for items which are not collected locally or through websites.
Price collection from brochures, reports and bulletins accounts for approximately 1.5% to 5% of the weight in CPIH, and is used for items not collected through local collection, websites or direct contact.
These price collections are referred to as ”central” collections.
Risk: Low
Whilst these collections have a small to medium, or medium weight in CPIH, there are a number of factors that reduce the risks substantially:
- All of the price collections are conducted in-house by staff in Prices Division. This gives us complete control over the process
- For all of these collections, there is a very clear and achievable course of action, should a data source become unavailable :
- if a retailer’s website becomes unavailable, then a new website can simply be identified, this is analogous to a shop closing in the local price collection, where we would simply find a new shop to collect the data from; it is extremely unlikely that more than one or perhaps two websites would close down in a given month, and so this is unlikely to cause issues for price collection
- if we are unable to continue collecting from a direct contact supplier then, again, we can simply identify a replacement supplier to collect the prices from
- should we be unable to source appropriate brochures, reports or bulletins, then we could simply identify alternative internet-based sources instead; many of the sources are purchase on annual subscription, so this provides some additional security for ongoing collections
The nature of these collections means that Price quotes will need to be manually entered into Prices processing systems. Robust quality assurance and validation procedures are in place for these processes, and are described in more detail in Annex A.
Profile: Low
None of the centrally collected items are of wider media or user interest, and are not economically or politically important. Whilst taken together their contribution to headline CPIH is large, they actually represent specific collections for many different items. Therefore we assign a low public interest profile to centrally collected data.
Assessment: Basic
Status: Achieved
The assessment of these sources is focussed on Prices own procedures, as these sources are essentially an in-house data collection conducted by Prices staff. This means that we are effectively both the supplier and the producer. We have robust quality assurance checks in place, and our data collection process is recognised under ISO9001: 2015, and supported by in-house staff training. Further information on these is presented in Annex A.
Back to table of contents6. Action plan
In the previous sections we have considered quality assurance for all data sources in our consumer price inflation measures. We assessed the required assurance levels by considering the risk of quality concerns for each data source, and the public interest profile of the item they are used to calculate. We then conducted the assessment based on four practice areas: operational context and data collection, communication with data supply partners, quality assurance (QA) checks by the supplier, and our own QA investigations. This information is detailed in Annex A.
Of the data sources we investigated, there are several that need further work to reach the level of assurance we are seeking.
For Household Final Consumption Expenditure (HHFCE), we would like a fuller understanding of how quality assurance has been applied to the source data used to construct expenditure estimates. HHFCE estimates are based on a complex array of data sources, and users should be aware that these are not necessarily fully understood. HHFCE data, however, remain the most suitable source of weighting information for consumer price indices, following international best practice. Their quality assurance and validation procedures should be comprehensive enough to identify any issues in the source data, and we have a good understanding of the data, given that they are also produced within ONS.
Finally, there are a number of data sources for which we have sought a basic level of assurance, and for which additional quality assurance information has been requested but, as yet, has not been provided. Moreover, contacts for some of these sources are out of date or unknown. We will continue to work with suppliers to better understand their processes. Users should be aware that our understanding of the data is incomplete; however, the risk to headline CPIH or CPI is minimal, as reflected in the basic assurance requirement.
To address these shortcomings, we will carry out further steps to improve our quality assurance. All outstanding actions are summarised below, with details on what actions we intend to take to rectify them.
HHFCE
HHFCE are in the process of completing a QAAD assessment of their data sources.
DfT
We will seek further information from DfT on data collection, methodology, and quality assurance procedures to allow us to make an assessment of these data sources.
Glasses
We plan to establish better lines of communication with Glasses by seeking a dedicated point of contact within the company. We will continue to request information on quality assurance procedures from Glasses.
Consumer Intelligence
We will clarify contact details for Consumer Intelligence. We will seek further quality assurance information from Consumer Intelligence.
Moneyfacts
We plan to establish a dedicated contact for Moneyfacts and keep this up to date. We will seek further quality assurance information from Moneyfacts.
Various
We will set up better communication mechanisms and establish firmer data delivery agreements with various data sources.
This version of the consumer price statistics QAAD is intended to act as a progress update. Over the next few months we intend to continue engaging with our data suppliers and, where appropriate, put in place firmer ongoing communications mechanisms and data delivery agreements. We will aim to publish an update to this QAAD in summer 2017. Importantly, this QAAD is not intended to serve as a final record of quality assurance. We view supplier engagement and feedback as an ongoing process, which we will continue to follow. We therefore intend to publish a review to this QAAD every 2 years.
Back to table of contents