We will continue to reduce our use of manual processes, legacy systems, and tools. The Office for National Statistics (ONS) coding policy mandates python and R as the core development languages for analysis workflows.
The ONS coding policy aims to achieve compliance with the Quality assurance of code for analysis and research guidance published by the Analysis Function. Tools for analysts must allow them to implement these standards on whatever platforms they use to do analysis.
ONS provides R and Python, integrated development environments and Git version control for developing Reproducible Analytical Pipelines (RAP) on the desktop and in our cloud platforms.
R and Python package libraries are mirrored using Artifactory and regularly updated. Some platforms do not support the latest versions of tools.
Version control is supported via internal GitLab and external GitHub repositories. Analysts can publish code internally on GitLab and externally on GitHub. Continuous integration tools are available but not deployed consistently.
We have also assessed our tools and how they meet the RAP strategy.
Analyst leaders will:
Work with security and IT teams to give analysts access to the right tools
Ensure that the Reproducible Analytical Pipelines (RAP) minimum viable product (MVP) is supported on ONS computers and platforms
Status: In progress. The tools required for the RAP MVP are available to analysts on all platforms.
Success criteria: Latest versions of tools available to analysts meet user needs on all platforms at end of 2024.
Ensure that R and python, and their toolchains, are supported by dedicated resource within ONS
Status: In progress Cloud based support is in place. We are working to improve desktop-based support by the end of 2024. This is currently provided by the analysis community.
Success criteria: Analysts know who to contact for technical support when using R and python at ONS.
Ensure that RAP further development is supported on ONS computers and platforms
Status: In progress
Success criteria: The tools required for RAP Further Development (Appendix C) are available to analysts on all ONS platforms at the end of 2024.
Work with security and IT teams to develop platforms that are easy for analysts to access, flexible and responsive to the needs of analysts
Write ONS-wide guidance on which platform to use and when
Status: Not started. We are awaiting new cloud platforms before this guidance can be completed.
Success criteria: Analyst teams know which platform to use for their development.
Analyst requirements to enable Integrated Data Programme Service (IDPS) platform to support RAP are implemented
Status: In progress
Success criteria: IDPS platform includes the tools needed to support the RAP MVP in first public beta release.
Active user forums for ONS platforms to ensure user needs are met
Status: In progress. Most platforms have well-established user communities who are feeding in user needs.
Success criteria: Users and platform development teams meet regularly so that user needs, issues and future requirements are well understood.
The strategy and timetable for ONS future platforms is developed, clearly communicated and rationalises the number of platforms
Status: In progress
Success criteria: Analysts understand how ONS platforms will be developed and managed, and can plan for transition from one platform to another.
Work with security, IT, and data teams to make sure that the data analysts need are available in the right place and are easy to access
Specify IDSP requirements to make sure that data versioning is implemented and fit for purpose
Status: In progress. Requirements have been specified and are being addressed by the Integrated Data Service (IDS) development teams.
Success criteria: IDPS includes dataset version management that meets user needs as defined in the Quality Assurance of Code for Analysis and Research guidance.
Analysts will:
Use open-source tools wherever and whenever appropriate
Transform analysis workflows to RAP workflows using open-source tools
Status: In progress
Success criteria: Priority RAP projects are successfully completed in 2024.
ONS coding policy prioritises open-source tools by default
Status: Completed
Open source their code
Publish guidance on how to open-source code responsibly
Status: Completed. See the Open sourcing analytical code guidance.
RAP projects are open sourced in 2024
Status: Not started
Success criteria: RAP projects in the ONS are open sourced by the end of 2024.
Work with data engineers and architects to make sure that source data are versioned and stored so that analysis can be reproduced
Specify IDSP requirements to make sure that data versioning is implemented and fit for purpose
Status: In progress
Success criteria: IDPS includes dataset version management that meets user needs.