Vanessa Cuthill worked at the ESRC for the past seven years, most recently as Deputy Director of Evidence, Impact and Strategic Partnerships. She now works as Director for Research and Enterprise at the University of Essex.
In this blog Vanessa explains how, over five years of working with researchers, statisticians, funders and policymakers, she has explored barriers that currently inhibit opportunities for social science researchers wanting to access and use administrative data, and how we are now closer than ever to improving the UK legal landscape for data sharing.
For the past five years the ESRC has been highlighting the opportunities and challenges facing researchers who seek to use ‘administrative data’ which is collected routinely by UK government departments for its operational and transactional purposes. Such data has potential value when used by researchers to answer public interest questions, such as: what factors reduce social inequalities and improve social mobility? what are the key influences on health and wellbeing pathways over the life course?; and how can we better understand UK productivity and growth?
When I started to think about the potential research use of administrative data I did not fully understand why such research was not always possible. Five years on, I have a much better understanding, but what progress in using administrative data has been made?
My baptism into the challenges being faced by researchers was in 2011 via an ESRC-led initiative, the Administrative Data Taskforce (“Taskforce”), which aimed to understand what were the opportunities and barriers to accessing and linking between government administrative data for research and policy purposes.
Uniquely, this Taskforce brought together key government departments and statisticians, and within a year published a report which had four main recommendations – one of which highlighted the need to change UK legislation (the focus of this blog).
While the potential value of using administrative data for research purposes inside and outside government is now better understood, the legal challenges surrounding data access and sharing remain complex. Government departments, Local Authorities, agencies and other public bodies face differing legal restrictions on the nature of the access they might provide for research using data that they control. Through the discussions of the Taskforce I rapidly came to appreciate the complexity of this legal landscape and that if progress is to be made this legal landscape must change.
For the past eighteen months I have been working with the Cabinet Office and Involve, a charity with expertise in public participation, exploring the suitability of proposals for new data sharing legislation. A novel Open Policy Making (OPM) process has been trialled by the Cabinet Office to bring together civil society organisations, privacy groups, academics and representatives from government departments to explore legislative proposals and, as a participant, I have found this a refreshing approach to policy development. Our OPM process culminated in a report that was endorsed by those who participated and enabled the Cabinet Office to shape its legislative options with a degree of confidence about their acceptability.
The report suggests several safeguards that should build public confidence, such as accreditation of researchers and those handling the data, a focus on the ‘public good’ of research projects, and the de-identification of data (by taking away personal information and replacing it with codes that can be linked).
The OPM participants learned how these safeguards are already being successfully utilised in an existing ESRC-funded data infrastructure, the Administrative Data Research Network (ADRN), which was set up by the ESRC following the 2011 Taskforce recommendations. Such safeguards also chime with the findings of reports  that ESRC and others have undertaken to explore public attitudes to data sharing, which found that researchers and statisticians are seen by respondents as ‘trusted’ in a way that private sector companies are not always, and that there can be broad public support for using this data for research purposes.
Other challenges arising from cultural barriers to sharing administrative data across and within government departments have been highlighted during our discussions. These cultural barriers still paralyse and in some cases prevent research projects progressing. Barriers such as data quality and accessibility, as well resource and time constraints, do impact on government departments decisions to share data with researchers; however risk aversion resulting from a few well publicised incidents of data breaches have led to a culture, in some departments, not to share data with researchers. New data sharing legislation won’t automatically address such cultural barriers, but they will be more exposed and harder to defend.
Until this summer it was not clear to me that any of these discussions and efforts would lead to changes in legislation. However I am delighted that proposals for new legislation are included in the Digital Economy Bill, which has now had its second Reading in Parliament. This offers the research community an opportunity that may not arise again for another decade. The proposals in the Bill do not alter pre-existing legal gateways, but offer a new legal route for government to share its administrative data with researchers, subject to safeguards being in place which protect privacy.
However, the proposals in the Bill do have a number of shortcomings: they exclude health data which will therefore continue to inhibit and complicate interdisciplinary research; and as the proposals offer only an optional legal gateway for use by government departments they stop short of requiring departments to share their data with researchers, leaving the question of how effective such permissive legislation will prove to be, and will the cultural barriers to sharing with the research community be addressed?
Nonetheless, the research community must seize such a rare legislative opportunity. My experience over the past five years suggests that we need to continue to highlight the research value of using such data, whilst we press for changes in government data sharing culture, and legislative change through this Bill. After five years of dreaming about legislative change, I am now hopeful that this is on the horizon and that working together we can change the data sharing landscape for the better and enable more data-driven research to inform important policy decisions.
 http://www.esrc.ac.uk/public-engagement/public-dialogues/public-dialogues-on-using-administrative-data/ and https://www.statslife.org.uk/news/1672-new-rss-research-finds-data-trust-deficit-with-lessons-for-policymakers and https://www.statslife.org.uk/news/2721-public-supports-sharing-health-data-for-research-poll-says and https://wellcome.ac.uk/what-we-do/our-work/public-engagement-and-trust