Cloud storage and archives: a match made in heaven?
With the rise in cloud services and storage, archivists are asking whether this can help address a growing digital preservation challenge. I am leading a team of myself (from Charles Beagrie Ltd), Paul Miller (from Cloud of Data), and Andrew Charlesworth (Senior Lecturer in IT Law, University of Bristol), which is funded by The National Archives, to address questions that archivists have raised about digital preservation and the cloud.
Archivists have understandable reservations around aspects of outsourcing to the cloud and whether it will meet their preservation needs. Some of the most common questions raised (but not an exhaustive list!), are:
- What do you do when your cloud provider fails? From the perspective of a long-term memory institution, how can an archive prepare for and manage sudden and catastrophic denial of access in the cloud such as bankruptcy, closure by financial regulators or by legal authorities? Such occurrences may be rare but will inevitably happen if you have to think in the long-term timeframe of an archive.
- How do we ensure any legal requirements and obligations relating to third party rights in the data to be stored (e.g. copyright, data protection) will be met? These may be placed upon us and our parent organisations by donors and funders via contracts and agreements or via legislation by Government.
- What level of data security can be provided? Will data be stored in the UK, within the European Economic Area, or elsewhere?
- What will it cost and how can you make a business case for it? Can it help us work in partnership to make digital preservation affordable? Are there other archives like us doing this and what experiences have they had?
We have been preparing guidance and case studies to assist the archives sector understand and share emerging best practice. The key requirements and needs in terms of the content of guidance for the sector have been collated via a series of interviews and a focus group with archive representatives.
Our team member Paul Miller has already posted a blog entry elsewhere during January called ‘Can the cloud do ‘in perpetuity’?‘ that introduces some of the key issues. As he notes there, the cloud can rarely be beaten as a way to get something up and running quickly, affordably, and with a minimum of fuss. But some of the most compelling attributes of the cloud are best suited to ephemeral or (relatively!) short-term use cases, whereas archives are there for the long-term. Are they compatible?
Earlier this month, I presented some of our key findings to around 50 Welsh archivists, records managers and students at the Archivematica Go-Live event held at the National Library of Wales. Of particular interest to that audience were the experiences of our case studies: Dorset History Centre, King’s College London, Parliamentary Archives, Tate Gallery, University of Oxford, and the Archives and Records Council Wales Digital Preservation Consortium (whose use of Archivematica in the cloud as a pilot project was the main focus of the event).
Each of these archives has established their requirements for cloud and/or local storage and has tested a range of potential solutions. Much of their work is at a very early stage but our case studies present where they have reached at the beginning of 2014, and we plan to update them again in around 12 months when various pilot projects will have concluded and additional valuable experience can be shared.
The guidance and case studies are being reviewed currently prior to their planned release in the second quarter of this year, when they will be published and announced on The National Archives’ website.
Whilst primarily targeted at public archives, the aim is to provide information that will be useful within a range of organisational contexts, and overarching advice that can also be translated into the private sector where relevant. It is intended to be accessible to individuals with a range of previous knowledge and experience.
To accompany the publication of the guidance and a future update, we are planning two webinars for archivists on digital preservation and the cloud. The first will be in mid-May 2014 and the second in early 2015. Keep an eye out for further announcements closer to the time that will provide details of dates/times and registration and an opportunity for you to put questions to us for the discussion.