‘Technology is best when it brings people together.’
Matt Mullenweg, creator of WordPress
Unifying national collections
The way we access information in the virtual space is changing. Discovery and exploration are no longer constrained by a keyword entered into a blank search bar. Instead museums, libraries, archives and galleries worldwide are welcoming a shift to generous interfaces, presenting their collections online in browsable and linkable networks of information that allow users to explore and discover new ideas through meaningful and contextualised relationships.
A transformative emerging technology in this virtual browsing landscape is ‘visual search’, an AI-based method of matching similar images based on their visual characteristics (colour, pattern, shape) rather than a keyword description.
The Deep Discoveries project will explore new machine learning methods to create a computer vision search platform that can identify and match images across digitised collections on a national scale. The research will focus on botanically-themed content, which will allow the researchers to test how far the recognition capabilities of the technology can be stretched. Can it recognise a rose in a textile pattern and the same flower in a herbarium specimen? How about on a ceramic vase?
The project is one of eight foundational projects in the Arts and Humanities Research Council (AHRC) ‘Towards a National Collection: Opening UK Heritage to the World’ programme. This is a five-year Strategic Priorities Funded programme that aims to benefit public audiences and researchers by making the UK’s unparalleled heritage collections increasingly discoverable as a unified ‘virtual collection’, dissolving barriers between different collections and extending access beyond the physical boundaries of their location.
Other foundational projects include projects on the integration of persistent identifiers and IIIF technology for digital assets, the management and preservation of born digital collections, involving citizen researchers in the creation, use and re-use of heritage data and integrating location and available information into the networked digitised collections.
All of the projects are led by Independent Research Organisations (organisations with the independent capability to undertake and lead research programmes) such as Tate, The British Library, the V&A, the Science Museum Group and other major cultural institutions and IROs.
‘Creating an integrated and accessible digital “collection” requires the investigation and implementation of many standards and new technologies (persistent identifiers, for example). But the initiative also opens the door to playful and creative new methods for search and discovery, like computer vision search, which might lead to unexpected ways for our audiences to engage with the digitised images.’
Lora Angelova
People: A cross-disciplinary approach
The project is a collaboration between The National Archives, the University of Surrey, Royal Botanic Gardens Edinburgh and the V&A. Along with project partners Gainsborough Silk Weaving Company, the Sanderson Design Archive, and the Museum of Domestic Design and Architecture, the team will assemble tens of thousands of images featuring botanical themes, from textile designs to herbarium specimen drawings. The University of Surrey’s team, based at the Centre for Vision Speech and Signal Processing, will use the images to fine-tune a computer vision search algorithm to meet the aims of the project.
Lora Angelova, Head of Conservation Research and Audience Development at The National Archives, is leading the project and will be working with a unique and diverse team which includes: Pip Willcox, Head of Research and Rachel Smillie, Head of Academic Partnerships (both at The National Archives), John Collomosse, Professor of Computer Vision at University of Surrey, Robert Cubey, Plant Records Officer at RBGE and Joanna Norman, Director of the V&A Research Institute. A post-doctoral researcher at the University of Surrey and two Digital Researchers at The National Archives and the V&A will also be central to the work.
The team’s expertise ranges from computation to curatorial, material science and audience engagement, with all investigators having worked in cross-disciplinary research environments and with a very broad variety of audiences. The team will work iteratively, and in parallel, addressing both challenging technological advancements and methodological questions around technology deployment, audience engagement and impact. Alongside these challenges, Deep Discoveries will contribute to an overall understanding of the criteria necessary for image collections across the UK to be linked, and survey the searching needs of diverse audiences.
Lora Angelova writes: ‘One of the most exciting aspects of Deep Discoveries is the involvement in our project of The Museum of Domestic Design & Architecture, The Sanderson Design Archive, and Gainsborough Silk Weaving Company. Working with partners that range from a private enterprise to a small museum associated with a university will allow us to better understand the possibilities of including a huge range of digitised image collections into the search platform, as well as the challenges faced by smaller organisations in digitising their collections.’
Possibilities: Pioneering search and discovery tools
Visual search holds huge potential for individuals and organisations and the development of visual search platforms for heritage collections are already underway globally (for example: ArtUK.org, ukiyo-e.org, pharosartresearch.org, and Google Arts & Culture initiatives).
As part of the ‘Towards a National Collection’ programme, Deep Discoveries is unique in that it will develop a method to network image collections specific to a nation rather than a single unifying collection type, such as woodblock prints, paintings or photographs. The insights developed could not only be extended to other large digitised image collections available via APIs (e.g. http://metmuseum.github.io) but will develop new scientific insights into best practices for constructing deep neural networks for visual style similarity.
Because of the richness, diversity and historical depth of the UK’s heritage collections, such a tool developed in the context of networking our national holdings would demonstrate the feasibility of the technology, positioning the UK as a leader in exploring the benefits and tackling unforeseen challenges in connecting image collections. It would serve as a pioneer in the creation of search and discovery tools within the virtual museum environment.
Crucially, the potential to uncover and generate links through visual search will create unforeseen insights into our shared past and present, allowing the emergence of new narratives around the complex tapestry of identity, while simultaneously creating bridges to nations around the globe whose histories are intricately linked with ours. The local connectivity of our collections, facilitated through empowering and user-friendly search technologies, will only serve to demonstrate that our heritage is inextricably interwoven with that of the global village.