If you’ve used our website in the last 18 months, you’ll be aware that we’re in the process of replacing our old Catalogue with a brand new shiny one. Discovery, our new catalogue, has been designed to provide a similar experience to the search tools found elsewhere across the web, with features like filters (to refine searches) and tagging (to help other people find records). It has also been designed with large volumes of data in mind – the old Catalogue was creaking under the weight of several million records, and with many more to come in the next few years (especially with the impending shift to 20-year rule, and the expected arrival of even more government records) we decided that the time had come to build a new one.
Right from the outset we knew that it would be an enormous undertaking. Our ambition has always been to replace not just the Catalogue, but also to incorporate data from other systems such as DocumentsOnline, the National Register of Archives and the ARCHON directory, in order to provide a ‘one-stop shop’ for anyone wanting to search our collection – and those held by other archives – and to download digital copies of records where available. The practicalities of doing this have been challenging, to say the least – with a number of different technical systems to deal with, along with enormous amounts of feedback from our many different user groups and staff, it has taken time to build a system that fulfils all our users’ needs.
Our approach has been to develop Discovery over time, gradually adding new features and functionality. Discovery is a very different system to the old Catalogue, and works in a different way – the iterative approach has allowed us to test it, help users get used to it and gather feedback that has helped us improve the system as we go. We’ve also been able to fix minor technical issues (bugs) as they are reported, releasing new versions of Discovery on a fairly regular basis.
The last major development came in September, when we launched the new browse feature (and switched off DocumentsOnline). We had a big push to ask for feedback that would help us identify any remaining ‘showstoppers’ (major issues) and decide when we could switch off the old Catalogue, without causing problems to our users. We received a mix of comments, complaints and suggestions – all really useful! We’ve already fixed some of the major sticking points, and are working on others, but we thought it would be helpful if we explained some of these in more detail.
How search results are displayed
The order in which search results are returned is probably the main bugbear of both users and staff. At the moment results are displayed in relevance order, using logic based on the search term used. This is how most search engines work, and is fairly standard across the web. However, this is not how the old Catalogue returns search results (based on catalogue reference logic). We’re trying to improve our understanding of each option, and the implications of each; we’re going to look at users’ motivations for searching, and how results may vary across different record series. We’re not sure how long this process will take, but it’s currently our highest priority – watch this space.
Searching digitised records
Since we moved our DocumentsOnline collections into Discovery, we’ve encountered a few problems. Many users have reported that when using the quick search boxes in our online records pages they’ve encountered odd search results; for example, users searching for the surname ‘Bedford’ in our wills collection have had lots of results for the place name Bedford.
Digitised collections have their own sets of metadata – the information attached to digital files that is used to search and browse – that doesn’t quite match up with the data contained in catalogue descriptions. This is because we have additional metadata which is more detailed and structured than the related catalogue descriptions are. In the digitised collections metadata, Bedford is marked as either a name or a place depending on the usage but this is not how the catalogue descriptions work. In short, the two sets of metadata don’t quite match up, and as a result the search functionality that’s been built into Discovery – to search catalogue metadata – can return unexpected results. Fixing this is a fairly complicated process, but we’re on the case.
On the whole we’ve received very positive feedback about the new browse feature, which we’re obviously delighted about! Some users (and staff) have asked to see a bit more text from the catalogue description, to help with the browsing process – at the moment only one line of text appears, and you have to click into the full description to see more information. Once again though everybody wants something different, so we’re probably going to run some user testing to make sure that the solution we propose will work for at least the majority of users.
The new release
This week we’ve released a new version of Discovery. This fixes some bugs that had been reported, as well as issues that were causing problems (in one case we were accidentally sending users round in circles, which we’ve hopefully resolved simply by removing a confusing link!).
Keep talking to us!
In all, we’ve already received thousands of comments, compliments, complaints and suggestions about Discovery, which are logged for review and prioritisation. We always intended Discovery to be a work-in-progress, to an extent – it has been built to be flexible, in order to accommodate the changing needs of our users, along with the expansion of our collection over the next few years. That’s why we need you to keep telling us what you think of it, because we can’t fix your problems if we don’t know what they are!
We’ll be back with another blog in a couple of weeks’ time, with news of the progress we’ve been making on integrating data from other archives (such as National Register of Archives and ARCHON) into Discovery – stay tuned!