#hackon12: We came, we saw, we didn’t sleep much

A few weeks ago when I talked about the purpose of hack days I promised I would report back after we had actually held one here at the Archives.

Last weekend, a large group of enthusiastic attendees joined us here at Kew for Hack on the Record. I think it might be the first hack day held actually inside a UK government department but I’d be happy to be proved wrong.

Approximately 12 hours in - only 12 to go!

The aim of the event was for the Archives and other organisations (such as the British Library) to show our latest work in opening our data to developers to build… whatever. Some of these datasets are colossal, like the systems which underpin Discovery or Legislation.gov.uk. But like any archival collection, they aren’t much good if they aren’t being used by people. We need to be encouraging their use and doing all we can to lower the barriers to working with our data to make it as easy as possible.

We pitched what we thought were some of the most promising datasets to participants. I have my own favourites – Discovery contains so much information which can be represented in all kinds of different ways.

We also invited the audience to challenge each other to solve specific problems. These ended up focusing around mapping, archiving social media and interpreting different sorts of records. My intention was to throw as many ideas around as possible to help the hackers understand the range of possibilities available. By 5 o’ clock we’d moved up to the reading rooms and a few dozen laptops were getting serious workouts.

By 10pm, most people were still working and a few hardened souls stayed up coding into the wee small hours. I confess I slunk home at about half eleven but a number of colleagues worked through the night and I’m very, very grateful because without them we wouldn’t have had an event. (And if I’d stayed over night, not even jump leads would have got me started on Sunday morning).

What I was building (with my meagre technical skills) was a map of the Victorian photographers listed in COPY 1, our collection of copyright registers from Stationer’s Hall. When photographers wished to register an image for copyright, they filled in their name and address and attached a print. The result is an extraordinary collection of images but the addresses allow us to build up a picture of the geographical spread of the whole collection. Take a look at the data here and click Visualize -> Map to see the mapped data.

At 4pm on Sunday, we reconvened in a conference room to present what we’d built. I’ll be posting a full list of all the hacks shortly but I’d like here to single out our winners.

First prize went to Jonathan Tweed and Kai En Ong (ably assisted by Michael Smethurst, Faith Mowbray and Paul Rissen), who developed a website based around the documents and data from the @ukwarcabinet Twitter feed. This uses Twitter to present Cabinet documents in ‘real time’, 70 years on and now provides the backbone to organise the information on their site.

Joint second prize went firstly to Tim Hodson who, built a visualisation of the relationships in SC 1, our collection of “ancient correspondence”, letters primarily from the medieval period. You can clearly see centres of power in the data: Henry III and his justiciar Hubert de Burgh sit like spiders at the heart of a web of parchment.

Jamie Mahoney’s Show me the money not only attempted to visualise the complex financial relationships in C131, which contains records of medieval lenders and debtors, but also included a crowdsourcing tool to clean up the data.

Third prize went to Crystal and Steven Hirschorn for their visualisations of Olympic data scraped from Wikipedia.

Extraordinary work by our own Aleks Drozdov showed us the future of Discovery – a mobile app capable of supplying documents and data to a phone. Peter Camfield explored time, and Sam Mbale, space. Not everything worked and there are many holes in systems produced in (well under) 24 hours. Even my simple hack contains numerous pins in the United States, almost all of which are errors.

But as proofs of concept, these systems show what is possible to achieve with cultural data. I was hugely inspired by all our hackers who gave up their time and I want to thank each and every one of them as well as all the hard working (and sleep deprived) National Archives staff who supported them.

Anyone fancy doing it again next year?