New Year Openings at The National Archives are a time for looking back at the world of 30 years ago, marvelling at how much has changed, or, as a recent blog post on Renewing the Values of SocietyÂ demonstrated, how much has stayed the same.
What government was doing about the web in 1982 hasnâ€™t received the publicity of the Falklands files. Mainly because, you might think, in 1982 the World Wide Web was little more than a gleam in the eye of Sir Tim Berners-Lee, called ENQUIRE. But while war was raging in the Falklands, a group of civil servants from the governmentâ€™s Central Computing and Telecommunications Agency (CCTA) were trying to second guess the future (BN 120/8 and BN 120/9).
Pre-web, there was plenty of computing going on in government departments; most of it hidden away on the one large machine each department had, with a scattering of terminals that connected staff in distant offices to the machine. The CCTA were trying to establish how much need, if any, there would be to transfer data from one government departmentâ€™s machine to another. Another 1982 anniversary was the official adoption of the TCP/IP protocol, building block of the internet, by the US Department of Defense. In 1982, data was being exchanged across the world, but the internet was still very much part of its US Cold War communications origins. In the UK civil servants were speaking not of nets, let alone internets, but packet switching. The climate for building networks was not very encouraging. There was a waiting list for new telephone lines â€“ which it was hoped the privatisation of British Telecom would address – meanwhile the first Data Protection Act (1984) was stirring in Parliament; Â there was an awareness of the publicâ€™s reluctance to have their personal information shared across departments by these worrying computers that sent you gas bills for Â£1,000,000,000.99p. Â And cost was a major factor: to saveÂ taxpayers money data from local benefits offices was sentÂ by the cheaper overnight tariff to the DHSS central computer in Newcastle.
But not only were the circumstances unpropitious for transmitting information, the CCTA files also demonstrate the difficulty of thinking about the future when you only have the vocabulary of the present: sending data in the sense of the repetitive number crunching done by the computer over a telephone line was refereed to as â€˜non-voice communicationâ€™. Looking forwards to the next ten years the CCTA surmised that there might be a dramatic increase in the number of data terminals (linked to the one departmental machine), perhaps in the order of 30,000 to 40,000, although Â some disagreed that â€˜non-voice communicationâ€™ would increase so rapidly.
At home, in schools and in small businesses, people were staring to use â€˜microcomputersâ€™ (the BBC Acorn BBC Micro was launched at the end of 1981 and the SinclairÂ ZX Spectrum in 1982), Â teaching themselves basic programming languages such as, er, BASIC. IT, whether at home or in the office, was flat, monochrome (generally green), text based, and static. You never seemed to be allowed to use more than eight characters to name anything. ‘Imagesâ€™ had to be created by manipulating keyboard characters; creativity was all about overcoming the constraints of what you couldnâ€™t do.
When the web, with all its liberating creativity and connectivity, came a decade or so later governments were early adopters of it. At first the archival world didnâ€™t quite know what to make of it; archives dealt with records, so there was a lot of debate about whether or not websites were records, there was a feeling that even if they were records, the stuff that archives wanted was the real solid stuff going into files in offices, the stuff of New Year Openings, not the ephemeral and open postings on websites. But as the web grew, it became apparent that whatever you wanted to call the material on websites, much of it only existed there, and was prone to disappearing pretty quickly; The National Archives has been systematically archiving the UK government web domainÂ since 2003, with many instances going back to the mid-90s.
Ministers and civil servants started blogging, and then tweeting, and people started answering back. In the UK Government Web Archive, weâ€™ve taken up the technical challenge to capture all this (even Twitter â€“ watch this space), to record the new ways that government is interacting with its citizens. But none of it is secret. In 2033, people might look in the UK Government Web Archive and think, Twitter, how quaint! (or even, â€˜websites, how quaint!), but they wonâ€™t be seeing anything they couldnâ€™t previously have looked at. There wonâ€™t be that frisson we get at New Year Openings when the answers to that great historical question, â€˜What was really going on?â€™ come tumbling out.
Imagine when you look atÂ material from the web archive, you didnâ€™t see this:
But something like this:
Researchers in Spain have usedÂ Twitter data to create this social graph of retweets from the 2011 Spanish election campaign. Analyses and visualisations such as this one demonstrated that electioneering tweeting went on almost exclusively within the party circles of its supporters; Twitter was not a means to engage with the opposing sides.
This is information you canâ€™t get from looking at one, or even lots of web pages.
There is endless scope for using the whole UKGWA collection as one set of Big Data. In some ways websites, with their mix of text, images, videos, seem the opposite of a dataset, but across the UK Government web domain there are a vast number of instances of the same kind of structured data: as well as Twitter feeds there are Â links to be analysed (see the work being done by Scott Hale at the Oxford Internet Institute); Â location information â€“ addresses, postcodes; Â job vacancies, even images: â€˜Who is shaking the Ministerâ€™s hand?â€™ may sound like a trivial question, but who knows what conclusions might be drawn from an Â image analysis of the thousands of photographs of such interactions we have (thanks to my colleague Claire Newing for suggesting this one).
Like the civil servants of 1982, we donâ€™t know what new technologies the future will bring, but already there is a growing community of tech savvy researchers, adept at data mashups and visualisations geared up to explore an uncharted world of hidden secrets in the UK Government Web Archive – and you wonâ€™t have to wait 30 years for the answers.
- 1.Â Pablo AragÃ³n, Karolin Kappler, Andreas Kaltenbrunner, Jessica G. Neff,Â David Laniado, and Yana Volkovich, Barcelona Media Foundation, Barcelona, Spain ^