Capturing content published on blogs has been part of our everyday work for some time. Other social media platforms, such as Twitter and YouTube, are a challenge for traditional web archiving technology. They are highly interactive and instantaneous, and much of the technology that underpins them changes regularly. Social media services are primarily designed with immediate use in mind and, because the content is forever changing and being deleted, it is at a high risk of being lost forever.
Government now uses a wide variety of social media platforms but we have limited this project to Twitter and YouTube as, although archiving them presented unique technical challenges, we were confident that it was feasible. In addition, Twitter and YouTube are heavily used across central government and contain a large amount of information that may not be accessible elsewhere. For comparison, Facebook is far too complex to archive effectively at scale while limiting to government accounts. Flickr is very similar. Now we have developed scalable solutions for Twitter and YouTube we will capture target accounts according to a regular schedule, and ensure this content remains available even after the original accounts have gone.
Continue reading »