Waiting for Lunchtime

For quite some time in the early 2000s the BBC had been running a messageboard on its "DNA" platform, asking for people to send in their memories of the Second World War, either from direct experience or from information given to them by parents or others who'd been alive at the time.

The project was enthusiastically promoted, and outreach teams travelled to libraries and communities around the country to try to find as many tales as possible. In many cases very elderly people, who'd never used a computer before, were taken through the steps needed to set up a DNA account and upload their words and photographs, thanks to very patient help and advice from hundreds of volunteer staff at libraries and community centres across the country. The Open University got involved, and use the information as part of their British History courses.

This information was then lovingly curated by editorial staff working at the online section of BBC History, and gradually a tremendous wealth of information about the War was being built up. And this wasn't the reminiscences of politicians or generals, this was the War as seen through the eyes of ordinary people, soldiers who were fighting across Europe and Africa, and civilians at home in bombed cities. What's more, because of the messageboard format of the system, people could reply to the memories of others, and reminisce about shared experiences that happened 60 years ago.

Then in 2006, as part of the Graf Report's review of BBC Online, it was decided that it was too expensive to pay for a couple of editorial staff to curate this information, and unfortunately it would have to close. But fortunately it was realised how valuable a resource this information was for the nation, and it was decided that although the "dynamic" messageboard system was too much effort to maintain, if the information could be published as "flat files" then it could remain on the BBC website forever.

This is where I got involved. Myself and another developer spent a few months setting up the necessary databases and conversion software to create a permanent archive of this information. We wanted to ensure that this information would be available forever, so we didn't just simply create a non-dynamic flat-file version of the DNA messageboard, we also converted all the information into XML files so that even in the future, if HTML stopped being the language of the internet, there would still be an easy way to convert this information to some other format.

We also wanted to tag and classify the data as best we could. This would be an impossible task to do manually over 100,000 files or more, so we wrote a Bayesian scanner which was run over the entire dataset to try to provide some degree of classification. It wasn't perfect, but it was an awful lot better than nothing.

It wasn't just the BBC who thought this was a worthy project. The British Library asked for a copy of the final XML files and the associated scanned and uploaded photographs. So did the National Archive. This really is what a public service broadcasting's online service is for -- producing things of permanent national interest, but which have very little commercial viability. If the BBC hadn't spent years obtaining this information, it would all have gone forever when those who remembered the War firsthand all eventually passed away.

And now?

Now ... once again the BBC has decided it needs to prune the amount of information it keeps online. And one of the sites it's said it'll be getting rid of is the People's War site. Yes, really.

Note that's getting rid of. They're not going to archive it. They're not going to hand it over to someone else. They're intending to completely delete this information. Forever.

It's not even as if this decision saves any money. The site is already an archive and there are no staff overheads involved in keeping it. The disk space required is minimal, a few hundred megabytes -- you can't even rent that small an amount from a commercial provider these days. It would fit comfortably in the memory of most people's mobile phones!

Yes, the data exists at the British Library and the National Archives. I know this, because I remember transmitting the data to them. But as far as I know it's still in the raw XML format we gave them. They've never unpacked it and turned it into another website. It's not accessible to the public. For all practical purposes this only exists at the BBC.

The People's War website is a genuine "national treasure". Please help save it! Write to Mark Thompson, Director General of the BBC. Write to Erik Huggers, Director of Future Media at the BBC (until he takes up his far more lucrative position with Intel, anyway). Write to your MP, and explain to them why this data needs to be saved. Please!

Lastly, here's the link to the WW2: People's War website itself. Have a look round, read some of the stories. Thanks.