Lifelong data: Imperial War Museum storage campaign

“When I started, I was shown an office with shelves with a 500GB hard drive.”

CIOs everywhere are obliged to procure long-term, secure storage for their organizations, but don’t hesitate to think of Ian Crawford, CIO of the Imperial War Museum (IWM). His mission is not only to store and store multimedia, which is the equivalent of five heritage sites containing highly sensitive data, but also to store it. eternally.. One of the oldest film archives in the world, a digitization program that contains about 24,000 hours of complex audio and video, as well as newly recorded material, and can handle assets of this size (and normal business storage requirements). Provides infrastructure) is not a small order.

“When I started [in 2007] I bowed in the middle, about 3 meters long, and was shown an office with shelves equipped with a 500GB hard drive. That was the collection. No backup … IT had just installed the first NAS box to start migrating data. “

The museum was informally digitized before he arrived, but Crawford knew there was still a lot to do. “At that time, we couldn’t go out and buy products that met all our needs, so it became a kind of hybrid of products from two different companies. [Axial] To catalog the objects in the collection.Then use a commercial digital asset management (DAM) company [Spectra Logic] To provide that side of the operation. Then I used a simple API to link and integrate the two. “

In March 2021, IWM deployed Spectra Logic’s Storcycle software as part of a large archiving infrastructure project to store existing digital information. We also continued to add to the collection through events, documentaries and shared projects. “We have a lot of experience working with Spectra for years. We have a tape library. I think I ordered the first library about 10 years ago. I still have it. , Upgraded. Duxford also has a second library physically separated, where most of the digital assets we generate are located. [are stored], Making a digital copy of the movie. “

Developing an infrastructure that combines the ability to store large amounts of data with “old-fashioned” catalogs has never been easier, but the structure Crawford has implemented can help a lot to do just that. “There are two T950 tape libraries, both using different tape technologies, one using LTO-7 and the other using IBM TS1150. Compliant with the OpenArchivalInformationSystem standard and basic. You need multiple copies, and you have to do some tricky things, such as making sure they aren’t physically next to each other. “

These tape libraries are stored in Duxford, but you can save them to both archives at the same time each time a copy is made. “Use uncompressed DPX. It’s basically like a string in an all-sequenced TIFF file. Then use Prores to make a kind of mezzanine copy from DivX or MPEG 4, H. Other files can be generated. 264, such. This is usually held on a rotating disk, so it also duplicates the rotating disk. Therefore, Duxford has a series of rotating disks (again, Spectra Logic). ), But this time London via Janet connection. “

The last frontier

Finding space for similar storage in London is a challenge that is unlikely to be solved so far. With an unlimited budget, that might be possible, but the ideal world the heritage sector dreams of is far from reality. Funding and space are the awards that museums and archives are always looking for, but recent global events also add their own challenges.

“We’ve been working on a project to set up a new gallery in a building on Lambeth Road for the past five to six years, but it’s actually taking up a lot of office space and abandoning another building. At this point we are trying to move all the kits from Lambeth Road to Daxford because we had a main server room on the same campus, so Covid is not in favor of delaying the project a bit. “

Ian Crawford
Ian Crawford, CIO, Imperial War Museum

Fortunately, the foresight on planning future workspaces meant that Covid hadn’t had a very serious impact on IWM. The shift to support flexible work and ensure access to those who need the most onsite access is the same loss of revenue seen on most other heritage sites during the pandemic. Nevertheless, it meant that the digitization project could continue.

Of course, the space issue is at the forefront of Crawford’s perception, as the project is still underway and there is historical data that requires continuous safe storage. One of the biggest challenges was finding a product that could not only retain existing archives, but also control how long data was retained.

No one tells me how long I want to keep it, it’s just assumed that we can keep it forever

“When you digitize a film, you may make multiple copies before you put it in your DAM system, and you have to save it somewhere while you’re working … no one tells me how long you want to keep it. It doesn’t … It’s a kind of assumption that we can keep it forever … As a result, like many organizations, we are very rarely accessed on rotating disks. I ended up with a lot of data, but we need to keep it. [StorCycle] The product really aims for that.It gives us a plate form that I am convinced we can save [data] It’s a tape, so even if it’s cheap, you can access it when you need it. “

2021 tape?

Tape storage via cloud storage may not be ideal for anyone outside the heritage sector. However, there have always been certain criteria that IWM does not want or cannot compromise on. For volatile materials such as nitrate film that require refrigeration (currently using BFI), and large numbers of digitized materials that require backup, it is clear that multiple storage types need to be available. .. In addition, the file size is so large that it causes problems with online storage.

“It’s a cost, you know, the amount we’re talking about, and some of the data we have is also confidential. Many of it is Crown Copyright and we Is the admin of it, so we feel it’s a bit safe to take care of ourselves … I just invested in two new 4K scanners. Imagine the size of the file you’re currently generating. In 4K, the one-hour promotional documentary in the collection is over terabytes. Size … A few years ago, I made some attempts to put some large files in the cloud, and based on them. It took me forever to get it back. “

British munitions factory during World War I
British munitions factory during World War I.Credit: Imperial War Museum

The move to the full cloud may not be the way to go for the foreseeable future, but some items should always remain in their original historical format, but Crawford keeps the product improving over time. We are aware of this and are planning to keep up with development. ..

“StorCycle is obviously aimed at more production and business type storage that has everything people usually have, such as Word documents and Excel, but it’s very exotic related to the exhibition. There’s a lot of stuff. Many movies are static, but what DAM doesn’t have … We know that about 70% of the data on existing platforms will eventually move to StorCycle. “

A new WWII and Holocaust gallery is scheduled to open in September / October, with the hopeful start of a financial recovery that accompanies the well-managed growth of IWM’s exponential data. Being fully accessible at all times is more important than ever. .. Ultimately, Crawford points out that “the more digital you are, the more you can do with that data.”

Lifelong data: Imperial War Museum storage campaign

Source link Lifelong data: Imperial War Museum storage campaign

Related Articles

Back to top button