The Bigger Picture: Visual Archives and the Smithsonian
Category: Behind the Scenes
The Smithsonian Channel produces award-winning television programming that engages viewers much in the same way as the Smithsonian's museums and galleries do throughout the United States with their visitors. Just as the Smithsonian is working to digitize its collections for greater access and preservation, the Smithsonian Channel and the Smithsonian Institution Archives are also undertaking various efforts to ensure the digital preservation of these television programs.
The reformatting workflow for this project has been dynamic, and it should be. During earlier accessions of Smithsonian Channel programming, the progams were transferred on DVDs, numbering in the hundreds. In order to preserve the files digitally and prepare them for ingest into the Smithsonian's Digital Asset Management System (DAMS), the DVDs undertook a lengthy workflow process to ensure the highest level of playability and playback quality.
As part of the project's workflow, and best practices at the Archives, each disc is individually scanned using virus detection software. While this process is lengthy, it is critical to ensuring the security of the Archives' IT infrastructure. The next step in the workflow is to individually create .ISO images of each disc, which retains each program's DVD menu functionality. After creation of the .ISOs, the individual .vobs are extracted and converted to a single .vob using a command prompt script. This single .vob is then converted to an .mpeg, also using command prompt, to ensure the greatest playability across multiple software programs. This process is individually repeated for every DVD within the collection and can take months to complete.
After creation of the mpegs, the associated metadata must be created for each individual file in preparation for ingest into the DAMs. The metadata is applied to each file using Adobe Bridge; however, the metadata cannot be embedded into the actual video files, thus creating a sidecar .xmp file is necessary to hold the associated video file's metadata. Once this process is complete, the .ISO, .mpg, and .xmp files are entered simultaneously into the DAMs to ensure to proper parent (.iso)/child (.mpg and .xmp) relationships are maintained.
Throughout the entire workflow, upon initial receipt, after each conversion, and after upload to the DAMs, each file has been viewed for quality assurance, furthering adding time to an already lengthy workflow. In total, processing the collection of 136 DVDs within the accession took roughly 300 hours to complete.
In an effort to simplify the workflow, archivists from the Smithsonian Channel and the Archives met to develop a plan to achieve maximum efficiency with the preservation of Smithsonian Channel's programming. During the meeting, it was decided to test a pilot program wherein the Smithsonian Channel would send a number of .mov files through a secure server to the Archives to develop a new workflow based solely on the digital transfer of the Smithsonian Channel's programs. While not eliminating the original DVD transfer yet, this process significantly decreased the workflow and time involved in the entire preservation process.
With the transfer of the .mov files, the conversion process was removed entirely from the workflow. Further, the metadata can be directly embedded into the file header of the .mov files, eliminating the need to create a separate file for the metadata. For DAMs ingest, only the .mov file is needed, as opposed to the .ISO, .mpg, and the .xmp file. In essence, what used to take nearly 300 hours to complete could essentially be completed in as little as a day for a collection of programming.
By making the process of preserving the Smithsonian Channel programs simpler and easier, programs can be preserved more quickly and with less files to work with and a more straightforward workflow there is less likelihood for errors to be made. The collaborative effort between the Smithsonian Channel and the Archives is a prime example of two institutions working together in the effort of digital preservation.
- What are You Watching?, The Bigger Picture blog, Smithsonian Institution Archives
- And Action: The Ins and Outs of DVD Video Preservation, The Bigger Picture blog, Smithsonian Institution Archives
- Digital Video Preservation: Further Challenges for Preserving Digital Video and Beyond, The Bigger Picture blog, Smithsonian Institution Archives
- Smithsonian Channel records at the Smithsonian Institution Archives
On Monday, October 27th, four of our finest were available on the Smithsonian's Facebook page to answer questions about preserving your own archival collections. The four archivists at the Q&A have specialties in the preservation and organization of audio/visual material, photos, and digital records (email, digital video, etc.)
This is our fourth year hosting this event and the questions have evolved into increasingly digital landscapes. We also have an increased number of resources online to point you to which we embed in the answers below. If you don't see the question you have, feel free to reach out to us in our forums. We enjoy getting to hear from you through this fun event each year!
Q: Effie Kapsalis, Smithsonian Institution Archives (SIA) - To get things rolling, would any of the archivists like to chime in with their favorite preservation project?
A: Lynda Schmitz-Fuhrig, SIA - I find it rewarding to make digital materials accessible again. I really enjoy researching possible solutions when it seems digital files can no longer be used. I recently was asked by a colleague if I could retrieve audio off a CD that had been mishandled by the addition of masking tape on the media (any type of adhesive on optical media can damage the contents. Also avoid writing on CDs/DVDs). Previous attempts to copy the file were unsuccessful, but I was able to access the majority of it by using some special software. Just because something seems unreadable is not always the case.
A: Dave Walker, Center for Folklife and Cultural Heritage, Ralph Rinzler Archives - Some of the most exciting preservation projects we have worked on are the home recordings and impromptu performances captured of artists like Lee Hays and Pete Seeger. A recent tape we uncovered featured Weavers’ singer Lee Hays playing guitar and singing with a little boy he was babysitting. This candid snapshot of sound helps to paint a more complete picture of an artist and humanize him in ways the records he performed on might not.
A: Michael Pahn, Archive Center, National Museum of the American Indian - I love seeing people connect with the materials in our collections, and preservation is an important step in making those connections. If a document isn't stabilized enough to handle, we can't digitize it and put it online. If a motion picture film has shrunk and deteriorated, no one will be able to watch it. Preservation is key to providing access. The most rewarding part of my job is hearing from people, often tribal community members and tradition-bearers, who have deep interests in, and knowledge about, the materials in our collections. NMAI recently preserved a series of silent films from 1923, and were able to share the newly-restored footage with the community. It has been extremely rewarding to hear how exciting people are to see the films, and to learn so much more about them from the community.
Q: Do you have any best practices for storing digital file
A: Lynda - We preserve digital records (both digitized and born-digital). The practices we follow include the files being stored on servers that are backed up regularly. We also have our materials on two sets of LTO tapes. For someone dealing with their own personal digital archive, it is wise to have two copies of the files. Try to store one of the copies somewhere else as well. CDs/DVDs and USB thumb drives have their limitations but can work as a temporary measure. It is important to check the files regularly and migrate them to other media to avoid obsolescence. Hard drives can be a good option. Digital preservation is an ongoing task. There also are cloud-based services, but be sure to read the terms of service carefully. You need to know what they plan on doing with your files and their plan in case they go out of business.
Q: We have two fine art photographs that are showing signs of buckling in their frames. This occurred after relocated to humid southeastern climate from New Mexico. Paper does not seem to be straightening out - can anything be done to re-flatten these two images?
A: Effie - Thanks for your question! Here is a post with some great tips on flattening out archival materials. It is a fairly labor-intensive, elaborate process, so if you need help from an conservator, you can locate one here.
Q: What is the best way to capture and save AVCHD video footage? I've been using Adobe Prelude but am not sure the best codec to use for archival storage and later editing in Premiere Pro.
A: Lynda - Digital video is tricky and is there no consensus on what the best codecs for preservation are. Those creating digital video should go for the highest quality possible: uncompressed and lossless is desirable. Files that are too compressed are difficult to preserve. Audio tracks on the video should strive for a minimum of 48 kHz and a minimum of 720 by 486 pixels at 30 frames per second.
Q: What is the best way to store audio tape (temperature, containers. etc.) to maintain integrity of the tape itself? I have nine cassette tapes of my grandfather narrating his life story. I've transferred them to CD and mp3, but I don't know if the original exists, so I'm concerned about keeping the tape's integrity for as long as possible.
A: Michael - Audio cassettes have the benefit of being made of polyester, which is one of the most stable plastics around. The basics for making sure that your audio cassettes last are: keep them in cases, keep them out of direct sunlight, and keep them somewhere in your house where the temperature and humidity do not fluctuate much (so not in your attic or basement). Changes in temperature and humidity have a major impact on the long-term viability of magnetic media. For some additional information, take a look at the Library of Congress' collections care page for sound recordings.
A: Dave - Magnetic audio tape does best in cold dry environments, ideally between 50 and 60 degrees Fahrenheit with a consistent relative humidity between 30% and 40%. You can extend the life of the tapes by keeping them away from strong magnetic sources (motors, electrical fixtures, loudspeakers, vacuum cleaners…etc.) and by storing them vertically, like books. Because the materials and dyes used to create consumer tape boxes are unknown unless tested, it is recommended that the original housing be replaced with enclosed inert plastic polypropylene cans. If stored well, open reel tapes can have an expected shelf life between 35-50 years and between five to 15 for audio cassettes.
Q: I am the head archivist for my college radio station at SUNY Fredonia and we have tons and tons on reels. We are in the process of getting our reel-to-reel machine fixed and I haven't worked with a reel-to-reel before, let alone digitizing old reels. I need to know a few things: what should I be most careful with when digitizing the reels and should I practice using a reel to reel on a separate machine? Also, is there any other advice or direction you can give me? Thank you very much!
A: Michael - First of all, be certain to have the reel-to-reel machine fixed and checked out by a qualified technician. It's the first step in your signal path, and it's going to be in direct contact with your original recordings, so it's very important that it be of a high quality and in tip-top shape. The other important detail is to get your output and input levels set well, which is going to take attention with each tape you digitize. Having the input level set too high, for instance, can result in digital distortion which, besides just sounding bad, is lost data. There are some great, and highly technical, resources online for audio preservation, such as this from Indiana University. Another great resource is the Sustainable Heritage Network, which targets tribal archives, libraries, and museums, but has very useful information for everyone.
A: Dave - Open reel tapes are fairly fragile so you’ll want to make sure to handle them with care prior to playback so as to not scratch or otherwise deform the plastic tape. As Michael suggested, it would be a good idea to have any machines serviced and calibrated by a knowledgeable technician so that you’re able to capture the most amount of information stored on the tape when it’s time to digitize. Before digitizing, you’ll want to put together a digitization plan that outlines what the goals of digitizing the reels are. Definitely check out the Sound Directions: Best Practices for Audio Preservation for a full walkthrough of what goes into audio digitization at Harvard and at IU-Bloomington. Another tip: if you find tapes that shed grey or black particles when handling, it may have the dreaded “sticky shed syndrome” which affects polyester backed tapes and should not be played back until a specialist can treat the tape! Hope that helps and feel free to reach out off-line for more specific guidance at: email@example.com.
Q: Where can I find a good manual to start to work with films archives? I work in a museum in Chile and it's very hard to get some material. Thanks for this opportunities.
A: Michael, A great place to start is the National Film Preservation Foundation's "Film Preservation Guide." Another great online guide is Film Forever, which is sponsored by the Association of Moving Image Archivists. It's focused on home movies, but you should check it out.
Q: Can you recommend me a text or book on how to assemble a catalog of a cinematic file, and if you know some place where I can find good examples of such catalagos.
A: Michael - There are many, many different ways to catalog moving image materials, and there isn't a single correct approach. The Association of Moving Image Archivists (AMIA) put together a publication called the "AMIA Compendium of Moving Image Cataloging Practice." This document pulls together responses to a survey that AMIA sent out about moving image cataloging, and includes guidelines and catalog records from several different institutions, each of which does things differently than the others. The AMIA Compendium is available for purchase from the Society of American Archivists.
A: Dave - A good place to start might by the “The FIAF Cataloguing Rules For Film Archives “ which covers the recommended basics for arranging and describing archival moving image materials. A hosted version can be found here which also includes some examples that you might find useful.
Q: I already have an MLS from a few years ago but with a focus in public service. How can I now shape a career path to become an archivist?
A: Marguerite Roby, Smithsonian Institution Archives - Hi, check out this great blog post regarding archival career advice. It includes links to several other resources as well.
Q: I've got approximately 150 very old vinyl records taken from my great-grandmother's house post-Katrina that were completely submerged in mud for at least three days following the storm. I've been cautious in my attempts to restore them, fearing the dried mud and dirt particles would scratch and damage the modulated grooves inscribed on the vinyl. Any advice from you guys on the proper way to restore and hopefully salvage an awesome record collection is greatly appreciated!
A: Dave – The records may be recoverable, and we’re glad you've been cautious. Given the dates your great-grandmother's collection may cover, our worry is that some of them may not be vinyl but made of other materials that are more vulnerable to solvents including distilled water! It is first important to identify what you have and sort them based on type before attempting any recovery. Do take a look at the following links, including one written for #presweek and more that get into the nitty-gritty, literally. This article “Preservation Week: Caring for Your LP Records” is a great short overview. The Library of Congress has an excellent tips sheet on caring for audio recordings, scroll down for cleaning recommendations. Do note that is meant for general cleaning, when the type of record is known. Dried mud can be very tenacious, so it may take several sessions to clean a record, starting with a soft brush to loosen the bulk of dirt before wetting. (The brush can also help you remove dirt from the covers.) Do also note that solvents may affect the labels and their adhesives, so take care to work a small section at a time. For disaster recovery in general, do take a look at guides on our disaster page.
Q: I've been making field recordings of musicians performing live in Rhode Island for the past few years and have accumulated days of material; my dream has been to establish some sort of online archive for these recordings & everyone I've talked to has been enthusiastic about the idea but I've been struggling with questions of how to actualize it, i.e. as more than just a dumping ground for files. Would love recommendations of existing online collections (I of course am already familiar with the Lomax archives!) that present material in dynamic ways that highlight interrelation, especially incorporating hypertext, as well as any other suggestions for making such an audio archive as interactive as possible.
A: Lynda - A few links you might find interesting from the National Museum of American History and the Center for Folklife and Cultural Heritage.
Q: I'd like to know what digital tools you use to help tease out the text on difficult to read documents - such as faded documents, or documents where the writing from the backside is interfering, etc.
A: Effie - We have a great summary of some steps you can take to enhance legibility of documents.
Q: I work in college archive that is working to preserve student blogs. How would you recommend going about this especially if the blog is still active?
A: Lynda - If the blog is still active you could capture it at regular intervals. There are both open-source and commercial services that provide web crawling. The WARC format is an established archival container format for web crawls. The International Internet Preservation Consortium has some great information as well. And you can read about our web and social media archiving efforts here.
Q: Hi - we need to transfer DAT audio to hard drive - any advice on this?
A: Dave - The best way to transfer a DAT to a hard drive is by going digital-to-digital. On most DAT machines, the AES/EBU output provides a stereo digital output which can be captured 1:1 in a digital audio workstation. You will need an interface attached to the computer with AES/EBU inputs and will need to clean the DAT machine prior to playback to avoid any errors and dropouts.
A: Lynda - You can also read about our DAT project here.
Q: What sort of degree/education should I pursue to specialize in Audio Digitization? From what I've read, this field is still fairly new and there exactly a direct pathway.
A: Dave - Digitizing archival audio is an activity which stems from traditional archival practice but one that requires strong technical backgrounds in audio engineering, material science, and digital preservation. For someone interested in working specifically with analog audio media, a Bachelors of Science in Electrical Engineering or Audio Engineering would provide a solid foundation for understanding the principles of recorded sound and would prepare someone for a career in troubleshooting equipment, dealing with problematic media, and obtaining the highest quality transfers. Since the field is still so new, there is no direct pathway for a career in audio digitization but having ample experience recording onto magnetic tape and operating studio equipment is an absolute must. For those interested in archival work, there are a number of universities which offer Masters of Library Science degrees within which someone could specialize in any number of special formats such as moving image film, obsolete photography, digital records, and audio media.
Q: I have an old color photograph from 1956 that had been partially exposed to light while in a frame so that now half the image is mostly white while the rest is very washed out. What is the process to restore old photographs, can old photos actually be restored to close to their original state, and do photography shops do this or do I need to find a SPECIALIST who does this?
A: Marguerite - Unfortunately there is no chemical process that can restore faded colors from a photograph. Your best option would be to scan the image and digitally restore color in a program such as Photoshop. I don't know that you would need a specialist to accomplish this, but you would have to pick up some technical expertise. If that is not a viable option, you can search for photo restoration services online where you can submit your digital file for color servicing
Q: Hello - I had a question regarding CD storage. How does the SIA deal with a large volume of CDs? Do they stay in the original jewel cases are they moved to something slimmer/another method of storage? My collection is getting a bit too large for the space I have and am looking for ways to condense the storage space without jeopardizing the CDs (many are out-of-print classical CDs).
A: Lynda - Be sure to store your CDs standing up as opposed to flat. Tyvek enclosures can be used to save some space and are archival. If we receive a CD missing a container or has a problematic enclosure (cracked, dirty, etc), a Tyvek envelope is used.
Q: I would like to know your opinion regarding cloud-based storage. Is it reliable? Is it a good alternative to a hard drive?
A: Lynda - Cloud-based storage can be a good alternative depending on your needs. You do need to read the Terms of Service from the provider very carefully. You want to note what they will do with the files (can they post your pictures on their website without permission) and what is their plan in case they shut down. Do you feel comfortable letting someone hold your materials? What type of access do they provide (24/7)? Can you easily retrieve/remove your files?
Q: How would I preserve paper/cards from the 1930's through the 1960's? I also have some war ration stamps from WWII, how would I preserve those?
A: Lynda - It is usually best to keep papers organized in archival paper folders and boxes. According to the British Postal Museum and Archive, if it is coated on the back with adhesive known as gum, humidity can cause stickiness and dryness can cause brittleness. Each sheet of stamps should have its own folder.
Q: Also, is there a different way to preserve newspaper?
A: Effie - We happen to have a forum post on preserving newspapers from our conservator on this! Check it out!
Q: I work in an archive with no collection management policy and we have a massive backlog as well as a constant uncontrolled influx of new materials. How do you manage a massive collection? How to you process backlog while capturing new items when they come in?
A: Lynda - A collection management policy is important and a first step. A solid inventory of the backlog and documenting what new materials are coming in would help. Depending on the types of materials, minimal processing might work until a plan is in place.
A: Dave - When dealing with audiovisual material, we find it helpful to reduce backlogs by documenting the peripheral information written on the media housing as the collection gets processed. Without playing back each disc or tape, it is difficult to know exactly what is on the media so any handwritten information (track list, artist info, location, date, tape speed…etc.) can be useful for researchers as they make their way through the collection. This cursory documentation can help to prioritize items for digitization as well. Hope that helps!
Ah, Halloween, my favorite time of year. With an upcoming photograph symposium on my mind, and the season getting drier, I'm inspired to make my costume up as a tightly wound vintage panoramic photograph. The fear of unrolling one never gets old, if you'll forgive the pun, because one never knows . . . what one might find lurking within (Meaning revealing prior damage, not the content – unless some creepy creature has taken up residence). Recently, I saw one rolled panorama so damaged by a historic water incident that the faces of the sitters had lifted right off the front of the print and stuck to the back of the paper facing it – leaving an eerie ghostlike effect on both sides similar in effect to the image below.
Truly, here at conservation HQ nothing actually strikes fear into our hearts more than unintentionally bad advice gone horribly awry. We've written a few posts on our conservative conservator approach to opening and unrolling rolled photographs, trying to strike a balance between too little and too much information (i.e. just enough to not get non-conservators in too deep). Some other authors online have taken it upon themselves to offer some pretty scary advice, such as steaming the roll with a steam iron or soaking the rolled photograph and then unrolling it in water. These both send shivers down my spine, and if you want to know why in great detail, you may dress up like a conservation student and read all about Properties and Stability of Gelatin Layers in Photographic Materials and their infinite variety of susceptibilities. Not all photographs are the same, and to give you an idea of what could happen, I'll just give you the cheat sheet by suggesting you imagine taking a steam iron or a hot soak to your favorite Halloween wiggly gelatin dessert. Bad idea, unless you want your dessert and photo to potentially look like The Blob or The Creature from the Black Lagoon.
In contrast to the unintentially bad advice gone wrong, there are some fairly on point instructions to be found online to help people humidify and flatten panoramic photographs. I found that an independent archivist has posted a photo-illustrated adaptation for the home user of a process written up by the National Park Service for their Museum Management Program, which does not give us the willies due to the care built into their setup. That said, for attempting either of these I would increase the safety factor further by choosing not to have free water available that could splash on to the photo if their container was bumped, but instead pour the water first onto clean absorbent blotter or bleed-proof toweling until that is saturated. I would also avoid the use of wax paper during drying, substituting silicon coated paper without creases or wrinkles. (Always note: consulting a conservator is best, especially if the object is resistant to movement, or if there appears to be writing on it or prior damage to the object.)
We recently had another panorama in the lab, and we put it through the gentle humidification process we describe in our posts listed below. This occasion also allowed us to addess what isn't discussed in any of these posts, namely how to store an awkwardly long photograph, say over six feet, afterwards if you do not have the space to store it flat between an archival folder and boards. Generally this technique is used for oversize flexible objects, such as textiles and massive works on paper. Occasionally when we must, we adapt the technique for smaller objects and roll onto large (to scale) archival paper core supports. This prevents crushing and increases the circumference versus rolling them around themselves like little cigarettes. If only Halloween pranksters would re-roll after toilet papering some unfortunate person’s home! Here is a short video of how we do it if a work is too large for flat file storage.
Happy and safe Halloween, everybody! May you take many pictures and keep them safe from becoming ghosts of themselves.
- Panoramic Panic! A Sticky Situation, Part 1, The Bigger Picture blog, Smithsonian Institution Archives
- Panoramic Panic! A Sticky Situation, Part 2, The Bigger Picture blog, Smithsonian Institution Archives
- Halloween Humidification Horrors!, The Bigger Picture blog, Smithsonian Institution Archives
- A Health Resort for Paper, The Bigger Picture blog, Smithsonian Institution Archives
In celebration of Archives Month, join us Monday, October 27th, 10am to 4pm ET, where four of our archivists specializing in audio/visual material, photos, and digital records (or electronic records) will be on the Smithsonian's Facebook page to answer questions about your own archival collections. Questions from our readers in the past have ranged from storing letter and diaries, to digitizing cassette tapes, to organizing digital photo archives.
Here are the folks who will be on-hand to answer your questions:
Michael Pahn is Head Archivist at the Smithsonian Institution’s National Museum of the American Indian Archive Center located in the museum’s Cultural Resources Center. Michael began at NMAI in 2003 as its Media Archivist, and has overseen preservation projects funded by the National Film Preservation Foundation, Save America’s Treasures, and the Smithsonian Collections Care and Preservation Fund. His prior experiences include Save Our Sounds Project Librarian at the Smithsonian Center for Folklife and Cultural Heritage, and Librarian at The Nature Conservancy. Michael is a member of the Society of American Archivists’ Native American Archives Roundtable Steering Committee. He has a BA in Anthropology from the University of Pittsburgh and an MLS from the University of Maryland.
Marguerite Roby is the Photograph Archivist at Smithsonian Institution Archives and manages several large photographic collections. Her work involves establishing intellectual and physical control over these collections as well as contributing efforts towards digitization and the management of digitized assets.
Lynda Schmitz Fuhrig, Electronic Records Archivist at the Smithsonian Institution Archives since 2005, specializes in preserving born-digital materials that include images, audio, video, websites, and email from across the Smithsonian. Her work involves using tools and creating methods that help digital objects remain accessible in the future.
Dave Walker is an Audio Digitization Specialist at the Smithsonian Center for Folklife and Cultural Heritage and specializes in the preservation and digital reformatting of analog audio media, especially open-reel tapes and instantaneous discs. He also participates in several national groups focused on best practices and standards in these areas, and on increasing access to preserved recordings.
We hope that you’ll join us on Facebook tomorrow, and we look forward to your questions! If you don't have a Facebook account, feel free to send us an email. See other progams happening at the Smithsonian related to Archives Month.