by Mike Casey, Director of Technical Operations, Audio/Video, Media Digitization and Preservation Initiative, Indiana University

The primary job of a digitization facility is, of course, to digitize. This is hands-on work that must be completed onsite. With the onset of the COVID-19 pandemic, our audio digitization engineers were facing weeks, if not months, of working from home. The prospect of developing work-at-home projects was beginning to seem a bit daunting when Tony Tadey, multimedia specialist at the Jacobs School of Music, reminded me of the Digital Audio Tape (DAT) transfer project from a couple of years ago that had ‘a few issues.’ These issues prompted the use of words such as unlistenable, glitchy, distorted, and ‘that awful screechy sound’ by staff and students listening to the files.

During the DAT transfer project, we learned that 30% or more of tapes held at IU Libraries Cook Music Library were either unusable or of limited use to researchers due to overwhelming digital errors that resulted in audio that can best be described using the colorful words in the paragraph above. It can be an extremely violent interruption of the intended audio. DAT has long been considered a problem format with degradation, machine alignment, and playback equipment obsolescence issues complicating the transfer to digital files.

We were saved by a decision made back in the 1990s when DAT was in heavy use, and another made five years ago.

Jacobs School staff were concerned about the longevity and stability of the DAT format from the beginning. To mitigate the risk of entrusting archival content to this format, many recitals and concerts were recorded onto two DAT tapes simultaneously. Some may have raised their eyebrows at this, but Jacobs School staff were prescient—and right.

At the beginning of the DAT transfer project in 2015, IU and its vendor, Sony Memnon, decided to use the NOA system which collects error rate, interpolation, and concealment data for each transfer. Nearly 9,000 DATs were transferred using NOA.

With two recordings of the same event, there exists the possibility that passages with errors in one may be fine in the other. With error information for all transfers, there also exists the possibility that we could predict which tapes have errors and their level of severity, rather than attempting the impossible task of listening to all 9,000 end-to-end.

So, QC specialist Glenn Hicks listened to 400 files, assigning each to a category representing the severity of its errors. From this, we were able to develop a metric that enabled us to predict the condition of the audio on each tape. Audio engineer Dan Figurelli then sampled the files and confirmed that, indeed, there were many occasions where good audio on one tape could replace bad audio on another.

To begin a project comparing files from two recordings and constructing a compilation file with the best of both, all we needed were the files, audio editing software, and audio editing chops.

Did I tell you about my four audio engineers who needed work that could be done at home? Bingo! A most worthwhile, not to mention substantial, work-at-home project was born. We can supply the files, the engineers have the editing chops, and they all have appropriate software in their home studios. (In fact, they all use Pro Tools at home.)

Productivity is something that we often prioritize within MDPI, but productivity is a lesser concern in this project. More important is the fact that our audio engineering staff is able to stay safe and healthy while also tackling work that moves our project forward. Many of these files are not usable in part or in toto. Every time we successfully construct a compilation file, we rescue content that becomes usable for research. Before COVID-19, no one had the resources available to work with the DAT content. COVID-19 has dealt all of us a very difficult hand. We are trying to make the best of it, saving audio file-by-file.

That is a victory!

(Above: A segment from a DAT with digital errors)

(Above: The same DAT after compilation)

(Above: Another segment from a DAT with digital errors)

(Above: The same DAT after compilation)

P.S. The title of this post references the fact that most of the content on these tapes is classical music. It also references this classic Saturday Night Live skit:

https://www.youtube.com/watch?v=fZLeaSWY37I

by Charlie Allen, Quality Control Specialist and Josh Brewer, Quality Control Specialist, Media Digitization and Preservation Initiative, Indiana University

In September 2019, Carmel Curtis, Josh Brewer, and Charlie Allen, IU staff members involved in the second phase of the Mass Digitization and Preservation Initiative, took a field trip to the Chicago suburb of Evanston, Illinois to check out a unique mobile audiovisual unit. This mobile audiovisual unit spent time in Baltimore, Maryland, in 2017 and Denver, Colorado, in November of 2018; New Orleans, Louisiana, is the next planned stop.

The Smith Fund Mobile Digitization Truck is the creation of the Smithsonian’s National Museum of African American History & Culture (NMAAHC) and is part of the Community Curation Program that is supported by the Robert Frederick Smith Explore Your Family History Center that seeks to preserve and share the African American history of various cities throughout the United States.

IU staff members spent two days observing the Smithsonian digitization team of archivists who helped to design and construct the mobile audiovisual unit. The archivists and consultants accepted various media formats, digitized them, and then returned the media along with USB drives containing digitized copies to their owners. Upon agreement with their clients, the Smithsonian will also keep a copy of the digital files and make them accessible online, enabling it to preserve vital and intimate collections that tell unique stories about African American communities and their migrations throughout the United States.

That gives you an overview of what went on, but the pictures tell the story better….

by Patrick Feaster, Media Preservation Specialist, Media Digitization and Preservation Initiative, Indiana University

Before you read any further, kindly stop and give the following piece of audio a listen.

Notice anything strange about it?

A little background: it’s one of the oldest known sound recordings made in Togo, captured on a phonograph cylinder by a German colonial administrator named Julius Smend around the year 1905.  The intensive, repetitive playback needed for transcription and study caused such soft wax cylinder recordings to wear out quickly under the heavy reproducers of the time, but Smend entrusted his recordings to the Berlin Phonogramm-Archiv, which had a strategy for overcoming this problem.  When wax cylinders came in from the field, a galvanoplastic process was used to form metal molds around them, destroying the originals but making it possible to cast as many duplicate copies afterwards as might be desired.  So that’s what happened to Smend’s Togolese recordings, which are available to us today only as they came out of the galvanoplastic molding process, and not as they went in.

Erich Moritz von Hornbostel—a pioneering ethnomusicologist who served as the Berlin Phonogramm-Archiv’s first director—later went through the various collections of field recordings in his care to choose representative examples for his “Demonstration Collection,” an anthology of world music he made available for sale to interested researchers and educators.  Several of the examples he picked were from Smend’s Togolese collection, including the one presented above: selection #78, identified as “chorus song with flutes and rattles, hunting song ‘tantana.’”  The Archives of Traditional Music at Indiana University holds two copies of it, one belonging to each of its two separate sets of the Demonstration Collection (purchased from Berlin by Henry Cowell and George Herzog respectively).

And both copies have the same, rather serious problem: when they’re played in the expected way, the audio runs backwards.  Lest you have any doubts about this, here’s the recording again, reversed to run the right way this time:

So what had gone awry?

Back in 1905, it would have been just about impossible for anyone to record or play an original cylinder backwards by mistake.  Standard phonograph motors were built to run in only one direction: forwards.  Meanwhile, the cylinders themselves were formed with a tapered internal bore having a larger internal circumference on one end than the other, so that when they were slipped onto a mandrel they could advance only so far before sticking snugly in place.  If someone had tried to put a cylinder onto the mandrel wrong-end first, either for recording or for playback, it wouldn’t have fit.  Cylinders could only be recorded in one direction, and afterwards they could only be played in one direction: spiraling from the end with the wider internal circumference towards the end with the narrower internal circumference.

But we’re not dealing here with an original cylinder; we’re dealing with a molded duplicate, and that makes all the difference.  What must have happened is that the original cylinder was positioned upside down when the metal mold was formed around it.  Later, when that mold was used to cast copies with the tapered bore running in the “right” direction, the resulting cylinders would all have come out backwards, with the groove spiraling from the narrower-circumference end to the wider-circumference end.  Standard phonographs at the time would only have been able to play these cylinders backwards.  And the original cylinder would have been destroyed in the process of making the mold, leaving no opportunity to check the copies against it.

Did you recognize that the recording was backwards when you first listened to it?  If so, congratulations on your phonographic savvy!  But if not, don’t feel bad: you’re in good company.

After all, Erich Moritz von Hornbostel himself must not have recognized the problem, since he chose to make this cylinder part of his Demonstration Collection as a representative specimen of Togolese music.  Nor did he stop there.  He also transcribed a segment of the music—backwards—and published his transcription in an academic article about the general characteristics of African music, accompanied by the following analysis (pp. 47-48):

An orchestra of [wind instruments] joins with the vocal chorus in the following quartet from North Togo (Kabure).  It is accompanied by rhythmic beats executed with rings which usually serve to stretch the bow; the beats are inaudible in the phonogram.  Here also the vocal part is the chief one.  The flute begins by moving parallel to it a fifth higher.  It is, however, far from being in harmony with it, it tuning being the result of non-musical factors, and, in fact, of mere chance.   But it follows the vocal part only half-way and even there not strictly, indulging in playful deviations; then, instead of going on to the second half it repeats the first one, thus transforming the tune into an ostinato.  The two (?) horns behave even more independently.  One more or less adapts its flourishes to the outline of the principal tune as far as is permitted by its restricted compass and its pitch, which is very discordant like that of the flute; the other confines itself to a drone on the tonic.  It is remarkable to what extent the instrumental idiom is reflected by the vocal melody; obviously the phrases in the third bar resembling a flourish of trumpets imitate the horns.

Of course, when Hornbostel writes that the flute “begins” by following the vocal part, but then departs from it in the “second half” by repeating the “first half,” he has the sequence of events turned around, just as the cylinder itself does.

Elsewhere in the same article, he expresses confidence in the phonograph’s ability to capture authentic sonic realities.  “As material for study, phonograms are immensely superior to notations of melodies taken down from direct hearing,” he writes, insisting that “only by means of the phonograph can we get the ‘real thing’” (p. 32).  Perhaps he had actually sensed something odd in the “tantana” recording, but if so, he might have chosen to ignore his misgivings as misleadingly ethnocentric.  When transcribing music from a cylinder, he claims, “a great amount of practice and constant revision are required to avoid mistaking what has really been sung or played for intervals and rhythms prevailing in our own music” (p. 33).  From this perspective, even if something in a field recording had sounded strange to him, he probably wouldn’t have questioned its truth.

The “tantana” cylinder poses an interesting dilemma when it comes to preparing files for preservation and access.  On one hand, if we think of the audio in terms of its documentation of Togolese music, then it ought surely to be presented in the direction in which it was originally recorded.  On the other hand, this cylinder has also had a long and significant history of being played in the other (reversed) direction.  That’s how Hornbostel listened to it while forming his views about African music, making his transcription, and deciding what to include in the Demonstration Collection.  And it’s also how Henry Cowell and George Herzog—as purchasers of copies of the Demonstration Collection—would have played it to their comparative musicology students.  So from the standpoint of twentieth-century scholarship, instruction, and publication, we might have good reason to present the audio in that direction.  Or it might be better yet to present it in both directions, leaving it to individual users to decide which one best suits their interests.  Meanwhile, our choice about which version(s) to make available—whatever it might be—should also be clearly and conspicuously spelled out, so that someone looking for one version of the audio wouldn’t unwittingly come away with the other.  And it’s a lot to have to explain!

Just another example of MDPI’s commitment to knowing our legacy media backwards and forwards.

By Darrell Myers, Post Production and Quality Control Manager – Film, Media Digitization and Preservation Initiative, Indiana University

IU Professor and Carmony Chair, Dr. Eric Sandweiss, recently partnered with the Indiana University Libraries Moving Image Archive and the Media Digitization and Preservation Initiative for his exhibit “800 Seasons: Change and Continuity in Bloomington, 1818-2018,” currently on display at the Mathers Museum of World Cultures on the IU-Bloomington campus.

“800 Seasons…” tells the story of Bloomington from the ground up. How do nature and history shape one another in the American Midwest—and how are Bloomington’s possibilities for tomorrow shaped by 200 years of community building in the southern Indiana hill country?

“We were looking for the widest variety of objects, images, and experiences possible to help people “feel” the ways in which Bloomington—and its environment–has persisted or changed over the last two centuries,” said Dr. Sandweiss. “It only made sense to consult IU’s great moving image archive.”

“Due to Moving Image Archive Director, Rachael Stoeltje, and Archivist Andy Uhrich’s deep knowledge of the Archive’s holdings, they were able to point Dr. Sandweiss towards a number of collections that fit in with his research needs,” said IULMIA Film Digitization Specialist Carmel Curtis.

Curtis coordinated access to digitized material, prioritizing others of interest for digitization through MDPI. Dr. Sandweiss then made several selections through IU Libraries’ Media Collections Online content management system.

“The staff of the Moving Image Archive were very helpful in listening to our goals and our interpretive needs,” said Sandweiss. “They suggested various clips from their collection and allowed me to browse the footage for ideas.  From those selections, I narrowed down according to the specific places where I could imagine moving images enhancing our story. From the start, I knew I wanted some kind of montage of scenes of life in and around Bloomington, and many of the available clips went into this opening presentation.”

Curtis then enlisted the help of MDPI Post Production and Quality Control Manager Darrell Myers to pull the films from the digital archive, digitally restore them, and create new clips. Dr. Sandweiss sat in for an editing session with Curtis and Myers at the Innovation Center to review and make suggestions. Mathers’ Manager of Exhibitions Matthew Sieber also provided technical expertise on how the content could be displayed in the museum space. The four kept communicating, making revisions as the project continued to take shape.

“It was easygoing and creative,” said Sandweiss. “The staff had many good ideas of their own, which I was pleased to incorporate; for my part, I was able to let them know what was most important, what fit my goals, and what was needed for the particular demands of exhibiting film in a gallery space.”

“It’s a unique service we can provide through MDPI,” said Myers. “We are all “in-house.” Films can be prioritized by IULMIA for delivery, then by our film scanning vendor Memnon, then by our Quality Control Staff, and lastly by myself in post. I can restore/edit, upload clips to IU Box for feedback, and repeat. The whole time we are collaboratively making it better.”

Four edited films are included in the exhibition. A compilation film, which depicts life in Bloomington and IU in a seasonal time space of a year, is projected near the entrance of the exhibit. “Home Begun,” from IULMIA’s Holland Collection depicts the building of a typical Midwestern family home, with text from the writings of Phillip Holland himself. Two Indiana University produced films are also on display. An excerpt from “Your Daughter at I.U.” depicts the life of  women attending college in the 1950s, and excerpts from “Food for Thought” depicts large-scale, industrial food production in the early 1980s.

Dr. Sandweiss kicked off the exhibit September 12^th with a one hour curator’s talk, “Circling the Square: The Nature of History in an Indiana Town.” He discussed the exhibition and how nature and history shape one another in the American Midwest.

“800 Seasons: Change and Continuity in Bloomington, 1818-2018,” will be on display at the Mathers Museum through July 31, 2020.

by Erica Titkemeyer: Co-Principal Investigator, Project Director

Since the late 1980s, staff at Wilson Library Special Collections within UNC Libraries at the University of North Carolina at Chapel Hill have worked diligently to migrate and digitize unique audiovisual recordings with the goal of long-term preservation and access. Beginning in 2014 and funded by a series of Andrew W. Mellon Foundation grants, grant staff began developing a central database that could collect and easily report on relevant data, including metadata that can aid in setting expectations for future digitization initiatives and storage growth. The database, Jitterbug, serves as the authoritative location for information about original analog items, preservation master files and transfers.

When considering digitization timelines, costs and digital storage needs, the two most useful data points are file duration and file size. Using MySQL queries, we are able to examine these data points by the item-level, by format, and by collection. Since we only began requiring these two fields in the last thee years of digitization, not all audio records report duration and file size. For this reason, the number of audio records examined vary between the two data points.

Finally, each record entered in our database represents a single file from an item. In most cases the number of files for an audio item correlates with the number of sides on that item (usually two sides for audiocassettes, audiotapes and audio discs), while video has a 1:1 ratio of item to file.

Table 1: Duration data by audio format

58,897 total records report audio duration

Table 2: File size (GB) data by audio format

30,996 total records report audio file size

In looking at both of these tables it is clear that file sizes and durations can vary wildly, making it difficult to predict digitization timelines, costs and storage needs. The longest open reel audiotape file, which runs three hours long (side two is equally long – 02:59:54) is a prime example of the unstandardized field recordings we can receive. These recordings are difficult to transfer with their inconsistent and slow speeds, but fortunately they do not make up the majority of our transfers.

Since we collect data at the file level, we could use Jitterbug to predict the percentage of double-sided recordings by format or even collection. This information could become helpful if we were to take on an addition to a collection and want to develop estimates for the total transfer hours or digital storage required based on what has already been digitized.

Table 3: Duration data by video format

Table 4: File size (GB) data by video format

Similar to audio, we have a wide range of durations and file sizes in digitized video recordings. While 6-hour long VHS tapes are relatively rare, they do appear in our collections. It would be interesting as we continue exploring our data to see if we can identify specific collections or types of collections that are more likely to contain long play (LP) or extended play (EP) tapes.

Graph 1: ½”, U-Matic and VHS file sizes v. durations

One final direction I considered when pulling data for this post was how we might be able to more accurately estimate file sizes based on video formats. Currently we assume 875MB per minute for our FFV1 encoded preservation files as a general rule, but that number can clearly change based on the quality of the analog original. After graphing all of the formats by their durations and file sizes, I saw that VHS data could stand in as an average, with a rising slope that cut through the center of all other format slopes (ignoring the LP and EP outliers).

The two formats that appeared to diverge the most from the VHS slope were ½” open reel video and U-Matic. Based on the ½” we have digitized thus far (all black and white), it’s clear that the resulting files are much smaller than the VHS average. As an example, one 30-minute ½” open reel videotape is 50% the size of a VHS tape of a similar duration. On the other end of the spectrum are our U-Matic tapes, which are more likely to produce larger file sizes. For example, one 30-minute U-Matic tape produced a file 25% larger than the same VHS mentioned above. Though we don’t always necessarily need to account for these minor differences, it could become helpful to have separate MB per minute estimates for formats that have proven over time to fall well below or above the average.

To date, we have digitization data for 29,173 audio items and 1,191 video items. While these records do not represent all content digitized, they do represent the bulk of work that has been completed in the last 3 years. As we continue to merge legacy data into the database, we can expand our dataset and continue to better predict costs and resources.

Grant Staff:
Erica Titkemeyer – Co-Principal Investigator, Project Director
Steve Weiss – Co-Principal Investigator, Curator (Southern Folklife Collection)
Anne Wells – Audiovisual Archivist
Sharon Luong – Applications Developer
Andrew Crook – Audiovisual Archives Assistant
Melanie Meents – Audiovisual Archives Assistant
Brian Paulson – Preservation Audio Engineer
Dan Hockstein – Preservation Audio Engineer
John Loy – Preservation Audio Engineer

by Rebecca Holte Manager, Audio and Moving Image Preservation, Digital Collections Services, New York Public Library

Special thanks to Mike Casey for inviting me to participate in this series, and to my NYPL colleagues: Genevieve Havemeyer-King, Media Preservation Coordinator (primary vendor liaison and specification and quality control manager), Ben Turkus, Assistant Manager – Audio and Moving Image Preservation (lab manager), and Nick Krabbenhoeft, Manager – Digital Preservation.

Introduction

The New York Public Library (NYPL), like IU Bloomington, Harvard University, and so much of our cultural heritage community, has recognized the risks inherent in its recorded audio and moving image collections. While our mass-digitization efforts really scaled up a couple of years after IU’s, we’ve made significant progress and are thrilled to finally be at a point where we’re able to share some cumulative data with the community.

To provide a little background, NYPL’s audio and video labs were born within performing arts curatorial divisions and were built up to fulfill both the preservation and access needs of those units. The labs were united under the Preservation Division roughly 10-15 years ago to serve all NYPL Research Centers (prioritizing special collections). Around the same time, digitization-to-file became a routine practice for audio, with video following a few years later.

In 2014, NYPL completed an audio and moving image (AMI) collection assessment funded by the Mellon Foundation, where we found that we held over 800,000 AMI recordings in the Research Centers, nearly 230,000 of which were a preservation priority. And as we’ve progressed with an item-level inventory and incorporated new acquisitions, that number has easily exceeded a quarter-million recordings. Our endeavor to ramp up our preservation efforts has been titled the NYPL Audio and Moving Image Initiative. By the end of 2019, we will have digitized more than 180,000 recordings, and mass digitization efforts work will continue for at least the next few years. The initial phase has focused on magnetic and optical media, with film digitization to begin in 2020 and grooved media (discs, cylinders) at a future date.

NYPL Collections

NYPL’s audio and moving image collections run the gamut from personal/private or experimental recordings, unreleased interviews and performances, performing arts rehearsals and productions, to radio or television broadcasts. To give just a few collection examples, some recent acquisitions that we’re really excited about include collections with material by Lou Reed, Sonny Rollins, and Fab 5 Freddy. NYPL also has a history of sponsoring original documentation, which includes the Louis Armstrong Jazz Oral History Project, dance original documentation and oral history projects, and recordings of Broadway, Off-Broadway, and regional theatre productions. A list of research divisions can be found at https://www.nypl.org/research-divisions/; filter by media to find divisions collecting “sound recordings” or “film & video”.

Collection materials can be accessed through three primary avenues: the Archives Portal, the Catalog, and Digital Collections. Archival and catalog records lead to Digital Collections, where you can stream recordings available globally or onsite at NYPL, depending on rights profiles. More and more recordings are added as we complete post-digitization processes, a multi-step effort that encompasses quality control, ingest to repository and streaming environments, archival/catalog processing, metadata conformance, and application of rights and permissions.

The data

The data described in this blog post is derived from a sample of 96,137 physical media items—73,208 audio and 22,929 video—across magnetic and optical formats. The data has been culled from sidecar JSON metadata files, though we also use MediaInfo directly on the files to cull/cross-check data in many scenarios.

What we’ve aggregated represents one information record per physical object.  We recognize that, in many cases, there could be multiple preservation master files per physical object; this is especially true of audio formats like compact cassettes or open reel audio. In our case, it’s also useful to recognize that a single item-level identifier may represent multiple physical objects, such as two compact cassettes stored within a single case or, in more confusing scenarios, audio and video media stored within the same case. At any rate, data sharing often comes with qualifiers.

Audio preservation masters presented here are WAV at 24 bits/96,000 Hz, but NYPL has since moved to lossless compression with FLAC. File format and codec for video preservation masters vary due to extraction method or specification. In 2017, NYPL began using lossless compression for video using FFV1/Matroska—both MOV and MKV are demonstrated in this data set. Current NYPL specifications can be found at https://github.com/NYPL/ami-specifications.

The variety of summaries or visualizations presented below highlights the difficulty in making generalizations, or even in using the typical format recording capacity for estimation purposes. (This may also be due to the types of collections we acquire, which demonstrate a variety of uses, few of which are standardized or broadcast.) Different analyses are more suitable in different situations, such as estimating vendor services, long-term digital storage, streaming services, usable hours of content, etc.

If we take averages at a high enough level—such as all audio objects versus all video objects—we’ve found that all-things-being-equal average durations are around the one hour mark. However, the minimums and maximums show a concerning range from zero minutes to over 30 hours! In terms of file sizes, the all-things-being-equal comparison is notably skewed for video, as we transitioned from uncompressed to lossless (FFV1) compression during the period in which this data was gathered.

We do still find high-level generalizations useful for broad planning purposes. Additionally, even at this bird’s eye view, the extreme ends of the spectrum have clued us into quality control issues that may have otherwise been missed. Why were some files listed as having no duration? Do we truly have recordings that are nearly 2,000 minutes long? (More than 30 hours?! What does that mean for the patron experience? Or for the cataloger?) We have since put checks in place to review outliers. For example, updates to our JSON metadata schema and MediaConch policies now help identify files with durations that are shorter than 10 seconds or longer than 10 hours (the shortest and longest expected durations, respectively).

“Simple” statistics (all formats in sample, duration and file size)

When we initially began looking at the specific formats, we kept it fairly simple and calculated straight averages and minimums/maximums. This representation begins to demonstrate the variances that we encounter across the formats, but it certainly serves as a better tool for estimating purposes.

Histograms for data distribution (select formats, duration)

A histogram is an even more precise way of reviewing data, in which you can see spikes where the most common durations occur as well as the outliers that may skew any averages or assumptions. In our experience, it’s definitely not a smooth bell curve distribution. We haven’t made any correlation between durations and specific collections, but it would certainly be interesting if we could make some additional predictions based on the nature of collection or creator (for example, interview versus rehearsal versus broadcast content, etc).

The below charts demonstrate duration distribution for formats where we have more than 1,000 items in our set: audio CD-R, compact cassette, DAT,quarter-inch open reel audio, Betacam SP, U-matic, and VHS. You can use your favorite program to automatically establish the frequency ranges, but to make these below charts easier to read, we’ve used 20 minute increments. Selecting your own frequency ranges also means that you can build your analysis to suit – for example, to reflect known manufacturing standards, or to reflect vendor pricing tiers, or to demonstrate how many media might take longer than one typical working day to transfer!

Conclusion, next steps, etc…

These summaries and visualizations represent only one use of our compiled data. At some point, we will look more closely at capture issues, or reasons digitization was not possible. It will be especially useful to look at any trends based on manufacturer stock (we have had an extremely bad run of Agfa tape) both to test validity of any previous assumptions and to pre-emptively flag media that may require additional labor. (Do we wish to send those to our in-house labs, or do we want someone else to spend the time on it? Do we think the collection items are important enough to warrant the additional resources?) We’re also debating what level of granularity is truly useful: do we include number of channels? Mono versus stereo? Side one / side two? Speed changes or extended play? In which cases would we also need to cross-reference the creator, type of collection, or stock in order to predict anything?

Stay tuned! This is really our first pass at data consolidation and sharing, and we hope to do more as our work progresses.

Media Digitization and Preservation Initiative
Dennis Cromwell, Executive Director
Adam Nickel, Processing and QC Specialist
Patrick Feaster, Media Preservation Specialist
Mike Casey, Director of Technical Operations, Audio/Video
Brian Wheeler, Senior Systems Engineer, IU Libraries

Over the past four years, Indiana University’s Media Digitization and Preservation Initiative has digitized more than 315,000 audio and video recordings. In the meantime, we’ve heard from many other folks who are planning digitization projects that predicting file size and recording length is difficult.  Of course, every collection is different, and nobody else will have exactly the same mix of holdings as we do at IU. Still, the general statistics we’ve gathered from the 315,000+ items we’ve reformatted to date may help provide others with at least some rough guidance for project planning.

The winner in the duration contest for video formats is VHS with an average running time of just about 100 minutes. This is not a surprise given that VHS is a fully consumer format. Consumers place higher value on the length of time available for recording than other characteristics, and so the format was developed with an emphasis on maximizing this variable. That’s why the EP (extended play) version of the format—that triples the possible recording time while reducing quality—was introduced. Over 3,000 of our VHS tapes are EP. One practical implication of this fact is that we must acquire and integrate consumer-level playback machines into our digitization signal chain, since professional decks do not support EP.

It may also be no surprise that the archenemy of VHS—Betamax—is close behind with an average duration of 94 minutes. Betamax is also a consumer format.

The more professional video formats – such as 1-inch, Betacam, and U-matic – tend to have shorter durations, in our experience.

On the audio side, Digital Audio Tape (DAT) is the clear winner with an average duration of 117 minutes. 120 minutes was a common length for the format. The majority of our DATs are recordings of concerts and recitals from the Jacobs School of Music. These events typically lasted on the order of two hours.

These durations, of course, impact the size of the digital files that are created. Tables 3 and 4 below list average file sizes.

In the tables above, the term ‘Package’ refers to the preservation package placed in long-term storage. It includes the preservation master file, all derivatives, metadata documents, and other items. In these tables, ‘Pres’ stands for preservation master file while ‘Prod’ is a production master file. Note that a mezzanine file (video) and a production master (audio) are the same in terms of role or function within the archive.

When planning for storage services, video is all important here, as file sizes are much larger than audio. For VHS, we expect to see an average preservation master file size of around 70 GB. MDPI uses the FFv1 format wrapped in Matroska  for video digitization. Other formats, such as uncompressed (using the v210 codec) or JPEG 2000, would yield different numbers.

The smallest average video preservation master size is 2.7 GB for the DVD format. The preservation master format in this case is an ISO disk image.

It’s worth bearing in mind that the averages shown above are mean averages, calculated as the total duration or size divided by the number of objects. This type of average may be the most useful one for large-scale project planning, but it may not provide the best insight into the “typical” example of a given format because it’s liable to be skewed by outliers at either end of the value range.  The median length was also evaluated for comparison. For most of the formats, the median and mean were fairly close (10%).   However, several video formats demonstrated larger differences.  For example, the 100-minute average for VHS reflects the fact that our holdings include a number of EP tapes with unusually long durations, as noted earlier.  In the case of VHS, the median duration turns out to be just 63.7 minutes.  So if you were going to bet on the duration of any one randomly chosen VHS tape at IU, your best guess would be 63.7 minutes.   You can see that the 7% of IU’s VHS tapes that were EP significantly skewed the numbers.

But the duration of playback isn’t the only variable that has a bearing on how much time and effort will be needed to complete a reformatting project.  Some carriers are more challenging and more time-consuming to deal with than others.

In the case of MDPI, Memnon Archiving Services, a Sony Company, transfers the bulk of our items in an onsite facility using (mostly, but not exclusively) parallel transfer workflows where one technician digitizes multiple recordings simultaneously. But some items aren’t a good fit for parallel transfers and instead need individualized attention. Take an open reel audio tape with speed changes, for example.  Field researchers will sometimes change recording speeds to conserve tape while working in remote areas.  The IU record is 34 speed changes on a single reel of tape!  Items that fail parallel transfer, or that are known from the start to be fragile or problematic, are sent to IU Media Digitization Studios, where staff can take the time to make one-on-one transfers. The table below provides a breakdown of the formats handled respectively by Memnon and IUMDS.

The next two tables show the percentage of recordings that were initially routed to a parallel transfer workflow but were then kicked out of it. In other words, these recordings required more than one straightforward attempt for digitization to be successful. Some intervention was required after the first digitization try. Typical interventions include tape repair, playback machine reconfiguration, stopping and starting playback any number of times to fix problems, baking more than one time, rehousing into a new shell, resolving disc tracking problems by hand, etc.

The relatively high failure rate of open reel audio tape can be attributed to the fact that our holdings in this format include large numbers of field recordings. Field collectors were more likely to change speeds and even track configurations within one tape than other users of the format.

The major causes of audiocassette failures include tapes breaking during transfer, transport issues where the tape will not move or play consistently, and tapes recorded at half speed or double the usual speed of 1.875 ips.

U-Matic video tapes are particularly challenging to digitize at this time due to both degradation and obsolescence issues. They have a high rate of mechanical and tape-related failures.

VHS failed for a number of reasons. Some changed speed in the middle—SP to EP or LP, for example, and sometimes back again. Others were unexpectedly PAL or SECAM. There were also a range of tape-related issues that required multiple cleaning passes or more than one baking session.

We will continue exploring our data in posts to come, one of which is tentatively titled “Life at the Extremes, or The Long and the Short of It.” We will also welcome guest posts from two other institutions in which they report on their digitization statistics. Will their findings be similar to ours or different? We don’t know yet and are very curious. Stay tuned!

Where are we?

We’ve been on hiatus, taking a summer break. But don’t worry, between lounging poolside, swimming in the ocean, and watching the sunset (not!), we have developed lots of interesting posts for the fall season. They are coming soon to an inbox near you. In the meantime, Mike Casey’s media preservation fairy tale is published in the latest issue of the Journal of the International Association of Sound and Audiovisual Archives (IASA Journal). It is available here:  http://journal.iasa-web.org/pubs/issue/view/IJ-50-08-2019/IJ-50-08-2019

Mike’s article on quality control may also be found in this issue.

When we start up again, we will lead with a post on statistics gleaned from our digitization project. Folks often ask us questions like “how large are your video preservation master files” or “what is the average duration of the average open reel tape,” and we mean to answer them. If you don’t know the duration (playback time) for a format that you hold, perhaps our data will get you in the ballpark for these critical preservation planning questions.

We will then broaden this inquiry by presenting basic statistics from another institution or two.

In the months to come, you will discover some heavy academic discourse as well as lighter explorations of our common work. For example, in one post we will answer the question “What do the hottest and coldest places on Earth have to do with media digitization?

Curious? So are we!

We will see you again on these pages soon.

by David Adamson, Audio Preservation Engineer/Quality Control Specialist, Media Digitization and Preservation Initiative, Indiana University

Audiocassettes encoded with Dolby noise reduction are notoriously difficult to digitize. Even if metadata makes clear the presence or absence of Dolby or the type used when it is present, it is often impossible to make it sound as intended during playback. This may be due to a number of factors, including years of degradation. In this post, MDPI audio preservation engineer David Adamson discusses MDPI’s strategy for accurate playback and long-term preservation.

—Mike Casey, Director of Technical Operations, Audio/Video, Media Digitization and Preservation Initiative, Indiana University

In November of 2018 MDPI began using a revised procedure for making preservation copies of Dolby-encoded audiocassettes. Up until that time, our attempts at Dolby-decoded transfers had been underwhelming (often with too much high frequency content missing) and A/B comparison seemed to suggest that in certain cases an undecoded transfer would be preferable to a Dolby decoded transfer. Was that really true? A combination of circumstances had to be deciphered, those of the conditions of the source material, and those of the engineers of the Dolby process itself.

Even before November, MDPI’s first step was to design a standard setup and workflow. This workflow was put together by MDPI audio engineers Melissa Widzinski and Dan Figurelli and director of technical operations Mike Casey. They created a Dolby workflow that used the Nakamichi NR-200, an external Dolby B/C noise reduction processor, for decoding. The earlier attempts at Dolby decoded transfers had relied on the internal Dolby circuits in the cassette decks. I was hired on, in part, for the purpose of integrating the new workflow and creating some good Dolby-decoded transfers. I began by working off of the MDPI audio staff’s workflow instructions. These instructions mentioned magnetic tape expert Richard Hess’ technique of using an “azimuth style” adjustment; to dial in the gain before the signal is sent to the Dolby circuit.  This proved to be a key element of creating a good Dolby transfer. I was also forwarded some email conversations between audio engineers with experience doing Dolby conversions for audiocassettes. In these emails it was expressed that the reason Dolby decoding can make something sound worse is because the Dolby encoding process tended to boost high frequencies and add compression that can actually sound pleasant in some cases, depending on the content. The emails went on to describe how Dolby as applied to audiocassettes was a simplified version of the Dolby technology (it was originally developed for open reel tape, with more detailed encoding for finer frequency bands). Dolby B in particular was designed to be less aggressive and even less noticeable if it happened to be played without decoding.

The Dolby picture came into even clearer focus when I reviewed the NR-200 manual. It includes graphs that illustrate when exactly the Dolby circuit is engaged, and confirmed that louder sections may not be affected at all. This reinforced the importance of selecting a correct gain setting on the Nakamichi processor, and suggested that a fixed setting on an internal Dolby circuit would not be ideal, particularly for digitizing large collections with disparate origins.

When working on our initial collection of Dolby tapes, I found that the best-sounding method for decoding was to use Richard Hess’ azimuth style adjustment to find the amount of gain that kept all intended high frequency information in the signal, while still removing a subtle layer of tape hiss. “Azimuth style” refers to making fine adjustments to the playback equipment to ensure that all of the original frequency information is captured in the transfer.To do this I would be physically raising and lowering the gain pot on the back of the NR-200 while riding the master level fader on the front to keep the overall level even in my headphones. While I do this I listen to the high frequency information in the audio signal. Increasing the gain causes the signal to hit the Dolby circuit’s threshold more frequently, and I’m looking for a “sweet spot” that mirrors how the original operator set their equipment. This first collection featured flutes as the main instruments; it was obvious when the NR-200 was removing too much high frequency information.

MDPI decided to create both a Dolby decoded preservation master-intermediate file and an undecoded preservation master, because when setting the gain a judgment call has to be made for each tape based on the sound quality of the content. Since we capture and preserve a corresponding “flat”, or undecoded, version of the tape, decisions like these can be revisited if need-be, further down the road. The two file types are created at the same time during one playback of the source cassette.

For me this process highlighted the necessity of continuing to tackle these specialty formats as soon as possible, and to make good use of the knowledge and perspectives of the audio preservation community to arrive at as accurate a transfer as possible.