MIT Video Productions Transcription and Summarization Pipeline

MIT Video Productions is the on-campus video production unit that video records faculty and department talks and symposia, Institute events and courses among a wide range of activities. Over its lifetime the unit has recorded over 45,000 tapes. Recently 17,000 tapes were selected for digitization (with the remaining tapes having been previously digitized or were initially recorded digitally). However, as a video production unit the metadata captured for each recording was limited to the date of recording, an event title, sometimes additional speaker or event information and some of the technical details. Most importantly, the contents of the recordings were not available for discovery.

With recent advances in machine learning and generative AI, we hypothesized that we could use contemporary tools to transcribe the video at a “good enough” level of accuracy to enable search within video contents and to automatically summarize the transcripts staff and archivists to use for discovery.