Machine learning algorithms are being put through their paces in more ways than we can fathom, gradually asserting more power in the digital entertainment landscape. One application which doesn’t spring immediately to mind or sound particularly glamorous is dubbing, where applying artificial intelligence software to the dubbing process can dramatically and easily increase global reach and slash costs for content and service providers.
For this reason, a press release from UK software firm Zoo Digital piqued our interest this week, announcing a grant from a UK Government-funded agency called Innovate UK, in partnership with the University of Sheffield. The project aims to build on “recent advances” in machine learning to streamline the dubbing process for TV and movie content. As we have become accustomed to when talking to any company claiming an AI edge, details are minimal and any elaboration on what specific advances have been made are a well-kept secret.
Nonetheless, the gist of the venture is to use machine learning to analyze the original and target language voices and provide feedback to improve the quality and speed of lip-sync dubbing. Crucially, the aim is not to replace the jobs of voice artists via automation, but to improve and diversify the dubbing process.
Thousands of budding AI start-ups out there today will either be acquired or go bankrupt, and eventually more and more responsibility will fall to the major internet giants and cloud titans. It therefore strikes us as strange that the likes of Google, IBM Watson, or the machine learning capabilities of AWS, are absent in the dubbing industry. IBM Watson last week launched a closed captioning service based on machine learning and has also used it to enhance metadata for search apps.
There has always been a market for dubbing, but the complexity and costs associated with the process means studios prefer subtitles. Yet not every country is fond of subtitles, particularly children and viewers with any impairment which makes reading difficult. With the help of machine learning, those costs could plummet. Zoo Digital cites that $1.4 billion was spent in EMEA on dubbing in 2016, expected to grow 10% annually, according to research from the Media Entertainment Services Alliance (MESA).
Zoo Digital claims the industry’s first cloud-based dubbing service, providing a platform for shifting responsibility away from technical audio engineers onto creative dubbing directors to deliver better performances from voice artists. Its ZOOdubs service allows voice artists to record live in the cloud anywhere, anytime, meaning materials no longer need pinging between studios – benefiting both voice artists and studios.
Combined with the best academic minds in machine learning from the University of Sheffield, plus an undisclosed funding grant, Zoo Digital is hoping to get a significant head start in a field where the major technology forces are yet to flex their muscles.
However, another reason the announcement stands out is, earlier this month, IBM launched Watson Captioning, applying AI and machine learning algorithms to automate the captioning process and increase accuracy over time. Customers can customize the product, to input a unique glossary of words and phrases for proper context and accuracy. IBM plans to update it with Watson-generated captions, delivering audio descriptions that can be edited and formatted in real-time. As far as we can see, Watson does not dabble in dubbing, but surely the natural language super computer has the clout to enter the dubbing market whenever it suits.
Zoo Digital CEO Stuart Green said, “With the recent growth in demand for dubbing, the entertainment industry faces a significant shortfall in talent used to create foreign language dubs. This research project will help pave the way for a substantial increase in capacity for high-quality dubbing. It will help us take another step towards making life easier for content creators looking to share their TV and movie content around the world. We’re delighted to be collaborating with leading academic research specialists. This is another opportunity to make an impact in the entertainment industry by rethinking how things can be done.”
Founded in the UK in 2006, starting out life providing DVD authorizing tools, Zoo Digital now has headquarters in Los Angeles and is seeing impressive growth, increasing revenues 63% in the first half of 2017 to $12.7 million. Its customer list boasts Disney, 20th Century Fox, Universal, Paramount, Warner Brothers, Sony Pictures, Netflix, Apple, BBC Worldwide, HBO, Lionsgate, Amazon Instant Video and many more.
Attracting the custom of the world’s largest media companies leaves us asking two questions, how long will Zoo Digital remain independent and why have these companies not developed copycat technologies in-house?