Your browser is not supported. Please update it.

5 December 2019

Sky building monster in-house personalization machine

A content-heavy Video Exchange in London, forged from the ashes of TV Connect, was naturally a little light on the technical nitty-gritty Faultline is used to soaking up. A panel on data, AI and machine learning did however throw up some disruptive ideas – mainly concerning the fact that Sky is bringing an increasing amount of personalization technology in-house having only 8 months ago formed a new dedicated innovation department.

Sky’s Innovation Lead, Melanie Moeller, summarized the newly formed department as addressing two sectors. Firstly, Sky’s innovation hub is creating tools to accelerate activities in machine learning to drive personalization activities, and secondly working with teams across the Sky business, such as the red hot addressable technology AdSmart.

Moeller was joined on the panel by CEO of Dutch AI-based video recognition company Media Distillery, Roland Sars. Moeller gave the example of working with companies “like Media Distillery” but then stressed that Sky was not currently working with Media Distillery. When pressed by Faultline, Moeller declined to provide an example of a vendor working in the innovation department with Sky.

Much of her work has been on how to manipulate sensors in the home to improve the user experience, such as microphones for voice control, although she was reluctant to elaborate over which AI-based voice assistants Sky is sharing this end user data with.

We know that the next generation Sky Q set top uses TiVo voice search across linear TV and VoD, powered by TiVo’s knowledge graph engine, inherited from Veveo. Sky has added a dedicated voice search button onto the side of its Ruwido touch remote controls for making search requests, using TiVo’s natural language search technology.

TiVo says its knowledge graph can understand conversations and trends, by crawling the internet looking for entertainment, then mapping relationships, discussions and number of mentions using machine learning. We sent the name ThinkAnalytics in Moeller’s direction, as a long-term recommendation engine supplier to Sky but again our comments fell on deaf ears.

All in all, despite Sky’s intention to drum up excitement about its futuristic developments in data and machine learning to enhance the video experience, the main takeaway was how the operator appears to be relying less and less on third-party suppliers since its takeover by Comcast. Bringing an increasing amount of technology in-house may be cause for concern for some vendor suppliers, although Sky’s in-house innovations could one day empower a new wave of start-ups.

Meanwhile, Media Distillery played a short clip of its platform in action. Frankly, it looked like a copycat of AWS Rekognition, which Sky – among other tier 1s – have used for broadcasts of major events.

We have observed the prolific efficiency of Rekognition firsthand as we’re sure many of our readers have too, but admittedly this comparison is harsh. But as an example, Sky used AWS AI technology to identify celebrity guests as they arrived at last year’s Royal Wedding in the UK, to spin up information and facts about each attendee for viewers via the new Who’s Who Live app – available inside the existing Sky news app and on the company website. That said, we hope Media Distillery can defend its own slice of the market faced with another market being commoditized by the many disruptive arms of AWS.

Sars gave the example of a viewer changing channels after just 15 seconds, about the time it takes for Media Distillery’s content tagging technology to create a comprehensive picture of what’s being viewed. “If a viewer leaves after 15 seconds then we can safely assume they didn’t like what they were seeing. Say it was a car advert, we can collect this data and personalize content accordingly,” he said. Although Sars suggested that a rapid channel change might be based on the type of car, not cars in general, which the database takes into account.

We have come across Media Distillery on a couple of occasions. In August it was tapped by YouSee, part of Danish pay TV operator TDC, to assist on a project identifying the true start and end times for content. Media Distillery uses EPG Correction to automatically make adjustments to TV timings as they are broadcast.

Media Distillery’s software can extract levels of detail down to the scene or even frame level in real time. It can identify actors by facial recognition applied to the relevant part of a frame, as well as logos and objects such as buildings and subtitles. It can also extract individual words and phrases from the audio stream, which is almost as important for metadata creation as the video.

This software can perform “intra” searches within whole content items such as movies or TV shows, which not only increases the search power but can also extract the clips it has found. Metadata as currently deployed rarely if ever goes deep enough to extract content at the clip level.

There is however scope to go further by scavenging the web for incidental information beyond the content itself, such as how it is trending, what feedback it has received and what mood it has generated. This could help exploit social media responses to say a TV show just after it has been broadcast, taking advantage of scores like Nielsen’s Total Content Ratings in the US to track viewing past the week window and exploit the full larger audience for advertising.

Media Distillery has also collaborated with German multiscreen software outfit 3 Screen Solutions (3SS) to develop a pre-integrated AI platform promising “unprecedented TV service personalization.” The duo claim operators will benefit from a wide combination of machine learning techniques to identify, understand and index aspects of content – such as faces, speech, topics and logos – to enable more diverse and accurate search results. The offering combines the 3Ready front end and UI product from 3SS with Media Distillery’s AI technology.

Elsewhere, something ruffled our feathers this week in the form of a misleading headline from TelecomTV – a news site which hasn’t been clued up about the video market for years. It wrote off the personalization market as a dud, based on increasing consumer reluctance to hand over data and “several” dubious demos in recent years. Faultline alone has had hundreds of demos in recent years, each getting more powerful and impressive. Yes, the personalization industry is one of many in the throngs of commoditization, but wider data-driven opportunities are only just getting started, and if you still cannot comprehend that if a service is free, then data is the currency, then you are in the wrong industry.