Your browser is not supported. Please update it.

16 August 2018

Hollywood teams with Linux Foundation to fix open source

Formation of a new industry body always begs the question why it was needed now and did not exist before and that certainly applies to the newly formed Academy Software Foundation (ASWF).

It has been established by the Academy of Motion Picture Arts and Sciences (AMPAS) and the Linux Foundation to foster open source software development in the movie and video production businesses. It already has strong backing with premier founding members including Cisco, Google Cloud, Intel, Animal Logic, Blue Sky Studios, DreamWorks, Walt Disney Studios and Weta Digital, who are a combination of major content creators and video infrastructure vendors. The focus is on animation, audio and visual effects, for which 84% of studios already use open source software, according to the ASWF. But that alone would not be sufficient motive for forming this group given that other bodies including the Linux Foundation already address training, certification and licensing, supporting numerous projects to enhance open source software development.

However AMPAS members had run into a number of issues, some peculiar to media applications and some still shared by other sectors across the open source movement. There was a growing sense that various factors were holding back open source development across the video and media industry, one being the siloed nature of development teams. This resulted from the complexities of dealing with disparate licensing requirements for some aspects of the technology, complex interdependencies between multiple versions of the software, and the general burden of support.

It had long been clear that there is no such thing as free software because of the hidden costs associated with support and integration. That itself had been accepted so long as open source delivered on its four core promises of being free to use as desired, free to modify, free to distribute at source code level and finally free to distribute with modifications. These four together were designed to give the enterprise using the software total freedom from any vendor with no lock-in and the ability to contribute additions or improvements to the open source community.

But in order to deal with this complexity, media companies were being forced to establish rigid or siloed development teams that opposed the industry trend towards microservices and the DevOps model whereby software could be created in much smaller components allowing continuous feedback between developers and operations teams. Microservices themselves increase complexity because there are more interacting components and that in turn makes problems harder to trace and communications within development teams more challenging. In other words microservices amplified the very problems associated with open source.

Another factor is that Hollywood in particular has come to the conclusion that there was a common interest in collaborating over the underlying technologies behind advanced audio and visual effects propelling the whole industry forward, with competition among studios then focused on how those effects are applied within the content. This led to establishment of various informal collaborations early in 2016 which have gradually solidified and finally blended into the new ASWF.

Walt Disney Animation Studios’ CTO Nick Canon has been one of the driving forces behind the ASWF, emphasizing the key role he hopes it will play establishing common foundation technologies. Five specific objectives were outlined, the first being to provide a neutral forum for coordination of cross-project developments and sharing of resources across the whole industry. Secondly it is setting up an open continuous integration (CI) and build infrastructure to stimulate and manage reference components that would provide building blocks for cross platform development to get away from the siloed development. On this front the ASWF is leaning heavily on the Linux Foundation to exploit expertise already available from other industry sectors.

The third goal is to provide developers and organizations with a clear path towards participation and code contribution. Fourthly the aim is to provide tools to streamline development for build and runtime environments through sharing of open source build configurations, scripts and recipes. The final objective is to provide clearer and more consistent licensing through a shared licensing template.

This last aim was triggered by various unfortunate experiences across the whole of the movement, notably patent trolls which have sucked out a lot of money and turned some businesses away from open source software as a result of predatory litigation for infringement of spurious intellectual property (IP) rights. One infamous case was that of Patrick McHardy, former head of the core development team at Netfilter, who has made several million dollars conducting litigation around alleged copyright infringement associated with technology employed by the Linux kernel for packet filtering, network address translation and port translation. These are key functions for directing packets through a network and blocking them from sensitive locations.

This was a case of an enemy within since McHardy and Netfilter had been part of the Linux open source development community for some time. The key point was that Linux is licensed under the GNU (General Public License) version 2, this giving end users freedom to run, study, share and modify the software.

However it also carries some terms which had appeared to be of academic interest until inevitably mammon kicked in and some unscrupulous developers started taking actions against companies or even individuals that appeared to have violated GPLv2. That is what McHardy did and as a result was suspended from the Netfilter core project team.

ASWF is obviously eager to avoid such contamination of open source in its community and has already come up with some measures to protect against patent trolls and what it calls ‘bad actors’. It has yet to give details but says it has written various measures into its legal small print to prevent such predatory litigation and ensure that all users of and contributors to open source software are as immune as possible from prosecution. Here ASWF is drawing on lessons already learnt from these earlier cases within the open source movement.

The ASWF has given more detail about how it is going to address the complexities of software maintenance and upgrading in an open source environment where no single entity has overall control. The starting point here is the existing web hosted service called GitHub for distributed version control, tracking changes in computer files and coordinating work on these among multiple teams. This includes various tools for bug tracking, feature requests and task management. Github now has over 40 million users and is the largest host of source code in the world, with its strategic importance for software development substantiated in June 2018 by Microsoft’s move to acquire it for $7.5 billion.

Yet despite its success ASWF deemed Github inadequate for its needs. According to Rob Bredow, head of visual effects company Industrial Light & Magic, who is also part of the ASWF’s Science and Technology Council and chair of its Open Source Investigation Committee, more structure and control is needed than Github provides. ASWF will use Github, but is adding the ability to ensure that when changes are submitted the code is generated automatically so that it is guaranteed they will be incorporated in the next version. This is the sort of roll back or forward mechanism that is common practice within single software projects outside open source.

The idea is that core longstanding projects in the effects area will be transferred to ASWF. This is likely to include OpenEXR, a high dynamic range image file format for use in digital visual effects production originally released by Industrial Light & Magic in 2003. OpenEXR is used for high-quality image processing and storage applications, storing HDR pixel data as either 16-bit or 32-bit floating-point numbers. Even with 16 bits, significantly higher dynamic range can be represented than is supported by most image capture devices. OpenEXR employs both lossless compression for the highest quality and lossy compression where reduction of file size is more important.

OpenEXR is still being enhanced in line with advances in image capture and display technology and could be submitted to the ASWF to look after, which would then analyze it and ensure it had the right resources to support it properly. If accepted, OpenEXR would remain open source but with additional resources to streamline development and enforce the sort of disciplines more associated with proprietary software development.

This is the first time the AMPAS has given its name to an entity it does not have full control over, which is a sign the big studios recognize the need to become more flexible and innovative in their approach to content creation and the associated technology. They face competition not just from each other but also the big Internet players like Netflix, Amazon and Google, so the launch of ASWF can also be seen in that context.