The issue of content moderation has been bubbling furiously beneath the surface of bodies politic throughout the Covid-19 pandemic and appears no closer to being resolved. This is relevant for operators, as well as platform providers, as they will be involved at least indirectly as implementers of impending legislation and technology, as so much of the content to be moderated is consumed on mobile devices.
The debate has been driven by concerns over content of various types, especially around harassment, bullying, child abuse, pornography, misinformation, self-harm and hate speech, also touching on combating of crime and terrorism. The perceived urgency led to a lot of legislation being cobbled together hurriedly, only to be followed by a kickback as impact on freedom of speech and privacy, in particular, became clear.
In Brazil, as one example, there was strong reaction from the human rights and Internet freedom lobby against a bill proposed just over a year ago on April 1 2020, entitled ‘Freedom, Responsibility and Transparency on the Internet’. A key paragraph in the objectors’ document stated that “measures restricting freedom of expression must comply with the principle of proportionality i.e. they must not unduly interfere with other rights of the persons targeted. In order to determine the proportionality of the restriction, it must be determined whether the sacrifice of freedom of expression that it entails is excessive or disproportionate to the advantages obtained through it.”
Similar sentiments have been expressed in other countries, as well as the EU (European Union). It is certainly possible to square this circle connecting freedom of speech with protection from abusive content at the legislative level, albeit not perhaps to the satisfaction of all parties. Yet the fundamental challenge of how to execute content moderation without infringing on individual privacy raises more intractable issues that are more logistical and technical.
These issues are rooted in the dichotomy between end-to-end encryption and access for content moderation. The main bone of contention is that many consumers have grown accustomed to secure end-to-end messaging via apps such as Facebook’s WhatsApp Messenger and Apple’s iMessage, through support for strong encryption (though this is only end-to-end within the Apple ecosystem for iMessage).
That has helped boost such services for sharing of rich media content including HD video as well as photos. Consumers felt secure posting content within friendship or user groups that they might not wish to share with a wider public, or be embarrassed to do so. This is where messaging started to intersect with social media sharing and raise the question of how content moderation could be enforced by the platforms themselves, or a legally sanctioned third party.
This issue has surfaced in various regions as part of that wider digital regulation debate that extends to curbing the big Internet platforms themselves, with both the EU and US agonizing over the issue. The Electronic Frontier Foundation (EFF), a major digital civil liberties defender, was quick to respond in March 2020 when the EARN IT Act was introduced by some US senators to tackle content moderation.
The EFF identified various ways in which the bill would be disastrous for free speech, privacy and security over the Internet. The main complaint was that all service providers and websites that supply content would be compelled to allow every message to be read by scanning software approved by the federal government, or else face crippling legal action. This was dubbed the Graham-Blumenthal attack on encryption, with reference to senators Lindsey Graham and Richard Blumenthal sponsoring the bill.
The situation was mirrored in Europe later in the year when the European Commission published its Digital Services Act (DSA) in December 2020. This came after several years agonizing over the contradictory requirements of privacy and content regulation among other things, which had resulted in several delays. The issue was perhaps even more contentious in Europe than elsewhere because of the strong advocacy for privacy protection as enshrined in the EU’s GDPR legislation enacted in May 2018, running up against increasing pressure to prohibit offensive and harmful content.
The Commission had to hurry up in the end to resolve divergence among member states that had been introducing their own legislation, impatient with the delays to DSA. Germany for example had its NetzDG, while the Avia Law countering online hate had been adopted France in May 2020.
From the mobile perspective, the DSA was welcomed with almost embarrassing obeisance by the GSMA with a statement whose sentiments were not shared by all the parties it claimed to represent.
“The European telecommunications industry welcomes today’s announcements by the European Commission,” it said in a statement, adding: “The Digital Services Act has the potential to restore citizens’ trust online and become a stepping stone to protect EU citizens by requiring that all service providers – regardless of their place of establishment – abide to certain standards if they want to offer their content, goods or services within the European Union. In this context, we welcome the Commission’s recognition that some digital actors have an increased responsibility to keep digital services free from illegal material and ensure that the rule of law is respected online with the same rigor as it is offline. We believe it is possible to achieve this goal while respecting Internet freedoms and protecting the basic principles enshrined in the eCommerce Directive.”
This was at best premature, and indeed the European Data Protection Supervisor, an independent authority set up to ensure that European institutions and bodies respect rights to privacy and data protection, expressed concern and responded with the statement: “Content moderation must be in line with the law, and profiling for content moderation purposes should be prohibited unless the online service provider can demonstrate that it is strictly necessary to address the systemic risks explicitly identified in the DSA. The EDPS also calls upon the legislators to further specify the information that must be provided to individuals, particularly when using automated means for content moderation.”
Meanwhile, a group of about 38 security experts within the Global Encryption Coalition (GEC) had already gone much further in criticizing not just the EU response but its grasp of the fundamental issues, after obtaining a leaked copy of a European Commission report on proposed technical measures to reconcile content moderation with privacy.
It was a damning inditement. The experts argued that the measures represented the worst of all worlds, actually placing users such as children at greater risk of harm, while breaking the line of trust established through end-to-end encryption. The main point was that by breaking end-to-end encryption to curb objectionable content online, one problem might be solved “by creating 1,000 more. Insecure communications make users more vulnerable to the crimes we collectively are trying to prevent”.
The complainants implied that the Commission had failed to employ anybody who had fully embraced the underlying issues.
There are various ways in which end-to-end encryption can be broken or interrupted for inspection of content, many of which were cited in that leaked Commission report. Content can be inspected before encryption in the user’s device, via client-side scanning where a piece of software scans message contents on the user’s phone, tablet or laptop and matches them against a database of prohibited content before allowing transmission. This scanning can similarly be performed after decryption at the server end, or by decrypting temporarily in transit by an authorized moderating third party. The latter involves key escrow, where a ‘middle box’ decrypts the data at a central server and then re-encrypts it for sending to the intended recipient.
All such measures come under the broad heading of back door access to encrypted communications, as opposed to direct access by parties specified by the user. There are also surrogate measures determined by surveillance not of the messages themselves but their sources and destinations, which has been employed to combat criminal and terrorist activity. Patterns emerge and users can be placed under suspicion on the basis of the people or places they regularly communicate with.
The methods differ in their implementation and technology but have in common the undermining of privacy that users had come to expect. The GEC contend that these back doors also increase risk of access by third party adversaries who might exploit them to perpetrate the very abuses the measures were meant to address by enabling access to the overall system.
This is in addition to the risks posed by governments themselves, whose exercise of control may amount to censorship in some eyes. The bottom lines then are firstly that it is impossible to create a back door for exploitation just by authorized parties without risk of breach, and secondly that the moment any authority can access a user’s content trust is broken.
In practice, all parties acknowledge that content moderation of some form must be enforced somehow, but there seems a dearth of ideas over how to reconcile the opposing camps of regulators and privacy or Internet freedom advocates. The idea of digital search warrants for law enforcement has been proposed or enacted, but that does not address spasmodic generation of offending content under protection of encryption. Most probably some form of client-based content moderation will be the least objectionable for users and freedom advocates, but still with questions over which party exercises control and jurisdiction.