Sarah T. Roberts: Symposium Brings Together Researchers and Industry to Discuss CCM

“All Things in Moderation” examined challenges and implications of commercial content moderation.

Commenting on an announcement made by Facebook in October that it would hire 1,000 employees to review ads and objectionable content, UCLA IS Professor Sarah T. Roberts, who examines CCM through the lens of social, political, legal, and cultural impact, said that tech firms like Facebook have only recently begun to release the numbers of their CCM employees, and that this change in the world of digital media signaled “an interesting cultural and political shift.”

David Kaye, U.N. Special Rapporteur on the right to freedom of expression and opinion, delivered the event’s keynote address.

“Researchers are often in a reactionary mode around this topic in its largest expression,” said Roberts during a roundtable discussion during the symposium, “All Things in Moderation: The People, Practices and Politics of Online Content Review – Human and Machine,” held at UCLA’s Charles E. Young Research Library, Dec. 6-7. Roberts, an assistant professor in UCLA’s Department of Information Studies, has pioneered the study of commercial content moderation – a phrase that she coined to describe the global corps of tech workers who monitor the internet for violent, disturbing, and other inappropriate content. This invisible workforce also happens to be underpaid, overworked, and poorly provided for in terms of mental and physical health care.

Roberts convened “All Things in Moderation” as an unprecedented forum to discuss CCM, delineate its challenges, and explore best practices and solutions. Participants from across academic, industry, and media fields explored the social, technical, legal, and policy aspects of content moderation, while delving into such issues as labor practices, human rights, and freedom of speech. David Kaye, the United Nations Special Rapporteur on the promotion and protection of the right to freedom of opinion and expression, delivered the keynote address.

Panel discussions centered on topics such as “User-Generated Moderation,” “Opacity, In/visibility and Activism,” Making Methodological Progress in Studying Content Regulation,” and “Managing a Diverse Distributed Moderation Workforce.” Speakers and panelists included Rasalyn Bowden, former content review training supervisor, Myspace; Nicholas Suzor, Queensland University of Technology; Emma Llanso, Center for Democracy & Technology; Mark Handel, Facebook; and Olivia Solon, The Guardian.

A plenary session on “(un)Covering Moderation, (un) Covering the Valley” was moderated by Safiya U. Noble, USC Annenberg School of Communication (far left) and Sarah T. Roberts, UCLA assistant professor of Information Studies (far right). Speakers were (L-R): David Ingram, Reuters; Olivia Solon, The Guardian; and Lauren Weber, The Wall Street Journal

Roberts’ work on CCM has been featured in numerous international publications and online media, including L’Humanitie, BBC4, The Atlantic, Rolling Stone, Quartz, and The New York Times. She said that while some tech firms are still resistant and even hostile to research, it has been interesting to watch as they decide from the perspectives of legality and public relations how to present their stances on CCM.

Professor Suzor, who chaired the discussion on “Making Methodological Progress in Studying Content Regulation,” noted that in order to understand the enormous scale of content moderation, researchers may need to investigate new techniques. He compared the rates of removal of removal of content that contained hate speech, sexual, or violent content, comparing the methods of YouTube, Twitter, Instagram, and several other online media purveyors.

“One of the core challenges that motivates me here is the problem of changing platforms,” said Suzor. “The YouTube of ten years ago is completely different from the YouTube of today. So, it’s really hard to be able to talk about moderation practices of any particular platform. We need to have longitudinal directing in order to understand how things are changing.”

UCLA doctoral student Morten Bay spoke on “How ‘black box’ responses to algorithmic censorship errors threaten the freedom of the press.”

“Our work is the study of a subject being a moving target … not in any way static but always evolving architecturally, policy-wise, and in every other way,” said Roberts during the discussion. “There are two dimensions to the study of this work. One is the outward-facing results of moderation and one is more internal about practices, techniques, tools. Here’s where a rupture between being an academic and following a socially relevant topic is difficult.”

To view the keynote by David Kaye of the U.N. and plenary sessions from All Things in Moderation, click here.

Click here for coverage of the event by The Atlantic.


Above: UCLA Assistant Professor of Information Studies Sarah T. Roberts (second from right) convened “All Things in Moderation” at UCLA, Dec. 6-7. The symposium featured discussions and panels on the social, political, legal, and cultural impact of commercial content moderation and its challenges.