The shadowy world of commercial content moderation is captured in “The Cleaners,” to be aired on PBS on Nov. 12.
“The Cleaners,” a German documentary on the workforce of online content moderators was shown at UCLA on Nov. 6 as part of the UCLA Graduate Students Association’s “Melnitz Movies” series. Sarah T. Roberts, assistant professor of UCLA’s Department of Information Studies, who was a consultant on the film and featured onscreen, participated in a Q&A with filmmakers Hans Block and Moritz Riesewieck, led by David Tenenbaum, founder of The Veteran Narrative and CEO of Honor Media. The event was sponsored by the UCLA Graduate School of Education & Information Studies and the UCLA Graduate Students Association.
Anne Gilliland, associate dean of UCLA’s Department of Information Studies, welcomed the audience of 150 – on Election Night, no less – to the James Bridges Theater in Melnitz Hall.
“Sarah really is a pioneer of critical work on the internet,” said Gilliland. “As we solidify UCLA Information Studies as the place where this kind of dialogue is put to our community [and] the larger public, this film continues that dialogue.”
A 2013 video of child abuse – that was shared on Facebook with 13,000 views and 4,000 “likes” – caught the attention of Block and Riesewieck, who found Roberts’ unprecedented work on commercial content moderation (CCM), a term that she coined.
“Not everything that is uploaded on social media is [always] visible there,” noted Riesewieck. “We questioned this fact and …[why] some content is accepted and other [content] is not. Is it algorithms? We soon found out … it’s done by humans and you need a lot of humans to do this work.
“Sarah was actually the starting point for this film. I mean that literally, because it was Sarah’s academic groundbreaking work about this topic which made us aware that there are humans actually responsible for what is deleted and what is kept on social media.”
“The Cleaners” was mainly shot in Manila, where agencies employed by social media companies hire CCM workers, who view objectionable and emotionally harmful content for hours a day, receive low wages, and whose requests and pleas for psychological support are ignored by their employers.
Block said that their Manila location revealed, “a jungle of outsourcing companies… a dark city. We tried to get in contact with these guys – no one knows about them.”
Riesewieck said that they were surprised to learn the secretive nature of the CCM work environment, and the gestapo tactics used to keep its employees from sharing what they see while on the job – and how they are treated.
“We decided to collaborate with a local group of students and they helped us in getting in touch with the people who do this job,” he recalled. “They use private policing around the companies to make sure none of the workers talks to anybody,” said Riesewieck. “They control even their Facebook accounts of the workers. The reprisals are quite high. They are told they will end up in prison if they talk to journalists. They are told there are high fines in case they talk to anybody.
“This made it really difficult to get to the point where we now are with the film. We wanted to dig deeper. We wanted to understand what influences their decisions, what influences their gut feelings. In many cases – and I hope this became clear [in] the film – they can’t just follow the [moderation] guidelines. They really need to interpret what they have in front of them. They really need to understand the context. They need to make decisions in a very short amount of time. It was really important to understand their [ideology] … their religious feelings, what is their mindset. That is what we wanted to take a look at.”
Roberts addressed the question using automation for CCM, and said that the nature of the content requires the reasoning of the human brain. She pointed to the Philippines as one of the main spots for CCM, and the history of colonial labor that has manifested itself in this industry.
“Despite what you might hear from the firms themselves about their technological capabilities, the computational thorny problems of getting machines to recognize … make meaning, and then take action with decisions around images, especially video – and never mind live streaming video – is incredibly difficult. Text is the easiest to automate, and this goes back decades where you can just have banned words lists. Once you get beyond text… the analysis of images and videos is so hard to get a computer to do.
“Unless you are a top firm, the chances of you being able to devote the computational and engineering resources to solving the problem is just very low. So, what’s cheaper than a computer? A human being. Where is the human being cheapest? In the global south. There is certainly a long history [of that] in the manufacturing sector, in the textile industry, and so on. Unfortunately, these are well-worn paths.”
Roberts said that the presence of objectionable content on the internet speaks mainly to the profitability of it for social media platforms, to the detriment of society at large.
“It’s actually the monetization of material intended to incite, intended to offend, intended to shock, intended to create derision and division, because that actually drives engagement,” said Roberts. “And engagement, for better or for worse – good, happy, angry engagement; people being offended [or] harassed en masse – that has monetary value to the platforms.
“It is a very worrisome prospect that these commercial entities over which we have pretty much zero control have [avoided responsibility] … in terms of the curation of material like potential war crimes. We should all be very concerned that the go-to place for the archiving and preservation of that material is YouTube. Why? Because YouTube can summarily decide to remove it. And YouTube, by the way, is a company that acts as one would expect it to do, in its own ideological orientation in the first place. It’s a company that comes from Silicon Valley… and takes policy decisions that are in support of the U.S. government’s foreign policy.”
Roberts noted that the notion of individuals’ predisposition for objectionable behavior despite the presence of social media is only amplified by the opportunity to share that behavior or actions through the internet.
“I think some of the bigger issues in the film are the way that things like sexism, racism, violence towards vulnerable groups, violence towards women, the Rohingya situation, is not only replicated in these media but it’s circulated and it’s monetized, giving frightening impetus for people to behave in these ways that perhaps they’re already somehow oriented to doing – but that has great and different consequences. It can be lucrative for the firms. That is a fundamental point of the film and of my research and I don’t want to lose it because I think we would be remiss. In fact, I think on Election Night, we should be thinking about that as much as possible.”
Roberts said that the chance to entwine her research with a creative domain such as film provided, “…a way for many people who for whatever reason, wouldn’t come into contact with academic work or even when the work is in popular press. It also has the benefit of being incredibly emotionally stirring and moving. This is work I’ve done for eight years and I know these stories intimately, and I know how disturbing they are and how powerful they are. And yet, when I see those kinds of stories… captured in this medium, I find myself moved to tears because it’s so powerful.”
In 2017, Roberts convened an unprecedented meeting at UCLA of scholars, journalists, and experts with “All Things in Moderation.” She is regularly quoted and featured in international media, including BBC’s “Newsnight,” The Columbia Journalism Review, Fortune, and The New York Times. In April, Professor Roberts was named a 2018 Carnegie Fellow.
“The Cleaners,” which is Block and Riesewieck’s directorial debut, was nominated for the Grand Jury Prize for a World Cinema Documentary, and won the Prix Europa for the Best European TV Documentary of the Year. The film has also won the Gilda Vieira de Mello Award at the International Film Festival and Forum on Human Rights and Honorable Mention at It’s All True – International Documentary Film Festival.
“The Cleaners” will premiere on PBS’s “Independent Lens” on Monday, Nov. 12 at 10 p.m., PST.
Above: Sarah T. Roberts, assistant professor of information studies, took part in a discussion on “The Cleaners,” an award-winning documentary by filmmakers Hans Block and Moritz Riesewieck at a screening in the James Bridges Theater on Nov. 6. The Q & A was led by David Tenenbaum, founder of The Veteran Narrative and CEO of Honor Media.
L-R: Professor Roberts, Riesewieck, Block, and Tenenbaum