Jigsaw, a unit within Google that forecasts and addresses emerging digital threats, today announced that Perspective, a free and open source API that uses machine learning to detect toxic comments, It processes 500 million requests daily and somehow reduces toxicity to improve online conversations at scale. The third issue of The Current, the company’s online research journal, published today, outlines the persistent and damaging problem of online toxicity and the role machine learning tools like Perspective can play in reducing it.
Launched in 2017 to combat online toxicity and bullying, Perspective scores comments based on their perceived impact on the conversation and helps platforms keep comment sections open to keep conversations engaging and productive.
“Internet toxicity is a pervasive problem that disproportionately affects marginalized groups, threatens independent journalism, and suppresses free speech and healthy dialogue,” said Jared Cohen, CEO of Jigsaw. “Our partners have come to realize that Perspective somewhat reduces the toxicity on their platforms, allowing them to create safer spaces for their users to exchange ideas. We are committed to working with our partners and academic institutions to continually train and re-educate our models.” with the goal of further improving toxicity identification, while minimizing bias and promoting healthier conversations.”
Jigsaw created Perspective using models developed in conjunction with Google’s anti-abuse tech team, and is now leveraging a variety of platforms to create a safe space for dialogue across multiple uses:
User Feedback: Perspective can provide feedback to users about potential perceived toxicity in their content in real time, generating significant results:
Coral by Vox Media, a major commenting platform used by media organizations around the world, uses this feature to give users the opportunity to change their wording before posting. A controlled study with McClatchy found that 40% of users changed their comments in response to that ad.
OpenWeb, a leading audience engagement platform that welcomes 100 million monthly active users, launched this feature after an OpenWeb study, conducted as part of a larger Jigsaw investigation, found that of all the users who comment and edit their language, 44.7% of these users replaced or removed the offending words, and 7.6% chose to completely rewrite their comment.
Helps human moderators: Perspective also serves to help human moderators work more efficiently and reduce toxicity on their platforms:
The New York Times created Moderator, a tool that leverages Perspective to prioritize comments for human moderation and authorize those most likely to survive manual review. The Times has increased the number of articles eligible for comment through Perspective, allowing it to offer readers a broader and more diverse selection of articles to comment on.
FACEIT, one of Europe’s largest gaming platforms, uses Perspective’s toxicity rating to warn or disable users, who can then turn to a human moderator for final review. As a result of this real-time moderation, FACEIT has achieved a 20% reduction in toxic messages.
Cofina, a Portuguese media company, managed to reduce the need for human moderation by 60%, saving their team at least 2 hours a day after integrating Perspective into their CMS to moderate commentary in their sports newspapers and financial.
“The Perspective API has greatly improved our ability to detect toxicity in the feedback we receive. It has made our moderation team’s work more impactful and efficient, and since implementing the tool we’ve been able to increase by about a third the amount of articles open to comments”. – Marcia Loughran, Communities Editor, The New York Times
“Working with Perspective has allowed us to foster positive interactions across hundreds of dynamic and engaged communities and tens of millions of users. The solutions are elegant and frictionless, supporting our vision of higher quality engagement with less toxicity.” – Nadav Shoval, CEO and co-founder of OpenWeb
“At Coral, we’re focused on simplifying and streamlining the essential work of moderators. Perspective helps us identify comments in multiple languages that require immediate attention. The Perspective team are fantastic partners in this mission.” –Andrew Losowsky, Coral Director by Vox Media
“We integrated Perspective into our CMS to moderate commentary in our sports and financial newspapers. We reduced human moderation by 60%. We saved at least two hours a day and are very pleased with the results. It also improved the quality of commentary in our websites and commitment to our users”. – Nuno Alves, Cofina Media Online Technical Director
“At FACEIT, we are on a mission to directly address toxicity and bullying in video games. As our gaming community continues to grow and evolve, we needed to create effective solutions that could work on a large scale. Perspective provided us with the toxicity model perfect that we use as a reference to develop our anti-toxicity engine”. –María Laura Scuri, Director of Business Intelligence at FACEIT
Through its partnerships, Jigsaw has also trained Perspective in new languages, allowing the machine learning model to be better positioned to improve online conversations on a global scale. It is currently available in English, Spanish, French, German, Italian and Portuguese. Some of Perspective’s partners include: El País, Le Monde, Cofina and other publishers who understand the importance of accepting high-quality conversations on their platforms and in multiple languages.
Jigsaw is also continually working with its partners to explore ways to mitigate bias in machine learning.
About Jigsaw
Jigsaw is a unit within Google that forecasts and confronts emerging threats, creating future-defining research and technology to make our world safer.
