OpenAI plans to use GPT-4 to build a content moderation system: Here's what we know

In a groundbreaking move, OpenAI has harnessed the power of its GPT-4 model to pioneer a new era of content moderation that promises scalability, consistency, and customisation. With content moderation being a persistent challenge for digital platforms, OpenAI’s innovative approach aims to streamline the process, but not without acknowledging the indispensable role of human involvement.

Content moderation has long been a complex issue, requiring the delicate balance of determining what content should be permissible on various online platforms. OpenAI‘s GPT-4 has emerged as a key player in addressing this challenge, offering the ability to not only make content moderation decisions but also contribute to the formulation and rapid iteration of policies. This could potentially reduce the time cycle for policy updates from months to mere hours.

OpenAI asserts that GPT-4 can decipher the intricacies of content policies, adapting instantaneously to any modifications. The result, according to the company, is a more consistent and accurate labeling of content, providing a positive vision for the future of digital platforms. According to Lilian Weng, Vik Goel, and Andrea Vallone of OpenAI, “AI can help filter online traffic according to platform-specific rules and ease the mental burden of a huge number of human moderators. 

The role of AI in alleviating the psychological strain on human moderators cannot be underestimated. The mental health impact of manually reviewing distressing content has been a concern, prompting Meta, among others, to compensate moderators for mental health issues stemming from reviewing graphic material. OpenAI’s implementation of AI intends to share the burden, offering AI-assisted tools to carry out approximately six months of work in a single day.

However, OpenAI is aware of the limitations of AI models. While many tech giants have already incorporated AI into their moderation processes, there have been instances of AI-driven content decisions going awry. The company acknowledges that GPT-4 is not infallible and that human oversight remains essential. The recognition of “undesired biases” and potential errors is a critical aspect that necessitates continued human review. Vallone from OpenAI’s policy team highlights the importance of keeping humans “in the loop” for validating and refining the output.

OpenAI’s approach is a step towards a more harmonious coexistence between AI and human moderators. By entrusting GPT-4 with routine aspects of content moderation, human moderators can focus their expertise on tackling complex edge cases that require nuanced understanding. The collaboration between AI and humans is envisioned to lead to more efficient and comprehensive content policies, reducing the risks associated with content-related pitfalls that other companies have faced.

— Nishtha Srivastava