Social Media Content Moderation: User-Moderator Collaboration and Perception Biases

Social media has emerged as a common platform for knowledge sharing and exchange in online communities. However, it has also become a hotbed for the diffusion of irregular content. Content moderation is crucial for maintaining a safe and healthy online environment by regulating the distribution of user-generated content (UGC).
Engaging users in content moderation fosters a sense of shared responsibility and empowers them to actively shape the environment of online communities. Leveraging the expertise of moderators leads to a deeper contextual understanding of content, thereby improving the overall consistency and legitimacy of content moderation in compliance with community or platform guidelines. Nevertheless, the collaborative effort of a more inclusive and community-driven moderation process remains unexplored by previous studies. While there is increasing attention to fairness, transparency, and ethics in content moderation, prior research often assesses content moderation perceptions of users, platforms, moderators, and bystanders in isolation. This results in a lack of comprehensive understanding of user perceptions in content moderation decision-making.
To address these limitations, this research proposes UMCollab, a user-moderator collaborative content moderation framework that incorporates the dynamics of user engagement and the domain knowledge of moderators into deep learning models to facilitate content moderation decision-making. Additionally, this research empirically investigates user perceptions of content moderation from the perspectives of content familiarity, content diversity, and user roles.
UMCollab leverages graph learning to model user engagement, which is further enhanced by the credibility and stance of users' online discussions. It also employs attention mechanisms to learn the domain knowledge of moderators based on their decisions regarding UGC per online community rules. Moreover, this study conducts an online user study by asking participants with diverse online engagement backgrounds and roles to complete a series of content moderation decision-making tasks and evaluate their perceptions of content moderation.
The findings of this dissertation research hold significant promise for promoting effectiveness, fairness, transparency, and community ownership in moderating UGC in social media, offering opportunities to improve the safety and success of online communities.