Building Multi-Modal Models for Content Moderation on Social Media

About

Social media platforms like Twitter, LinkedIn, and Facebook manage diverse content, including articles, messages, images, and videos. To ensure a safe and appropriate environment, these platforms utilize various methods to filter out content that violates their policies, focusing on dangerous, objectionable, and abusive material. Audio, a significant component of multimedia content, plays a critical role in identifying whether a video is spam or legitimate. Key audio elements like gunshots, explosions, screams, and hate speech, combined with corresponding video frames, can significantly improve the accuracy and precision of multimedia content moderation.

In this session, we will explore using deep neural networks, particularly Convolutional Neural Networks (CNNs), to extract relevant audio features from spectrograms and other audio representations. CNN-based models are adept at identifying local patterns and key audio elements essential for effective content moderation. We will investigate alternative machine learning models, including Support Vector Machines (SVMs) and Random Forests.

Key Takeaways:

  • Multimedia Classification Importance: Understanding the critical role of multimedia classification in maintaining safe social media environments.
  • Feature Extraction Techniques: Learning hand-crafted and CNN-based feature extraction methods to identify key audio elements.
  • Multimodal Modeling: Delving into techniques significantly enhancing classification performance by amalgamating different data types (audio, video, text).Hands-On Experience: Gaining practical experience through EDA, feature extraction, model training, and performance analysis.
  • Model Performance and Regularization: Analyzing various models' performance metrics and techniques to handle overfitting, ensuring robust content moderation systems.

Speaker

video thumbnail
Book Tickets
Stay informed about DHS 2025

Download agenda

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details