Peacemaker — Multimodal Cyberbullying Detection System

A production-grade harassment detection system that analyzes text, images, and audio across social media platforms (Instagram, Telegram).

Tech Stack

• Text: DistilBERT fine-tuned on cyberbullying toxicity dataset
• Images: ViT (spatial features) + CLIP (image-text overlays)
• Audio: OpenAI Whisper for speech-to-text transcription
• Fusion: Late fusion — each model scores independently, merged at final layer
• Backend: FastAPI REST endpoints
• Frontend: index.html
• Deployment: Render
• Models: Hosted on Hugging Face Hub

Why ViT + CLIP Together

ViT alone cannot extract text embedded inside images — memes, screenshots with captions, image overlays. CLIP handles image-text relationships while ViT processes visual features. Using both gives complete visual understanding that neither model achieves alone.

Why Late Fusion

Each model (text, image, audio) produces its own confidence score independently. Scores are merged at the final layer. This means each model is independently replaceable without retraining the entire system.

Architecture

Text Input → DistilBERT → Text Score Image Input → ViT + CLIP → Image Score → Late Fusion → Final Score Audio Input → Whisper → Text → DistilBERT → Audio Score

Social Media Integration

• Instagram: Custom Safari binary cookie parser for authentication
• Telegram: Bot API integration

Installation

1. Clone the repository: git clone https://github.com/KishoharS/peacemaker.git

2. Set up virtual environment: python -m venv venv source venv/bin/activate

3. Install dependencies: pip install -r requirements.txt

4. Models are hosted on Hugging Face: KishoharS/peacemaker-bert KishoharS/peacemaker-vit

5. Run backend: python backend_api.py

6. Open index.html in browser