🚨 GuardianDrive: An AI-Enabled Smart Vehicle Emergency Response System

Tata Hackathon Project | Submission for Edge AI for ADAS and Autonomous Systems

---

📖 Overview

GuardianDrive is an intelligent in-vehicle emergency response system that uses real-time computer vision and AI to monitor driver health, detect critical conditions (drowsiness, distraction, unresponsiveness), and autonomously take over vehicle control to ensure passenger and public safety.

The system integrates seamlessly with the CARLA autonomous driving simulator and leverages MediaPipe, YOLOv8, and a custom multi-threaded decision engine to provide a complete emergency co-pilot solution.

---

🎯 Key Features

Feature	Description
🧠 Driver Drowsiness Detection	Real-time Eye Aspect Ratio (EAR) & blink monitoring using MediaPipe FaceMesh
😮 Yawn Detection	Mouth Aspect Ratio (MAR) analysis to detect fatigue-induced yawning
👀 Head Pose Estimation	6-DOF head pose via PnP solver — detects distraction (looking left, right, up, down)
📊 Fatigue Scoring	Composite fatigue index (0–100%) from eye closure, yawning, blink frequency & head movement
🚗 Autonomous Takeover	AI safely decelerates, applies hazard lights, and brings the vehicle to a stop
🚑 Emergency Dispatch	Automatically alerts EMS, Police, emergency contacts, and nearby vehicles (V2V)
🔊 Voice Alerts (TTS)	Real-time audio warnings to driver using pyttsx3
🚨 Ambulance Spawn	Spawns a Ford Ambulance actor in CARLA near the ego vehicle on emergency
🚶 V2V Crowd Response	Nearby simulated vehicles stop and pedestrians walk towards the emergency scene
🖥️ HUD Dashboard	Glassmorphism-style OpenCV dashboard with speedometer, ADAS status, GPS & alert logs

---

🏗️ System Architecture

SmartVehicleEmergencySystem/
│
├── main.py                    # Main orchestration loop & HUD rendering
│
├── ai/                        # Computer Vision AI Modules
│   ├── driver_monitor.py      # MediaPipe face mesh, EAR/MAR/head pose
│   ├── yolo_detector.py       # YOLOv8 environment object detection
│   ├── ear_detection.py       # Eye Aspect Ratio utilities
│   ├── eye_detection.py       # Eye landmark detection
│   └── face_mesh.py           # Face mesh wrapper
│
├── future/                    # Decision & Control Modules
│   ├── emergency_manager.py   # Emergency decision engine & protocol
│   ├── vehicle_controller.py  # CARLA vehicle control (autopilot, emergency stop)
│   ├── notification_system.py # Alert dispatcher (EMS, police, contacts)
│   └── dashboard.py           # Flask/web dashboard thread & shared state
│
├── templates/                 # Web dashboard HTML templates
├── yolov8n.pt                 # Pre-trained YOLOv8 nano model weights
├── camera_sensor.py           # CARLA camera sensor utilities
├── spawn_vehicle.py           # Vehicle spawning helper
├── preview_ui.py              # UI preview utilities
└── requirements.txt           # Python dependencies

---

🧪 How It Works

1. Driver Monitoring Pipeline

Webcam Feed → MediaPipe FaceMesh → EAR / MAR / Head Pose → Driver State Classification

Driver States:

• ATTENTIVE — Eyes open, looking forward
• BLINK — Short eye closure (< 300ms)
• DROWSY — Eyes closed ≥ 2 seconds
• UNRESPONSIVE — Eyes closed ≥ 4 seconds (triggers emergency)
• DISTRACTED — Head turned away > 3 seconds (triggers emergency)
• FAINTED — Head turned ≥ 5 seconds (triggers emergency)

Fatigue Score Formula:

Fatigue = (Eye Closure × 30%) + (Yawning × 30%) + (Blink Freq × 20%) + (Head Movement × 20%)

2. Emergency Protocol Sequence

UNRESPONSIVE/FAINTED Detected
    → 3× Progressive Voice Warnings (2s apart)
    → Emergency Takeover:
        [1] Voice alert + Siren (winsound)
        [2] Dispatch notifications (EMS, Police, Contacts, V2V)
        [3] CARLA vehicle: reduce speed → hazard lights → safe stop
        [4] Spawn Ambulance actor behind ego vehicle
        [5] Nearby V2V vehicles stop; pedestrians walk to scene

3. Environment Perception (YOLOv8)

• Third-person CARLA RGB camera feed → YOLOv8 nano model
• Detected classes: Cars, Pedestrians, Motorcycles, Traffic Lights, Road Signs, Bicycles, Heavy Vehicles
• Detection counts updated live in the HUD

---

🖥️ Dashboard HUD Layout

┌──────────────────────────────────────────────────────────────────────┐
│  LEFT PANEL (360px)  │  CENTER (1000px)         │  RIGHT PANEL (400px)│
│                      │                           │                     │
│  • Speedometer       │  CARLA Camera Feed        │  • Driver Monitor   │
│  • Throttle/Brake    │  (YOLOv8 Detections)      │  • Driver Status    │
│  • ADAS Status       │                           │  • Fatigue Level    │
│  • Alert Logs        │  ─────────────────────    │  • Emergency Info   │
│                      │  Bottom Panel (300px)     │  • Health Profile   │
│                      │  • Objects Summary        │                     │
│                      │  • System Alerts          │                     │
│                      │  • Comm Status  │ GPS     │                     │
└──────────────────────────────────────────────────────────────────────┘

Keyboard Controls:

Key	Action
ESC	Shutdown system
Q	Toggle Autopilot ON/OFF
W	Throttle forward
S	Reverse
A / D	Steer left / right
X	Brake
C	Center steering

---

⚙️ Prerequisites

System Requirements

• OS: Windows 10/11 (64-bit) (winsound dependency)
• Python: 3.8 – 3.10
• GPU: NVIDIA GPU recommended for YOLO inference
• Webcam: Required for driver monitoring

Software Requirements

• CARLA Simulator 0.9.13+ running on localhost:2000
• Python virtual environment (recommended)

---

🚀 Installation & Setup

Step 1 — Clone the Repository

git clone https://github.com/priyachaudhary4/SmartVehicleEmergencySystem.git
cd SmartVehicleEmergencySystem

Step 2 — Create Virtual Environment

python -m venv yolo_env
yolo_env\Scripts\activate

Step 3 — Install Dependencies

pip install carla
pip install opencv-python
pip install mediapipe
pip install ultralytics
pip install scipy
pip install numpy
pip install pyttsx3
pip install flask

Step 4 — Start CARLA Simulator

Launch CARLA before running the system:

# From your CARLA installation directory
CarlaUE4.exe -quality-level=Low -windowed -ResX=1280 -ResY=720

CARLA must be running on localhost:2000

Step 5 — Run the System

python main.py

---

📦 Key Dependencies

Package	Purpose
carla	CARLA Python API — simulator connection, vehicle control
opencv-python	Camera feed processing, HUD rendering
mediapipe	Face mesh landmarks for driver monitoring
ultralytics	YOLOv8 object detection
scipy	Euclidean distance for EAR/MAR calculation
numpy	Numerical array operations
pyttsx3	Text-to-speech voice alerts
winsound	System beep/siren alerts (Windows)
flask	Web dashboard backend
threading	Concurrent emergency protocol execution

---

🔧 Configuration

Key parameters can be tuned in ai/driver_monitor.py:

EAR_THRESHOLD = 0.20        # Eye Aspect Ratio — below this = eyes closed
MAR_THRESHOLD = 0.60        # Mouth Aspect Ratio — above this = yawning
# Timing thresholds in process_frame():
closed_duration >= 4.0      # → UNRESPONSIVE
closed_duration >= 2.0      # → DROWSY
distracted_duration >= 5.0  # → FAINTED
distracted_duration > 3.0   # → DISTRACTED

Traffic manager settings in main.py:

tm.vehicle_percentage_speed_difference(vehicle, 0.0)  # 100% speed limit
tm.distance_to_leading_vehicle(vehicle, 8.0)          # 8m following distance

---

🗺️ CARLA Map

The system loads Town05 by default:

world = client.load_world('Town05')

Town05 features a multi-lane highway layout suitable for emergency scenario simulation.

Simulation Setup:

• 40 background NPC vehicles spawned via Traffic Manager
• Ego vehicle: Tesla Model 3 blueprint
• Third-person RGB camera: x=-6.5, z=3.2, pitch=-15°
• Synchronous mode: fixed_delta_seconds = 0.05 (20 FPS)

---

🤝 Team

Built for the Tata Hackathon by Team GuardianDrive.

---

📄 License

This project is developed for hackathon purposes. All simulator assets belong to their respective owners (CARLA, Ultralytics YOLOv8).