Simplified MPEG-4 Video Encoder Pipeline

Multimedia Systems Mini Project — A complete MPEG-4-inspired video codec implemented from scratch in Python, covering every stage from raw frames to a compressed binary bitstream and back.

---

Pipeline Overview

Input frames (.png / .jpg)
        │
        ▼
┌─────────────────────┐
│  1. Pre-processing  │  BGR → YCbCr  +  4:2:0 chroma subsampling
└────────┬────────────┘
         │
         ▼
┌─────────────────────┐
│  2. I-frame coding  │  8×8 DCT  →  Quantisation  (intra, no reference)
└────────┬────────────┘
         │
         ▼
┌─────────────────────┐
│  3. P-frame coding  │  Three-Step Search motion estimation
│                     │  Motion-compensated residual  →  DCT  →  Quantisation
└────────┬────────────┘
         │
         ▼
┌─────────────────────┐
│  4. Entropy coding  │  struct packing  +  bz2 compression  →  video.bin
└────────┬────────────┘
         │
         ▼
┌─────────────────────┐
│  5. Evaluation      │  PSNR · Compression ratio · Pipeline visualisation
└─────────────────────┘

---

Project Structure

.
├── mpeg_codec.py        # Core library — all codec logic (Parts 1–5)
├── run.py               # CLI: encode | decode | viz | sweep
├── viz.py               # Pipeline stage visualisation figure
├── extract_frames.py    # Extract frames from a video file (mp4 → png)
├── gen_test_frames.py   # Generate synthetic test frames (bouncing sprites)
│
├── real_frames/         # Input frames extracted from video.mp4
├── video.mp4            # Source video used for the real test
├── video.bin            # Compressed bitstream (synthetic frames)
├── video_real.bin       # Compressed bitstream (real video frames)
│
├── pipeline.png         # Stage visualisation — synthetic frames
├── pipeline_real.png    # Stage visualisation — real video frames
└── experiments.png      # QF sweep + GOP sweep plots

---

Installation

pip install numpy opencv-python matplotlib

Python 3.9+ required. No other dependencies.

---

Quick Start

Step 1 — Get your frames

Option A — extract from a video file:

python extract_frames.py
# Edit video_path and output_folder at the top of the script first.
# Outputs 12 frames resized to 128×96 in real_frames/

Option B — generate synthetic test frames:

python gen_test_frames.py -o sample_frames -n 12 --width 128 --height 96

Step 2 — Encode

python run.py encode real_frames -o video.bin --gop 8 --q 50 --search 8

Flag	Default	Description
--gop	8	Group of Pictures size (I-frame every N frames)
--q	50	Quality 1–100 (higher = better quality, larger file)
--search	8	Motion search window ±pixels
--no-chroma-subsample	off	Disable 4:2:0 subsampling

Step 3 — Decode

python run.py decode video.bin -o decoded/ --ref real_frames
# --ref is optional: enables PSNR computation against originals

Step 4 — Visualise the pipeline

python run.py viz real_frames video.bin -o pipeline.png

Step 5 — Run experimental sweeps

python run.py sweep real_frames -o experiments.png --gop 8 --q 50

Sweeps quality (Q=10→90) and GOP size (1→16) and saves a plot.

---

Results

Synthetic frames (128×96, 12 frames, Q=50, GOP=8)

Metric	Value
Raw size	~540 KB
Compressed	~8 KB
Compression ratio	~65×
I-frames	2
P-frames	10

Real video frames (128×96, 12 frames, Q=50, GOP=8)

Metric	Value
Raw size	~405 KB
Compressed	~7–10 KB
Compression ratio	~50–60×

Quality vs Compression ratio (GOP=8)

Higher quality → lower compression ratio (larger file). The codec achieves 185× compression at Q=10 and 25× at Q=90.

Pipeline visualisation (real frames)

Design Choices

Colour space — BT.601 YCbCr

Separates luma (Y) from chroma (Cb, Cr), enabling independent compression and chroma subsampling. This is the same standard used in MPEG-2, MPEG-4 Visual, and H.264.

4:2:0 chroma subsampling

Halves chroma resolution in both dimensions, reducing chroma data by 75% with minimal perceptible quality loss. Human vision has ~4× lower spatial acuity for colour than brightness.

JPEG quantisation matrices

Standard JPEG luma and chroma matrices scaled by a quality factor derived from the libjpeg formula. Coarser quantisation at high frequencies (less perceptible) and finer at low frequencies (most visible energy).

Three-Step Search (TSS) motion estimation

A fast O(log S) block-matching algorithm that visits 9 candidate positions per step and halves the step size each iteration. Much faster than full search while finding near-optimal motion vectors for most content.

16×16 macroblocks

Standard MPEG macroblock size — balances motion granularity against vector overhead. 8×8 DCT blocks inside each macroblock for the residual.

bz2 entropy coding

bzip2 (block-sorting + Huffman) provides strong general-purpose lossless compression after DCT quantisation produces sparse coefficient arrays. Simpler to implement than CABAC while still achieving substantial size reduction.

---

Module Reference

mpeg_codec.py

Function	Description
encode(frames, params)	Encode list of BGR frames → bytes
decode(blob, output_shape)	Decode bytes → list of BGR frames
make_qtables(quality)	Build luma + chroma quantisation tables
bgr_to_ycbcr(bgr)	BT.601 colour conversion
chroma_down(plane)	4:2:0 box-filter downsample
three_step_search(cur, ref, mb, S)	TSS motion estimation
motion_compensate(ref, vectors, mb)	Build prediction from motion vectors
psnr(a, b)	PSNR in dB between two frames
frame_breakdown(records)	Count I-frames and P-frames

Params named tuple

from mpeg_codec import Params
params = Params(
    gop=8,          # I-frame every 8 frames
    quality=50,     # 1 (worst) – 100 (best)
    block=8,        # DCT block size
    macroblock=16,  # Motion estimation macroblock size
    search=8,       # Search window ±pixels
    subsample=True, # Enable 4:2:0 chroma subsampling
)

---

Frame Requirements

• Format: .png or .jpg
• Size: width and height must be multiples of 16 (macroblock size)
  • Recommended: 128×96, 128×128, 256×144, 320×240
• extract_frames.py automatically resizes to 128×96

---

Authors

Group project — Multimedia Systems, 2026.