Skip to content

armsp/smart_incident_management

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

41 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Smart Incident Management System - developed on 7.03.26 at the Claude Builders Hackathon, ETHZ

An AI-powered internal portal that transforms how engineering teams handle IT incidents. When a user reports an issue, an intelligent agent automatically triages it — classifying severity, identifying the affected repository and files using real git analysis (pydriller + gitpython), matching it against past github issues via semantic search, and generating a fully-formed change request proposal with code-level diffs, git-blame-derived developer assignments, and knowledge based approvers. All without ever leaving the portal.

This is based on real SDLC research based on Knowledge Maps, code churn and socio-technical congruence. We use pydriller to mine the git repository. Using semantic search we can find if a new incident has been seen before or not. As a first step the agent determines if a set of incidents warrant a new Change Request. Once a Change Request is created, the agent determines using real contextual data on real organizational repositories, which project or team would be best suited for handling the incident. It also finds which developer would be the best fit to be assigned to work on resolving the incident. It also identifies the files that may need to be changed based on historical commits and similar issues in the repository. Then based on metrics from pydriller on the main authors of those files, it finds where knowledge is concentrated and those authors become the approval chain. For severe incidents multiple approvers might be needed.

The system is smart enough to evolve over time. At a determined frequency - weekly or monthly - it analyzes all the incidents and change requests an this helps understand the brittle parts of the architecture and suggests improvements which the system could progressively refine such as:

  • Architecture
  • Backlog
  • Test Definitions

What Makes This Different

Most incident management tools stop at ticket tracking. This system goes from "something broke" to "here's exactly what to fix, who should review it, and why" — automatically.

Traditional Workflow This System
User files a ticket User files a ticket
Manual triage by on-call engineer Agent auto-classifies severity, category, impacted repo
Engineer searches codebase manually Agent identifies exact files via keyword maps + semantic issue matching
Someone guesses who should review Git blame analysis surfaces the actual code owners with line-level ownership data
PR created separately on GitHub Change Request generated in-portal with affected files, proposed changes, and approval tracking
Ad-hoc approval over Slack/email Policy-driven approvals — CRITICAL needs 2 reviewers including module owner; SECURITY adds a mandatory security reviewer

Architecture Overview

                          ┌──────────────────────────────────────────┐
                          │           Frontend (Next.js)             │
                          │  ┌──────────────┐ ┌───────────────────┐  │
                          │  │ Incident     │ │ Change Request    │  │
                          │  │ Reporting    │ │ Review Portal     │  │
                          │  │ (public)     │ │ (developer auth)  │  │
                          │  └──────┬───────┘ └────────┬──────────┘  │
                          └────────┼──────────────────┼──────────────┘
                                   │   REST API       │
                          ┌────────┼──────────────────┼──────────────┐
                          │        ▼                  ▼              │
                          │           FastAPI Backend                │
                          │                                          │
                          │  ┌─────────────────────────────────────┐ │
                          │  │        Triage Pipeline               │ │
                          │  │                                     │ │
                          │  │  1. Issue Classifier                │ │
                          │  │     ├─ Semantic Search (MiniLM)     │ │
                          │  │     └─ LLM Judgment (Claude Haiku)  │ │
                          │  │                                     │ │
                          │  │  2. Triage Agent                    │ │
                          │  │     ├─ Severity/Category (LLM/Rules)│ │
                          │  │     ├─ Repo Identification          │ │
                          │  │     ├─ File Matching (keyword maps) │ │
                          │  │     ├─ Git Blame Analysis            │ │
                          │  │     └─ Reviewer Selection            │ │
                          │  │                                     │ │
                          │  │  3. Change Request Generation       │ │
                          │  │     ├─ Target files + line ranges   │ │
                          │  │     ├─ Approval policy computation  │ │
                          │  │     └─ Audit log recording          │ │
                          │  └─────────────────────────────────────┘ │
                          │                                          │
                          │  ┌─────────────┐  ┌──────────────────┐   │
                          │  │   SQLite     │  │  3 Real OSS     │   │
                          │  │   (async)    │  │  Repos (git     │   │
                          │  │             │  │  submodules)     │   │
                          │  └─────────────┘  └──────────────────┘   │
                          └──────────────────────────────────────────┘

Key Features

1. Multi-Stage Intelligent Triage Pipeline

Every incident goes through a sophisticated multi-stage pipeline:

Stage 1 — Issue Deduplication (Semantic Search + LLM)

  • Embeds the incident using all-MiniLM-L6-v2 sentence-transformers
  • Searches against GitHub issues - for a real world demo we provide indexed issues from Flask, Celery, and Werkzeug
  • Top-k candidates are enriched with full issue bodies and sent to Claude Haiku for root-cause reasoning
  • Claude decides: LINK_TO_EXISTING (duplicate of a known issue) or CREATE_NEW (novel problem)
  • Falls back to cosine similarity threshold (0.65) when API is unavailable

Stage 2 — Classification & Repo Identification

  • LLM path: Claude Haiku classifies severity (LOW → CRITICAL) and category (BUG, SECURITY, PERFORMANCE, OUTAGE)
  • Rule-based fallback: Keyword matching against curated severity/category dictionaries — works fully offline
  • Identifies the target repository by scoring incident text against repo descriptions and keyword maps

Stage 3 — File-Level Analysis

  • Cross-references matched GitHub issues with their linked commit SHAs to find historically relevant files
  • Falls back to keyword-map-based file matching when no issue links exist
  • Runs git blame on the top matched files to extract per-author line ownership
  • Selects reviewers based on code ownership percentage and recency of commits

Stage 4 — Change Request Generation

  • Auto-creates a change request in SQLite with target files, risk level, and approval requirements
  • Applies severity-based approval policies (CRITICAL/HIGH = 2 approvals, MEDIUM/LOW = 1)
  • SECURITY-category incidents automatically add an extra security reviewer requirement
  • Records full audit trail of every action

2. Real Codebase Analysis (Not Mocked)

The system analyzes real open-source repositories included as git submodules:

Repository Team What the Agent Analyzes
Flask (pallets/flask) Web Platform Team Sessions, routing, templating, config, blueprints
Celery (celery/celery) Infrastructure Team Task workers, result backends, beat scheduler, retry logic
Werkzeug (pallets/werkzeug) Core Libraries Team Security utils, routing, multipart parsing, debugger, HTTP

The RepoAnalyzer service provides:

  • Commit history traversal via pydriller (configurable 180-day window, up to 5000 commits/repo)
  • Fix-commit detection — regex matching against commit messages for "fix", "bug", "hotfix", "patch", "CVE", etc.
  • Git blame — per-author line ownership with percentage, last commit date, and commit hash
  • File churn analysis — commit count, fix ratio, contributor count, and a composite brittleness score
  • Contributor stats — commit counts, files touched, fix commit ratio per developer

3. GitHub Issue Integration

  • SQLite storage — full issue metadata including title, body, labels, linked commits, linked PRs
  • Semantic embedding indexall-MiniLM-L6-v2 encodes issue text (title + labels + body), stored as NumPy matrix with JSON metadata
  • Bidirectional commit↔issue linking — from GitHub timeline data AND by parsing #NNN references in local commit messages
  • Query-time scoring — keyword match count + boost for issues with linked commits + boost for bug/security labels

4. PR-Like Change Request Review Portal

The /changes page provides a GitHub-PR-like review experience entirely within the portal:

  • Developer authentication — sign in as a registered developer (profiles mined from git history)
  • Affected files view — shows file paths, line ranges, current code, proposed changes, and reasons
  • Reviewer cards — git-blame-derived reviewers with line ownership stats and commit dates
  • Approval tracking — real-time approval count vs. required, with approve / request changes / reject actions
  • Approval history — timestamped log of every reviewer decision with comments
  • Policy enforcement — the system won't transition to APPROVED until the required number of approvals is met
  • Filterable list — filter change requests by status and risk level

5. Configurable Approval Policies

CRITICAL severity  → risk_level HIGH   → 2 approvals required, must include module owner
HIGH severity      → risk_level HIGH   → 2 approvals required
MEDIUM severity    → risk_level MEDIUM → 1 approval required
LOW severity       → risk_level LOW    → 1 approval required
SECURITY category  → always adds +1 security reviewer requirement

Tech Stack

Layer Technology Why
Frontend Next.js 15 (App Router) + shadcn/ui + Tailwind CSS Fastest path to a polished, accessible UI with file-based routing
Backend FastAPI (Python 3.11+) Async-native, auto-generated OpenAPI docs, Pydantic validation
Database SQLite via SQLAlchemy (async + aiosqlite) Zero setup, single file, plenty fast for the use case
Repo Analysis pydriller + gitpython Pydriller for commit traversal, gitpython for blame — no external API needed
Semantic Search sentence-transformers (all-MiniLM-L6-v2) + NumPy Fast local embedding + cosine similarity via dot product
LLM Intelligence Anthropic Claude Haiku 4.5 Incident classification, issue deduplication reasoning, structured JSON output
Incident Storage JSONL (append-only log) Simple, human-readable, git-diffable incident log

Quick Start

1. Clone with submodules

git clone --recurse-submodules git@github.com:armsp/smart_incident_management.git
cd smart_incident_management

# If you already cloned without submodules:
git submodule update --init --recursive

2. Environment variables

Create a .env file in the project root:

ANTHROPIC_API_KEY=sk-ant-...       # optional — enables LLM triage, falls back to rule-based without it
REPOS_BASE_PATH=/path/to/repos     # absolute path to the repos/ directory

3. Fetch and index GitHub issues (one-time setup)

cd backend
pip install -r requirements.txt

# Fetch closed issues from GitHub (optional GITHUB_TOKEN for higher rate limits)
python scripts/fetch_issues.py

# The semantic index is built automatically on first startup

4. Backend

cd backend
uvicorn main:app --reload --port 8000

The API will be available at http://localhost:8000 (interactive docs at http://localhost:8000/docs).

5. Frontend

cd frontend
npm install
npm run dev

The UI will be available at http://localhost:3000.


Pages

Route Access Description
/ Public Report incidents & view incident list with real-time agent analysis results
/changes Developers only Change request review portal with file diffs, approval tracking, and reviewer management

The /changes page requires signing in as a registered developer (from data/developers.json).


Project Structure

├── backend/
│   ├── main.py                    # FastAPI app — incident pipeline, CORS, lifespan
│   ├── config.py                  # Repo registry, keyword maps, approval policies, API keys
│   ├── database.py                # Async SQLAlchemy + SQLite
│   │
│   ├── models/                    # SQLAlchemy ORM models
│   │   ├── incident.py            # Incident tracking
│   │   ├── change_request.py      # Change requests with target files, blame, reviewers
│   │   ├── approval.py            # Per-reviewer approval decisions
│   │   ├── audit_log.py           # Full audit trail
│   │   └── github_issue.py        # Cached GitHub issues with search text
│   │
│   ├── schemas/                   # Pydantic request/response schemas
│   │   ├── incident.py
│   │   └── change_request.py
│   │
│   ├── routers/                   # API route handlers
│   │   ├── incidents.py           # CRUD + state transitions
│   │   ├── change_requests.py     # CRUD + filtering
│   │   ├── approvals.py           # Approve/reject/request-changes
│   │   ├── analysis.py            # On-demand repo analysis
│   │   ├── trends.py              # Trend reports + health scores
│   │   └── audit.py               # Audit log queries
│   │
│   ├── services/                  # Business logic layer
│   │   ├── triage_agent.py        # Multi-stage triage (LLM + rules + git analysis)
│   │   ├── repo_analyzer.py       # Pydriller/gitpython analysis engine
│   │   ├── issue_linker.py        # Bidirectional commit↔issue linking
│   │   ├── issue_classifier.py    # Semantic + LLM deduplication
│   │   └── semantic_issue_search.py # MiniLM embeddings + cosine similarity
│   │
│   ├── policies/                  # Configurable policy definitions
│   ├── scripts/
│   │   ├── fetch_issues.py        # GitHub issue fetcher (pre-demo setup)
│   │   └── test_api.py            # API smoke tests
│   └── .cache/                    # Auto-generated
│       ├── issues/                # Fetched GitHub issues (JSON per repo)
│       └── embeddings/            # Sentence-transformer embedding matrix + metadata
│
├── frontend/
│   ├── app/
│   │   ├── layout.tsx             # Root layout
│   │   ├── page.tsx               # Incident reporting + list view
│   │   └── changes/page.tsx       # CR review portal (developer auth gate)
│   ├── components/ui/             # shadcn/ui components
│   └── lib/                       # Shared utilities
│
├── data/
│   ├── developers.json            # Developer profiles (mined from git history)
│   └── incidents.jsonl            # Append-only incident log (auto-generated)
│
└── repos/                         # Git submodules (real OSS projects)
    ├── flask/                     # pallets/flask
    ├── celery/                    # celery/celery
    └── werkzeug/                  # pallets/werkzeug

API Endpoints

Incidents

Method Endpoint Description
POST /api/incidents Submit incident — triggers full triage pipeline
GET /api/incidents List all (filterable by status, severity)
GET /api/incidents/{id} Detail view with classification + triage results

Change Requests

Method Endpoint Description
GET /api/changes List all (filterable by status, risk level)
GET /api/changes/{id} Detail with target files, approvals, reviewers
PATCH /api/changes/{id}/status Transition state

Approvals

Method Endpoint Description
POST /api/changes/{id}/approve Approve with optional comment
POST /api/changes/{id}/reject Reject with reason
POST /api/changes/{id}/request-changes Request changes with comment

Developer Auth

Method Endpoint Description
GET /api/developers List registered developers
POST /api/developers/login Name-based developer authentication

How the Triage Pipeline Works (End to End)

User submits: "Users getting 403 Forbidden when accessing dashboard after login"
        │
        ▼
  ┌─ Stage 1: Issue Classifier ──────────────────────────────────────────┐
  │  Semantic search finds top-3 similar issues from Flask/Celery/Werkzeug│
  │  Claude Haiku reasons: "This is a novel auth issue, not a duplicate"  │
  │  Decision: CREATE_NEW (confidence: 0.85)                              │
  └───────────────────────────────────────────────────────────────────────┘
        │
        ▼
  ┌─ Stage 2: Triage Agent ──────────────────────────────────────────────┐
  │  Classification: severity=HIGH, category=SECURITY                     │
  │  Repo match: "flask" (score: 14, matched: session, 403, auth, login)  │
  │  File match: src/flask/sessions.py, src/flask/wrappers.py             │
  │  Git blame: David Lord owns 65% of sessions.py (142 lines)            │
  │  Reviewer: David Lord — "142 lines in src/flask/sessions.py (65.4%)"  │
  └───────────────────────────────────────────────────────────────────────┘
        │
        ▼
  ┌─ Stage 3: Change Request ────────────────────────────────────────────┐
  │  CR created: "Fix: Users getting 403 Forbidden..."                    │
  │  Risk: HIGH → 2 approvals required                                    │
  │  Category: SECURITY → +1 security reviewer                           │
  │  Target files: [sessions.py, wrappers.py]                             │
  │  Suggested reviewers: [David Lord, Armin Ronacher]                    │
  │  Audit log: "agent created CR from INC-A1B2C3D4"                     │
  └───────────────────────────────────────────────────────────────────────┘

Design Decisions

  1. Real repos, not mocked data — The system analyzes actual Flask, Celery, and Werkzeug repositories with real commit histories. Git blame returns real contributor names. This isn't a simulation.

  2. Dual-mode intelligence (LLM + rules) — Everything works offline with rule-based fallbacks. The LLM layer adds reasoning quality but isn't required. This makes the demo reliable regardless of network conditions.

  3. Semantic search for issue deduplication — TF-IDF or keyword matching misses paraphrased duplicates. Sentence-transformer embeddings capture semantic meaning, and the LLM layer prevents false positives by reasoning about root cause vs. surface symptoms.

  4. JSONL for incidents, SQLite for everything else — Incidents use an append-only log (simple, auditable, human-readable). Change requests, approvals, and GitHub issues use SQLite via async SQLAlchemy for relational queries.

  5. Git submodules for reproducibility — The analyzed repos are pinned to specific commits, ensuring consistent demo results across machines.

  6. Policy-driven approvals — Rather than hardcoding "2 approvals", the system derives requirements from severity and category. This is configurable in config.py and extensible to org-specific rules.

About

No description or website provided.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors