Skip to content

[AI Evaluation] Redesign the generated HTML report #7593

Description

@grafanaKibana

Background and motivation

The HTML report we generate (via HtmlReportWriter, rendered by the React app under TypeScript/components) has picked up a lot of features over time: metric cards, chat history,
search, trending, tags, and the layout has gotten busier but not more convenient. Once you have a decent number of
scenarios and metrics it's harder than it should be to scan results and drill into a specific
metric or conveniently compare cases/runs/metrics.

I've got a mockup of a cleaner design (screenshots below) using FluentUI 2 design system, and wanted to agree on direction before
I sink time into implementation. The work is a visual refresh only — no data or API changes:

  • Agree on the design direction (mockup attached).
  • Rebuild the layout in the existing components, still reading the same Dataset /
    ScenarioRunResult / EvaluationMetric data via ReportContext.
  • Keep what already works: scenario tree, run history, search, markdown, pass/fail bars,
    metric and diagnostics drill-downs.
  • Keep the build/output identical: single self-contained file, the ##DATASETS## placeholder
    HtmlReportWriter relies on, and the existing CSP.
  • Re-check accessibility against the new design ([AI Evaluation] Run accessibility checks on the HTML report #6215).
  • Confirm the Azure DevOps report tab still renders correctly (same components).

Screenshots

Overview tab

Light Dark
Image Image

Cases tab

Light Dark
Image Image

History tab

Light Dark
Image Image

Comparison tab

Light Dark
Image Image

Standalone mockup to play with

AI Evaluation Report v3.1 (standalone).html

Risks

Mostly visual-regression risk in the shared components, the same React app drives both the
standalone report and the AzDevOps extension, so both need verifying. No API or data-shape
changes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions