Skip to content

demos: standalone system-services-monitor demo - #1383

Draft
dmvevents wants to merge 3 commits into
NVIDIA:mainfrom
dmvevents:demo-system-services-monitor
Draft

demos: standalone system-services-monitor demo#1383
dmvevents wants to merge 3 commits into
NVIDIA:mainfrom
dmvevents:demo-system-services-monitor

Conversation

@dmvevents

Copy link
Copy Markdown

Per #891 split (5 of 5). Marked draft — depends on PR landing first.

Series progress

What this contains

  • demos/system-services-monitor/ standalone demo subtree

What this does NOT contain

  • Implementation (impl PR)
  • Helm chart (PR 4)

CR-ignored findings addressed

  • daemonset.yaml image pin to SHA
  • livenessProbe httpGet check
  • servicemonitor.yaml alert spec
  • Dockerfile python:3.13 (CVE)

cc @XRFXLP

@coderabbitai

coderabbitai Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 6514e64c-8b07-47f8-9bfc-bfa55fcaa750

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@copy-pr-bot

copy-pr-bot Bot commented Jun 10, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions

Copy link
Copy Markdown
Contributor

@dmvevents this PR has been inactive for 14 days. Do you need help finishing it, or should we close it for now? Feel free to reopen anytime.

Lands the demo subtree (NVIDIA#891 split, 5 of 5). Standalone — depends on
the implementation in <impl-pr-num> but doesn't gate any production
functionality.

Includes CR-ignored fixes from the umbrella PR:
- daemonset.yaml: image pinned to SHA tag (not :0.1.0)
- daemonset.yaml: livenessProbe httpGet validated
- servicemonitor.yaml: FabricManagerFlapping alert spec
- Dockerfile: python:3.13 base for CVE remediation

Signed-off-by: Anton Alexander <dmvevents@users.noreply.github.com>
…validation

The demo re-bundled CUDA context/memory validation as a polling daemon
check — the exact model deesharma24 rejected on NVIDIA#891, where a long-running
daemon contends for GPU memory with active workloads. That review is why
CUDA validation re-landed as a preflight init-container (NVIDIA#1384), which runs
once before workloads schedule.

Remove the daemon check from this demo to avoid contradicting the accepted
design: delete checks/cuda_validation.py and drop all references from
monitor.py, config.py, metrics.py, the k8s ConfigMap/PrometheusRule, and the
tests. README now points at preflight-checks/cuda-validation/ (NVIDIA#1384) as the
supported form.

Also note in the DaemonSet that the system-services-monitor:0.1.0 tag is a
demo-local pin, independent of the Helm chart appVersion.

Signed-off-by: Anton Alexander <dmvevents@users.noreply.github.com>
The system-services-monitor design doc is ADR-049; ADR-030 upstream is the
unrelated grpc-tls-authentication design. Corrects the scope docstring only.

Signed-off-by: Anton Alexander <dmvevents@users.noreply.github.com>
@dmvevents
dmvevents force-pushed the demo-system-services-monitor branch from 9d55deb to 28ed8ad Compare July 24, 2026 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant