Skip to content

fix: clash agents had no microphone — expose input sinks as real capture sources#77

Merged
guohai merged 1 commit into
mainfrom
fix/clash-agent-mic-remap
Jul 17, 2026
Merged

fix: clash agents had no microphone — expose input sinks as real capture sources#77
guohai merged 1 commit into
mainfrom
fix/clash-agent-mic-remap

Conversation

@guohai

@guohai guohai commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Problem

After the new-headless fix (#76), both clash agents were still silentdebug_agent_*_out.raw = 0.000000.

Root cause (found in the live runner container)

The runner gives each agent browser PULSE_SOURCE=Sink_A_In.monitor, but Chromium/WebRTC does not enumerate PipeWire .monitor sources as microphones. A secure-context probe against the real site showed:

Sink_A_In.monitor → getUserMedia: NotFoundError, inputs: []

So the agent's voice web app (Agora ConvoAI) can't get a mic → never starts its call → no greeting → the broadcaster publishes pure silence.

"The agents heard the moderator" was a red herring: Sink_*_In had the moderator audio, but that was the runner's own parec dump — the browser never captured it.

Fix

pipewire-setup.sh remaps each input-sink monitor into a real capture source (module-remap-sourceMic_A / Mic_B), and clash-runner.ts points the browsers at Mic_A/Mic_B instead of the raw monitors.

Validation (live container)

Source Chromium sees getUserMedia
Sink_A_In.monitor inputs: [] ❌ NotFoundError
Mic_A (remap) inputs: ["Default","Mic_A"] state: live

Plus audio flow-through: a tone played into Sink_A_In was captured on Mic_A at RMS 0.38.

Coverage

  • Adds a Docker audio-pipeline regression check: Mic_A/Mic_B exist and carry audio played into Sink_*_In.
  • Updates CLASH_DESIGN.md (mic-exposure section).
  • Typecheck clean; 64 clash unit tests pass.

Deploy

Runner-side — image rebuilds on merge (CI), then vox-upgrade.sh on the runner host. No server/schema changes.

Generated with SMT smt@agora.io

…ure sources

Even with new headless, both agents stayed silent. Root cause: the runner gave
each browser PULSE_SOURCE=Sink_A_In.monitor, but Chromium/WebRTC does NOT
enumerate PipeWire `.monitor` sources as microphones. In the live container,
getUserMedia({audio:true}) returned NotFoundError with inputs:[], so the agents'
voice web app (Agora ConvoAI) never started its call → no greeting → the
broadcaster published pure silence.

("Agents heard the moderator" was a red herring: Sink_*_In had the moderator
audio, but that was the runner's own parec dump — the browser never captured it.)

Fix: pipewire-setup.sh remaps each input-sink monitor into a real capture source
(module-remap-source → Mic_A / Mic_B), and clash-runner.ts points the browsers
at Mic_A/Mic_B instead of the raw monitors.

Verified in the live runner container:
- Sink_A_In.monitor → getUserMedia NotFoundError, inputs:[]
- Mic_A (remap)     → inputs:["Default","Mic_A"], track state:live
- tone into Sink_A_In → captured on Mic_A at RMS 0.38 (audio flows through)

Adds a Docker audio-pipeline regression check (Mic_A/Mic_B exist and carry
audio) and updates CLASH_DESIGN.md. Runner image must be rebuilt + redeployed.

🤖 Built with SMT <smt@agora.build>
@github-actions

Copy link
Copy Markdown

🤖 Codex Code Review

No findings.

I reviewed the PR diff for the audio setup, runner wiring, browser env, and regression test. The change looks logically consistent: Mic_A/Mic_B are created from the input sink monitors before browser launch, the runner points PULSE_SOURCE at those real capture sources, and setup fails fast if the sources are missing.

Residual note: I did not run the audio integration test in this read-only environment.

@github-actions

Copy link
Copy Markdown

🤖 Claude Code Review

I reviewed the full diff plus the surrounding context in pipewire-setup.sh, test-audio-pipeline.sh, and browser-agent.ts.

Summary

This is a clean, well-scoped fix. The root-cause diagnosis is correct — Chromium/WebRTC does not enumerate a PulseAudio/PipeWire .monitor as a capture device, so getUserMedia against Sink_*_In.monitor fails with NotFoundError. Remapping each input-sink monitor into a real named source (module-remap-sourceMic_A/Mic_B) and pointing PULSE_SOURCE at those is the right approach.

Correctness of the audio graph holds up: both signal paths that must reach an agent's mic write into Sink_*_In — the moderator via pacat (broadcaster.ts) and the opponent via the cross-wire loopback (Sink_X_Out.monitor → Sink_Y_In). Both surface on Sink_*_In.monitor, which is exactly what Mic_* remaps, so no path is lost by the rename. The isolation invariant (step 7) is also unaffected, since the remap is a passive read-only consumer of the monitor.

Test coverage is genuinely strong here — the new "Mic_A carries audio played into Sink_A_In" assertion (line 115) exercises the remap end-to-end with an RMS check rather than just asserting the source exists, which is what actually proves the fix works (channel-map/upmix correctness included).

Setup robustness is consistent with the existing pattern: under set -euo pipefail a failed load-module aborts loudly, and the follow-up grep -qx verification loop (lines 93–98) mirrors the existing sink-verification block, so a half-built graph fails fast rather than producing silent agents.

No security concerns: these are container-internal shell scripts with no user-controlled input (no injection surface), no credential handling, and --no-sandbox was pre-existing.

Minor observations (non-blocking)

  • browser-agent.ts:76-78 / doc comment attribution. The --headless=new change and its doc-comment justification are presented (in both the design doc and code comments) as part of this fix, but that flag already landed in commit fix: clash agents silent — use Chromium new headless so browser audio renders #76. Not a code defect — just note the narrative conflates two fixes if that matters for changelog accuracy.
  • pipewire-setup.sh:88-89. The load-module return (module ID) isn't captured, so there's no clean programmatic unload handle for the remap sources. That's fine given they live for the container's lifetime and never need per-match teardown (unlike the loopbacks), but worth a one-line comment noting the asymmetry with the cross-wire modules if a future reader wonders why these aren't tracked.

Overall: correct, well-tested, safe to merge.

@guohai
guohai merged commit 4ea6e59 into main Jul 17, 2026
3 checks passed
@guohai
guohai deleted the fix/clash-agent-mic-remap branch July 17, 2026 12:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant