feat(voice): opt-in audio fade-out on interruption (#6165)#6410
Open
dorukdumlu wants to merge 1 commit into
Open
feat(voice): opt-in audio fade-out on interruption (#6165)#6410dorukdumlu wants to merge 1 commit into
dorukdumlu wants to merge 1 commit into
Conversation
Author
|
Ended up implementing this in #6410 went with a zero-latency approach inside _ParticipantAudioOutput instead of the holdback buffer, details in the PR description |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #6165
What
Adds an opt-in linear fade-out applied to the agent's audio when it gets interrupted, instead of cutting playback abruptly:
Default is
0.0(disabled) — existing behavior is unchanged.How (and why not the holdback buffer I proposed in the issue)
In my comment on #6165 I proposed holding back a small tail buffer before forwarding frames downstream. I dropped that approach while implementing: it adds first-frame playback latency equal to the tail size, which is the one metric a voice agent shouldn't pay for a cosmetic feature.
Instead, the fade is built inside
_ParticipantAudioOutput, which already knows everything needed at interruption time:_forward_audiokeeps a small rolling history (~400ms: queue size + fade + margin) of frames already pushed to thertc.AudioSource.AudioSource.queued_durationtells us how much of that history hadn't played yet, so the exact playback position isend_of_history - queued_duration.clear_queue(), a single frame is re-pushed: the nextfade_out_on_interruptionseconds of the actual audio from the playback position, with a linear 1 → 0 gain ramp.This gives zero added latency and no content discontinuity — the fade is the same audio the user was hearing, just ramped down. The history bookkeeping is only active when the option is enabled.
playback_positionaccounting is untouched: the faded tail replays audio that was already subtracted viaqueued_duration, so reported position stays correct for chat-history truncation.Notes
_forward_audiois blocked inAudioSource.capture_frameunder backpressure when the interruption fires, that in-flight frame can land afterclear_queue(). Today it plays at full gain after the cut; with the fade enabled it may play before the faded tail. Fixing that needs coordination between_forward_audioand_wait_for_playoutand felt out of scope here.Testing
_build_fade_out_frameis a pure function; added hermetic unit tests (tests/test_interruption_fade_out.py, markedunit): ramp shape/monotonicity, correct offset from playback position, clamping to unplayed duration and to available history, stereo, and the no-op cases.--unitsuite passes (same result asmainbaseline on my machine).ruff check/ruff formatclean; no new mypy errors inroom_io.