Skip to content

feat(api-server): expose real-time audio spectrum over REST and WebSocket - #4612

Open
stranzero wants to merge 3 commits into
pear-devs:masterfrom
stranzero:feat/api-spectrum
Open

feat(api-server): expose real-time audio spectrum over REST and WebSocket#4612
stranzero wants to merge 3 commits into
pear-devs:masterfrom
stranzero:feat/api-spectrum

Conversation

@stranzero

@stranzero stranzero commented Jul 26, 2026

Copy link
Copy Markdown

Summary

  • Hook the player Web Audio graph (peard:audio-can-play) with an AnalyserNode and publish log-spaced frequency bands (0–255).
  • Add GET /api/v1/spectrum and an opt-in WebSocket SPECTRUM stream (SUBSCRIBE_SPECTRUM / UNSUBSCRIBE_SPECTRUM).
  • Drive sampling from the main process so it stays responsive when the Pear window is unfocused.

API

  • REST: GET /api/v1/spectrum{ bands, peak, timestamp } (204 if no data yet)
  • WebSocket: send { type: SUBSCRIBE_SPECTRUM } to receive { type: SPECTRUM, bands, peak, timestamp } frames
  • Config: spectrumEnabled (default true), spectrumBands (16), spectrumFps (20)

Test plan

  • Enable API Server, play audio, confirm GET /api/v1/spectrum returns non-empty bands that move with the track
  • Connect WebSocket, send SUBSCRIBE_SPECTRUM, confirm frames at ~configured fps
  • Unfocus / minimize Pear and confirm spectrum updates continue (no Chromium background throttle)
  • Send UNSUBSCRIBE_SPECTRUM and confirm frames stop for that socket
  • Disable spectrumEnabled and confirm REST returns 204 / WS stops emitting

Summary by CodeRabbit

  • New Features
    • Added real-time audio spectrum streaming with new configuration: enablement, band count, and update rate.
    • Added GET /api/{API_VERSION}/spectrum to fetch the latest spectrum frame; returns no content when disabled/unavailable.
    • Added optional WebSocket spectrum subscriptions for live spectrum updates.
    • Spectrum frames now include band levels, a rolling peak metric, and timestamps, and are integrated into the server start/stop lifecycle.
  • Improvements
    • Spectrum polling and streaming adapt to configuration changes automatically.

…cket

Hook the player Web Audio graph with an AnalyserNode and publish log-spaced
bands (0-255) via GET /api/v1/spectrum and an opt-in SPECTRUM WebSocket stream.
Main-process polling keeps sampling responsive when the window is backgrounded.
@coderabbitai

coderabbitai Bot commented Jul 26, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 7289706d-09a5-4f8d-b2d0-cf3f4987677c

📥 Commits

Reviewing files that changed from the base of the PR and between f977ea7 and 27b5baa.

📒 Files selected for processing (1)
  • src/plugins/api-server/renderer.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/plugins/api-server/renderer.ts

📝 Walkthrough

Walkthrough

The API server adds configurable audio spectrum capture, renderer-side FFT sampling, backend IPC polling, an HTTP spectrum endpoint, and opt-in WebSocket spectrum broadcasts.

Changes

Audio spectrum streaming

Layer / File(s) Summary
Spectrum configuration and data contracts
src/plugins/api-server/config.ts, src/plugins/api-server/backend/types.ts, src/plugins/api-server/backend/routes/websocket.ts
Configuration defaults, backend state, polling methods, and the SpectrumData payload are added.
Renderer spectrum capture
src/plugins/api-server/renderer.ts, src/plugins/api-server/index.ts
Renderer lifecycle hooks configure an analyser, calculate FFT bands and peak data, and send frames over IPC.
Backend polling and HTTP access
src/plugins/api-server/backend/main.ts, src/plugins/api-server/backend/routes/control.ts
Backend lifecycle code polls spectrum data, stores the latest frame, restarts polling on configuration changes, and exposes GET /api/{API_VERSION}/spectrum.
Opt-in WebSocket streaming
src/plugins/api-server/backend/routes/websocket.ts
Clients can subscribe or unsubscribe to spectrum frames, which are broadcast only to subscribed sockets and removed on close.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant RendererAudioGraph
  participant BackendSpectrumPolling
  participant HTTPClient
  participant WebSocketClient
  BackendSpectrumPolling->>RendererAudioGraph: request-spectrum
  RendererAudioGraph-->>BackendSpectrumPolling: audio-spectrum frame
  BackendSpectrumPolling->>HTTPClient: return latest spectrum
  BackendSpectrumPolling-->>WebSocketClient: broadcast subscribed spectrum frame
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: exposing real-time audio spectrum data over REST and WebSocket.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/plugins/api-server/backend/main.ts`:
- Around line 90-102: Update startSpectrumPolling to clear this.spectrum
whenever spectrumEnabled is false, before returning after stopping the polling
timer. Ensure the getSpectrum route observes the cleared cache and returns 204
when spectrum streaming is disabled, while preserving the existing polling
behavior when enabled.

In `@src/plugins/api-server/renderer.ts`:
- Around line 130-145: Update onRendererLoad and onRendererUnload to store the
named peard:request-spectrum callback, then remove that callback from the IPC
object during unload before clearing rendererIpc. Preserve the existing
audio-can-play listener cleanup and state reset while ensuring each renderer
reload leaves no spectrum IPC listener behind.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 7b89fdbf-305e-4019-bbca-9dc84cdf8768

📥 Commits

Reviewing files that changed from the base of the PR and between 1a84f1d and da790fa.

📒 Files selected for processing (7)
  • src/plugins/api-server/backend/main.ts
  • src/plugins/api-server/backend/routes/control.ts
  • src/plugins/api-server/backend/routes/websocket.ts
  • src/plugins/api-server/backend/types.ts
  • src/plugins/api-server/config.ts
  • src/plugins/api-server/index.ts
  • src/plugins/api-server/renderer.ts

Comment thread src/plugins/api-server/backend/main.ts
Comment thread src/plugins/api-server/renderer.ts
Return 204 after spectrumEnabled is turned off by dropping the cached frame,
and remove the peard:request-spectrum listener on renderer unload so reloads
do not stack handlers.
@stranzero

Copy link
Copy Markdown
Author

Addressed the review notes:

  • Clear this.spectrum when spectrumEnabled is off (and on backend end) so GET /api/v1/spectrum returns 204 instead of a stale frame
  • Remove the peard:request-spectrum IPC listener on renderer unload via removeAllListeners

Cap log bands around 4.2 kHz, lift treble with a stronger shelf, and prefer
peak/loud-bin energy on upper bands so sparse highs aren't RMS-diluted into
silence. Deepen the analyser floor slightly for quiet HF.
@stranzero

stranzero commented Jul 26, 2026

Copy link
Copy Markdown
Author

Pushed a spectrum balance tweak so the right-side bands actually move on typical tracks:

  • Cap log range ~4.2 kHz so the last bars aren’t empty air up to 14 kHz
  • Stronger treble shelf + peak-biased upper bands (avoids RMS dilution)
  • Slightly deeper analyser floor (-90 dB)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant