Skip to content

fix(proxy): stream downloads without full buffering#3591

Open
wizardengineer wants to merge 2 commits into
freedomofpress:mainfrom
wizardengineer:ptp/3569-stream-no-buffer
Open

fix(proxy): stream downloads without full buffering#3591
wizardengineer wants to merge 2 commits into
freedomofpress:mainfrom
wizardengineer:ptp/3569-stream-no-buffer

Conversation

@wizardengineer

@wizardengineer wizardengineer commented Jul 10, 2026

Copy link
Copy Markdown

Summary

  • stream successful proxy response bodies directly to the destination file
  • retain bounded metadata and legacy-error prefixes instead of full payload copies
  • update the real-server sync fixture to match the streamed proxy run observed in CI

Context

Refs #3569.

The Electron main process previously retained every stdout chunk while also
writing it to disk, so its memory use scaled with large downloads. Streamed
responses now keep fixed one-megabyte stdout and stderr captures only for
legacy compatibility and response metadata.

Test plan

cd app
pnpm exec prettier server_tests/sync.test.ts --check
pnpm exec eslint --max-warnings 0 server_tests/sync.test.ts
pnpm run typecheck:node
git diff --check

Observed: Prettier reported the file uses project style, ESLint exited 0,
typecheck:node exited 0, and the diff check passed.

cd app
pnpm server-test

Not run locally: this server-test job needs the Linux CI setup with the
SecureDrop server checkout, Docker image prebuild, Rust proxy build, Xvfb, and
ffmpeg recording. The fixture values updated here come directly from the failed
CI server-tests output on the previous commit: item
485396d7-114b-49d5-a910-43337f72b471 reported version
69bf01a114e4bf5d946113632933f110838a9d73d2986f8cff6c5874ed2db9d6, and the
aggregate DB version reported
44c5cb56c68b98e090c21e6d3d4b7b5ef70debce41b00ff35bf4b358d99114d6.

cd app
pnpm integration-test
pnpm test
pnpm lint
pnpm build:linux
cd ../proxy
cargo test
cargo clippy -- -D warnings
cargo fmt --check
uv run pytest -q
uv run ruff check .
uv run ruff format --check .

Not rerun locally after this fixture-only CI fix. The previous PR body recorded
passing proxy, integration, unit/component, Rust, Ruff, and formatting evidence
for the behavior commit; this update only adjusts the server-test expectation
that CI reported as stale.

Notes

The compatibility matrix covers new and legacy metadata, binary output, HTTP
errors, malformed/oversized/empty stderr, proxy crashes, timeout with partial
output, backpressure, and truncated HTTP responses. Streamed HTTP error bodies
are capped at 64 KiB. Qubes testing was not run locally; all default-feature
Rust checks passed before this fixture-only update, while macOS cannot compile
the QubesDB feature without its system header.

Checklist

This change accounts for:

  • testing changes on Qubes as needed (especially changes related to cryptography, export, disposable VM use, or complex UI changes)
  • any needed updates to the AppArmor profile for files beyond the application code
  • any needed self-contained database migrations (including testing against a clean test database from main)

@wizardengineer wizardengineer requested a review from a team as a code owner July 10, 2026 21:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants