Skip to content

fix: raise eval push payload limit to 100 MiB (match prime#818) - #2222

Open
hubert-marek wants to merge 1 commit into
mainfrom
fix/raise-eval-push-payload-limit-75mib
Open

fix: raise eval push payload limit to 100 MiB (match prime#818)#2222
hubert-marek wants to merge 1 commit into
mainfrom
fix/raise-eval-push-payload-limit-75mib

Conversation

@hubert-marek

@hubert-marek hubert-marek commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Companion to PrimeIntellect-ai/prime#818.

_MAX_SAMPLES_PAYLOAD_BYTES is the per-sample ceiling push_traces (verifiers/v1/utils/platform.py) enforces before POSTing eval samples to /evaluations/{id}/samples; its comment notes it mirrors the Prime Evals client's request ceiling. prime#818 raises that client ceiling 25 → 100 MiB, so this mirror is raised to match.

Why: computer-use environments (cua_world_v1, osworld_v2_v1) embed a full 1920×1080 screenshot per step (~1.5–3.5 MiB each), so a single rollout's sample exceeds 25 MiB and this check raises, skipping the upload — uv run eval --push logs:

  • sample 0 is too large to upload (53564890 > 26214400 bytes) — cua_world (~51 MiB)
  • sample 0 is too large to upload (78391638 > 26214400 bytes) — osworld_v2 (~75 MiB @ 50 steps)

so those runs get no hosted rollout link. 100 MiB (with prime#818) gives real headroom for screenshot-heavy rollouts without compressing traces.

🤖 Generated with Claude Code


Note

Low Risk
Single constant change with no logic edits; slightly larger HTTP bodies on upload, bounded by the same checks as before.

Overview
Raises _MAX_SAMPLES_PAYLOAD_BYTES in platform.py from 25 MiB to 100 MiB so the verifiers --push path stays aligned with the Prime Evals client request ceiling (prime#818).

push_traces still uses this constant for per-sample rejection and batch splitting before POSTing to /evaluations/{id}/samples; only the ceiling changes, so screenshot-heavy computer-use rollouts that were skipped as “too large” can upload instead of failing the check locally.

Reviewed by Cursor Bugbot for commit 066420a. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Raise eval push payload limit to 100 MiB in _MAX_SAMPLES_PAYLOAD_BYTES

Increases _MAX_SAMPLES_PAYLOAD_BYTES in platform.py from 25 MiB to 100 MiB to match the limit set in prime#818. Any payload size enforcement using this constant now allows up to 100 MiB.

Macroscope summarized 066420a.

@hubert-marek
hubert-marek marked this pull request as ready for review August 2, 2026 07:44
@hubert-marek
hubert-marek requested a review from xeophon August 2, 2026 07:44
@macroscopeapp

macroscopeapp Bot commented Aug 2, 2026

Copy link
Copy Markdown

Approvability

Verdict: Approved 066420a

Single-line configuration change increasing a payload size limit constant from 25 MiB to 100 MiB. The change is minimal, self-contained, and has clear intent with no complex runtime implications.

You can customize Macroscope's approvability policy. Learn more.

macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Aug 2, 2026
Mirror of PrimeIntellect-ai/prime#818. Computer-use envs (cua_world, osworld_v2)
embed a full screenshot per step, so one rollout's sample exceeds the 25 MiB
push ceiling and is skipped ("sample 0 is too large to upload"): cua_world
~51 MiB, osworld_v2 ~75 MiB at 50 steps, higher for longer rollouts. Match the
Prime Evals client ceiling raised in prime#818.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@hubert-marek hubert-marek changed the title fix: raise eval push payload limit to 75 MiB (match prime#818) fix: raise eval push payload limit to 100 MiB (match prime#818) Aug 2, 2026
@hubert-marek
hubert-marek force-pushed the fix/raise-eval-push-payload-limit-75mib branch from 3af1f66 to 066420a Compare August 2, 2026 07:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant