Skip to content

feat: support macOS local benchmark runs - #2

Merged
waple0820 merged 1 commit into
mainfrom
agent/macos-local-pilot
Jul 12, 2026
Merged

feat: support macOS local benchmark runs#2
waple0820 merged 1 commit into
mainfrom
agent/macos-local-pilot

Conversation

@waple0820

Copy link
Copy Markdown
Collaborator

What changed

  • add a process-tree profiler for macOS with CPU lifetime accounting, RSS/Chrome RSS, host-memory guard, and throughput
  • make the benchmark wrapper Bash 3 compatible and select the correct platform profiler
  • run local comparisons with an explicit system Chrome executable and accurate machine labels
  • continue evaluation/summarization when individual benchmark tasks fail, while still stopping on a resource guard
  • use Lexmount pagination totals when verifying active-session cleanup

Why

The 5090 host was not available, so the GPT-5.5 LexBench comparison needed a reproducible macOS fallback without presenting RSS as PSS or controller metrics as server-side utilization. The local adapter fix is already merged in browseruse-agent-bench#100.

Validation

  • 10 tests passed
  • Ruff passed
  • Bash syntax check passed
  • real local LexBench task completed and judged successfully
  • paired 8-task smoke and paired 20-task pilot completed on both backends
  • c5 local pilot stayed above the 6 GiB host-memory guard

@github-actions

github-actions Bot commented Jul 12, 2026

Copy link
Copy Markdown

Claude finished @waple0820's task in 2m 56s —— View job


PR Review

  • Gather context
  • Review changed files for bugs and security issues
  • Post findings

Findings

1 bug found. No security issues.


Bug: len(response) is evaluated unconditionally — probe_lexmount_sessions.py:77

"page_size": int(getattr(pagination, "page_size", len(response))),

Python evaluates all arguments before calling a function, so len(response) is computed every time pagination is not None — even when page_size exists on the pagination object. If the real Lexmount SDK response doesn't implement __len__, this raises TypeError on every paginated list call.

The test masks this because FakeResponse defines __len__, but the real response object likely doesn't.

Fix: don't use len(response) as an eager default. Either fall back to a safe sentinel or compute it lazily:

"page_size": int(getattr(pagination, "page_size", 0)),

Fix this →


@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Claude review completed. Review run: https://github.com/lexmount/LexBrowserEnv/actions/runs/29203667689. Please check Claude's inline feedback and summary comment for details.

@waple0820
waple0820 merged commit 857a30d into main Jul 12, 2026
1 check passed
@waple0820
waple0820 deleted the agent/macos-local-pilot branch July 12, 2026 18:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant