Skip to content

feat(eval): 评估qwen3-8B在lexbench和Online-mind2web两个benchmark上的分数 - #5

Merged
xianghuisun-svg merged 4 commits into
mainfrom
sxh_phase1
Jul 13, 2026
Merged

feat(eval): 评估qwen3-8B在lexbench和Online-mind2web两个benchmark上的分数#5
xianghuisun-svg merged 4 commits into
mainfrom
sxh_phase1

Conversation

@xianghuisun-svg

Copy link
Copy Markdown
Contributor
  • 新增 Qwen3-8B 的 LexBench、Online-Mind2Web、WebArena-Lite 一键评测启动器,并保留 GPT-5.5 LexBench 流程。

  • 新增 Qwen3-8B 实验配置,以及 LexBench / Online-Mind2Web 的评测报告与资源效率报告。

  • README.md 中补充一键启动命令、评测范围和报告入口。

  • uv run --extra dev ruff check src

  • uv run python -m compileall -q src

  • bash -n scripts/*.sh

  • 四个公开 Shell 启动器的 --help 检查。

@github-actions

github-actions Bot commented Jul 13, 2026

Copy link
Copy Markdown

Claude finished @xianghuisun-svg's task in 3m 31s —— View job


Code Review

  • Gather PR context
  • Review diff for bugs and security issues
  • Post findings

Two actual bugs found:

1. Credential leak — req_llmapi.py:32

OPENAI_API_KEY = require_env("OPENAI_API_KEY")   # line 28
...
print(OPENAI_API_KEY)   # line 32 — API key written to stdout

The API key is printed unconditionally at module level every time the script runs. This exposes the secret in any terminal scroll-back, CI/CD job log, or shell history that captures stdout. Fix this →

2. Dead code / wrong API call — req_llmapi.py:53–54

_seed = random.randint(0, 1000000)   # computed but never used
completion = self.client.chat.completions.create(messages=messages, model=self.model)

_seed is generated but never passed to the create() call, so it has no effect. If the intent is to make the call reproducible/deterministic via a seed parameter, it should be passed as seed=_seed. If reproducibility is not the goal, the dead assignment should be removed. Fix this →


No other bugs or security issues found in the remaining ~7 700 lines of new code.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Claude review completed. Review run: https://github.com/lexmount/LexBrowserEnv/actions/runs/29229397715. Please check Claude's inline feedback and summary comment for details.

@github-actions

github-actions Bot commented Jul 13, 2026

Copy link
Copy Markdown

Claude finished @xianghuisun-svg's task in 3m 44s —— View job


Code Review

  • Gather PR context
  • Check if previous bugs were fixed
  • Review diff for new bugs and security issues
  • Post findings

Previous issues from the last review: the req_llmapi.py file (which had print(OPENAI_API_KEY) and the dead _seed variable) has been excluded from the PR via commit 9dc8db9. Both prior bugs are resolved.

One new issue found:

API key exposed in process table — online_mind2web/cli.py:1161–1162

def build_judge_command(..., api_key: str) -> list[str]:
    return [
        str(python),
        str(osu_checkout / "src/run.py"),
        ...
        "--api_key",
        api_key,   # ← secret in argv
        ...
    ]

The judge API key is passed as a CLI argument value. While run() at line 149 correctly redacts it from log output via _redact_command(), the raw key is still exposed in /proc/<pid>/cmdline and ps aux for the lifetime of the subprocess — readable by any user on the same host.

If the OSU run.py harness can also accept the key via an environment variable, prefer that. If the interface is fixed (CLI-only), a workaround is to write the key to a tempfile.NamedTemporaryFile and pass --api_key_file <path> — but only if the upstream harness supports it. Fix this →


No other bugs or security issues found.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Claude review completed. Review run: https://github.com/lexmount/LexBrowserEnv/actions/runs/29229774195. Please check Claude's inline feedback and summary comment for details.

@xianghuisun-svg
xianghuisun-svg merged commit 4622b80 into main Jul 13, 2026
1 check passed
@waple0820
waple0820 deleted the sxh_phase1 branch July 14, 2026 11:36
@waple0820
waple0820 restored the sxh_phase1 branch July 14, 2026 11:36
@waple0820
waple0820 deleted the sxh_phase1 branch July 14, 2026 11:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant