Skip to content

Pull requests: strands-agents/evals

New pull request New
26 Open 207 Closed
26 Open 207 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

feat: add support for Claude agents to OpenInference mapper area-tracing Trace/session ingestion: providers, session mappers, extractors, telemetry/OTEL enhancement New feature or request
#340 opened Jul 31, 2026 by liramon2 Contributor • Draft
9 tasks done
4
feat: add integration tests for ADK mapper area-tracing Trace/session ingestion: providers, session mappers, extractors, telemetry/OTEL chore Maintenance tasks, dependency updates, CI changes, refactoring with no user-facing impact
#339 opened Jul 31, 2026 by liramon2 Contributor • Draft
9 tasks done
6
fix: scope tools to owning agent in multi-agent traces area-evaluators Evaluators: output, trajectory, tool use, interactions, and LLM-as-judge quality metrics area-tracing Trace/session ingestion: providers, session mappers, extractors, telemetry/OTEL bug Something isn't working
#336 opened Jul 30, 2026 by liramon2 Contributor Loading…
9 tasks done
20
ci: bump pypa/gh-action-pypi-publish from 1.14.0 to 1.14.2 area-community Repo health, governance, contributor process, release process, and CI dependency bumps chore Maintenance tasks, dependency updates, CI changes, refactoring with no user-facing impact dependencies Pull requests that update a dependency file github_actions Pull requests that update GitHub Actions code
#333 opened Jul 30, 2026 by dependabot Bot Loading…
feat: add skill-level evaluators for skill-equipped agents area-evaluators Evaluators: output, trajectory, tool use, interactions, and LLM-as-judge quality metrics enhancement New feature or request
#330 opened Jul 28, 2026 by sangminwoo Collaborator Loading…
9 tasks done
13
feat(experiment): add opt-in model routing for evaluators area-core Core eval framework: Case, Experiment, task handler, evaluation data stores area-evaluators Evaluators: output, trajectory, tool use, interactions, and LLM-as-judge quality metrics enhancement New feature or request
#325 opened Jul 23, 2026 by pdebjyot Contributor Loading…
4 tasks done
1
ci: update pre-commit requirement from <4.6.0,>=3.2.0 to >=3.2.0,<4.7.0 area-community Repo health, governance, contributor process, release process, and CI dependency bumps chore Maintenance tasks, dependency updates, CI changes, refactoring with no user-facing impact dependencies Pull requests that update a dependency file python Pull requests that update python code
#321 opened Jul 22, 2026 by dependabot Bot Loading…
fix: convert Langfuse AGENT observations regardless of parent area-tracing Trace/session ingestion: providers, session mappers, extractors, telemetry/OTEL bug Something isn't working
#314 opened Jul 15, 2026 by poshinchen Contributor Loading…
7 of 9 tasks
1
6
fix: add GenericGenAISessionMapper for dict spans with unrecognized scope area-tracing Trace/session ingestion: providers, session mappers, extractors, telemetry/OTEL bug Something isn't working
#309 opened Jul 13, 2026 by ybdarrenwang Collaborator Loading…
9 tasks done
24
test(redteam): cover RedTeamCase metadata sync and _build_system_prompt assembly area-redteam Red teaming: adversarial generation, attack strategies, attack success evaluation chore Maintenance tasks, dependency updates, CI changes, refactoring with no user-facing impact
#301 opened Jul 7, 2026 by zeroshotmind Loading…
9 of 11 tasks
1
feat(redteam): reframe attacker prompts to survive aligned attacker models area-redteam Red teaming: adversarial generation, attack strategies, attack success evaluation enhancement New feature or request
#298 opened Jul 2, 2026 by kevmyung Contributor Loading…
6 of 9 tasks
1
fix(redteam): separate errored attacks from breaches in ASR area-redteam Red teaming: adversarial generation, attack strategies, attack success evaluation bug Something isn't working
#296 opened Jul 2, 2026 by kevmyung Contributor Loading…
7 of 9 tasks
1
fix(redteam): reset target session between PAIR/SequentialBreak iterations area-redteam Red teaming: adversarial generation, attack strategies, attack success evaluation bug Something isn't working
#292 opened Jul 1, 2026 by kevmyung Contributor Loading…
9 tasks done
1
12
feat(redteam): redesign risk categories for agent-centric evaluation area-redteam Red teaming: adversarial generation, attack strategies, attack success evaluation enhancement New feature or request
#290 opened Jul 1, 2026 by kevmyung Contributor Loading…
9 tasks done
1
ci: update opentelemetry-instrumentation-langchain requirement from <0.62.0,>=0.40.0 to >=0.40.0,<0.63.0 area-community Repo health, governance, contributor process, release process, and CI dependency bumps chore Maintenance tasks, dependency updates, CI changes, refactoring with no user-facing impact dependencies Pull requests that update a dependency file python Pull requests that update python code
#288 opened Jun 29, 2026 by dependabot Bot Loading…
2
feat: add model-output corruption effects area-chaos Chaos/fault injection: experiments, recovery strategy, partial completion, failure communication enhancement New feature or request
#284 opened Jun 24, 2026 by venkatkrish543re Loading…
1 of 9 tasks
44
ci: update langfuse requirement from <4,>=2.0.0 to >=2.0.0,<5 area-community Repo health, governance, contributor process, release process, and CI dependency bumps chore Maintenance tasks, dependency updates, CI changes, refactoring with no user-facing impact dependencies Pull requests that update a dependency file python Pull requests that update python code
#283 opened Jun 24, 2026 by dependabot Bot Loading…
2
ci: update mypy requirement from <2.0.0,>=1.15.0 to >=1.15.0,<3.0.0 area-community Repo health, governance, contributor process, release process, and CI dependency bumps chore Maintenance tasks, dependency updates, CI changes, refactoring with no user-facing impact dependencies Pull requests that update a dependency file python Pull requests that update python code
#282 opened Jun 24, 2026 by dependabot Bot Loading…
chore(detectors): added async detectors execution (WIP) area-detectors Failure detection and root cause analysis of agent sessions chore Maintenance tasks, dependency updates, CI changes, refactoring with no user-facing impact
#277 opened Jun 17, 2026 by poshinchen Contributor Loading…
7 of 9 tasks
8
feat(experiment): add verbosity-aware stack traces to evaluation error reasons area-core Core eval framework: Case, Experiment, task handler, evaluation data stores area-devx Developer experience: papercuts, confusing public APIs, error messages, ergonomics, usability enhancement New feature or request
#268 opened Jun 15, 2026 by AndyMc629 Loading…
9 tasks done
1
Feature/skills aggregator: add SkillEvalAggregator for batch evaluation comparisons area-evaluators Evaluators: output, trajectory, tool use, interactions, and LLM-as-judge quality metrics enhancement New feature or request
#229 opened May 14, 2026 by venkatkrish543re • Draft
feat: Add EvaluationPlugin for agent invocation evaluation and retry area-core Core eval framework: Case, Experiment, task handler, evaluation data stores area-evaluators Evaluators: output, trajectory, tool use, interactions, and LLM-as-judge quality metrics enhancement New feature or request
#166 opened Mar 18, 2026 by afarntrog Contributor Loading…
5 of 7 tasks
3
feat: add OTel test semantic convention attributes to Experiment spans area-core Core eval framework: Case, Experiment, task handler, evaluation data stores area-tracing Trace/session ingestion: providers, session mappers, extractors, telemetry/OTEL enhancement New feature or request
#131 opened Feb 10, 2026 by anirudha • Draft
7
feat: Optional Case specific Goal for GoalSuccessRateEvaluator area-evaluators Evaluators: output, trajectory, tool use, interactions, and LLM-as-judge quality metrics enhancement New feature or request
#75 opened Dec 17, 2025 by dbermuehler • Draft
7 tasks
feat: add ContextualFaithfulnessEvaluator area-core Core eval framework: Case, Experiment, task handler, evaluation data stores area-evaluators Evaluators: output, trajectory, tool use, interactions, and LLM-as-judge quality metrics enhancement New feature or request
#64 opened Dec 7, 2025 by stefanoamorelli Loading…
7 tasks done
ProTip! Type g i on any issue or pull request to go back to the issue listing page.