Skip to content

Normalize observation-only steps without blank agent messages #295

Description

@neubig

Some standardized samples represent environment/tool observations as blank source: agent steps with only an observation field. This is schema-valid, but semantically awkward because the environment, not the agent, authored the observation.

Examples from the audit/heuristics include web/mobile datasets such as WebLINX, Wonderbread, Mind2Web, Go-Browse-WA, TurkingBench, WebArena, AndroidControl, and related browser/mobile traces.

Acceptance criteria:

  • Decide the canonical representation for standalone observations when no preceding tool call is available.
  • Prefer attaching observations to the triggering tool-call step when possible.
  • If standalone observation steps remain necessary, document the convention clearly or add schema support for environment-authored steps.
  • Regenerate affected samples after the representation is settled.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions