Skip to content

Stop dropping Context collections from the agent catalog - #208

Open
AshishKumar4 wants to merge 1 commit into
mainfrom
fix/collections-truncated
Open

Stop dropping Context collections from the agent catalog#208
AshishKumar4 wants to merge 1 commit into
mainfrom
fix/collections-truncated

Conversation

@AshishKumar4

@AshishKumar4 AshishKumar4 commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

With hundreds of skills, the agent gets a truncated view of its Context Library and collections go missing. Missing a collection is a much bigger problem than missing a skill: collections are how the agent finds anything at all, and individual skills can be enumerated on demand through the session's list()/search().

There were two causes.

AGENT_CATALOG_MAX_ENTRIES was 25, which a deployment with a handful of public collections already exceeds on its own.

The bigger one: normalizeAgentCatalog sorted the combined list by title before clamping it, so which entries survived was decided purely by alphabetical position. Collections competed against skills on title. Raising the cap alone would only have moved the threshold at which that reappears, so this fixes the drop policy instead.

What does this change?

  • normalizeAgentCatalog clamps from the tail and sorts the survivors, so the gatekeeper's ordering is the priority signal rather than the alphabet
  • The Context gatekeeper lists its collections ahead of its skills, so a large skill set can no longer push them out
  • AGENT_CATALOG_MAX_ENTRIES 25 → 200
  • boundAgentCatalog() and AgentCatalogRequest are removed

200 rather than something larger because the catalog is inlined in the system prompt on every turn and compaction never reaches it, so the entry count is a direct and permanent context cost — roughly 80 KB at 200 entries and the field caps.

On the removal: request.limit was always AGENT_CATALOG_MAX_ENTRIES, a constant both sides already import from this module, so the parameter carried no information, and the provider-side clamp only duplicated the one the Workshop applies anyway. That leaves a single bound, in the kernel, on untrusted input. This changes the getAgentCatalog signature, which both gatekeepers validate through capnweb-validate, so it assumes the Workshop and an installed gatekeeper are never left at mismatched versions.

Why is this obviously correct and trivially verifiable?

The behavioural change is one moved .toSorted() and one reordered array concatenation. The new test in agent-catalog.test.ts fails on the old sort-then-slice ordering and passes on the new one; the id length bound that the deleted boundAgentCatalog suite used to assert is now covered against normalizeAgentCatalog, which is the only bound left.

Checklist

Checking every item does not guarantee acceptance. Maintainers determine whether
a pull request meets the contribution policy.

  • This is a small, concrete change; it is not a feature, refactor, or low-value cleanup.
  • I understand that maintainers decide whether the change is obviously correct and trivially verifiable.
  • I have read and followed the contribution guidelines.

@github-actions

github-actions Bot commented Aug 14, 2026

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@github-actions github-actions Bot added the gatekeeper Changes to a gatekeeper integration label Aug 14, 2026
@AshishKumar4

Copy link
Copy Markdown
Contributor Author

I have read the CLA Document and I hereby sign the CLA

github-actions Bot added a commit that referenced this pull request Aug 14, 2026
Comment thread packages/gatekeeper-context/src/library-gatekeeper.ts Outdated
Comment thread packages/gatekeeper-context/src/library-gatekeeper.ts Outdated
Comment thread packages/gatekeeper-context/src/library-gatekeeper.ts Outdated
@github-actions github-actions Bot added the workshop/shared Changes to shared Workshop APIs label Aug 14, 2026
Comment thread packages/gatekeeper-context/src/library-gatekeeper.ts Outdated
Comment thread packages/gatekeeper-context/src/library-gatekeeper.ts Outdated
Comment thread packages/gatekeeper-context/src/library-gatekeeper.ts Outdated
@AshishKumar4 AshishKumar4 changed the title Fix: agents seeing highly truncated list of collections and skills Draft: Fix: agents seeing highly truncated list of collections and skills Aug 14, 2026
@AshishKumar4
AshishKumar4 marked this pull request as draft August 14, 2026 15:58
@github-actions github-actions Bot added the kernel Changes to the Workshop kernel label Aug 14, 2026
@cloudflare cloudflare deleted a comment from ask-bonk Bot Aug 14, 2026
@cloudflare cloudflare deleted a comment from ask-bonk Bot Aug 14, 2026
Comment thread packages/gatekeeper-context/src/library-gatekeeper.ts Outdated
Comment thread packages/gatekeeper-context/src/library-gatekeeper.ts Outdated
Comment thread packages/gatekeeper-context/src/library-gatekeeper.ts Outdated
Comment thread packages/workshop-backend/src/agent-catalog.ts Outdated
Comment thread packages/workshop-backend/src/agent-catalog.ts Outdated
Comment thread packages/workshop-shared/src/gatekeeper.ts
@AshishKumar4 AshishKumar4 changed the title Draft: Fix: agents seeing highly truncated list of collections and skills Stop dropping Context collections from the agent catalog Aug 14, 2026
@cloudflare cloudflare deleted a comment from ask-bonk Bot Aug 14, 2026
The agent saw a truncated Context Library: with hundreds of skills,
collections went missing from its discovery catalog.

Two causes. `AGENT_CATALOG_MAX_ENTRIES` was 25, which a library with a
handful of public collections already exceeds. And `normalizeAgentCatalog`
sorted the combined list by title *before* clamping it, so which entries
survived was decided by alphabetical position — collections competed with
skills on title, and a collection titled "Zulu" lost to every `aa-*` skill
name. Raising the cap alone would only move that threshold.

So the clamp now drops from the tail and sorts the survivors, making the
gatekeeper's ordering the priority signal, and the Context gatekeeper lists
its collections ahead of its skills. Collections are the agent's entry
points; individual skills past the cap stay reachable through the session's
list()/search().

The cap goes to 200 rather than higher because the catalog is inlined in the
system prompt on every turn and compaction never reaches it, so the entry
count is a direct, permanent context cost (~80 KB at 200 entries).

`boundAgentCatalog()` and `AgentCatalogRequest` go away with it. The request
limit was always AGENT_CATALOG_MAX_ENTRIES, a constant both sides already
import, so it carried no information, and the provider-side clamp only
duplicated the Workshop's. One bound, in the kernel, on untrusted input.
@AshishKumar4
AshishKumar4 force-pushed the fix/collections-truncated branch from 7180f71 to 2835253 Compare August 14, 2026 21:45
@ask-bonk

ask-bonk Bot commented Aug 14, 2026

Copy link
Copy Markdown

Confirmed: skills are internally sorted, collections are internally sorted, and they're concatenated collections-first. Since the Workshop drops from the tail, collections survive unless collection count alone exceeds 200 — the intended guarantee.

I've completed a thorough review of this PR. Here's my assessment:

Summary

The change is coherent, correct, and well-contained. It fixes the "collections dropped from the agent catalog" bug via two mechanisms:

  1. normalizeAgentCatalog switches from sort-then-slice to slice-then-sort (agent-catalog.ts:43-45). This makes the gatekeeper's provider ordering the priority signal, so tail-drop discards the last-listed entries rather than the alphabetically-last ones.
  2. The Context gatekeeper lists collections before skills (library-gatekeeper.ts:338), so a large skill set can no longer push collections past the cap.

Verification

  • No dangling references: boundAgentCatalog/AgentCatalogRequest are fully removed with no remaining importers (checked repo-wide).
  • Signature change is consistent across all three sides: both gatekeepers (scheduler.ts:272, library-gatekeeper.ts:319) drop the request param, and the overseer call site (overseer.ts:4866) drops the argument. The derived CatalogGatekeeperFacet type flows correctly with no wire-format change.
  • Logging follows conventions: module-scoped createWorkshopLogger, concrete dot-separated event: "agent.catalog.truncated", and size is a real member of WorkshopObservabilityFields (observability.ts:27).
  • All prior review findings resolved: no unused VENDOR_ID/obsContext imports, doc comments updated (gatekeeper.ts:716-723), security/defense rationale retained and clarified (agent-catalog.ts:19-27).
  • The new test genuinely distinguishes old vs new behavior (agent-catalog.test.ts:41-56): the "keep" entry with a last-sorting title survives the cap only under slice-then-sort. The deleted boundAgentCatalog suite's id-length assertion is now covered by the extended normalizeAgentCatalog test (agent-catalog.test.ts:24).
  • PR body matches the code as provided to me (25→200, symbols removed) — the stale-metadata note in earlier bot comments referred to an intermediate commit.

The one non-blocking design tradeoff (if skills alone reach 200, collections could still be dropped) is genuinely mitigated by collections-first ordering: collections survive unless the collection count itself exceeds 200, a much stronger guarantee than before.

No new actionable code defects found.

LGTM!

github run

@cloudflare cloudflare deleted a comment from ask-bonk Bot Aug 14, 2026
@cloudflare cloudflare deleted a comment from ask-bonk Bot Aug 14, 2026
@AshishKumar4
AshishKumar4 marked this pull request as ready for review August 14, 2026 21:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gatekeeper Changes to a gatekeeper integration kernel Changes to the Workshop kernel workshop/shared Changes to shared Workshop APIs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant