Skip to content

PRDCT-490: new top-level CLI section — kbagent#1015

Open
Iamfle4ka wants to merge 13 commits into
mainfrom
PRDCT-359-cli-section
Open

PRDCT-490: new top-level CLI section — kbagent#1015
Iamfle4ka wants to merge 13 commits into
mainfrom
PRDCT-359-cli-section

Conversation

@Iamfle4ka

@Iamfle4ka Iamfle4ka commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

New section: the kbagent CLI

Adds a top-level CLI section for kbagent (github.com/keboola/cli) — Keboola's agent-friendly command-line interface. Help docs previously had no CLI section, only outbound links to the legacy Keboola-as-Code CLI on developers.keboola.com. kbagent is a different, newer tool; the legacy links are kept (that tool is still supported).

Pages (src/content/docs/cli/)

  • Overview — what kbagent is, why it's different (workflow-oriented, multi-project, dev-branch propagation, GitOps sync, agent sandboxing), who it's for, and how it relates to MCP server / AI Kit / Kai.
  • Getting started — install (curl / uv tool install), connect single project / whole org, doctor, first commands.
  • Commands — the command groups (project/config/job/storage/flow/sync/branch/workspace/agent/…).
  • Workflows — dev branches, GitOps sync, Storage files, encryption, the permission firewall, and real use cases.
  • For AI agents — Claude Code plugin + /keboola subagent, read-only sandboxing, kbagent context, conversation ID.

Verified live (locally, kbagent v0.66.0)

--version, --help (all command groups), per-group --help, and doctor (pre-auth) — real output captured. The firewall flags (--deny-writes / --deny-destructive) and command-group structure are verified against v0.66.0.

Flagged for owner review (TODO(human-review) / VERIFY(owner))

Auth-dependent facts (single-project token type, org-setup flags), exact flags on job run / config search / workspace from-transformation, the plugin install string, and the legacy-CLI deprecation timeline. Real output of authenticated read-only commands (project/config/job list against a demo project) still to be captured — the CLI needs a token, which the author will supply locally.

Nav

New top-level CLI group after Transformations. Section name "CLI" is a default — open for Jordan.

Verification

npm run build clean (263 pages, +5); node scripts/audit-phase2.mjs MISSING IMAGES 0, no new broken links.

Draft — do not merge.

🤖 Generated with Claude Code

Note for reviewers: overview/index.md shows as fully rewritten due to CRLF→LF line-ending normalization — the only content change there is the new Keboola CLI subsection.

Nikita and others added 3 commits July 13, 2026 12:26
…agent)

New section for kbagent, Keboola's agent-friendly command-line interface
(github.com/keboola/cli) — distinct from the legacy Keboola-as-Code kbc.
Overview (what it is, why, who for, how it relates to MCP/AI Kit/Kai) and
getting-started (install, connect single/org, doctor, first commands).
Install + version + doctor output verified locally (v0.66.0). Nav: new
top-level CLI group after Transformations.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- commands: the command groups (project/config/job/storage/flow/sync/
  branch/workspace/agent/…) verified against v0.66.0 kbagent --help.
- workflows: dev branches, GitOps sync, storage files, encryption, the
  --deny-writes/--deny-destructive firewall (verified), + real use cases.
- for-agents: Claude Code plugin + /keboola subagent, read-only
  sandboxing, kbagent context, conversation ID.
Facts not verifiable without auth flagged VERIFY(owner).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Point the overview 'Keboola CLI' section and the AI tooling pages at the
new /cli/ section; keep the legacy Keboola-as-Code link (different tool).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@linear-code

linear-code Bot commented Jul 13, 2026

Copy link
Copy Markdown

PRDCT-359

PRDCT-490

@vercel

vercel Bot commented Jul 13, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
connection-docs Ready Ready Preview, Comment Jul 24, 2026 6:49pm

Request Review

Replaced illustrative examples with real read-only output captured
against project 264 (L0 - Shopify), all via --deny-writes:
- getting-started: connected doctor (all PASS), real job list, and the
  global-search-not-enabled -> --search-type config-based behavior.
- commands: real config-based search result + a --json project list
  block showing machine output.
Write/create commands (job run, workspace from-transformation) stay
flagged VERIFY — not executed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Screenshots covering the key steps so a reader sees how kbagent works:
- Terminal (real output rendered with charmbracelet/freeze, project 264):
  doctor-connected in getting-started, job list in getting-started,
  config-based search in commands.
- New page cli/web-ui.md: the optional 'kbagent serve --ui' browser
  dashboard, with real screenshots (dashboard / configs / jobs) captured
  from the running Web UI against project 264. Nav entry added.
All from the dummy-data demo project; no real-customer data.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per review — pages were thin and the split wasn't cleanly Diátaxis.
Restructured into a linear sequence, one type per page, each fuller:
- Overview (hub + tool comparison + map)
- Get Started (tutorial: numbered first run + multi-project/org)
- How It Works (NEW concepts page: connection/config model,
  multi-project, dev-branch propagation, GitOps, permission firewall,
  agent-first design)
- How-to Guides (recipe-structured: org onboard, dev branch, GitOps,
  audit, job monitor, CI/CD tokens, encrypt, safe/sandboxed run)
- Use with AI Agents
- Command Reference (fuller: global flags, every group + real
  subcommands verified against v0.66.0, error-code categories)
- Web UI
Each page ends with a Next/Related link so the set reads in order.
Screenshots preserved. Nav reordered to match the sequence.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Overview: expanded from a stub into a real landing page (what you can
  do, in one minute, why a CLI vs the UI, section map).
- Get started: per-OS install (macOS/Linux curl, Windows PowerShell uv,
  any-OS uv, uvx one-off) so Windows users aren't stuck; Windows/uvx
  lines flagged for owner confirmation.
- For AI agents: Claude Code plugin install is now a prominent callout;
  noted it installs via marketplace command (not a downloadable file),
  and code blocks already have copy buttons.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Install step is now interactive macOS/Linux/Windows tabs (Starlight
<Tabs syncKey='os'>) instead of stacked subsections — pick your OS, see
the matching commands. Code blocks inside the tabs keep syntax
highlighting + copy buttons.

Enables MDX site-wide to allow the component: added @astrojs/mdx (already
present transitively) after starlight() in astro.config so
expressive-code still applies; getting-started.md -> .mdx with HTML
comments converted to MDX. Build + audit clean; markdown-remark pin
unaffected.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Iamfle4ka Iamfle4ka changed the title docs(cli): new top-level CLI section — kbagent PRDCT-490: new top-level CLI section — kbagent Jul 15, 2026

@padak padak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: CLI section vs. kbagent v0.66.1 source

I reviewed the rendered preview (Vercel) and cross-checked every factual claim against the actual kbagent source at keboola/cli HEAD (v0.66.1). Overall this is accurate and well-structured — the vast majority of claims check out against the code, including several the author flagged as unverified. Below: 4 factual fixes, answers to the open TODO(human-review) / VERIFY(owner) markers (most can now be resolved), and a few nits.

Factual fixes needed

  1. web-ui.md — "kbagent prints the local URL (default http://127.0.0.1:8001/) and opens it"
    kbagent serve --ui does not auto-open a browser — there is no webbrowser call anywhere on the serve path (only lineage server opens a browser). The port, the HttpOnly session cookie, and localhost-only default are all correct. Fix: "…prints the local URL (default http://127.0.0.1:8001/) — open it in your browser."

  2. commands.md — "kbagent defines ~52 codes across 14 categories"
    The ErrorCode enum currently has 66 members; the linked docs/error-codes.md lists 46 (it has drifted — we'll regenerate it on the CLI side, tracked separately). The 14 categories are right. Recommendation: drop the number entirely ("kbagent defines stable string codes across 14 categories … the full list is in docs/error-codes.md") so this page can't go stale the same way.

  3. getting-started.mdx Step 4 — kbagent config list # configurations in the default project
    Without --project, config list (and job list) fan out across all connected projects, not the default project — resolve_projects(None) returns every connection. With one project connected the output is identical, but the comment teaches the wrong mental model (and contradicts concepts.md, which correctly says reads fan out). Suggest: # configurations across all connected projects.

  4. workflows.md "Work in a development branch" — kbagent sync branch-link --project analytics
    branch-link requires a sync workspace initialized in git-branching mode; outside one it fails with Git-branching mode is not enabled. Run 'sync init --git-branching' first. Add the prerequisite (kbagent sync init --project analytics --git-branching, inside a git repo) or link the GitOps recipe. Bonus worth stating: with no --branch-id/--branch-name, it maps the current git branch name to a same-named Keboola branch, creating it if absent — that's the whole point and currently implicit.

Answers to the open TODO/VERIFY markers (verified against source)

  • Plugin install string (for-agents.md): /plugin install kbagent@keboola-agent-cli is correct — marketplace name is keboola-agent-cli (.claude-plugin/marketplace.json), plugin name kbagent. The /keboolakeboola-expert subagent description is accurate too. VERIFY(owner) can be removed.
  • Single-project token type (getting-started.mdx Step 2): a Storage API token is correct and sufficient for browsing. Worth one added sentence: some commands need an admin (master) token — e.g. token create requires canManageTokens, and branch/workspace creation needs admin privileges.
  • org setup flags (getting-started.mdx, workflows.md): all verified correct — --allow-env-manage-token is required to read KBC_MANAGE_API_TOKEN (default-deny since 0.29.0, not 0.28.0 as the hidden comment says), and --org-id / --project-ids / --yes are all real. Note --token-expires-in lives on project refresh, not on org setup.
  • config search flags (workflows.md): --query / --ignore-case correct; cross-project fan-out when --project is omitted confirmed. VERIFY can be removed.
  • token create/refresh (workflows.md): correct — --expires-in is seconds (omit = never expires), secret shown once, token refresh --token-id rotates. VERIFY can be removed.
  • encrypt values (workflows.md): the required flags are --component-id ID and --input JSON|@file|-. Suggest completing the example: kbagent encrypt values --project prod --component-id keboola.ex-db-snowflake --input '{"#password": "secret"}'.
  • branch create / branch merge (workflows.md): both claims verified correct — branch create auto-activates the new branch, and branch merge generates the KBC UI merge URL (it also resets the active branch back to main afterwards, which might be worth a half-sentence).
  • uvx invocation (getting-started.mdx): uvx --from 'git+https://github.com/keboola/cli' kbagent is the right form; the console entry point is kbagent.
  • Windows (getting-started.mdx): install.sh is bash-only, so the uv-in-PowerShell path is the right recommendation. TODO can be resolved.
  • --version output: kbagent v0.66.0 format confirmed (typer.echo(f"kbagent v{__version__}")).

Verified correct (spot-checked against source — no action)

Global-flags table including the exact firewall semantics (--deny-writes spans write+destructive+admin; --deny-destructive deliberately leaves admin/write allowed — matches permissions.py); macOS config path via platformdirs and the .kbagent/ walk-up precedence; init --from-global --read-only (exists, and read-only also chmods the config); config delete (exists); search defaulting to Global Search with the --search-type config-based fallback; serve port 8001 / HttpOnly cookie / localhost-only; KBAGENT_CONVERSATION_IDX-Conversation-ID header; doctor reporting plugin installation; the {"status": "ok", "data": …} JSON envelope and the project list example keys; the four example error codes; install.sh at repo root; the keboola-cli[server] extras install line (matches README; --force for reinstall is a good touch).

Nits

  • Sidebar labels (src/sidebar.mjs): entries are bare slugs, so Starlight falls back to full page titles — the sidebar reads "Get started with kbagent / How kbagent works / kbagent how-to guides / kbagent command reference / kbagent Web UI" (five "kbagent"s). _data/navigation.yml already defines the short titles (Get Started / How It Works / How-to Guides / …); adding matching label:s in sidebar.mjs would align the two.
  • concepts.md config path: only the macOS path is given. Add the Linux one (~/.config/keboola-agent-cli/config.json) or note that kbagent doctor prints the active path.
  • overview/index.md: the whole file shows as rewritten because of a CRLF→LF normalization; the only content change is the Keboola CLI subsection. Fine to keep, but worth one line in the PR description so reviewers don't hunt through 177 changed lines.
  • getting-started.mdx "(auto-updates, version-pinnable)" next to the uv install: the auto-update is kbagent's own startup hook and applies to install.sh installs equally — the parenthetical reads as a uv feature.
  • commands.md agent group: consider adding run-detail, run-events, and prompt-improve to the list (all exist).

Nice work overall — especially the discipline of flagging unverified claims inline instead of shipping guesses; nearly all of them turned out to be correct.

@padak

padak commented Jul 20, 2026

Copy link
Copy Markdown
Member

Tone & positioning feedback (separate from the factual review above)

The section is accurate and well-organized, but it reads as a reference, not as "I have to try this." The copy is safe — and it undersells the capabilities that are objectively unique. This isn't a request for marketing adjectives; the tone should stay sober (it's help docs). The opportunity is to surface the genuinely differentiating features, which are currently buried in lists. Ordered by impact:

1. The hero story is buried

The overview opens with "one tool to manage all your Keboola projects from the terminal" — a generic CLI pitch. The actual differentiator appears halfway down the page: hand your whole org to an AI agent, safely. No other data-platform CLI is agent-first; that should be the first paragraph. Something like:

Your data platform now has a command line an AI can drive. Connect your whole organization, put a firewall around what the agent may touch, and let it audit, fix, and build across every project at once — while you keep the veto on anything destructive.

2. The coolest feature is one word in a bullet list

The agent group — scheduled AI agents running on cron against your projects (kbagent agent create --type ai_agent --cron "0 6 * * 1" --prompt "...") — is a single bullet under "Development" in commands.md. This is a capability competitors don't have: "every Monday at 6am, an AI agent reviews the weekend's failed jobs and posts a triage summary." It deserves its own how-to recipe in workflows.md, possibly its own page.

3. Lineage is a demo-able wow with six words of coverage

kbagent lineage does cross-project, column-level lineage with mermaid/HTML/ER output and a local interactive server — and in the docs it's just a name in the list "component, data-app, stream, sharing, lineage, kai". One screenshot of the HTML lineage graph would sell more than a page of text.

4. The recipes lack a story with scale

The Goal → commands → Result format in workflows.md works, but none of the recipes has the "aha" moment. Consider one capstone recipe:

Find every config across 40 projects that still points at the old warehouse, fix them in a dev branch, and review the change as a PR — five commands. (searchsync pull → edit → sync diffsync push)

That's precisely the thing the UI can't do, and it justifies the whole section's existence.

5. Smaller opportunities

  • workspace query --sql — "debug your transformation's SQL from the terminal without touching production" — is practically absent.
  • kai ask — "ask Keboola's AI about your project without leaving the shell" — is just "(beta)" in parentheses.
  • The Web UI page mentions the "ask the local AI" box in one sentence — that's a local AI dashboard over your whole org.
  • The "In one minute" block ends on a command, not a result — show a line of output as the payoff.
  • The terminal screenshots are static tables; an asciinema/GIF of Claude Code driving kbagent would be the strongest asset in the section (best fit: the for-agents page).

padak added a commit to keboola/cli that referenced this pull request Jul 20, 2026
…c in check-error-codes (#490)

docs/error-codes.md had drifted 20 codes behind the ErrorCode enum (missing
all Data Apps, Developer Portal, Flow, serve-envelope, and newer Sync/Auth
codes). The doc is publicly linked from the new help-docs CLI section
(keboola/connection-docs#1015), so completeness now matters.

- Add the 20 missing codes to their category sections; new sections: Flow,
  Data Apps, Developer Portal. DATA_APP_INVALID_REPO and
  DATA_APP_REPO_VALIDATION_BLOCKING are marked reserved (defined in the enum
  since 0.28.0 but not currently emitted; validate-repo signals via exit 1).
- Extend scripts/check_error_codes.py (make check-error-codes) with an
  enum-vs-doc completeness check: every ErrorCode member must appear in the
  doc and vice versa, so the doc cannot silently drift again. --list now
  reuses the shared enum parser.
- Tests: TestErrorCodesDocCompleteness in tests/test_integration.py (doc
  matches enum; missing-code and stale-code drift both detected).
…kers, tone pass

Factual (per review 4734182992): serve --ui doesn't auto-open the browser;
error-code count dropped (enum drifts); config list fans out across all
connected projects; branch-link needs sync init --git-branching and maps the
current git branch by default.

VERIFY/TODO markers resolved per owner verification (plugin string, token
types, org setup flags incl. 0.29.0 default-deny, config search, token
create/refresh, encrypt values example, uvx, Windows). Only Jordan's
legacy-CLI-timeline TODO remains.

Tone (comment 5021425109): agent-first hero on the overview; new recipes —
scheduled AI agents (cron), fix-a-config-everywhere capstone, workspace SQL
debugging; lineage gets its own reference bullet; kai ask surfaced;
"In one minute" ends on the result. Sidebar labels via frontmatter
sidebar.label (survives gen:sidebar).

New runnable examples fact-checked against keboola/cli source; required
flags added (agent create --name, workspace --component-id/--workspace-id,
kai ask -m). Lineage/GIF assets tracked as a follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Iamfle4ka

Copy link
Copy Markdown
Collaborator Author

@padak thanks for the source-level review — applied in 27ec0d8. Rundown:

Factual fixes (all 4):

  • web-ui.md — no auto-open; now "…prints the local URL — open it in your browser."
  • commands.md — error-code count dropped entirely, categories + docs/error-codes.md link stay.
  • getting-started.mdx Step 4 — config list comment now says "across all connected projects."
  • workflows.md branch recipe — added sync init --project analytics --git-branching prerequisite + the current-git-branch default behavior (agreed, that's the whole point — now stated).

Open markers: all VERIFY/TODO resolved per your verification (plugin string, token types + new admin-token sentence, org setup flags with the 0.29.0 correction, config search, token create/refresh, encrypt values completed with your example, uvx, Windows). Only Jordan's legacy-CLI-timeline TODO remains — that one's his call.

Nits: sidebar labels fixed via frontmatter sidebar.label (survives gen:sidebar regeneration); Linux config path + doctor note added; CRLF note added to the PR description; uv parenthetical reworded (auto-update correctly attributed to kbagent's startup hook); agent group list completed with run-detail / run-events / prompt-improve.

Tone comment — applied the text parts:

  • Overview hero now leads with the agent-first story (your wording, lightly trimmed).
  • New recipe Schedule an AI agent (your Monday-triage scenario) + agent bullet in the reference links to it.
  • New capstone recipe Fix a config everywhere at once (config search → branch → sync pull/diff/push → merge).
  • New recipe Debug transformation SQL from the terminal (workspace from-transformation + query --sql).
  • lineage got its own reference bullet (cross-project, column-level, mermaid/HTML/ER + lineage server); kai ask -m surfaced on the agents page; Web-UI AI box called out; "In one minute" now ends on the result.

New runnable examples were fact-checked against the source before pushing — that pass added the required flags your review didn't cover verbatim: agent create --name, workspace from-transformation --component-id, workspace query --project --workspace-id, and kai ask -m (the question is an option, not positional).

Follow-ups (not in this PR): lineage HTML-graph screenshot and an asciinema/GIF of Claude Code driving kbagent (needs a live run against the demo project) — will land as a separate PR; docs/error-codes.md regeneration is on the CLI side per your note.

Build clean (265 pages), audit: 0 new broken links / 0 missing images.

🤖 Generated with Claude Code

@Iamfle4ka
Iamfle4ka marked this pull request as ready for review July 20, 2026 11:54

@keboola-pr-reviewer-bot keboola-pr-reviewer-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: needs_human (risk 3/5) · profile docs

New CLI docs section that changes build config, site navigation, and makes many author-flagged-unverified product claims — needs human review.

Concerns:

  • astro.config.mjs: Build config changed: adds @astrojs/mdx integration affecting how the site builds.
  • _data/navigation.yml: Navigation/structure data changed: new top-level CLI group in nav and sidebar.mjs.
  • src/content/docs/cli/getting-started.mdx: Unverified product-behaviour claims; author-flagged auth facts and command flags not confirmed.
  • + 1 more concern — see Check Run summary.

Suggested reviewers: @keboola/docs

@padak

padak commented Jul 20, 2026

Copy link
Copy Markdown
Member

Re-review of 27ec0d8 — all prior points addressed; 3 broken commands in the new agent recipe

Verified the update against the preview and the CLI source again. All 4 factual fixes, the TODO/VERIFY resolutions, the nits (the frontmatter sidebar.label approach is the right call — survives gen:sidebar), and the tone changes are applied correctly, and the preview is rebuilt. workspace from-transformation / workspace query / kai ask -m examples all check out.

One new issue: the Schedule an AI agent recipe has three commands that exit 2 as written (workflows.md):

  1. agent create … --type ai_agent requires --cli. The flag validation (commands/agent.py::_action_from_flags) fails with "ai_agent action requires --cli {claude|codex|gemini} and --prompt TEXT." Add e.g. --cli claude.
  2. kbagent agent test (bare) fails"--type is required (one of: ai_agent, cli_command, mcp_tool) or pass --from-file." agent test previews an ad-hoc action definition; it takes no task ID and can't dry-run an already-created task. To run the created task immediately, the command is kbagent agent run <task-id>.
  3. kbagent agent runs (bare) fails — a task ID is required (positional or --id): kbagent agent runs <task-id>.

Corrected block:

kbagent agent create --name "Weekend triage" --type ai_agent \
  --cli claude --cron "0 6 * * 1" \
  --prompt "Review the weekend's failed jobs and post a triage summary"
kbagent agent run <task-id>      # run it once right now (create prints the ID)
kbagent agent list               # what's scheduled
kbagent agent runs <task-id>     # past runs (drill in with run-detail / run-events)

One operational caveat worth a sentence in the recipe's result: scheduled firing requires kbagent serve running — task CRUD and ad-hoc agent run are offline-first, but the cron loop that fires tasks on schedule lives in the server. "No human in the loop" holds only while serve is up.

Tiny follow-up on the error-codes paragraph in commands.md: it still says "across 14 categories" — that count will drift the same way the code count did (keboola/cli#490 adds Flow, Data Apps, and Developer Portal sections, making it 17). Suggest dropping the number there too.

Nikita and others added 2 commits July 20, 2026 14:54
…eenshot

agent create gets required --cli claude; agent test (ad-hoc only) replaced
with agent run <task-id>; agent runs takes the task ID; result notes that
scheduled firing needs kbagent serve running. Error-codes paragraph drops
the category count too (17 incoming via keboola/cli#490).

Adds the interactive lineage-browser screenshot (demo-project pipeline:
extractor tables -> two transformations -> BDM outputs) to the lineage
reference bullet.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Iamfle4ka

Copy link
Copy Markdown
Collaborator Author

@padak fixed in 39b761d — thanks for catching the agent recipe before a reader did:

  • agent create example now includes --cli claude
  • agent test (ad-hoc only) swapped for kbagent agent run <task-id> with a note that create prints the ID
  • agent runs <task-id> — ID added
  • Result line now states the cron loop lives in the server: scheduled firing needs kbagent serve running; task CRUD and ad-hoc agent run work without it
  • Error-codes paragraph: category count dropped too ("grouped into categories (…, and more)") — nothing left there to drift when docs: complete error-codes.md (46 -> 66 codes) + enforce enum-doc sync cli#490 lands

Bonus in the same commit: the lineage browser screenshot you suggested is in — the demo project's pipeline (extractor tables → two transformations → BDM outputs, 15 nodes) on the lineage reference bullet. The Claude-Code-driving-kbagent GIF is parked for now.

Build clean, preview rebuilding.

🤖 Generated with Claude Code

@padak padak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified 39b761d against the CLI source and the rebuilt preview: the agent recipe now runs as written (--cli claude present, agent run/runs take the task ID, the kbagent serve requirement for scheduled firing is stated), the error-codes paragraph has no counts left to drift, and the lineage screenshot renders (HTTP 200 on the preview). All findings from the source-level review, the tone pass, and both delta re-reviews are addressed. Approving.

@jordanrburger

Copy link
Copy Markdown
Contributor

@padak - WDYT about figuring out how to keep the kbagent docs as the canonical source of truth? I.e. should we slightly modify the docs in your repo so we can borrow them more directly instead of duplicating them? My fear is that it will get outdated quickly.

@jordanrburger

Copy link
Copy Markdown
Contributor

@padak - WDYT about figuring out how to keep the kbagent docs as the canonical source of truth? I.e. should we slightly modify the docs in your repo so we can borrow them more directly instead of duplicating them? My fear is that it will get outdated quickly.

@claude - Can you review this PR from this perspective? What are our options to not be duplicating content?

@padak

padak commented Jul 20, 2026

Copy link
Copy Markdown
Member

@jordanrburger Good instinct — the drift fear is justified (my review of this PR caught exactly that kind of drift in the other direction: the CLI repo's own docs/error-codes.md was 20 codes stale). But "borrow directly" is the right answer for only part of this content. The useful split is by drift speed:

Content class Drift speed Examples in this PR Right strategy
Command signatures & flags Fast (every CLI release) commands.md group/subcommand lists Generate from the CLI, never hand-write
Behavioral facts Medium ports, config paths, firewall semantics, defaults Hand-write + CI freshness check
Concepts, positioning, recipes Slow index.md, concepts.md, workflows.md Hand-write here; different audience than repo docs

One honest correction to "borrow from the CLI repo"

The CLI repo's docs (guide.md, use-cases.md, commands-reference.md, even kbagent context) are also hand-maintainedcontext is a curated f-string, not generated from the CLI's command tree. Importing them wholesale would swap one hand-maintained copy for another, plus they're written for contributors and AI agents, not help-docs readers. The only zero-drift source of truth that exists today is the Typer app itself (--help introspection). So single-sourcing the reference means building a small generator first, not just borrowing files.

Options considered

  1. Link-only (what this PR already does for error codes): cheapest, no drift, but the help docs stop being self-contained and GitHub markdown isn't styled/searchable for end users. Right for long tails (full error list), wrong for the main reference page.
  2. Build-time import of repo docs (git submodule / raw fetch / npm package): moves the duplication, doesn't remove it — the imported files are still hand-written, and coupling the docs build to another repo's HEAD makes builds non-hermetic. Only worth it after the source files are generated.
  3. Hybrid: generated reference + hand-written prose + freshness CI — my recommendation, below.

Proposed mechanism

CLI side (keboola/cli — I'll file the issue and can build it):

  • New scripts/gen_command_reference.py that introspects the Typer app and emits command-reference.md (groups, subcommands, flags, help text) — attached as a GitHub Release asset on every release, alongside the wheel. Since it's derived from the same objects that produce --help, it cannot drift by construction.
  • docs/error-codes.md is already guarded as of docs: complete error-codes.md (46 -> 66 codes) + enforce enum-doc sync cli#490 (enum↔doc completeness check in CI).

Docs side (connection-docs):

  • A scheduled GitHub Action fetches the artifact from the latest release tag (not main — docs should describe what users actually have) and opens an auto-PR when it changed. Reviewable diffs, hermetic builds, no runtime coupling. commands.md becomes a thin curated wrapper (the grouping prose + screenshots) around the generated core.
  • For the hand-written pages, a weekly freshness job: run kbagent --help / <group> --help in CI and grep that every documented flag still exists; open an issue on mismatch. That's the automated version of the source-level review this PR just went through, and it covers the "medium drift" class that generation can't.

What stays canonical where: the CLI repo remains canonical for facts (code, generated reference, error codes — updated in the same PR as the code, which is the whole point of keeping docs next to code); connection-docs remains canonical for the end-user narrative (concepts, recipes, positioning). Prose duplication between the two is fine when the audiences differ — it's only facts that must never be written twice, and the mechanism above makes every fact either generated or CI-checked.

Near-term: I'd ship this PR as-is (approved) — it's accurate today and the freshness risk starts at the next CLI release, which is exactly when the generator + auto-PR loop should be in place. I'll open the keboola/cli issue for the generator and link it here.

@padak

padak commented Jul 20, 2026

Copy link
Copy Markdown
Member

Generator issue filed as promised: keboola/cli#498 (introspect the Typer app → command-reference.md release asset + drift CI). Once it ships, the scheduled auto-PR loop on this side can be wired up.

@padak

padak commented Jul 20, 2026

Copy link
Copy Markdown
Member

Update: the generator is implemented, not just filed — keboola/cli#500 (closes keboola/cli#498). It introspects the Typer command tree and the release workflow attaches command-reference.md as a Release asset generated from the exact wheel being shipped (238 commands / 27 groups at v0.70.1). A release is being cut today; if #500 lands before the tag, the first version these help docs describe already carries the zero-drift asset, and the auto-PR loop on this side can be wired against it.

padak added a commit to keboola/cli that referenced this pull request Jul 20, 2026
…yper app (#498) (#500)

* feat(release): generate command-reference.md release asset from the Typer app (#498)

The CLI's command reference exists only as hand-maintained copies (kbagent
context, CLAUDE.md, commands-reference.md, and now the help.keboola.com CLI
section from keboola/connection-docs#1015) -- every one a documented
silent-drift risk. This adds the zero-drift source: a generator that
introspects the live Typer/Click command tree, so the output cannot disagree
with the shipped CLI.

- scripts/gen_command_reference.py: walks the command tree (duck-typed via
  commands/repl._is_group -- Typer >=0.25 vendors Click, so isinstance
  against plain click classes fails), emits deterministic markdown: global
  options, one section per group, per-command option tables with required
  markers. Hidden commands/aliases (sl) are skipped, matching
  check_command_sync.py. 238 commands / 27 groups at v0.70.1.
- release.yml: after the wheel build, generate the reference FROM the built
  wheel (uv run --no-project --with dist/*.whl) and upload
  command-reference.md as a release asset next to the wheel, so consumers
  fetch it pinned to a released version.
- Makefile: gen-command-reference target.
- cli.py: fix --allow-env-manage-token help text: default-deny shipped in
  0.29.0, not 0.28.0 (changelog is authoritative) -- this exact string was
  the source of a wrong version in the help-docs PR.
- tests/test_gen_command_reference.py: determinism, completeness
  cross-checked against check_command_sync.collect_commands(), required
  markers, hidden-alias and --help exclusion, version header.

Closes #498.

* fix(release): address Devin review -- backfill-safe workflow + dataclass walker

1. release.yml: the job checks out the release tag, so a workflow_dispatch
   backfill of a tag predating scripts/gen_command_reference.py would fail
   the generation step and block the wheel upload (the documented purpose of
   the manual trigger). The step now skips cleanly when the script is absent,
   and command-reference.md uploads only when actually produced.
2. gen_command_reference.py: _walk returned a bare 3-tuple against the
   CONTRIBUTING.md multi-value-return rule; replaced with a frozen
   LeafCommand dataclass. Output byte-identical.
3. Test loader registers the script module in sys.modules before exec --
   @DataClass under 'from __future__ import annotations' resolves its module
   namespace via sys.modules at class-creation time.
@padak

padak commented Jul 20, 2026

Copy link
Copy Markdown
Member

How the zero-drift loop works now that keboola/cli#500 is merged

For @jordanrburger and @Iamfle4ka — here's exactly what happens per release and what this repo can wire against it.

What the CLI side now does automatically (shipped)

On every published release of keboola/cli, the release workflow checks out the exact tagged commit, builds the wheel, and runs the new generator against that built wheel — introspecting the same Typer command tree that renders --help. The result is attached as a release asset, so it cannot disagree with the shipped CLI by construction. Two stable URLs from then on:

  • https://github.com/keboola/cli/releases/latest/download/command-reference.md — always the latest stable release
  • https://github.com/keboola/cli/releases/download/v<X.Y.Z>/command-reference.md — pinned

Content: global options, every visible command group (27 today), every command (238 today) with an option table incl. required markers. Hidden aliases are excluded. Output is deterministic, so diffs between releases read as clean changelogs ("v0.71.0 added flow schedule --timezone").

What this repo can wire against it (proposed, not yet built)

A scheduled action that keeps a committed copy in sync and opens a reviewable PR when a release changes it — no silent drift, no manual copying:

# .github/workflows/sync-cli-reference.yml
name: Sync kbagent command reference
on:
  schedule: [{ cron: "0 7 * * *" }]
  workflow_dispatch:
jobs:
  sync:
    runs-on: ubuntu-latest
    permissions: { contents: write, pull-requests: write }
    steps:
      - uses: actions/checkout@v5
      - name: Fetch latest released reference
        run: |
          curl -fsSL -o /tmp/command-reference.md \
            https://github.com/keboola/cli/releases/latest/download/command-reference.md
      - name: Open PR if changed
        uses: peter-evans/create-pull-request@v7
        with:
          add-paths: <wherever the imported copy lives>
          branch: sync/cli-command-reference
          title: "sync: kbagent command reference from latest release"
          commit-message: "sync: kbagent command reference from latest CLI release"

Design decisions that are yours to make on this side:

  1. Where the imported content lands. Options: (a) render it as a subpage under cli/commands/ (full generated reference, linked from the curated page), or (b) keep commands.md fully hand-written as it is now — curated groups, screenshots, prose — and use the fetched file only as a CI freshness check (fail/notify when a documented command or flag no longer matches). Given how good the curated page already is, I'd start with (b): lower effort, keeps the editorial quality, still kills silent drift. (a) can come later if you want flag-level detail on the site.
  2. Latest vs pinned. releases/latest is the right default — the auto-update hook keeps virtually all users on the latest stable, so that's what the docs should describe.
  3. Review stays human. The action opens a PR; a human merges. The diff is small and legible by construction.

What stays hand-written (deliberately)

The narrative pages (overview, concepts, recipes, for-agents) — different audience, slow drift. The generated asset covers the one content class that drifts every release: command signatures. For the narrative pages the guard is the periodic fact-check (the thing this PR's review did manually); that can be a follow-up freshness job here or on the CLI side.

Happy to draft the sync workflow as a PR against this repo once you pick option (a) or (b).

@padak

padak commented Jul 20, 2026

Copy link
Copy Markdown
Member

First command reference https://github.com/keboola/cli/releases/tag/v0.71.0

@Iamfle4ka Iamfle4ka added the dev-docs-migration developers.keboola.com → help.keboola.com migration label Jul 21, 2026
- getting-started: inline token how-to (Project Settings -> API Tokens ->
  New Token) + live screenshot from demo project 264; tokens concept page
  demoted to a trailing details link (PRDCT-575)
- new /cli/troubleshooting/ page: 10 likely failures, symptom -> cause ->
  fix, with --json error codes; wired into nav + cross-linked from
  getting-started and the commands error-codes section (PRDCT-576)
- concepts: editable "how it works" SVG diagram (actors -> kbagent with
  config store and serve -> permission firewall -> stacks) (PRDCT-577)
- install: curl script marked Recommended on macOS/Linux, uv on Windows;
  landing aligned; provisional pending Padak's canon call (PRDCT-578)

All kbagent invocations pass the CLI reference gate (v0.76.1, 0 findings
after it caught a missing required --project on sync init); fact-checked
vs keboola/cli (29 claims confirmed, incl. serve powering cron agents).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Iamfle4ka

Copy link
Copy Markdown
Collaborator Author

Jerabek's review applied (ca843ebd) — the four in-docs items from his Slack pass:

  1. Token flow — Get Started now walks the token creation inline (Project Settings → API Tokens → New Token, Full Access, copy-once warning) with a live screenshot from the demo project; the generic tokens page is a trailing details link instead of the main path. His preferred fix — kbagent login browser auth — is a CLI-side feature request (no such command in v0.76.1), being raised with @padak separately.
  2. Troubleshooting — new /cli/troubleshooting/ page: the 10 most likely failures, each symptom → cause → fix with its --json error code, cross-linked from Get Started and the Commands error-codes section.
  3. Concepts diagram — editable SVG (you/agent → kbagent with config store and serve → permission firewall → stacks, org-wide fan-out) at the top of How It Works.
  4. Install canoncurl … install.sh marked Recommended on macOS/Linux, uv on Windows; landing snippet aligned. Provisional pending @padak's call (flagged VERIFY(owner) in-page) — flip to uv-first is a two-line change.

Item 1 of his review (per-command generated reference at Stripe/GitHub-CLI level) is tracked as PRDCT-557 — its own PR after this one merges, rendering the auto-synced _data/cli/command-reference.md that already ships with the zero-drift loop (#1037).

Verified: build clean (266 pages); audit at baseline; every kbagent invocation passes the CLI reference gate against v0.76.1 (the gate caught one real error during writing — sync init without its required --project); all 29 factual claims fact-checked against keboola/cli source, including the riskiest one (scheduled agents run inside kbagent serve — README-confirmed).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dev-docs-migration developers.keboola.com → help.keboola.com migration

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants