Skip to content

feat(analyse): add Bash language support - #92

Merged
ubmarco merged 3 commits into
mainfrom
feat/bash-support-48
Jul 29, 2026
Merged

feat(analyse): add Bash language support#92
ubmarco merged 3 commits into
mainfrom
feat/bash-support-48

Conversation

@ubmarco

@ubmarco ubmarco commented Jul 10, 2026

Copy link
Copy Markdown
Member

Adds bash as a supported comment_type for marker extraction. Closes #48.

What

  • New CommentType.bash, discovering .sh, .bash, .zsh and .ksh files.
  • Bash has only # line comments, so extraction reuses the existing hash-comment path: a (comment) @comment tree-sitter query over the tree-sitter-bash grammar, with function_definition scope nodes.
  • Adds the tree-sitter-bash>=0.25.1 dependency.

Tests & docs

  • A default_oneliner_bash case in the shared oneline.yaml extraction fixture, with its captured snapshot.
  • Discovery coverage for the new extensions; the comment-type enum error-message tests updated.
  • Documented as a new FE_BASH feature, with the configuration / analyse / changelog pages updated.

Not included

Fish shell — it is not POSIX-compatible and no tree-sitter-fish grammar is published on PyPI, so it cannot be wired into this package. Can be revisited if a PyPI grammar becomes available.

@ubmarco

ubmarco commented Jul 11, 2026

Copy link
Copy Markdown
Member Author

Review

Must fix — the supported-styles table in configuration.rst is corrupted. The new Bash row was inserted mid-cell into the JSONC row: the continuation line (e.g. the mode line // -*- mode: jsonc -*-) now renders inside the Bash row's "discovered file types" cell, and the JSONC cell loses its example. Move the Bash row below that line. The RST stays syntactically valid, so the docs build can't catch this — worth an eyeball of the rendered table.

Test coverage

  • features.rst:255 guarantees that shebang lines never produce spurious markers, but nothing asserts it. Cheapest cover: a second bash case in the declarative oneline.yaml fixture whose source starts with #!/bin/bash. The function name { } definition form is likewise unexercised (the fixture uses greet() { … }).
  • FE_BASH lists "association of comments with function definitions" as a key capability, backed by SCOPE_NODE_TYPES:48; the fixture executes that path but nothing asserts the resulting scope.

Otherwise clean. Wiring is complete and consistent: enum + file-type map, query + init_tree_sitter branch, both hard-coded comment_type error-message tests updated, discovery test covers all four extensions, docs and changelog updated. The change is additive — the only user-visible behavior change besides the feature is the schema-validation error string now listing bash. CI is green across the matrix; no earlier review comments to address.

@codecov-commenter

codecov-commenter commented Jul 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.45455% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 92.38%. Comparing base (327f65b) to head (f73507f).

Files with missing lines Patch % Lines
src/sphinx_codelinks/analyse/utils.py 80.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #92      +/-   ##
==========================================
+ Coverage   92.34%   92.38%   +0.04%     
==========================================
  Files          43       43              
  Lines        3721     3743      +22     
  Branches      380      381       +1     
==========================================
+ Hits         3436     3458      +22     
  Misses        172      172              
  Partials      113      113              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ubmarco

ubmarco commented Jul 11, 2026

Copy link
Copy Markdown
Member Author

Review findings addressed in f604413 and fe82065:

  1. Table fixed (f604413) — the JSONC example continuation line is back in the JSONC row's cell, and the Bash row now follows it cleanly: configuration.rst:325-332.
  2. Shebang and function keyword form covered (fe82065) — new declarative case shebang_oneliner_bash with its captured snapshot: the shebang comment yields no marker and no warning, the need anchors to line 2, and function greet { echo hi; } parses.
  3. Scope association asserted (fe82065) — new test_find_associated_scope_bash following the existing per-language pattern: comments associate with the function_definition for both definition syntaxes, plus the enclosing-scope fallback for a comment inside a function body.

Verified with the full tox suite (267 passed, 34 snapshots), mypy, and the pinned pre-commit hooks on the changed files; CI is green on fe82065.

HartmannNico added a commit to HartmannNico/sphinx-codelinks that referenced this pull request Jul 22, 2026
…attern

- Fix test_source_discover + test_src_trace: add 'markdown' to expected
  comment_type validation error message list
- Add FE_MARKDOWN feature need to docs/source/components/features.rst
  (fixes docs build warnings for IMPL_MD_1/IMPL_MD_3/IMPL_LANG_1)
- Add default_oneliner_markdown fixture to tests/data/extraction/oneline.yaml
  with end_sequence: " -->" config
- Generate snapshot for the markdown extraction fixture
- Add init_markdown_tree_sitter fixture + test_extract_comments_markdown
  + test_init_tree_sitter_markdown to tests/test_analyse_utils.py
- Add 'markdown' entry to LANG_MAP in tests/test_extraction_fixtures.py
- Update docs: configuration.rst (supported values + table row),
  analyse.rst (language support list), change_log.rst (Unreleased entry)
ubmarco added 3 commits July 28, 2026 11:46
Add `bash` as a supported comment-type for marker extraction, discovering
`.sh`, `.bash`, `.zsh` and `.ksh` files. Bash has only `#` line comments,
so it reuses the existing hash-comment extraction path (BASH_QUERY +
`function_definition` scope nodes) via a tree-sitter-bash grammar.

A `default_oneliner_bash` case is added to the shared `oneline.yaml`
extraction fixture with its captured snapshot, and the feature is
documented under `FE_BASH`.

Fish shell is intentionally excluded: no tree-sitter-fish grammar is
published on PyPI, so it cannot be wired into this Python package.

Closes #48
… row

The bash row was inserted between the JSONC row's last cell and its
continuation line, so the mode-line example rendered inside the bash
"discovered file types" cell and the JSONC cell lost it.
…ssociation

- shebang_oneliner_bash extraction fixture: the shebang comment yields no
  marker or warning, the marker anchors to line 2, and `function name { }`
  definitions parse
- test_find_associated_scope_bash: comments associate with the
  function_definition for both definition syntaxes, including the
  enclosing-scope fallback for comments inside a body
@ubmarco
ubmarco force-pushed the feat/bash-support-48 branch from fe82065 to f73507f Compare July 28, 2026 09:47
@ubmarco
ubmarco marked this pull request as ready for review July 28, 2026 09:47
@ubmarco
ubmarco merged commit 935d8df into main Jul 29, 2026
13 checks passed
@ubmarco
ubmarco deleted the feat/bash-support-48 branch July 29, 2026 07:37
ubmarco added a commit that referenced this pull request Jul 30, 2026
Follow-up to #97. Adds the traceability artifacts for the substantial
features that landed since 1.3.0 without any.

## Audit

Every merged PR in the `1.3.0` → `1.4.0` window, checked against
`features.rst` and the `@` markers in `src/`:

| PR | Substantial feature? | Traceability |
|---|---|---|
| #92 Bash language support | yes | ✅ `FE_BASH` + `FAULT_BASH_1/2` +
`IMPL_BASH_1/2/3` + added to `IMPL_LANG_1` |
| #85 Preprocessor-aware C/C++ engine (libclang) | yes | ❌ **none** — no
feature need, no markers |
| #89 Marker anchoring fix | no (fix to existing traced code,
`IMPL_OLP_1`) | n/a |
| #83 Declarative fixture tests | no (test infra, no `src/` change) |
n/a |
| #84 / #86 Dependency caps | no | n/a |

So the libclang engine was the one gap: 4 new modules and 353 lines in
`analyse/preproc/`, plus wiring in `analyse.py` / `config.py` / `cmd.py`
/ `src_trace.py`, with zero traceability.

## Changes

**`docs/source/components/features.rst`** — new `FE_PREPROC` feature
need, placed after the language-support block, with three `fault`
children:

- `FAULT_PREPROC_1` — objects in inactive branches are extracted (false
positives, the defect the engine exists to prevent)
- `FAULT_PREPROC_2` — objects in active branches are dropped (false
negatives)
- `FAULT_PREPROC_3` — flags resolved from the wrong compilation database
entry, so the wrong branches evaluate as active

**Six `@` markers**, one per capability listed in the need, so the
mapping is 1:1 and reviewable:

| Marker | Site | Capability |
|---|---|---|
| `IMPL_PREPROC_1` | `analyse.py::create_src_objects_libclang` | engine
entry / opt-in |
| `IMPL_PREPROC_2` |
`preproc/libclang_parser.py::extract_active_comments` | drop
inactive-branch comments |
| `IMPL_PREPROC_3` | `preproc/compile_db.py::find_compile_db` | walk-up
database discovery |
| `IMPL_PREPROC_4` | `preproc/compile_db.py::load_flags_map` | per-file
compiler flags |
| `IMPL_PREPROC_5` | `preproc/compile_db.py::defines_to_args` |
standalone header parsing |
| `IMPL_PREPROC_6` | `preproc/loader.py::load_clang_cindex` |
optional-dependency guard |

Comments only — no logic touched.

## Verification

A green `-nW` build alone would not prove the markers work: a marker
that fails to parse produces no need and no error. So the built output
was checked directly.

```
$ tox -e docs-clean          # sphinx-build -nW --keep-going
build succeeded.

$ grep -o 'id="IMPL_PREPROC_[0-9]"' docs/_build/html/development/traceability.html | sort -u
id="IMPL_PREPROC_1" … id="IMPL_PREPROC_6"        # all six needs materialized

$ grep -o 'href="[^"]*#FE_PREPROC"[^>]*title="IMPL_PREPROC_[0-9]"' … | sort -u
IMPL_PREPROC_1 … IMPL_PREPROC_6                  # all six link to the feature
```

- `tox -e py312-sphinx8-needs5` — 345 passed, 1 skipped, 58 snapshots
passed
- `tox -e mypy` — no issues in 20 source files
- `pre-commit run --files …` — all hooks pass

## Deliberately out of scope

Pre-existing gaps, all predating the 1.3.0 → 1.4.0 window this PR was
scoped to. Flagging rather than fixing, since each is a separate
judgment call:

- **`FE_CSHARP` is an orphan.** It has no incoming `impl` link — it is
missing from the `IMPL_LANG_1` list in `analyse/utils.py:110` even
though `init_tree_sitter` handles `CommentType.cs` (utils.py:122). C# is
the only supported language whose implementation is untraced. One-word
fix if you want it here.
- **No feature need for the `src-trace` directive, nor for source-link
URL generation** (`local-url` / `remote-url`), while the CLI has three
(`FE_CLI_*`). This asymmetry may be intentional — capability-level needs
covering *what* is traced regardless of interface — so changing it is a
feature-model decision, not a gap fix.
- **The test-needs layer is empty.** `docs/ubproject.toml` defines a
`test` type and `development/traceability.rst` runs a `src-trace` over
the `tests` project, but `tests/` carries no `@` markers, so zero
`TEST_` needs exist.

## Note on sequencing

No changelog entry here, deliberately: this PR is based on `main`, where
the `Unreleased` heading still holds the Bash entry that #97 converts
into the dated `1.4.0` section. Editing it would conflict with the open
release PR. Since these artifacts document a feature already listed in
the 1.4.0 notes, either fold a line into #97 before merging it, or let
the next release's notes pick this up — your call on the order.
patdhlk pushed a commit that referenced this pull request Jul 30, 2026
Prepares the **1.4.0** release.

## Release changes

- ⬆️ Bump version `1.3.0` → `1.4.0` in `pyproject.toml`.
- 📝 Finalize the changelog: convert the `Unreleased` heading into a
dated `1.4.0` section (released 30.07.2026). Only 1 of the 6 PRs merged
since 1.3.0 (#92) had added a changelog entry, so this also documents
the 5 that were missing one, plus the traceability artifacts from #98.
- 🗺️ No roadmap changes — none of the merged PRs completed a listed
roadmap item.

Minor bump: two new user-facing features, no removals or config-breaking
changes.

## Changelog for 1.4.0

**New and Improved**

- ✨ Added Bash language support for the `analyse` module (#92)
- ✨ Added an opt-in preprocessor-aware C/C++ extraction engine, powered
by libclang (#85)
- 📚 Traced the preprocessor-aware C/C++ engine in the feature
documentation (#98)
- 🧪 Added a declarative fixture and snapshot test layer for marker
extraction (#83)

**Fixes**

- 🐛 Anchor newline-terminated one-line markers to the start of the
comment (#89, fixes #88)
- 🐛 Pinned `typer` and `sphinxcontrib-typer` to keep the documentation
build working (#84, #86)

## Behavior change worth calling out

#89 is released as a fix, but it does narrow marker matching: a
newline-terminated one-line marker (default `@…`) is now only recognized
when nothing but comment decoration and whitespace precedes the start
sequence. A project that deliberately embedded such markers *after*
prose on the same line will stop seeing them — the fix is what stops `@`
in prose from producing bogus need IDs and `InvalidNeedException`.
Explicitly bounded markers (e.g. `[[ … ]]`) are unaffected.

## Verification

Rebased onto `main` at #98, so this branch carries the full 1.4.0
content.

- `tox -e py312-sphinx8-needs5` — 345 passed, 1 skipped (58 snapshots
passed).
- `tox -e docs-clean` (`sphinx-build -nW --keep-going`) — build
succeeded, 0 warnings.
- `pre-commit run --files docs/source/development/change_log.rst
pyproject.toml` — all hooks pass.

## Not included

Pre-existing traceability gaps are left for a follow-up, since all
predate 1.3.0: `FE_CSHARP` has no incoming `impl` link; the five
enum-site markers carrying a trailing `;` (`IMPL_BASH_1`, `IMPL_GO_1`,
`IMPL_RUST_1`, `IMPL_RUST_2`, `IMPL_JSONC_1`) are inert and produce no
needs, so file-extension discovery is untraced for every language; and
`IMPL_C_1` / `IMPL_PY_1` / `IMPL_GO_2` are anchored on
`COMMENT_MARKERS`, which has no readers. See the discussion in #98.

## After merge

- Tag `1.4.0` on the merge commit (matching the existing `1.0.0`–`1.3.0`
tags). The `Release` workflow triggers on that tag pattern and publishes
to PyPI.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bash Support

3 participants