diff --git a/README.md b/README.md index abeab9c..d6c3048 100644 --- a/README.md +++ b/README.md @@ -14,26 +14,24 @@ The project is split across: ## ✨ Features -### Implemented - -- [x] **Go to Definition** — resolves signals, variables, parameters, templates, functions, and - components, including **cross-file** jumps through `include` statements, and jumping straight - into an included library file from its `"path.circom"` string. -- [x] **Hover** — shows the symbol kind and its declaration signature (header only for block-bodied - defs like `template`/`function`/`bus`). -- [x] **Completion** — in-scope body symbols, file top-level names, reserved keywords, and **member - completion** (`component.`) that resolves a component's template across files. -- [x] **Find References** — every occurrence of a symbol, resolved *semantically* (not text-matched), - so shadowing is respected. -- [x] **Rename** — scope-aware rename with `prepareRename` support; refuses keywords, include-path - strings, illegal names, and unresolved member-access fields. -- [x] **Error-recovering parser** — keeps working on invalid/partial circom files. -- [x] **Lazy, cached analysis** — parsing and symbol tables are memoized and invalidated only on real - edits; includes are read from disk once. -- [x] **Sandboxed includes** — `include` resolution is confined to workspace roots - (path-traversal / symlink-safe). - -> See [`TODO.md`](./TODO.md) for the roadmap of features not yet implemented. +CCLS provides **Go to Definition** (cross-file via `include`), **Hover**, **Completion** +(including `component.` member completion), semantic **Find References**, scope-aware +**Rename**, an **error-recovering parser**, lazy/cached analysis, and **sandboxed includes** +(path-traversal safe). + +See [`docs/features.md`](./docs/features.md) for the full list with per-feature details, and +[`docs/roadmap.md`](./docs/roadmap.md) for what is not yet implemented. + +--- + +## 📚 Documentation + +In-depth docs live in [`docs/`](./docs/README.md): + +- [Features](./docs/features.md) — what CCLS can do, and how each feature resolves symbols. +- [Architecture](./docs/architecture.md) — workspace layout, resolution core, source DB, VFS. +- [Roadmap](./docs/roadmap.md) — capabilities not yet implemented. +- [Per-crate deep dives](./docs/crates/) — `parser`, `syntax`, `lsp`, `vfs`. --- @@ -100,24 +98,16 @@ Optional, but recommended for snapshot testing. ## 🏗️ Architecture -``` -circom-language-server/ -├── crates/ -│ ├── parser/ # `logos` lexer + event-driven parser with markers -│ ├── syntax/ # `rowan` lossless syntax tree + typed AST -│ ├── vfs/ # Virtual file system (existence, text, change log) -│ └── lsp/ # LSP server: handlers, global state, resolver, semantic index -├── editors/code/ # VS Code extension (TypeScript, `circom-plus`) -└── xtask/ # Build & install tasks (`cargo xtask install …`) -``` - -Key design notes: - -- **Resolution core** (`resolver.rs` + `semantic.rs`) is name-based and shared by goto-definition, - hover, references, and rename — so the same symbol resolves consistently across features. -- **Source database** (`source_db.rs`) memoizes parse / file DB / symbol table per file, drained from - the VFS change log so editing file A never recomputes file B. -- Includes are loaded once from disk, cached, and confined to workspace roots. +A multi-crate Rust workspace: `parser` (lexer + event-driven parser), `syntax` (`rowan` +lossless tree + typed AST), `vfs` (in-memory virtual file system), and `lsp` (the language +server), plus the TypeScript VS Code extension in `editors/code/` and build tasks in `xtask/`. + +The resolution core (`resolver.rs` + `symbol_table.rs`) is name-based and shared by +goto-definition, hover, references, and rename, over a salsa-shaped source DB that is +invalidated by draining the VFS change log. + +See [`docs/architecture.md`](./docs/architecture.md) for the full design, and +[`docs/crates/`](./docs/crates/) for per-crate deep dives. --- diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..3dfd96f --- /dev/null +++ b/docs/README.md @@ -0,0 +1,24 @@ +# CCLS Documentation + +This folder holds the in-depth documentation for **CCLS**, the Circom language server. + +For the quick-start (clone, install Rust, build, run tests), see the root +[`README.md`](../README.md). For the VS Code extension package, see +[`editors/code/README.md`](../editors/code/README.md). + +## Contents + +| Document | Description | +| --- | --- | +| [Features](./features.md) | What CCLS can do today, and how each feature resolves symbols. | +| [Architecture](./architecture.md) | Workspace layout, the resolution core, the source DB, and the VFS. | +| [Roadmap](./roadmap.md) | Capabilities not yet implemented and follow-ups on existing features. | + +### Per-crate deep dives + +| Crate | Document | Responsibility | +| --- | --- | --- | +| `parser` | [crates/parser.md](./crates/parser.md) | `logos` lexer + event-driven parser with markers and grammar modules. | +| `syntax` | [crates/syntax.md](./crates/syntax.md) | `rowan` lossless syntax tree and typed AST. | +| `lsp` | [crates/lsp.md](./crates/lsp.md) | LSP server: handlers, global state, resolver, source DB, symbol table. | +| `vfs` | [crates/vfs.md](./crates/vfs.md) | In-memory, path-interned virtual file system with a change log. | diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..19eb570 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,111 @@ +# Architecture + +CCLS is a multi-crate Rust workspace plus a TypeScript VS Code extension. This page describes +the workspace layout and the design of the three load-bearing subsystems: the **resolution +core**, the **source database**, and the **virtual file system** (including sandboxed +includes). + +## Workspace layout + +``` +circom-language-server/ +├── crates/ +│ ├── parser/ # `logos` lexer + event-driven parser with markers +│ ├── syntax/ # `rowan` lossless syntax tree + typed AST +│ ├── vfs/ # Virtual file system (existence, text, change log) +│ └── lsp/ # LSP server: handlers, global state, resolver, semantic index +├── editors/code/ # VS Code extension (TypeScript, `circom-plus`) +└── xtask/ # Build & install tasks (`cargo xtask install …`) +``` + +See [crates/](./crates/) for per-crate deep dives. + +## Server entry point + +`crates/lsp/src/main.rs` opens a stdio LSP connection, advertises +[`server_capabilities`](https://microsoft.github.io/language-server-protocol/) (definition, +hover, completion, references, documentSymbol, formatting, rename with `prepareSupport`), and +runs a `main_loop` that routes each `Message::Request` to `GlobalState::handle_request` and +each `Message::Notification` to `GlobalState::handle_notification`. Workspace roots are +captured from the `initialize` handshake (modern `workspace_folders`, falling back to legacy +`root_uri`), canonicalized, and handed to the VFS for include confinement. + +## Resolution core + +The resolution core lives in `crates/lsp/src/resolver.rs` and +`crates/lsp/src/symbol_table.rs`. It is the single name-resolution engine shared by +goto-definition, hover, references, and rename — so a given symbol resolves identically across +all features. + +- **`SymbolTable`** (`symbol_table.rs`) is a per-file, scope-aware index of declarations by + name: the file top-level (template/function/bus names) plus each body scope + (params/signals/vars/components). Each `Symbol` carries a `def_range` (the **name token** — + what goto-def/rename/references jump to) and a `decl_range` (the **whole declaration** — + what hover shows), plus a `type_name` (the instantiated template, for `component c = T();`). +- **`resolver`** (`resolver.rs`) is pure: given a token under the cursor and a file's + `SymbolTable`, it returns the `ResolvedSymbol`s the token refers to. It owns no I/O and no + LSP request types — callers map each `def_range` into a file-scoped `lsp_types::Location`. + +Resolution is **purely name-based** against the table. This is what makes it sound across +files: the same name resolves in each file's own table without reusing a foreign token's +identity (the legacy `hash(text)` index was structurally unsound for cross-file lookups). + +## Source database + +`crates/lsp/src/source_db.rs` is a salsa-shaped source database layered over the in-memory +[`Vfs`](#virtual-file-system-vfs). Each `SourceDatabase` method maps 1:1 to a future salsa +query, so adopting salsa later is an implementation swap, not a redesign. + +The split (rust-analyzer style): + +- [`Vfs`](./crates/vfs.md) owns **file existence + text + change log**. +- `ContentCacheDb` owns **derived caches** — parse, AST, `FileDB` (offset/line bookkeeping), + and `SymbolTable` — keyed by `FileId`, behind a single `RefCell`. + +### Change-log-driven invalidation + +Invalidation is driven by **draining the VFS change log**, not by content hashing. When a query +runs, the DB drains `Vfs::take_changes()` and drops only the caches for changed ids. The +guarantees this gives: + +- **Editing file A never recomputes file B** — only changed `FileId`s are invalidated. +- Includes are read from disk and parsed **once**, then cached. +- A client resending **identical text records no change** (pointer-equal then content-equal + short-circuit in the VFS) and so triggers no reparse. + +## Virtual file system (VFS) + +`crates/vfs/src/lib.rs` is a pure in-memory, path-interned VFS in the rust-analyzer style. +It deliberately does **no disk I/O** — path resolution and `fs::read_to_string` stay in the +LSP layer, which feeds text in via `Vfs::set_file_contents`. Keeping the VFS I/O-free makes it +unit-testable without touching the filesystem. + +Responsibilities: + +- **Path interning**: an absolutized `VfsPath` ↔ `FileId` (`u32`). Aliased paths + (`/a/../a/c` vs `/a/c`) absolutize to one `VfsPath` → one `FileId`. +- **Text storage**: a current `Arc` per `FileId`; `None` text means absent/deleted. +- **Change log**: every mutation records a `ChangedFile` (`Create`/`Modify`/`Delete`), drained + via `Vfs::take_changes()` — the invalidation signal for the source DB and the future hook + for a file-watcher. + +See [crates/vfs.md](./crates/vfs.md) for the full API. + +## Sandboxed includes + +`include "…"` resolution is confined to workspace roots as a path-traversal defense: + +- `Vfs::set_workspace_roots` stores already-canonicalized root paths during the `initialize` + handshake. +- `Vfs::is_confined(canonical)` is a **pure** containment check — `canonical.starts_with(root)` + against the roots — with **no disk I/O**. The LSP layer canonicalizes the candidate + (resolving `..` and symlinks) before calling, which keeps the VFS I/O-free. +- The check is **fail-closed**: with no roots configured, nothing is confined, so the server + refuses to load any include rather than risk an arbitrary read. + +## Extension + +The TypeScript client in `editors/code/` (`circom-plus`) is a thin LSP client: it spawns the +server binary, forwards the standard text-document requests/notifications, and wires the +results into VS Code's goto/hover/completion/references/rename UI. See +[`editors/code/README.md`](../editors/code/README.md). diff --git a/docs/crates/lsp.md b/docs/crates/lsp.md new file mode 100644 index 0000000..d4211c8 --- /dev/null +++ b/docs/crates/lsp.md @@ -0,0 +1,119 @@ +# `lsp` crate + +The Language Server Protocol implementation, built on +[`lsp-server`](https://docs.rs/lsp-server) and +[`lsp-types`](https://docs.rs/lsp-types). This is the crate that becomes the `ccls` server +binary; it owns document state, request dispatch, and the resolution/index machinery that all +features ride on. + +> Also see [`architecture.md`](../architecture.md) for how the pieces fit together, and +> [`features.md`](../features.md) for the user-facing feature list. + +## Layout + +``` +crates/lsp/src/ +├── main.rs # binary entry: stdio connection, capabilities, main loop +├── global_state.rs # GlobalState: document state + request/notification dispatch +├── handler.rs # handler module registry (one module per feature) +├── handler/ +│ ├── goto_definition.rs # textDocument/definition (implemented) +│ ├── hover.rs # textDocument/hover (implemented) +│ ├── completion.rs # textDocument/completion (implemented) +│ ├── references.rs # textDocument/references (implemented) +│ ├── rename.rs # textDocument/rename (+ prepare) (implemented) +│ ├── document_symbol.rs # textDocument/documentSymbol (placeholder: returns None) +│ └── formatting.rs # textDocument/formatting (placeholder: returns None) +├── resolver.rs # pure token -> definition resolver (shared resolution core) +├── symbol_table.rs # per-file, scope-aware declaration index +├── source_db.rs # salsa-shaped source DB over the Vfs (parse/ast/file_db/symbol_table) +└── file_db.rs # per-file offset/line bookkeeping + FileId re-export +``` + +## Entry point (`main.rs`) + +`main()` opens a stdio `Connection`, serializes `server_capabilities()`, runs the +`initialize` handshake, and enters `main_loop`. **All logging goes to stderr** — stdout is +the LSP message channel. + +`server_capabilities()` advertises: + +- `text_document_sync: FULL` (full document on every change — incremental sync is a roadmap + item). +- `definition`, `hover`, `references`, `documentSymbol`, `documentFormatting` providers. +- `completion` with the `.` trigger character. +- `rename` with `prepareSupport` (the client consults the server before opening the rename + box, so keywords/strings never become renamable). + +`workspace_roots(params)` extracts workspace folders (modern `workspace_folders`, falling +back to legacy `root_uri`), converts each from a `file:` URI to a path, and canonicalizes +them. These roots feed the VFS's include-confinement check. + +## Main loop + +`main_loop` reads `Message`s off the connection: + +- `Message::Request` — if it isn't `shutdown` (handled by the transport), routed to + `GlobalState::handle_request`, which dispatches to the matching `handler::*::handle` by + request type and sends back a `Message::Response`. +- `Message::Notification` — routed to `GlobalState::handle_notification` (e.g. + `didOpen` / `didChange`, which update document text in the source DB). +- `Message::Response` — ignored (the server sends no requests of its own). + +## Global state (`global_state.rs`) + +`GlobalState` holds the `ContentCacheDb` (the source DB) and provides the dispatch surface. +It also exposes the shared cursor prologue every read-handler uses — `cursor_context(uri, +position)` resolves the URL to a `FileId`, fetches the AST and `FileDB`, and converts the LSP +position to a byte offset, returning a `CursorContext { id, ast, file_db, offset }`. + +The handler contract (see `handler.rs`): each feature module exposes +`fn handle(state: &GlobalState, params: P) -> Result>`. + +> Note: the `handler.rs` module doc-comment currently says "the rest are placeholders", but +> `hover`, `completion`, `references`, and `rename` are in fact fully implemented (only +> `document_symbol` and `formatting` remain placeholders). Trust the individual handler files. + +## Handlers (`handler/`) + +Each handler rides the shared resolution core. Highlights: + +- **`goto_definition`** — resolves signals/vars/params/templates/functions/components, with + cross-file jumps via `include` and a `jump_to_lib` path for include-path strings. +- **`hover`** — kind + declaration signature (header only for block-bodied defs). +- **`completion`** — in-scope names, top-level names, keywords, and member completion + (`c.`) that resolves a component's template across files. +- **`references`** / **`rename`** — occurrences found by *resolving* candidates (not + text-matching), so shadowing is correct; both are in-file today (cross-file is a roadmap + item). + +## Resolution core (`resolver.rs` + `symbol_table.rs`) + +- `SymbolTable` (`symbol_table.rs`) — per-file index of declarations by name: the file + top-level plus each body scope. A `Symbol` carries `def_range` (name token), `decl_range` + (whole declaration), and `type_name` (the instantiated template for components). +- `resolver` (`resolver.rs`) — pure token → `ResolvedSymbol` lookup over a `SymbolTable`. + Shared helpers: `token_at_offset`, `identifier_at`, `token_ancestors`, `component_field`. + +See [`architecture.md` § Resolution core](../architecture.md#resolution-core). + +## Source DB (`source_db.rs`) + +`SourceDatabase` is a salsa-shaped trait (`file_text` is the input; `parse`/`ast`/`file_db`/ +`symbol_table` are derived, memoized via interior mutability). `ContentCacheDb` is the +`Vfs`-backed implementation. Invalidation is change-log-driven: draining +`Vfs::take_changes()` drops only changed ids, so editing file A never recomputes file B. See +[`architecture.md` § Source database](../architecture.md#source-database). + +## File DB (`file_db.rs`) + +`FileDB` is per-file offset/line bookkeeping: the `FileId` (re-exported from `vfs`), the +`file:` URL, the byte offset of every `\n`, and the source text. It converts between LSP +positions (UTF-16 code-unit counts) and byte offsets — the bridge between the protocol and +rowan's byte-based `TextSize` ranges. + +## Tests + +Handler and resolver tests use `insta` snapshots under `crates/lsp/src/handler/snapshots/` +and fixtures under `crates/lsp/src/test_files/` (including a `with_include/` cross-file +case). Run with `cargo test -p lsp`. diff --git a/docs/crates/parser.md b/docs/crates/parser.md new file mode 100644 index 0000000..5c5b517 --- /dev/null +++ b/docs/crates/parser.md @@ -0,0 +1,91 @@ +# `parser` crate + +The Circom **lexer**, an **event-driven parser**, and the **grammar** rules. This crate turns +raw `.circom` source text into a flat sequence of parser events that the +[`syntax`](./syntax.md) crate builds into a lossless syntax tree. + +The grammar rules follow the official circom compiler grammar +(`iden3/circom` → `parser/src/lang.lalrpop`). + +## Layout + +``` +crates/parser/src/ +├── lib.rs # crate root: re-exports the modules below +├── lexer.rs # logos-based tokenizer -> Vec +├── token_kind.rs # TokenKind enum: infix()/prefix()/postfix(), is_trivial(), keywords +├── parser.rs # the Parser struct + markers, fuel, depth guard, trivia handling +├── event.rs # Event { Open, Close, Token } emitted by the parser +└── grammar/ # one module per language construct + ├── block.rs + ├── bus.rs + ├── declaration.rs + ├── definition.rs + ├── expression.rs + ├── function.rs + ├── include.rs + ├── list.rs + ├── main_component.rs + ├── pragma.rs + ├── statement.rs + └── template.rs +``` + +## Lexing (`lexer.rs`) + +Uses [`logos`](https://docs.rs/logos) to scan source text into a `Vec>`, where each +`Token` carries its `TokenKind` and a text slice. Whitespace and comments are kept as tokens +(trivia) rather than dropped, so the downstream tree stays **lossless**. + +## Token kinds (`token_kind.rs`) + +`TokenKind` is the source of truth for operator parsing. Its associated functions drive +Pratt / precedence-climbing in the expression grammar: + +- `infix()` — binary operator bindings (precedence + associativity). +- `prefix()` — unary prefix operators. +- `postfix()` — unary postfix operators. +- `is_trivial()` — whitespace/comments, handled by `wrap_trivia()`. + +Reserved keywords are also enumerated here; `crates/lsp/src/handler/completion.rs` mirrors +them for keyword completion. + +## Event-driven parser (`parser.rs` + `event.rs`) + +The parser does **not** build a tree directly. It emits a flat `Vec` of: + +- `Event::Open` / `Event::Close` — node boundaries, identified by a `Marker`. +- `Event::Token` — a leaf token (by position into the token slice). + +Markers (`Marker::Open`, `Marker::Close`) let grammar code open a node, parse its children, +and close it with a specific kind — the same model rust-analyzer uses. This indirection is +what enables **error recovery**: a partially-parsed node can be closed early without aborting +the whole parse. + +### Guards + +- **`fuel`** — a budget (reset to 256 whenever the parser advances on non-trivia). If the + parser stops advancing, fuel runs out and recovery kicks in; this catches non-advancing + infinite loops. +- **`depth`** — the current expression-nesting depth, incremented in the Pratt recursive + core. Bounds recursion on deeply-nested untrusted input so a pathological file cannot + overflow the stack (fuel only catches non-advancing loops, not deep ones). + +### Trivia + +`wrap_trivia()` emits leading whitespace/comments at/after the current position and returns +the next non-trivial token kind. `current()` wraps it. `peek()` does lookahead **without** +emitting trivia, so it is safe to call for dispatch decisions before/after `current()` without +double-emitting. + +## Grammar (`grammar/`) + +Each language construct has its own module. Expressions use Pratt / precedence climbing via +the `TokenKind` operator methods. Error recovery is supported through the +`error_report()` mechanism, which records parse errors into the event stream while continuing +to parse. + +## Tests + +Lexer and grammar tests use `insta` snapshots under `crates/parser/src/snapshots/`. Run them +with `cargo test -p parser`. diff --git a/docs/crates/syntax.md b/docs/crates/syntax.md new file mode 100644 index 0000000..6e9947a --- /dev/null +++ b/docs/crates/syntax.md @@ -0,0 +1,82 @@ +# `syntax` crate + +Builds a **lossless**, typed syntax tree over circom source using +[`rowan`](https://docs.rs/rowan). It consumes the flat event stream produced by the +[`parser`](./parser.md) crate and exposes both a generic `SyntaxNode` tree and a typed AST. + +"Lossless" means whitespace, comments, and even parse errors are preserved in the tree, so it +round-trips to the original source exactly. + +## Layout + +``` +crates/syntax/src/ +├── lib.rs # crate root +├── node.rs # CircomLanguage, SyntaxNode/Token aliases +├── tree.rs # syntax_tree() / build_green(): events -> rowan green tree +├── tree/test_utils.rs # view_ast() snapshot helper +├── abstract_syntax_tree/ # typed AST nodes (AstNode impls) +│ ├── mod.rs +│ ├── program.rs +│ ├── definition.rs +│ ├── declaration.rs +│ ├── statement.rs +│ ├── expression.rs +│ ├── block.rs +│ └── name.rs +├── test_files/ # circom fixtures (happy/ + error_recovery/) +└── snapshots/ # committed insta snapshots +``` + +## The language type (`node.rs`) + +`rowan` is parameterized by a `Language`. `CircomLanguage` binds rowan's `SyntaxKind` to the +parser's `TokenKind` (a `#[repr(u16)]` enum), with a compile-time +`size_of::() == 2` assertion guaranteeing the `transmute` in `kind_from_raw` is +sound. The module then defines the conventional aliases: `SyntaxNode`, `SyntaxToken`, +`SyntaxElement`, `SyntaxNodeChildren`, `PreorderWithTokens`. + +## From events to a green tree (`tree.rs`) + +- `syntax_tree(source)` tokenizes, parses as a whole circom program, and builds the tree. +- `syntax_node_from_source(source, scope)` parses starting from a specific entry `Scope` + (used by targeted grammar tests). +- `build_green(tokens, events, builder)` drives a `rowan::GreenNodeBuilder` straight from the + parser's event stream (`Open` / `Close` / `Token` / `ErrorReport`). + +Design points worth knowing: + +- **Fresh `NodeCache` per parse** — identical tokens/subtrees are deduplicated *within* one + parse, but nothing accumulates across parses. A process-global cache would leak every + distinct identifier ever parsed for the whole server lifetime (unbounded in a long LSP + session); a per-parse cache bounds memory to one parse's working set. +- **Defense-in-depth against malformed streams** — `build_green` is robust against a stray + `Close`, an unclosed `Open`, or a stream with no root `Open`, so `GreenNodeBuilder::finish` + (which asserts exactly one top-level node) never panics even on a grammar bug. An empty + stream yields an empty `ParserError` root. +- **Error nodes are zero-width** — `Event::ErrorReport(msg)` becomes a childless `Error` + node. The message is diagnostic metadata, **not** source text: emitting it as a token would + make rowan size the node by the message length and break offset/range math. `has_error` / + `parses_clean` key off the node kind, not text. +- **Tokens are wrapped** — each consumed token becomes a single-child node of the same kind; + this wrapping is load-bearing for `AstNode::cast`. + +## Typed AST (`abstract_syntax_tree/`) + +The generic `SyntaxNode` tree is lossless but awkward to walk. The AST modules add typed +wrappers (`AstCircomProgram`, `AstTemplateDef`, `AstSignalDecl`, `AstComponentDecl`, +`AstIdentifier`, etc.) that implement rowan's `AstNode` trait, so consumers (the LSP +`resolver` and `symbol_table`) navigate the tree ergonomically via `cast` and accessor +methods rather than raw `kind()` checks. + +## Tests and snapshots + +- Fixtures live in `test_files/happy/` (valid) and `test_files/error_recovery/` (malformed). +- The `test_syntax!(path, scope)` macro parses a fixture and snapshots both the tree and an + AST view via `view_ast()`. +- Snapshots are committed under `snapshots/`. +- Error-recovery tests pin behavior on missing `;`, unclosed `{`, and stray top-level tokens. +- `build_green_tests` synthesize raw event streams to assert `build_green` never panics on + malformed input. + +Run them with `cargo test -p syntax`. diff --git a/docs/crates/vfs.md b/docs/crates/vfs.md new file mode 100644 index 0000000..35c634a --- /dev/null +++ b/docs/crates/vfs.md @@ -0,0 +1,64 @@ +# `vfs` crate + +A pure in-memory, **path-interned** virtual file system in the rust-analyzer style. It is the +single source of truth for file existence and text, and it emits the change log that drives +cache invalidation in the [`lsp`](./lsp.md) source DB. + +This crate does **no disk I/O** — by design. Path resolution and `fs::read_to_string` stay in +the LSP layer, which feeds text in via `Vfs::set_file_contents`. Keeping the VFS I/O-free +makes it unit-testable without touching the filesystem. + +Source: `crates/vfs/src/lib.rs` (the whole crate is one file). + +## Core types + +- **`FileId(u32)`** — path-interned file identity. Two aliased paths share one id because + they absolutize to one `VfsPath`. Stable for the lifetime of the `Vfs` that allocated it. +- **`VfsPath(PathBuf)`** — an *absolutized* path, the interning key. Aliased paths + (`/a/../a/c` vs `/a/c`) collapse to one `VfsPath` (built via `path-absolutize`). +- **`ChangeKind`** — `Create` / `Modify` / `Delete`. +- **`ChangedFile { file_id, change_kind }`** — one recorded change, drained by the source DB. +- **`Vfs`** — the store: `path_to_id` map, a `Vec` (path + `Option>` + text), a pending change log, and canonicalized workspace roots. Single-threaded by design + (mutation takes `&mut self`); a `RwLock`/channel arrives only with the deferred + file-watcher, and the change-log surface is identical either way. + +## API surface + +| Method | Purpose | +| --- | --- | +| `VfsPath::from_abs_path(path)` | Absolutize a `&Path` into a `VfsPath` (`None` if it fails). | +| `Vfs::set_workspace_roots(roots)` | Store canonicalized workspace roots (called at `initialize`). | +| `Vfs::is_confined(canonical)` | **Pure**, fail-closed containment check (no disk I/O). | +| `Vfs::set_file_contents(path, text)` | Intern/update/delete a file; records a `ChangedFile`. | +| `Vfs::take_changes()` | Drain and return the pending change log. | +| `Vfs::has_changes()` | `true` if there are pending changes. | +| `Vfs::file_id(path)` / `Vfs::path(id)` / `Vfs::file_text(id)` | Identity + text lookups. | + +## Key behaviors + +- **No-op on identical text** — `set_file_contents` short-circuits when the new text is + pointer-equal (`Arc::ptr_eq`) and falls back to a content compare. A no-op records *no* + change, so a client resending identical content triggers no downstream invalidation. +- **`None` text = delete** — setting `text = None` on an existing file records a `Delete` + (forward-looking for the file-watcher; no deletions occur today). +- **Change log as the invalidation signal** — the source DB drains `take_changes()` to drop + only the caches for changed ids. There is no per-entry content hash; the change log is the + only signal. + +## Sandboxed includes + +`is_confined` is the path-traversal defense for `include "…"` resolution: + +- The LSP layer canonicalizes a candidate include path (resolving `..` and symlinks) before + calling, then checks `is_confined`. +- `is_confined` compares `canonical.starts_with(root)` against the stored roots — pure, no + disk I/O. +- **Fail-closed**: with no roots configured, nothing is confined, so the server refuses to + load any include rather than risk an arbitrary read. + +## Tests + +`lib.rs` includes unit tests covering path aliasing, create/modify recording, change-log +draining, text round-tripping, the identical-text no-op, and `None`-as-delete. Run with +`cargo test -p vfs`. diff --git a/docs/features.md b/docs/features.md new file mode 100644 index 0000000..8420f25 --- /dev/null +++ b/docs/features.md @@ -0,0 +1,87 @@ +# Features + +What CCLS provides today, and how each feature resolves symbols under the hood. The roadmap +of features **not yet** implemented lives in [roadmap.md](./roadmap.md). + +All features below share a single **resolution core** (see +[architecture.md](./architecture.md)): a name-based resolver over a per-file symbol table. +That means the same symbol resolves consistently whether you go-to-definition, hover, +find-references, or rename it. + +## Implemented + +### Go to Definition + +Resolves signals, variables, parameters, templates, functions, and components — including +**cross-file** jumps through `include` statements, and jumping straight into an included +library file from its `"path.circom"` string. + +- Handler: `crates/lsp/src/handler/goto_definition.rs` +- It uses `token_at_offset` (not `identifier_at`) because an include-path `CircomString` is + also a valid jump target. + +### Hover + +Shows the symbol kind and its declaration signature (header only for block-bodied definitions +like `template` / `function` / `bus`). + +- Handler: `crates/lsp/src/handler/hover.rs` +- Member-access fields (`c.x`) are not resolved by the flat resolver and yield `None` + (consistent with rename/references). + +### Completion + +In-scope body symbols, file top-level names, reserved keywords, and **member completion** +(`component.`) that resolves a component's template across files. + +- Handler: `crates/lsp/src/handler/completion.rs` +- Member mode detects `receiver.`, resolves the receiver's template (in-file or via + `include`), and offers that template's signals. +- Normal mode returns `is_incomplete: false` — the client filters by the typed word. +- The `.` trigger character is advertised in server capabilities. + +### Find References + +Every occurrence of a symbol, resolved *semantically* (not text-matched), so shadowing is +respected. + +- Handler: `crates/lsp/src/handler/references.rs` +- Returns the declaration plus every in-scope use as `Location`s. **In-file by design** — + cross-file references are a follow-up (see roadmap). + +### Rename + +Scope-aware rename with `prepareRename` support; refuses keywords, include-path strings, +illegal names, and unresolved member-access fields. + +- Handler: `crates/lsp/src/handler/rename.rs` +- Occurrences are found by *resolving* each candidate (not text-matching), so shadowing is + correct. In-file (the symbol's defining file); cross-file rename is a follow-up. +- `prepareSupport` is advertised so the client consults the server (not its own textual word + check) before opening the rename box. + +### Error-recovering parser + +Keeps working on invalid or partially-typed circom files. See +[crates/parser.md](./crates/parser.md) for the marker-based recovery model. + +### Lazy, cached analysis + +Parsing and symbol tables are memoized and invalidated only on real edits; includes are read +from disk once. A client resending identical text records no change and triggers no reparse. +See [architecture.md](./architecture.md#source-database) for the change-log-driven +invalidation. + +### Sandboxed includes + +`include` resolution is confined to workspace roots — path-traversal and symlink-safe. With no +roots configured the server refuses to load any include rather than risk an arbitrary file +read (fail-closed). See [architecture.md](./architecture.md#sandboxed-includes). + +## Registered but not yet implemented + +Two handlers are advertised in server capabilities but currently return an empty result. See +[roadmap.md](./roadmap.md) for details: + +- **Document Symbol / Outline** — `crates/lsp/src/handler/document_symbol.rs` +- **Formatting** — `crates/lsp/src/handler/formatting.rs` diff --git a/TODO.md b/docs/roadmap.md similarity index 59% rename from TODO.md rename to docs/roadmap.md index 0638d11..bd85093 100644 --- a/TODO.md +++ b/docs/roadmap.md @@ -1,22 +1,23 @@ # Roadmap -Features not yet implemented in CCLS. Implemented features are listed in the -[README](./README.md#-features). +Features not yet implemented in CCLS. Implemented features are listed in +[features.md](./features.md). ## Placeholders (capability registered, returns empty) These handlers exist but currently return `None`: -- [ ] **Document Symbol / Outline** — `textDocument/documentSymbol`. Walk the program AST and emit one - symbol per template / function / signal / variable / component with its location and kind. - See `crates/lsp/src/handler/document_symbol.rs`. +- [ ] **Document Symbol / Outline** — `textDocument/documentSymbol`. Walk the program AST and + emit one symbol per template / function / signal / variable / component with its + location and kind. See `crates/lsp/src/handler/document_symbol.rs`. - [ ] **Formatting** — `textDocument/formatting`. Re-tokenize the document and normalize - whitespace/indentation into `TextEdit`s. See `crates/lsp/src/handler/formatting.rs`. + whitespace/indentation into `TextEdit`s. See + `crates/lsp/src/handler/formatting.rs`. ## Not yet implemented -- [ ] **Diagnostics** — syntax/semantic error reporting (`textDocument/publishDiagnostics`). The - parser already produces errors via `error_report()`; surface them to the client. +- [ ] **Diagnostics** — syntax/semantic error reporting (`textDocument/publishDiagnostics`). + The parser already produces errors via `error_report()`; surface them to the client. - [ ] **Semantic Highlighting** — `textDocument/semanticTokens`. - [ ] **Signature Help** — `textDocument/signatureHelp`. - [ ] **Code Actions / Quick Fixes** — `textDocument/codeAction`. @@ -27,10 +28,10 @@ These handlers exist but currently return `None`: ## Existing features — follow-ups -- [ ] **Cross-file Rename & References** — both are currently in-file only. A workspace-wide symbol - graph is needed instead of name/`def_range` matching across files (which both misses real - cross-file usages and can collide when two files define a same-named symbol at the same - line:column). +- [ ] **Cross-file Rename & References** — both are currently in-file only. A workspace-wide + symbol graph is needed instead of name/`def_range` matching across files (which both + misses real cross-file usages and can collide when two files define a same-named symbol + at the same line:column). - [ ] **Doc-comment parsing** — richer hover derived from circom comments. -- [ ] **Incremental sync** — document sync is currently `Full`; switch to incremental `didChange` - ranges. +- [ ] **Incremental sync** — document sync is currently `Full`; switch to incremental + `didChange` ranges.