Skip to content

Commit f05ab1e

Browse files
committed
feat: support per-role database storage roots
- Replace the host-wide root with an ordered allowlist and persist each role's exact selection - Require explicit placement during direct creation when multiple roots exist - Reject overlapping roots, unlisted selections, referenced root removal, and changes to generated data binds BREAKING CHANGE: Config now requires host.data_roots and data_root on every Postgres and KV role; host.data_root is no longer supported
1 parent d6abe8c commit f05ab1e

20 files changed

Lines changed: 432 additions & 158 deletions

File tree

Lines changed: 34 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -1,60 +1,60 @@
11
## Context
22

3-
`Paths.databases` currently fixes database files under `<state>/databases`, and `Database.data` derives
4-
every engine bind source from it. Generated assets, backup staging, and locks also derive from `Paths`,
5-
but they must remain in the canonical state tree when database files move to dedicated storage.
6-
7-
The source configuration is strict and human-owned. evdb does not retain configuration history or move
8-
database data, while generated Compose already records the last bind source used for each managed role.
3+
`Paths.databases` supplies the canonical database root. The first implementation moved that choice to
4+
one `Host.data_root`, but one host-wide value cannot place roles on different disks. Generated assets,
5+
backup staging, and locks must remain in the canonical state tree regardless of role placement.
96

107
## Goals / Non-Goals
118

129
**Goals:**
1310

14-
- Store one explicit host-wide data root and use it for every database role.
15-
- Make fresh guided and direct setup support a mounted-disk directory.
16-
- Reject unsafe roots and prevent an existing generated role from silently switching to an empty path.
17-
- Keep the canonical root as the setup default.
11+
- Store an ordered allowlist of host database roots and one exact selection on every role.
12+
- Collect roots during fresh setup and choose placement during database creation.
13+
- Permit later catalog edits while preventing removal of referenced roots.
14+
- Reject unsafe roots and prevent an existing generated role from silently changing placement.
1815

1916
**Non-Goals:**
2017

21-
- No per-project or per-role data paths.
18+
- No arbitrary per-role paths outside the host catalog.
2219
- No mount creation, persistence, startup ordering, or availability monitoring.
2320
- No data copy, relocation, import, compatibility default, or source migration.
2421
- No change to generated assets, backup staging, locks, or repositories.
2522

2623
## Decisions
2724

28-
### Put the root on Host and keep Paths as the canonical default
25+
### Store exact paths in an ordered catalog and on each role
26+
27+
Replace the single root with required `Host.data_roots: tuple[Path, ...]`. Add required
28+
`data_root: Path` to `Postgres` and `KV`, and derive `Database.data` from the role setting. Exact paths
29+
avoid an alias layer and make placement visible beside engine settings. `Paths.databases` remains only
30+
the initial default. List order is preserved, but multiple roots have no implicit direct-creation default.
2931

30-
Add required `Host.data_root: Path` and serialize it as `host.data_root`. `Database.data` uses this field,
31-
while `Paths.databases` remains the default selected by fresh initialization. This keeps operator policy
32-
in source configuration without making all runtime paths configurable.
32+
### Validate the catalog as one unit
3333

34-
### Validate a dedicated safe directory
34+
Every root must be a unique normalized absolute path with no symlinked or non-directory existing
35+
component. Roots must not overlap one another, evdb configuration, generated assets, Traefik, backups,
36+
locks, or a local Restic repository. For a non-canonical root, the immediate parent must already exist.
37+
Every role selection must exactly match one catalog entry. Initialization prepares every root.
3538

36-
The value must be a normalized absolute path with no symlinked or non-directory existing component. It
37-
must not be a broad filesystem ancestor or overlap evdb configuration, generated assets, Traefik,
38-
backups, locks, or a local Restic repository. For a custom root, the immediate parent must already exist;
39-
initialization creates or converges only the selected leaf as a private managed directory.
39+
Fresh guided setup collects roots until the operator is done and shows the ordered list in review.
40+
Operators may later edit `config.yml` and rerun initialization. Adding or removing an unused root is
41+
valid; removing a referenced root fails source validation.
4042

41-
The setup review states the selected path. The operator is responsible for mounting storage before evdb
42-
or Docker starts; evdb does not imply that a custom directory is a persistent mount.
43+
### Select placement during creation and keep it immutable
4344

44-
### Treat the root as immutable after role generation
45+
Guided database creation asks for a root when the catalog has multiple entries and includes it in review.
46+
Direct creation uses the sole root automatically; with multiple roots it requires `--data-root PATH`.
47+
Creation writes the exact path under the role. Idempotent add rejects a different supplied path.
4548

46-
Fresh setup and source loading require the field. Existing-host initialization accepts only the stored
47-
value. Before replacing an existing role Compose file, database rendering compares the current primary
48-
data bind with the desired bind. A different source fails before creating the new data directory or
49-
rewriting Compose. This catches both supported command misuse and direct source edits without adding a
50-
machine-state file.
49+
Before replacing existing role Compose, rendering compares the current primary data bind with the
50+
desired bind. A different source fails before creating the new data directory or rewriting Compose. The
51+
database settings editor never exposes placement.
5152

5253
## Risks / Trade-offs
5354

54-
- [A configured disk is not mounted when Docker starts] -> State clearly that mounts and startup ordering
55-
are operator-managed; evdb validates the directory but does not claim mount availability.
56-
- [A source edit points at an empty directory] -> Compare the generated primary data bind before any
57-
render mutation and reject a changed source.
58-
- [Existing source lacks the required field] -> Reject it under the existing strict-schema policy; the
59-
operator must provide current source explicitly.
55+
- [A configured disk is not mounted when Docker starts] -> Mounts and startup ordering remain
56+
operator-managed; evdb validates directories but does not claim mount availability.
57+
- [A source edit points at an empty directory] -> Compare generated primary data binds before mutation.
58+
- [A catalog entry is removed while in use] -> Require every role selection to remain in the catalog.
59+
- [Existing source lacks catalog or role placement] -> Reject it under the strict-schema policy.
6060
- [Generated Compose is absent] -> Allow rendering because no prior managed bind exists to preserve.
Lines changed: 19 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,21 @@
11
## Why
22

3-
Database files are fixed under `/var/lib/evdb/databases`, which prevents an operator from placing them
4-
on a dedicated mounted disk during host setup. The host needs one explicit storage root while preserving
5-
the existing project and role layout.
3+
Database files are fixed under `/var/lib/evdb/databases`, which prevents an operator from distributing
4+
roles across dedicated storage paths. The host needs an allowlisted root catalog and an explicit
5+
creation-time placement for each database role.
66

77
## What Changes
88

9-
- **BREAKING**: require `host.data_root` in `config.yml`; fresh setup writes
10-
`/var/lib/evdb/databases` by default.
11-
- Add guided and direct initialization input for one host-wide database data root.
12-
- Derive every database bind source as `<data_root>/<project>/<role>/data`.
13-
- Validate the selected root as a safe normalized absolute path and reject overlap with other managed or
14-
local repository paths.
15-
- Reject a changed data bind source when an existing generated Compose file records another root. evdb
16-
does not move database data.
17-
- Keep mount configuration and startup ordering under operator control.
9+
- **BREAKING**: require a non-empty ordered `host.data_roots` list and an exact `data_root` selection on
10+
every Postgres and KV role; no compatibility values are synthesized.
11+
- Let fresh guided setup collect one or more roots, beginning with `/var/lib/evdb/databases`.
12+
- Let guided database creation choose an allowlisted root. Direct creation uses the sole root
13+
automatically or requires an explicit selection when multiple roots exist.
14+
- Derive every database bind source as `<selected-root>/<project>/<role>/data`.
15+
- Validate each root, reject duplicates and overlaps, and reject role selections outside the catalog.
16+
- Allow operators to edit the catalog in `config.yml` and rerun initialization; referenced roots cannot
17+
be removed.
18+
- Reject changed role data binds when generated Compose records another root. evdb does not move data.
1819

1920
## Capabilities
2021

@@ -24,12 +25,12 @@ None.
2425

2526
### Modified Capabilities
2627

27-
- `config`: Make the host database data root explicit, safe, and immutable after database generation.
28-
- `host-setup`: Collect and prepare the host-wide database data root during initialization.
29-
- `deploy`: Allow database data outside `/var/lib/evdb` while keeping all other production paths fixed.
28+
- `config`: Define an allowlisted root catalog and explicit immutable placement for every database role.
29+
- `host-setup`: Collect, validate, and prepare all configured roots and select placement during creation.
30+
- `deploy`: Allow role data outside `/var/lib/evdb` while keeping all other production paths fixed.
3031

3132
## Impact
3233

33-
This changes the host source schema, initialization CLI and review, database path derivation, generated
34-
Compose safety checks, documentation, and focused configuration and database tests. It adds no runtime
35-
dependency and does not change backup staging or repositories.
34+
This changes the host and role source schema, initialization and database-creation CLI, path derivation,
35+
generated Compose safety checks, and focused tests. It adds no runtime dependency and does not change
36+
backup staging or repositories.
Lines changed: 26 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,30 @@
11
## ADDED Requirements
22

3-
### Requirement: Database data root safety and immutability
4-
`host.data_root` SHALL be a normalized absolute non-symlink path for one dedicated evdb database tree.
5-
It SHALL NOT overlap source, generated assets, Traefik assets, backups, locks, or a local Restic
6-
repository. A custom root's immediate parent SHALL already exist and be safe. evdb SHALL reject a desired
7-
database bind source that differs from the source recorded in an existing generated role Compose file,
8-
and SHALL NOT move database data.
3+
### Requirement: Database data root catalog and immutable placement
4+
`host.data_roots` SHALL be a non-empty ordered list of unique normalized absolute non-symlink paths.
5+
Entries SHALL NOT overlap one another, source, generated assets, Traefik assets, backups, locks, or a
6+
local Restic repository. A non-canonical root's immediate parent SHALL already exist and be safe. Every
7+
Postgres and KV role SHALL store one exact catalog entry as `data_root`. evdb SHALL reject a desired role
8+
bind source that differs from the source recorded in existing generated Compose and SHALL NOT move data.
99

10-
#### Scenario: Custom root uses mounted storage
11-
- **WHEN** `host.data_root` is `/mnt/database-volume/evdb`
12-
- **THEN** project `example-prod-01` Postgres data resolves to `/mnt/database-volume/evdb/example-prod-01/postgres/data`
10+
#### Scenario: Two roles select different roots
11+
- **WHEN** Postgres selects `/data` and KV selects `/var/lib/evdb/databases`
12+
- **THEN** each role derives `<selected-root>/<project>/<role>/data`
1313

1414
#### Scenario: Root overlaps backup storage
15-
- **WHEN** `host.data_root` is equal to or contains the managed backup path
15+
- **WHEN** one catalog entry is equal to or contains the managed backup path
1616
- **THEN** validation fails before source, directories, or services change
1717

18+
#### Scenario: Catalog roots overlap
19+
- **WHEN** the catalog contains `/data` and `/data/fast`
20+
- **THEN** validation rejects the catalog before mutation
21+
22+
#### Scenario: Role selects an absent root
23+
- **WHEN** a role selects `/removed` and that path is not in `host.data_roots`
24+
- **THEN** source validation rejects the exact project and role
25+
1826
#### Scenario: Existing generated bind differs
19-
- **WHEN** a role's generated Compose file records a data bind under one root and source selects another
27+
- **WHEN** a role's generated Compose records one data root and source selects another
2028
- **THEN** rendering fails before creating the new data directory or replacing Compose
2129

2230
## MODIFIED Requirements
@@ -26,22 +34,22 @@ Non-secret source SHALL live at `/etc/evdb/config.yml` and secret source at `/et
2634
An rclone repository SHALL reference one private native rclone file in place; a local repository SHALL
2735
omit `host.backup.rclone_config`. Generated database assets SHALL live under
2836
`/var/lib/evdb/projects/<project>/<role>`, dedicated Traefik assets under `/var/lib/evdb/traefik`, and
29-
mutable local backups and locks under `/var/lib/evdb`. Database data SHALL live under the required
30-
`host.data_root` using `<project>/<role>/data`; fresh initialization SHALL default that root to
31-
`/var/lib/evdb/databases` and write it explicitly.
37+
mutable local backups and locks under `/var/lib/evdb`. Database data SHALL live under each role's
38+
required allowlisted `data_root` using `<project>/<role>/data`; fresh initialization SHALL begin the root
39+
catalog with `/var/lib/evdb/databases` and write all selections explicitly.
3240

3341
#### Scenario: Default Postgres paths are derived
34-
- **WHEN** fresh setup creates project `example-prod-01` with a Postgres role using the default data root
42+
- **WHEN** project `example-prod-01` Postgres selects `/var/lib/evdb/databases`
3543
- **THEN** its Compose path is `/var/lib/evdb/projects/example-prod-01/postgres/compose.yaml` and its data path is `/var/lib/evdb/databases/example-prod-01/postgres/data`
3644

3745
#### Scenario: Custom Postgres data path is derived
38-
- **WHEN** `host.data_root` is `/mnt/database-volume/evdb` and project `example-prod-01` has a Postgres role
39-
- **THEN** its data path is `/mnt/database-volume/evdb/example-prod-01/postgres/data` while its Compose path remains canonical
46+
- **WHEN** Postgres selects `/data` from `host.data_roots`
47+
- **THEN** its data path is `/data/<project>/postgres/data` while its Compose path remains canonical
4048

4149
#### Scenario: Local repository is configured
4250
- **WHEN** `host.backup.repository` is a normalized absolute local path
4351
- **THEN** `config.yml` contains no rclone configuration field
4452

4553
#### Scenario: Operator locates host inputs
46-
- **WHEN** an operator reviews `/etc/evdb`, `/var/lib/evdb`, and `host.data_root`
54+
- **WHEN** an operator reviews `/etc/evdb`, `/var/lib/evdb`, and `host.data_roots`
4755
- **THEN** evdb's desired settings and managed credentials are separated from generated runtime and database data

openspec/changes/customize-database-root/specs/deploy/spec.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,11 @@
22

33
### Requirement: Production paths
44
Canonical evdb source SHALL live under `/etc/evdb`; generated services, Traefik assets, local backups,
5-
and locks SHALL live under `/var/lib/evdb`; database data SHALL live under the explicitly configured
6-
`host.data_root`; and the verified tool SHALL be the regular executable `/usr/local/bin/evdb`. No copied
5+
and locks SHALL live under `/var/lib/evdb`; database data SHALL live under each role's selection from
6+
`host.data_roots`; and the verified tool SHALL be the regular executable `/usr/local/bin/evdb`. No copied
77
rclone file, deployment machine state, activity record, restore staging, or transaction tree SHALL be
88
created.
99

1010
#### Scenario: Tool version changes
1111
- **WHEN** the verified installer atomically replaces `/usr/local/bin/evdb`
12-
- **THEN** every database continues using stable source, generated, credential, backup, and configured data paths
12+
- **THEN** every database continues using stable source, generated, credential, backup, and selected data paths
Lines changed: 28 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,34 @@
11
## ADDED Requirements
22

3-
### Requirement: Host database root selection
4-
Fresh guided initialization SHALL ask for one host-wide database data root and default it to
5-
`/var/lib/evdb/databases`. Direct initialization SHALL accept `--data-root PATH`. The redacted review
6-
SHALL show the selected root, and initialization SHALL prepare it before database roles are rendered.
7-
evdb SHALL leave filesystem mounting and startup ordering to the operator.
3+
### Requirement: Host database root catalog
4+
Fresh guided initialization SHALL collect one or more database roots, beginning with
5+
`/var/lib/evdb/databases`, and SHALL show the ordered catalog in review. Direct initialization SHALL
6+
accept repeated `--data-root PATH`. Initialization SHALL validate and prepare every configured root.
7+
Operators MAY edit the catalog in `config.yml` and rerun initialization; evdb SHALL leave filesystem
8+
mounting and startup ordering to the operator.
89

9-
#### Scenario: Guided setup accepts the default
10-
- **WHEN** the operator accepts the database data root default
11-
- **THEN** source records `host.data_root: /var/lib/evdb/databases`
10+
#### Scenario: Guided setup accepts one root
11+
- **WHEN** the operator accepts the canonical first root and adds no others
12+
- **THEN** source records `host.data_roots` with only `/var/lib/evdb/databases`
1213

13-
#### Scenario: Guided setup selects a mounted-disk directory
14-
- **WHEN** the operator enters `/mnt/database-volume/evdb` whose immediate parent is safe and present
15-
- **THEN** review shows that path and initialization prepares it as the host database root
14+
#### Scenario: Guided setup adds another root
15+
- **WHEN** the operator adds `/data`
16+
- **THEN** review shows both ordered paths and initialization prepares both roots
1617

17-
#### Scenario: Source omits the data root
18-
- **WHEN** initialization loads `config.yml` without `host.data_root`
18+
#### Scenario: Source omits the root catalog
19+
- **WHEN** initialization loads `config.yml` without `host.data_roots`
1920
- **THEN** it rejects the unsupported source instead of synthesizing a compatibility default
21+
22+
### Requirement: Database root selection during creation
23+
Guided database creation SHALL select one configured root and include it in confirmation. When exactly
24+
one root exists, guided and direct creation SHALL use it automatically. When multiple roots exist,
25+
guided creation SHALL ask and direct creation SHALL require `--data-root PATH`. Creation SHALL reject any
26+
path outside the host catalog and SHALL persist the exact selection on the role.
27+
28+
#### Scenario: Guided creation has multiple roots
29+
- **WHEN** the host catalog contains `/var/lib/evdb/databases` and `/data`
30+
- **THEN** the operator chooses one before confirming database creation
31+
32+
#### Scenario: Direct creation omits a required selection
33+
- **WHEN** direct database creation omits `--data-root` while multiple roots exist
34+
- **THEN** creation fails before source or generated files change
Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
1-
## 1. Host Configuration
1+
## 1. Root Catalog
22

3-
- [x] 1.1 Add the required host data root to source loading, validation, serialization, guided setup, direct initialization, and review.
3+
- [x] 1.1 Replace the single host root with a validated ordered catalog and explicit allowlisted placement on every database role.
44

5-
## 2. Database Storage
5+
## 2. Creation Flow
66

7-
- [x] 2.1 Derive database paths and managed directories from the configured root, and reject changes that conflict with an existing generated Compose bind.
7+
- [x] 2.1 Collect roots during initialization and select immutable placement during guided and direct database creation.
88

99
## 3. Verification
1010

11-
- [x] 3.1 Cover default, custom, unsafe, missing, and immutable-root behavior with focused tests and validate the change artifacts.
11+
- [x] 3.1 Cover catalog validation, per-role path derivation, creation selection, catalog edits, and immutable generated binds with focused checks.

0 commit comments

Comments
 (0)