feat!: make guided setup reviewed and verifiable

- Embed the pinned lego provider catalog and verify DNS credentials by
  issuing one wildcard certificate reused by every database route
- Support local and rclone Restic repositories with preflight and safe
  non-root execution
- Preserve configurable Postgres identities across runtime, backups,
  PgBouncer, and connection details
- Add append-only review and edit flows with contextual output and help

BREAKING CHANGE: Postgres roles must declare username and database
fields.
DNS settings must match the bundled provider catalog. Rclone
repositories
must use 'rclone:REMOTE:PATH', while local repositories must omit
'rclone_config'.

Author: evannotfound

AGENTS.md

@@ -8,6 +8,3 @@
 - Never commit `.env`, backup files, dumps, rclone config, or resolved secrets.
 - Source config contains `op://` references only.
 - Tests use disposable containers and local Restic repositories only.
-- Do not run deployment, Restic, Docker, cron, or systemd changes on `montreal-01` while
-  implementing `build-db-system`.
-- Production migration is a separate OpenSpec change.

Makefile

@@ -48,6 +48,7 @@ integration-collect:
 
 binary:
 	$(PYINSTALLER) --clean --noconfirm --onefile --name evdb --paths src \
+		--add-data src/evdb/dns_providers.json:evdb \
 		--add-data src/evdb/units:evdb/units src/evdb/__main__.py
 
 require-db:

README.md

@@ -39,20 +39,30 @@ Choose a host ID and base domain. For a host ID of `example-01` and a base domai
 `storage.example.com`, arrange for `*.example-01.storage.example.com` to resolve to the server from
 your clients. evdb uses DNS-01 to issue certificates but does not create DNS records.
 
-Create a private file containing the environment variables Traefik requires for your DNS provider,
-one `KEY=VALUE` per line. Keep its path for initialization.
+Guided initialization includes every DNS provider supported by the pinned Traefik release. It shows the
+provider's documented variables and help URL, accepts credentials with masked input, and verifies them by
+obtaining one certificate for `*.<host-id>.<base-domain>`. For non-interactive setup, prepare a private
+file containing documented `KEY=VALUE` lines and pass it with `--dns-file`.
 
-Configure your backup remote as the normal non-root user who will own remote access:
+For remote backups, configure rclone as the normal non-root user who will own remote access:
 
 ```sh
 rclone config
 rclone config file
 rclone lsd remote:
 ```
 
-Replace `remote` with the configured remote name. Set the reported rclone configuration file to mode
-`0600`; evdb requires its absolute path and uses it without copying it. A typical Restic repository is
-`rclone:remote:evdb/example-01`.
+Set the reported rclone configuration file to mode `0600`; evdb uses it in place. Guided setup reads its
+configured remotes and asks you to select one, then defaults the repository path to `evdb/<host-id>`.
+
+For a local Restic repository, create only its parent as the non-root backup user. For example:
+
+```sh
+install -d -m 0700 "$HOME/restic"
+```
+
+Guided setup can then use an absent child such as `$HOME/restic/example-01`. Local mode does not require
+rclone and derives the Restic process identity from that safe parent.
 
 ## Install evdb
 
@@ -72,17 +82,18 @@ Start guided initialization:
 sudo evdb init
 ```
 
-Enter the host ID, base domain, ACME email, Traefik DNS provider name, Restic repository, DNS
-credential file, and absolute rclone configuration path prepared above. Leave the initial Restic
-password blank to generate one.
+The append-only setup flow validates each answer, searches supported DNS providers, collects documented
+credentials, selects rclone or local storage, and shows a redacted review before Apply. Leave the initial
+Restic password blank to generate one. Setup waits for the wildcard certificate and verifies or creates
+the Restic repository before enabling automatic backups.
 
 Initialization stores configuration under `/etc/evdb`, managed data under `/var/lib/evdb`, starts the
 TLS router, initializes the Restic repository, and enables automatic daily backups.
 
 ## Create your first database
 
-Project names must end in `-dev-N`, `-test-N`, or `-prod-N`. Create Postgres and retrieve its
-TLS-secured connection URL:
+Project names must end in `-dev-N`, `-test-N`, or `-prod-N`. Create Postgres with a generated managed
+login and retrieve its TLS-secured connection URL:
 
 ```sh
 sudo evdb database add notes-prod-01 postgres
@@ -96,6 +107,18 @@ postgresql://default:<password>@notes-prod-01.example-01.storage.example.com:543
 Use the URL with any standard Postgres client. `database info` prints complete credentials and
 therefore requires a terminal.
 
+Guided creation has an Advanced option for preserving an existing username, database name, and password
+when you import data yourself. Automation can provide the same creation-only identity without exposing
+the password in process arguments:
+
+```sh
+sudo evdb database add imported-prod-01 postgres \
+  --username app_user --database-name app_db --password-file /private/postgres-password
+```
+
+evdb does not import the source database or rotate that identity after creation; use standard Postgres
+dump and restore tools for the data transfer.
+
 ## CLI usage
 
 Run `sudo evdb` for the guided terminal interface. Direct commands execute immediately and are useful

openspec/changes/archive/2026-07-25-build-db-system/design.md

@@ -1,6 +1,6 @@
 ## Context
 
-`montreal-01` currently has 25 managed database instances under `/home/ubuntu/databases`: 14 Postgres, 7 Dragonfly, and 4 Redis. The current scripts back up only 6 Postgres and 4 KV instances. They delete prior local files before upload is known to be good, handle each engine as one batch, do not check Restic results well enough, and cannot produce a safe Dragonfly RDB.
+`production-host` currently has 25 managed database instances under `/home/ubuntu/databases`: 14 Postgres, 7 Dragonfly, and 4 Redis. The current scripts back up only 6 Postgres and 4 KV instances. They delete prior local files before upload is known to be good, handle each engine as one batch, do not check Restic results well enough, and cannot produce a safe Dragonfly RDB.
 
 Read-only Docker inspection confirmed that the managed database, PgBouncer, HTTP sidecar, and native Traefik containers have no Docker CPU, memory, cpuset, or PID limits. Source config records that current resource contract as `unlimited`, and generated Compose preserves it by adding no resource limit keys.
 
@@ -27,7 +27,7 @@ Production uses Ubuntu 22.04 ARM64, Python 3.10, Docker 29, Compose 2.40, Restic
 
 **Non-Goals:**
 
-- Deploying any part of this repository to `montreal-01`.
+- Deploying any part of this repository to `production-host`.
 - Replacing cron, changing Compose ownership, restarting databases, changing images, upgrading Restic, rotating secrets, or touching the OneDrive repositories.
 - Changing any running Redis instance to Dragonfly.
 - Moving database data or redesigning Traefik.
@@ -65,7 +65,7 @@ src/evdb/
 Related deployment and test files use the same names:
 
 ```text
-config/montreal-01/{host,postgres,kv}.yml
+config/production-host/{host,postgres,kv}.yml
 compose/{postgres,redis,dragonfly,traefik}.yml.j2
 ansible/{hosts,backup,databases,restore}.yml
 ansible/roles/{base,app,backup,postgres,kv,traefik}/
@@ -78,7 +78,7 @@ The alternative flat layout was shorter but would mix engine backup and restore
 
 ### Source YAML and runtime JSON
 
-Humans edit three small YAML files for `montreal-01`. Ansible and local config tools render one JSON file per instance. Production Python reads only JSON.
+Humans edit three small YAML files for `production-host`. Ansible and local config tools render one JSON file per instance. Production Python reads only JSON.
 
 Each instance keeps two simple sections:
 
@@ -181,11 +181,11 @@ Compose templates are separate for Postgres, Redis, and Dragonfly. They preserve
 Traefik is the only service that publishes host ports 5432 and 6379. Postgres, PgBouncer, Redis, and Dragonfly stay internal.
 
 ```text
-<instance>.postgres-montreal-01.storage.evanovation.com:5432
+<instance>.postgres-production-host.storage.evanovation.com:5432
   -> Traefik TLS HostSNI
   -> <instance>-pgbouncer-1:5432
 
-<instance>.kv-montreal-01.storage.evanovation.com:6379
+<instance>.kv-production-host.storage.evanovation.com:6379
   -> Traefik TLS HostSNI
   -> <instance>-redis-1:6379
 ```
@@ -199,7 +199,7 @@ Every enabled KV HTTP sidecar runs in its instance's Compose project, connects t
 Public HTTP is owned by an external proxy:
 
 ```text
-https://<instance>.kv-montreal-01.storage.evanovation.com
+https://<instance>.kv-production-host.storage.evanovation.com
   -> external proxy :443
   -> 127.0.0.1:133xx
   -> serverless-redis-http
@@ -214,7 +214,7 @@ The `app` role installs a root-owned release at `/opt/evdb/releases/<git-sha>` a
 
 The service account owns state and secret files and belongs to the Docker group. Documentation calls out that Docker access is root-equivalent.
 
-Production is never the default target. Make and Ansible require an explicit target, and any production apply also requires an explicit production flag. The integration suite creates disposable local inventory and does not use `montreal-01`. HTTP tests exercise only disposable sidecars and never contact an external proxy.
+Production is never the default target. Make and Ansible require an explicit target, and any production apply also requires an explicit production flag. The integration suite creates disposable local inventory and does not use `production-host`. HTTP tests exercise only disposable sidecars and never contact an external proxy.
 
 ### Systemd jobs and status
 

openspec/changes/archive/2026-07-25-build-db-system/proposal.md

@@ -13,7 +13,7 @@ Evanovation's database setup is spread across host files, shared Compose templat
 - Add Restic upload, snapshot tracking, retention, prune, and repository check support while keeping repository format v1.
 - Add Ansible, Compose templates, and systemd units for the finished system and test them on disposable local hosts.
 - Add local integration tests, CI, secret scanning, and operator documentation.
-- Keep production unchanged. Deployment to `montreal-01`, replacement of cron, Restic upgrades, image changes, secret rotation, container restarts, and Compose takeover will be handled by a later production move.
+- Keep production unchanged. Deployment to `production-host`, replacement of cron, Restic upgrades, image changes, secret rotation, container restarts, and Compose takeover will be handled by a later production move.
 
 ## Capabilities
 

openspec/changes/archive/2026-07-25-build-db-system/specs/config/spec.md

@@ -59,18 +59,18 @@ Validation SHALL reject duplicate instance ids and Compose projects within the s
 - **THEN** validation accepts the shared listeners and checks instance routes by unique domain
 
 ### Requirement: HTTP settings
-Each KV instance SHALL record whether its serverless HTTP sidecar is enabled, its unique loopback port, intended external domain, pinned image, token reference, and connection limit. The initial `montreal-01` config SHALL enable the sidecar for all 11 KV instances.
+Each KV instance SHALL record whether its serverless HTTP sidecar is enabled, its unique loopback port, intended external domain, pinned image, token reference, and connection limit. The initial `production-host` config SHALL enable the sidecar for all 11 KV instances.
 
 #### Scenario: External route data is recorded
 - **WHEN** a KV instance defines its intended HTTP domain and loopback port
 - **THEN** validation checks the contract without inspecting or changing the external proxy
 
 #### Scenario: Initial HTTP coverage is checked
-- **WHEN** the initial `montreal-01` config is validated
+- **WHEN** the initial `production-host` config is validated
 - **THEN** all 11 KV instances have enabled HTTP settings with unique loopback ports
 
 ### Requirement: Standard HTTP domains
-Target HTTP domains SHALL use `<instance>.kv-montreal-01.storage.evanovation.com`. Current legacy domains MAY be recorded as current facts, but the target SHALL NOT use `kv-na01.storage.evanovation.com`.
+Target HTTP domains SHALL use `<instance>.kv-production-host.storage.evanovation.com`. Current legacy domains MAY be recorded as current facts, but the target SHALL NOT use `kv-na01.storage.evanovation.com`.
 
 #### Scenario: Legacy HTTP domain is recorded
 - **WHEN** a current route uses the old `kv-na01.storage.evanovation.com` suffix

openspec/changes/archive/2026-07-25-build-db-system/specs/deploy/spec.md

@@ -65,7 +65,7 @@ Ansible, rendered Compose, JSON config, and systemd units SHALL be tested on dis
 - **THEN** the test host can run config validation and local backup and restore commands without production credentials
 
 ### Requirement: Production guard
-Production SHALL NOT be a default Ansible target. Any future production run SHALL require the production inventory and an explicit apply flag. This change SHALL NOT run those playbooks against `montreal-01`.
+Production SHALL NOT be a default Ansible target. Any future production run SHALL require the production inventory and an explicit apply flag. This change SHALL NOT run those playbooks against `production-host`.
 
 #### Scenario: Deploy command has no target
 - **WHEN** a deploy target is omitted

openspec/changes/archive/2026-07-25-build-db-system/specs/http/spec.md

@@ -43,7 +43,7 @@ The repository SHALL record each sidecar's intended domain and loopback port as
 The missing `oai-co-prod-02` route and two current `kv-na01.storage.evanovation.com` domains SHALL be recorded only as informational follow-ups for the external proxy owner during the later production move.
 
 #### Scenario: Initial HTTP contract is rendered
-- **WHEN** all `montreal-01` KV config is rendered
+- **WHEN** all `production-host` KV config is rendered
 - **THEN** it contains 11 unique intended domains and 11 unique loopback ports without a proxy route plan
 
 ### Requirement: HTTP authentication

openspec/changes/archive/2026-07-25-simplify-db-manager/design.md

@@ -21,7 +21,7 @@ The implementation must continue to use subprocess argument arrays, keep product
 
 **Non-Goals:**
 
-- Deploying this redesign to `montreal-01` or changing any running production container.
+- Deploying this redesign to `production-host` or changing any running production container.
 - Providing a web dashboard, persistent control-plane service, or public API.
 - Managing the external HTTP proxy.
 - Purging database data, backup history, or 1Password items.
@@ -41,7 +41,7 @@ This is preferred over YAML anchors, profiles, or inheritance because those mech
 
 ### Generated lock and observed state
 
-`host.lock.json` will be a tool-owned, secret-free file beside `host.yml`. It will record the platform-specific digest resolved for each host-level image and stable assigned values that cannot be recomputed safely, initially HTTP loopback ports. Existing Montreal ports will seed the first lock; new ports will use the next free value in the configured range. Operators may review and commit the lock but never edit it manually.
+`host.lock.json` will be a tool-owned, secret-free file beside `host.yml`. It will record the platform-specific digest resolved for each host-level image and stable assigned values that cannot be recomputed safely, initially HTTP loopback ports. Existing production ports will seed the first lock; new ports will use the next free value in the configured range. Operators may review and commit the lock but never edit it manually.
 
 Observed Docker facts will not be source configuration. `evdb plan` will obtain the active release manifest and live status over SSH and compare them with the normalized desired model. This replaces the public `current` and `target` sections.
 
@@ -67,7 +67,7 @@ Items are not deleted when config writes, applies, rollbacks, or retirements fai
 
 `evdb show <database>` will combine derived configuration, active release facts, and live engine status into one detail view. For Postgres it will show engine and image versions, state, hostname, port, username, database name, TLS requirement, 1Password item, data path, container name, backup summary, and a complete `postgresql://` URL. For Redis and Dragonfly it will show the corresponding native TLS `rediss://` URL and, when enabled, the public HTTP endpoint and token.
 
-The command intentionally reveals the full connection URL and HTTP token every time, matching the existing Montreal manager behavior selected by the operator. It will resolve credentials from 1Password on the local controller, percent-encode URL components, and print them directly to the terminal. Credentials will not be requested from the remote host, included in SSH or child-process arguments, emitted to structured logs, or persisted in state. If local 1Password authentication cannot resolve a required field, the command will fail rather than print a misleading partial URL.
+The command intentionally reveals the full connection URL and HTTP token every time, matching the existing production manager behavior selected by the operator. It will resolve credentials from 1Password on the local controller, percent-encode URL components, and print them directly to the terminal. Credentials will not be requested from the remote host, included in SSH or child-process arguments, emitted to structured logs, or persisted in state. If local 1Password authentication cannot resolve a required field, the command will fail rather than print a misleading partial URL.
 
 This explicit details command is the only normal output allowed to contain credentials. Status, plan, logs, backup history, release history, and all JSON machine output remain secret-free.
 
@@ -130,7 +130,7 @@ Commands accept a plain database name when it is unique. If the same name exists
 ## Migration Plan
 
 1. Add the new source parser and normalizer alongside fixtures that prove concise entries expand to the current 25-instance deployment contract.
-2. Convert Montreal source config to one `host.yml`, delete the two instance files, and seed `host.lock.json` with current image digests and HTTP ports.
+2. Convert production source config to one `host.yml`, delete the two instance files, and seed `host.lock.json` with current image digests and HTTP ports.
 3. Add the local controller, remote JSON protocol, 1Password manager, and status/lifecycle commands against fakes and disposable hosts.
 4. Adapt Ansible and Compose rendering to normalized input, then add staged release activation and rollback on disposable Docker projects.
 5. Extend isolated restore code with persistent candidates and promotion, including forced failure recovery tests.

openspec/changes/archive/2026-07-25-simplify-db-manager/proposal.md

@@ -9,7 +9,7 @@ The current system exposes migration inventory and deployment internals instead
 - Add a generated lock file for resolved image digests and stable allocated values such as HTTP ports; operators do not edit this file.
 - Add a short `evdb` command that runs locally and uses SSH to manage the configured production host.
 - Add plan, apply, create, start, stop, restart, status, and log workflows while keeping Ansible as an internal deployment mechanism.
-- Add a database details command that shows live metadata, native and HTTP endpoints, and complete usable connection URLs like the existing Montreal manager.
+- Add a database details command that shows live metadata, native and HTTP endpoints, and complete usable connection URLs like the existing production manager.
 - Manage convention-based database passwords and tokens in 1Password from the local controller without exposing secret values in arguments, config, state, or logs; the details command intentionally reveals requested credentials only in terminal output.
 - Present backup creation, backup history, and full restore verification as direct operator workflows.
 - Add deployment release history and rollback without changing database data.
@@ -35,9 +35,9 @@ None. The repository has no archived main capability specs; these capabilities s
 
 ## Impact
 
-- Replaces the source configuration model and command syntax in `src/evdb/config.py`, `src/evdb/cli.py`, the checked-in Montreal configuration, tests, and documentation.
+- Replaces the source configuration model and command syntax in `src/evdb/config.py`, `src/evdb/cli.py`, the checked-in production configuration, tests, and documentation.
 - Adds local SSH transport, 1Password item management, desired-versus-deployed planning, Compose lifecycle control, and deployment release manifests.
 - Changes Ansible roles and Compose rendering to consume normalized generated configuration rather than three human-authored inventory files.
 - Extends existing backup, Restic, status, and isolated restore code instead of replacing their validated data-handling paths.
 - Requires a write-capable local 1Password session or service account and SSH/sudo access to the managed host.
-- Does not deploy to, restart, or otherwise write to `montreal-01` as part of this change.
+- Does not deploy to, restart, or otherwise write to `production-host` as part of this change.