Cairn

Self-hosted, S3-compatible object storage.

Cairn speaks the S3 API, stores object data as plain files on a normal filesystem, and keeps all metadata in an embedded SQLite database. There is no external database, clustering layer, or separate storage daemon to run. Point any S3 client at it, or use the built-in web console. It runs as one binary, and a distroless container image is also published.

Cairn is for people who want to host their own object storage on a homelab machine, a VPS, or a small production node, with the S3 API and a console but without operating a distributed system.

Releases are CI-gated and publish static linux/amd64 and linux/arm64 binaries with a SHA256SUMS manifest, plus a multi-arch image at ghcr.io/harsh-2002/cairn. Verify a download with sha256sum -c SHA256SUMS.

The console

The S3 wire protocol is a commodity; the console is the reason to pick Cairn. It ships inside the same binary — nothing extra to deploy.

^{Storage, compression and per-bucket usage across the node.}

Bucket browser — preview, download, tag, copy, share or delete any object.	In-place preview — images, video, audio, PDF, Markdown, JSON, CSV and text, without downloading.
Access — a validated bucket-policy editor, object ownership, Block Public Access.	Data protection — encryption at rest, Object Lock, lifecycle rules.
Credentials — mint short-lived scoped keys with a policy builder that explains them.	Metrics — requests, errors, throughput and latency from Cairn's own stored series.

More screens on the project site.

Features

• S3 API: buckets and objects, byte-range and conditional reads, checksums (CRC32, CRC32C, CRC64NVME, SHA-1, SHA-256), multipart upload, copy, bulk delete, and v1/v2 plus version listings. Versioning, tagging, CORS, lifecycle expiration, object lock (WORM), and presigned URLs. SigV4 (header and streaming-chunked) and Bearer authentication. The aws CLI and the standard AWS SDKs work against it.
• Web console: on its own port, manage buckets and users; browse, upload, download and share objects; preview images, video, audio, PDFs, Markdown, JSON, CSV and text in place without downloading them; mint access keys scoped by a policy; view storage, compression, and replication status; jump anywhere with a command palette; and see when a newer release is available (an hourly, opt-out check).
• Access control: bucket policies, ACLs, Block Public Access, Object Ownership, and short-lived credentials, mintable through the AWS-STS wire surface (AssumeRole, GetSessionToken) served on the S3 port or through the management API.
• Durability: writes are staged, fsynced, atomically renamed, and acknowledged after a single metadata commit. On restart Cairn reconciles and reclaims any orphaned data. Acknowledged writes survive power loss.
• Security: native TLS, AES-256-GCM encryption of secrets at rest with online master-key rotation, and server-side object encryption: per-bucket SSE-S3, an aws:kms request surface (label-only, see Scope), optional mandatory-encryption enforcement, and transparent encryption of all objects at rest (CAIRN_ENCRYPT_AT_REST).
• Storage efficiency: optional per-bucket block compression, with range reads that touch only the blocks they need.
• Operations: Prometheus metrics (with a ready-to-import dashboard for an external monitoring stack), liveness and readiness endpoints, structured logs, asynchronous bucket replication to another node or S3 endpoint, webhook event notifications, one-shot streaming import/migration of buckets and objects from another S3-compatible store (MinIO/Garage/R2/AWS/another Cairn) into this node, and a CLI for bootstrap, config validation, integrity checks, and backup and restore.

Install

The install script sets Cairn up on a host or with Docker, and updates an existing installation when you run it again:

curl -fsSL https://raw.githubusercontent.com/Harsh-2002/Cairn/main/install.sh | sudo sh

It detects Docker and offers it (or installs the binary with a systemd or OpenRC service), generates the master key and admin credentials, can enable TLS, and stores data in a Docker named volume or under /var/lib/cairn. The Docker setup lives in /opt/cairn so you can edit the compose file. Run sh install.sh --help for options such as --docker, --host, --update, and --uninstall.

Quickstart

To set it up by hand with Docker instead:

docker build -t cairn .

docker run -d --name cairn \
  -p 7373:7373 -p 7374:7374 \
  -v cairn-data:/data \
  -e CAIRN_MASTER_KEY="$(openssl rand -hex 32)" \
  cairn serve

• S3 API on :7373. Point any S3 client at http://localhost:7373.
• Web console on :7374. Open http://localhost:7374 in a browser.
• Default login is cairn / cairnadmin. Override it with CAIRN_ROOT_ACCESS_KEY and CAIRN_ROOT_SECRET_KEY before exposing a node. Liveness is at /healthz, readiness at /readyz, and Prometheus metrics at /metrics, all on the S3 port.

Smoke-test with the AWS CLI (configure it with the access key and secret above, any region):

aws --endpoint-url http://localhost:7373 s3 mb s3://demo
echo 'hello cairn' | aws --endpoint-url http://localhost:7373 s3 cp - s3://demo/hi.txt
aws --endpoint-url http://localhost:7373 s3 cp s3://demo/hi.txt -

CAIRN_MASTER_KEY is a 32-byte hex key that seals every secret at rest. Generate it once, keep it constant for the life of your data, and store it outside the database backup. Without it, a fixed development key is used, which is not safe for production.

To run without Docker, build the cairn binary from source and run cairn serve with the same environment. See CONTRIBUTING.md for the toolchain.

Configuration

Configuration is environment-only. Every setting is a CAIRN_* variable, validated on startup with cairn validate-config. There is no config file and there are no flags. Common variables:

Variable	Default	Purpose
CAIRN_DATA_DIR	./data	Root of staging and per-bucket object files
CAIRN_DB_PATH	./data/cairn.db	SQLite metadata file (same filesystem as the data)
CAIRN_MASTER_KEY	dev key	32-byte hex key sealing secrets at rest; set it in production
CAIRN_LISTEN_ADDR	0.0.0.0:7373	S3 API listener
CAIRN_WEB_ADDR	0.0.0.0:7374	Console and management API; set off to run headless
CAIRN_REGION	us-east-1	Region label and SigV4 scope
CAIRN_TLS_CERT_PATH / CAIRN_TLS_KEY_PATH	unset	Enable built-in TLS when both are set
CAIRN_MASTER_KEY_RING	unset	Key ring for online master-key rotation

The full reference is in docs/configuration.md. CLI subcommands: cairn serve (default), validate-config, bootstrap, integrity [--repair], backup, and restore.

Deploying

• Two listeners. Expose the S3 port to clients, and keep the console and management port on a trusted interface, firewalled, or disabled with CAIRN_WEB_ADDR=off. Do not expose a plaintext interface to an untrusted network.
• TLS. Terminate TLS at Cairn (set the cert and key paths; SIGHUP reloads) or at a reverse proxy in front.
• Redundancy. Put the data filesystem on redundant storage such as ZFS, RAID, or a replicated block volume. For cross-host redundancy, run a second node and enable bucket replication to it.
• Backups. Use cairn backup and cairn restore. See docs/backup-restore.md.

Operator guides: docs/operations.md, docs/deployment-kubernetes.md, docs/upgrade-rollback.md, docs/scaling-limits.md, docs/disaster-recovery.md, and docs/troubleshooting.md.

Performance

A like-for-like comparison against MinIO, both on the same 2 vCPU / 16 GB host, using MinIO's own warp benchmark. Both run single-node, single-drive, over plaintext HTTP; each figure is the median of three runs.

Throughput (higher is better; obj/s for small objects, MiB/s for large):

Operation	Object size	Cairn	MinIO	Ratio
PUT	4 KiB	1696 obj/s	1308 obj/s	Cairn 1.30×
PUT	8 MiB	115 MiB/s	128 MiB/s	MinIO 1.12×
GET	4 KiB	3431 obj/s	1815 obj/s	Cairn 1.89×
GET	8 MiB	1463 MiB/s	801 MiB/s	Cairn 1.83×
STAT	4 KiB	3494 obj/s	1824 obj/s	Cairn 1.92×
DELETE	4 KiB	7442 obj/s	5236 obj/s	Cairn 1.42×
LIST	4 KiB	21183 obj/s	10288 obj/s	Cairn 2.06×
MIXED	1 MiB	285 MiB/s	271 MiB/s	Cairn 1.05×

Resource use while serving the same workload (lower is better):

	Mean CPU	Peak memory
Cairn	76%	106 MB
MinIO	88%	1072 MB

Absolute numbers depend on the host and vary between runs; the ratio is what to read. Reproduce it with the harness CI runs on every push (bench-compare):

BIN=target/release/cairn bash conformance/bench_compare.sh

Scope

Cairn is single-node by design: one process, one data filesystem, one metadata database. Cross-host redundancy comes from asynchronous bucket replication, which is eventually consistent with observable lag, rather than from clustering. Drive redundancy is left to the storage underneath. Server-side encryption is supported — SSE-S3 (per-bucket AES-256 default encryption, optionally mandatory), an aws:kms request surface for SDK compatibility (the key id is a validated label over the same node master key, not independent key material, so it is not cryptographic tenant isolation), and optional transparent encryption of every object at rest (CAIRN_ENCRYPT_AT_REST). A dedicated external KMS with distinct per-key material is not implemented. The target is homelab and small-to-mid production workloads that want the S3 API and a console without running a distributed system.

Roadmap

Planned work, tracked against the architecture in docs/delivery.md (Phase 15). These are additive and do not change the S3 or management API.

• Integration with an external KMS (distinct per-key material and tenant isolation); the current aws:kms surface is a label over the node master key, and transparent encryption of all objects at rest already ships (CAIRN_ENCRYPT_AT_REST).
• Lifecycle transition to a remote cold tier, with a restore-from-cold workflow.
• Zero-copy reads with kernel TLS, building on the existing sendfile fast path.
• Signed release artifacts (cosign) and SBOM attestation.

Documentation

The engineering specification and operator guides are in docs/; start at docs/CLAUDE.md for the index. docs/s3-api-matrix.md lists which S3 operations are supported, partial, or out of scope.

License

Apache-2.0 (LICENSE). The license is permanent; there is no enterprise edition or open-core split. Governance and maintenance are described in GOVERNANCE.md. See SECURITY.md for the security policy, CODE_OF_CONDUCT.md for community expectations, and CONTRIBUTING.md for the developer workflow.