amazee.ai

This repository contains the backend and frontend services for the amazee.ai application. The project is built using a modern tech stack including Python FastAPI for the backend, Next.js with TypeScript for the frontend, and PostgreSQL for the database.

🚀 Tech Stack

• Backend: Python FastAPI
• Frontend: Next.js + TypeScript
• Database: PostgreSQL
• Testing: Pytest (backend), Jest (frontend)
• Containerization: Docker & Docker Compose
• Orchestration: Kubernetes with Helm

📦 Releasing and deploying

Three steps, all of them yours to start.

1. Run the Release workflow from the Actions tab. Pick patch, minor or major; optionally give a commit on dev to release instead of its tip.
2. Review and merge the Deploy vX.Y.Z to production pull request it opens.
3. Promote main to prod in Lagoon, by hand. Nothing before this touches production.

What the Release workflow does

• Reads the newest vX.Y.Z tag and raises the part you chose.
• Cuts release/vX.Y.Z from main, merges dev into it, then writes the new version into every file that carries one.
• Tags that commit and creates the GitHub Release, with user-facing notes.
• Publishes the Helm charts to GHCR from that branch, so the chart versions are the bumped ones.
• Opens release/vX.Y.Z → main with a description read from git.

It branches from main rather than dev on purpose. main carries the bump commit from the previous release and dev does not, so from the second release onward a branch cut from dev conflicts on all six version files, and would need a third pull request carrying main back to dev to avoid it.

Because the head is a release branch and not dev, its description keeps describing the code it was written for however long the review takes.

The deploy pull request description

Generated by .github/scripts/deploy-pr-body.sh from git log and git diff --name-status. It names the schema migrations that alembic will run, then every changed file that decides how production runs — .lagoon.yml, docker-compose.yml, helm/, the Dockerfiles, scripts/ cron entrypoints, app/core/config.py. No model is involved, so it cannot be truncated and cannot report a deploy as carrying no migrations when it carries three.

The GitHub Release notes

Written by a model through the LiteLLM proxy, from the diff between the previous tag and the new one. Prose earns its place here: the reader is a user of the API, not a deploy approver. The prompt is .github/prompts/changelog-user.md and the step needs the LITELLM_API_KEY secret. Without it the release still happens, with GitHub's own generated notes. If nothing user-visible changed, the model says so and only the generated notes are used.

The public GitHub Releases page is the changelog. There is no CHANGELOG.md.

Where the version lives

.github/scripts/bump-version.sh writes it into:

• app/__version__.py — served by /version and the OpenAPI schema
• helm/Chart.yaml — chart version, appVersion, and the subchart pins
• helm/charts/backend/Chart.yaml, helm/charts/frontend/Chart.yaml
• frontend/package.json, frontend/package-lock.json

In the YAML and Python files the version lines carry an x-release-please-version marker, so the script only ever rewrites a value that is meant to be a version. Do not edit these by hand.

One consequence to know: dev never receives the bump commit, so the dev environment reports the version of the release before last. The tag and main are authoritative.

📋 Prerequisites

• Docker and Docker Compose
• Make (for running convenience commands)
• Node.js and npm (for local frontend development)
• Python 3.x (for local backend development)

🛠️ Setup & Installation

1. Clone the repository:

   git clone [repository-url]
   cd [repository-name]

2. Install node dependencies

   cd frontend
   npm install
   cd ../

3. Environment Setup:

   • Copy any example environment files and configure as needed
   • Ensure all required API keys are set
   • Ensure you have set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY variables

4. Start the services:

   docker-compose up -d

   This will start:

   • PostgreSQL database (port 5432)
   • Backend service (port 8000)
   • Frontend service (port 3000)
   • litellm service (port 4000)

🧪 Running Tests

Backend Tests

make backend-test       # Run backend tests
make backend-test-cov   # Run backend tests with coverage report
make backend-test-regex # Waits for a string which pytest will parse to only collect a subset of tests

LiteLLM Integration Tests

make integration-test                     # Against the default proxy image
make integration-test LITELLM_TAG=v1.95.0 # Gate a specific LiteLLM version
make integration-test LITELLM_IMAGE=ghcr.io/amazeeio/litellm-lagoon-base-database LITELLM_TAG=v1.96.2

Runs tests/integration/ against real LiteLLM proxies (budgets, spend tracking, caps, budget cycles incl. the 31d/1mo special cases, lifecycle, user sync, model sync). Uses docker-compose.integration.yml under its own compose project, so it never touches your dev containers or volumes. Requires docker compose >= 2.24. CI: the "LiteLLM Integration Gate" workflow (manual dispatch to gate a version bump + weekly cron); see litellm-integration-tests-plan.md for design and prerequisites.

Frontend Tests

make frontend-test    # Run frontend tests if they exist

All Tests

make test-all        # Run both backend and frontend tests

Cleanup

make test-clean      # Clean up test containers and images

🧹 Cleanup

To clean up test containers and images:

make test-clean

🚀 Local Development

1. Start all services in development mode:

   docker-compose up -d

   Local Compose automatically picks up docker-compose.override.yml, which swaps the Postgres services to the pgvector image so private AI key creation can run locally without changing Lagoon deployment config.

2. View logs for all services:

   docker-compose logs -f

3. View logs for a specific service:

   docker-compose logs -f [service]  # e.g. frontend, backend, postgres

4. Restart a specific service:

   docker-compose restart [service]

5. Stop all services:

   docker-compose down

The development environment includes:

• Hot reloading for frontend (Next.js) on port 3000
• Hot reloading for backend (Python) on port 8800
• PostgreSQL database on port 5432

Access the services at:

• Frontend: http://localhost:3000
• Backend API: http://localhost:8800

💳 Limit Precedence and Key Ownership

Private key creation supports two ownership modes:

• owner_id: user-owned key.
• team_id: team-owned shared key.
• owner_id and team_id are mutually exclusive.
• If both are omitted, key ownership defaults to the current user (owner_id=current_user.id).

Limit controls and precedence:

• Team cap: PUT /spend/{region_id}/team/{team_id}/budget
• Team member cap (user within team): PUT /spend/{region_id}/team/{team_id}/member/{user_id}/budget
• Key cap: PUT /spend/{region_id}/key/{key_id}/budget
• Effective enforcement is the strictest applicable gate for the request context.

Scenario A: Team cap $5 shared across users/keys

• Use team-owned keys (team_id) for shared team usage.
• Create keys via POST /private-ai-keys with team_id.
• Set the team budget cap via PUT /spend/{region_id}/team/{team_id}/budget.
• All team keys/users spend from the same team budget pool until the team cap is reached.

Scenario B: User cap $2 within a team (across that user's keys)

• Use user-owned keys (owner_id) for keys tied to a specific user.
• Create keys via POST /private-ai-keys with owner_id.
• For users inside a team, set user budget with the team-member endpoint (not user-only endpoint).
• Use PUT /spend/{region_id}/team/{team_id}/member/{user_id}/budget.
• The member cap applies across that user's keys in the specified team.

Scenario C: Per-key cap $2 for each key

• Set key budgets directly via the key spend endpoint.
• Use PUT /spend/{region_id}/key/{key_id}/budget.
• Each key is enforced independently.
• Team and team-member limits can still apply as additional ceilings.

Note: Spend enforcement in LiteLLM is evaluated on spend updates, so the blocking request is typically the first request after crossing a cap.

♻️ Team Lifecycle & Hard Delete

Teams go through a three-stage lifecycle managed by background workers.

Stages

Stage	Trigger	What happens
Active	Team created	Normal operation
Soft-deleted	>76 days inactive (no API activity) + 14-day grace after warning email; or manual POST /teams/{id}/soft-delete	deleted_at set; all LiteLLM keys expired (duration=0d); users deactivated. POOL teams are exempt from automatic soft-delete.
Hard-deleted	deleted_at is ≥ 90 days ago	All data permanently removed (GDPR requirement)

Hard-delete cascade order

When hard_delete_expired_teams() runs (daily at 03:00 via cron), it deletes each expired team's data in this order to respect FK constraints:

1. limited_resources (team + user rows)
2. LiteLLM keys (remote call, best-effort)
3. spend_caps (team-, user-, and key-scoped)
4. ai_tokens (private AI keys) from DB
5. api_tokens, user_admin_regions (user FK tables — no ON DELETE CASCADE)
6. audit_logs.user_id set to NULL (rows preserved for audit history)
7. user_spend_cache (email-keyed stale cache)
8. users
9. team_products, team_regions
10. Audit log entry written (action=team.hard_delete)
11. teams (cascades team_metrics automatically)

Restore

A soft-deleted team can be restored by a system admin via POST /teams/{id}/restore. The restore:

• Clears deleted_at and reactivates all users
• Re-provisions the LiteLLM team and users in every active region (idempotent)
• Un-expires all keys in LiteLLM

If LiteLLM re-provisioning fails for any region, the team is still marked restored in the DB and the response includes a "warning" field listing the affected regions. Check the audit_logs table (action=team.restore) for the full litellm_failed_regions detail.

Manual trigger

# Trigger the hard-delete job manually on the backend container
python scripts/trigger_hard_delete_job.py

If you have a database dump, you can restore it into your local PostgreSQL service following these steps:

1. Extract the dump:

   mkdir -p ./restore-data
   tar -xf the-postgres-database-dump.tar -C ./restore-data

2. Prepare the restore script: The dump should contain a restore.sql file, which then contains placeholders and likely a different database name. Update it for your local environment:

   # Replace the data path placeholder
   sed -i '' 's/\$\$PATH\$\$/\/tmp\/restore/g' ./restore-data/restore.sql
   # Replace the dumped database name (`dumped-database-example-name`) with your local one (e.g. `postgres_service`)
   sed -i '' 's/dumped-database-example-name/postgres_service/g' ./restore-data/restore.sql

3. Stop the backend: To prevent active connections during the restoration, stop the backend container:

   docker compose stop backend

4. Get the name of the postgres container Copy the name e.g. amazeeai-postgres-1 and replace <postgres-container-name> in the following commands.

   docker compose ps

5. Transfer and restore: Copy the files to the database container, fix permissions, and run the restoration:

   # Create directory and copy files
   docker exec <postgres-container-name> mkdir -p /tmp/restore
   docker cp ./restore-data/. <postgres-container-name>:/tmp/restore/
   
   # Fix permissions so the postgres user can read the .dat files
   docker exec <postgres-container-name> chown -R postgres:postgres /tmp/restore
   
   # Run the restoration script
   docker exec <postgres-container-name> psql -U postgres -f /tmp/restore/restore.sql

6. Restart and Clean up:

   # Start the backend again
   docker compose start backend
   
   # Optional: remove temporary files from the container
   docker exec <postgres-container-name> rm -rf /tmp/restore

🛠️ Development Workflow

We follow a structured branching and deployment process to ensure stability across environments.

1. Feature Development

• Default Branch: dev is the default branch, and it is linked to the dev environment on Lagoon.
• Branching: Always create new feature branches from dev. Bugfixes can potentially be created from the main branch if they need to be merged into main and prod faster than in-progress dev work.
• Review: Create a Pull Request (PR) back into dev. All PRs must be reviewed and tested locally before merging.

2. Testing & Staging

• Dev Testing: After merging, verify your changes on the dev environment.
• Staging: Once verified on dev, create a PR from dev to main. The main branch serves as our Stage environment.

3. Production Deployment

• Lagoon: Deployments are managed via Lagoon.
• Promotion: Deploy to Prod by promoting the build from the main branch directly on the Lagoon Dashboard or via Lagoon CLI.

👥 Contributing

1. Create a new branch from dev: git checkout -b feature/my-feature
2. Make your changes and commit.
3. Run the test suite: make test-all
4. Submit a pull request to the dev branch.

📁 Project Structure

.
├── app/                   # Backend Python code
├── docs/                  # Documentation around design decisions
├── frontend/              # React frontend application
├── tests/                 # Backend tests
├── scripts/               # Utility scripts
├── docker-compose.yml     # Docker services configuration
├── Dockerfile             # Backend service Dockerfile
├── Dockerfile.test        # Test environment Dockerfile
└── Makefile               # Development and test commands

🔑 Environment Variables

Backend

• DATABASE_URL: PostgreSQL connection string
• SECRET_KEY: Application secret key
• DYNAMODB_ROLE_NAME: role to assume for accessing DDB resources (created by terraform)
• SES_ROLE_NAME: Role to assume for SES access (created by terraform)
• SES_SENDER_EMAIL: Validated identity in SES from which emails are sent
• ENV_SUFFIX: Naming suffix to differentiate resources from different environments. Defaults to dev.
• SES_REGION: Optional, defaults to eu-central-1
• DYNAMODB_REGION: Optional, defaults to eu-central-2
• MOAD_API_KEY: API key for MOAD service authentication
• PERIODIC_TOPUP_EXPIRY_DAYS: Days before periodic top-up budget expires (default: 365)

Frontend

• NEXT_PUBLIC_API_URL: Backend API URL

👥 Contributing

1. Create a new branch for your feature
2. Make your changes
3. Run the test suite
4. Submit a pull request

📄 License

This project is licensed under the Apache License, Version 2.0 - see below for details:

Copyright 2024 amazee.io

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

For the full license text, please see http://www.apache.org/licenses/LICENSE-2.0