Skip to content

feat: DB Migrations - #1661

Open
andypols wants to merge 5 commits into
finos:mainfrom
qube-rt:feat/db-migrations
Open

feat: DB Migrations#1661
andypols wants to merge 5 commits into
finos:mainfrom
qube-rt:feat/db-migrations

Conversation

@andypols

Copy link
Copy Markdown
Contributor

Description

This PR provides adds DB migrations. #1656 adds dateCreated and lastModified to repository records to support sorting or repository by date. This cannot be used without a corresponding migration of existing data.

Each migration is tracked by its own unique ID, rather than relying on a single global “current database version”. This means two branches can add separate migrations and be merged in either order. When both reach main, GitProxy checks the complete registry and runs whichever migration IDs have not yet been applied.

This gives us:

  • a clear history of data changes
  • one shared implementation across database backends
  • a record of which changes have already run
  • safe restart behaviour after a partial migration
  • optional rollback support
  • a consistent pattern for future schema and data changes

Example migration

For example a migration for #1656 could be implemented like this.

import type { Repo } from '../types';
import type { Migration } from './index';

export const addRepoDates: Migration = {
  id: '20260722-add-repo-dates',
  up: async (sink) => { 
    const repos = await sink.getRepos(); 
    for (const repo of repos) {
      if (repo.dateCreated && repo.lastModified) { 
        continue; 
      }

      const dateCreated =
        repo.dateCreated ?? (await sink.deriveCreatedAt(repo._id)) ?? new Date().toISOString();

      await sink.updateRepo({
          ...repo, 
          dateCreated, 
          lastModified: repo.lastModified ?? dateCreated, 
      });
    } 
  }, 
};

andypols added 4 commits July 28, 2026 14:53
Signed-off-by: Andrew Pols <andrew.pols@qube-rt.com>
Signed-off-by: Andrew Pols <andrew.pols@qube-rt.com>
Signed-off-by: Andrew Pols <andrew.pols@qube-rt.com>
Signed-off-by: Andrew Pols <andrew.pols@qube-rt.com>
@andypols
andypols requested a review from a team as a code owner July 28, 2026 19:29
@netlify

netlify Bot commented Jul 28, 2026

Copy link
Copy Markdown

Deploy Preview for endearing-brigadeiros-63f9d0 canceled.

Name Link
🔨 Latest commit f5e1eab
🔍 Latest deploy log https://app.netlify.com/projects/endearing-brigadeiros-63f9d0/deploys/6a75e322d54d9a00083ca07b

@linux-foundation-easycla

linux-foundation-easycla Bot commented Aug 7, 2026

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

@codecov

codecov Bot commented Aug 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.89744% with 47 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.08%. Comparing base (d68c99a) to head (f5e1eab).

Files with missing lines Patch % Lines
src/db/mongo/repo.ts 6.25% 15 Missing ⚠️
src/db/mongo/migrations.ts 56.52% 10 Missing ⚠️
src/db/file/migrations.ts 79.54% 9 Missing ⚠️
src/db/file/repo.ts 76.00% 6 Missing ⚠️
src/db/migrations/index.ts 90.47% 3 Missing and 1 partial ⚠️
src/db/migrations/example.ts 90.32% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1661      +/-   ##
==========================================
- Coverage   90.40%   90.08%   -0.32%     
==========================================
  Files          88       93       +5     
  Lines        8680     8874     +194     
  Branches     1608     1645      +37     
==========================================
+ Hits         7847     7994     +147     
- Misses        802      848      +46     
- Partials       31       32       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

The db mock in testProxy.test.ts was missing runMigrations, which
src/proxy/index.ts now imports and calls on startup, breaking CI.

Signed-off-by: Andrew Pols <andrew.pols@qube-rt.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant