Skip to content

PG19: fix backend crash on ALTER PUBLICATION ... SET ALL TABLES EXCEPT and restore EXCEPT membership across Citus conversions - #8741

Open
ibrahim halatci (ihalatci) wants to merge 5 commits into
pg19-supportfrom
ihalatci-fix-pg19-publication-crash
Open

PG19: fix backend crash on ALTER PUBLICATION ... SET ALL TABLES EXCEPT and restore EXCEPT membership across Citus conversions#8741
ibrahim halatci (ihalatci) wants to merge 5 commits into
pg19-supportfrom
ihalatci-fix-pg19-publication-crash

Conversation

@ihalatci

Copy link
Copy Markdown
Contributor

Closes #8730

Problem

On PostgreSQL 19, the backend SEGFAULTs when Citus is installed and a user runs:

ALTER PUBLICATION testpub_foralltables_excepttable SET ALL TABLES EXCEPT (TABLE testpub_tbl2);

This is reproducible in check-vanilla (publication test), where the crash also cascades into a long tail of unrelated vanilla failures. postmaster.log shows the backend terminated by signal 11.

Symbolized stack:

quote_identifier (rdi = 0x0)
  AppendPublicationObjects
  DeparseAlterPublicationStmtExtended
  QualifyPublicationObjects
  PreprocessAlterPublicationStmt
  citus_ProcessUtility

Root cause

AppendPublicationObjects() in deparse_publication_stmts.c special-cased only PUBLICATIONOBJ_TABLE. Every other object kind fell through into the TABLES IN SCHEMA branch, which does:

appendStringInfo(buf, "%s", quote_identifier(publicationObject->name));

PG19 adds PUBLICATIONOBJ_EXCEPT_TABLE, which carries its relation in publicationObject->pubtable and leaves publicationObject->name == NULL. Citus therefore passed NULL to quote_identifier() — a straight NULL dereference.

The crash is only the most visible symptom. Once EXCEPT tables are deparsed correctly, three further layers of Citus publication handling turn out to be unaware of exclusions.

Provenance — this is a pre-Beta1 PG19 feature, not a Beta2 regression

Upstream commit Date Change
fd366065 2026-03-04 Allow table exclusions in publications via EXCEPT TABLE
493f8c64 2026-03-20 Add support for EXCEPT TABLE in ALTER PUBLICATION (the exact crashing path)
96b37849 2025-10-09 Add ALL SEQUENCES support to publications (for_all_tables / for_all_sequences)

All three are present in both REL_19_BETA1 (2026-06-01) and REL_19_BETA2 (2026-07-13). This is not Beta2-introduced — it is a PG19 feature that predates Beta1 and was missed by the original PG19 compatibility sweep. It surfaced now only because the crash became visible in a fresh check-vanilla run.

The four correctness layers

1. Deparse and qualify EXCEPT-table objects (the crash)

deparse_publication_stmts.cAppendPublicationObjects() now handles PUBLICATIONOBJ_EXCEPT_TABLE explicitly, reading the relation from pubtable and emitting proper ALL TABLES EXCEPT (TABLE ...) syntax. Local (non-distributed) tables continue to be filtered out of worker-bound commands, and AppendPublicationAllObjects() accounts for PG19's for_all_tables / for_all_sequences reconstruction semantics.

qualify_publication_stmt.cQualifyPublicationObjects() schema-qualifies PUBLICATIONOBJ_EXCEPT_TABLE relations so the deparsed command is unambiguous on workers.

This is a real semantic fix, not a NULL guard: the object kind is deparsed and qualified through its own path.

2. Metadata-activation dependency traversal

metadata/dependency.c — for a FOR ALL TABLES publication, GetPublicationRelationsDependencyList() was calling the included-relations APIs. On PG19 those assert !alltables, and even without asserts they miss the EXCEPT tables entirely, so an excluded table could be absent from the node snapshot when the publication is created on a worker.

It now collects explicit exclusions with GetExcludedPublicationTables(pubid, PUBLICATION_PART_ROOT) and orders them correctly, so activate_node_snapshot() emits the excluded relation before the publication that references it.

3. Partition-root preservation, and lifecycle when an excluded table is later distributed

commands/publication.c — EXCEPT reconstruction now always requests PUBLICATION_PART_ROOT. Under publish_via_partition_root = false the previous PUBLICATION_PART_LEAF choice dropped the partitioned root OID, but EXCEPT syntax must preserve the explicitly listed root.

metadata/metadata_sync.cAddTableToPublications() previously consulted only included publications. A local table excluded via ALL TABLES EXCEPT (t) and later distributed would begin being published on workers while the coordinator still excluded it. It now also detects excluded publications and rebuilds a complete SET ALL TABLES EXCEPT (...) on workers, preserving local-table filtering.

Note: EXCEPT membership cannot be amended with ALTER PUBLICATION ADD/DROP TABLE — the whole object list must be re-SET, which is why these paths reconstruct rather than patch.

4. OID-rewriting conversions

commands/publication.cGetAlterPublicationDDLCommandsForTable() also queried only included publications, so any conversion that changes a relation's OID silently lost EXCEPT membership:

  • citus_add_local_table_to_metadata() renames the original OID to <name>_<shardid> and creates a new shell relation. The surviving pg_publication_rel exclusion row follows the OID, so the publication ended up excluding the shard while the new shell table was published.
  • ReplaceTable() / drop-CASCADE (e.g. undistribute_table()) deletes the exclusion row outright; the post-load replay restored only included memberships.

The function now, under #if PG_VERSION_NUM >= PG_VERSION_19 and only for the add direction, emits one complete SET ALL TABLES EXCEPT (...) per publication returned by GetRelationExcludedPublications(relationId).

This works because deparsed commands name tables by name, not by OID: capturing the command before the rewrite and replaying it after the replacement exists re-points the exclusion at the new relation, and because SET replaces the whole object list wholesale it simultaneously evicts the stale shard-named exclusion — no extra DROP is required.

GetAlterPublicationExcludedTablesDDLCommand() gained an includeLocalTables parameter: worker propagation must filter local tables (workers cannot name them), while the coordinator-local restore must keep them, or restoring would silently drop local excluded tables from the coordinator's EXCEPT list.

Negative controls

Each layer was validated by surgically disabling only that fix, rebuilding, and confirming that exactly the corresponding assertions flip — no more, no less.

The layer-4 control is the clearest. Same binary, same test; only difference is the restore loop present vs. replaced with NIL:

############ WITH FIX ############
 FLOW1 | t_undist
 FLOW2 | t_local

############ WITHOUT FIX ############
 FLOW2 | t_local_9600002
 flow | excluded_relname
---------------------------------------------------------------------
(0 rows)
Flow Mechanism Without fix With fix
undistribute_table ReplaceTable / drop-CASCADE 0 rows — exclusion deleted t_undist
citus_add_local_table_to_metadata OID renamed to shard t_local_9600002 — stale shard OID t_local

Tests

New coverage lives in a dedicated pg19 regression test (with a _0.out skip stub so it is a no-op on PG < 19). publication.sql / publication.out are left byte-identical to upstream, and no expected output anywhere was altered to mask the crash.

Scenarios covered:

  • ALTER PUBLICATION ... SET ALL TABLES EXCEPT (...) deparse/qualify (the original crash)
  • metadata activation / node-snapshot ordering, asserting the publication is emitted after its excluded root — with the partitioned child excluded explicitly so the assertion cannot pass by accident
  • excluding a partitioned root under default publish_via_partition_root settings
  • an excluded local table that is subsequently distributed, verified against the worker catalogs
  • both OID-rewrite conversions, asserting exclusion_retained_after_undistribute, exclusion_moved_to_shell_table, no_exclusion_left_behind_on_shard, and exclusion_present_on_workers

Validation

Environment publication pg19
PG19 Beta2 pass, no diffs pass, no diffs
PG18.4 pass, no diffs pass (skip stub)
PG17.10 pass, no diffs pass (skip stub)

All builds complete with 0 warnings and 0 errors. Expected-output files were md5-verified before and after each run to prove the harness never regenerated them.

Diff

8 files changed, 716 insertions(+), 16 deletions(-) — 4 source files, 2 header/metadata files, 2 test files.

 src/backend/distributed/commands/publication.c              | 131 ++++++++++-
 src/backend/distributed/deparser/deparse_publication_stmts.c |  86 ++++++-
 src/backend/distributed/deparser/qualify_publication_stmt.c  |   6 +-
 src/backend/distributed/metadata/dependency.c                |  22 +-
 src/backend/distributed/metadata/metadata_sync.c             |  45 +++-
 src/include/distributed/commands.h                           |   6 +
 src/test/regress/expected/pg19.out                           | 251 +++++++++++
 src/test/regress/sql/pg19.sql                                | 185 +++++++++

All PG19-specific behavior is version-gated behind PG_VERSION_NUM >= PG_VERSION_19; PG17 and PG18 code paths are unchanged.

@codecov

codecov Bot commented Aug 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 64.86486% with 26 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.69%. Comparing base (ff8f1f8) to head (5a32aa4).
⚠️ Report is 1 commits behind head on pg19-support.

❌ Your patch check has failed because the patch coverage (64.86%) is below the target coverage (75.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@               Coverage Diff                @@
##           pg19-support    #8741      +/-   ##
================================================
- Coverage         88.80%   88.69%   -0.11%     
================================================
  Files               288      288              
  Lines             64416    64478      +62     
  Branches           8100     8111      +11     
================================================
- Hits              57206    57191      -15     
- Misses             4873     4955      +82     
+ Partials           2337     2332       -5     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

ihalatci-msft and others added 5 commits August 8, 2026 14:11
PostgreSQL 19 adds table exclusions to publications (upstream commits
fd366065 "Allow table exclusions in publications via EXCEPT TABLE." and
493f8c64 "Add support for EXCEPT TABLE in ALTER PUBLICATION."), plus
independent ALL TABLES / ALL SEQUENCES flags (96b37849). All of these
predate REL_19_BETA1.

Citus' publication deparser only special-cased PUBLICATIONOBJ_TABLE and
treated every other object kind as a schema object, reading
publicationObject->name. For the new PUBLICATIONOBJ_EXCEPT_TABLE the
relation lives in ->pubtable and ->name is NULL, so deparsing an
EXCEPT-table object called quote_identifier(NULL) and segfaulted the
backend. This is reproducible with the vanilla regression suite:

    ALTER PUBLICATION testpub_foralltables_excepttable
        SET ALL TABLES EXCEPT (TABLE testpub_tbl2);

Handle EXCEPT tables as real tables rather than adding a NULL guard:

* Qualification and deparsing now match PUBLICATIONOBJ_EXCEPT_TABLE
  alongside PUBLICATIONOBJ_TABLE, so the relation is schema-qualified and
  emitted through the table branch.
* A new PG19-only helper emits the ALL TABLES [EXCEPT (...)][, ALL
  SEQUENCES] object list, shared by the CREATE and ALTER paths. Local
  (non-distributed) EXCEPT relations are still filtered out when local
  tables are not included.
* Reconstructing CREATE PUBLICATION from the catalogs now carries
  puballsequences, and reads the excluded relations via
  GetExcludedPublicationTables() when puballtables is set, emitting them
  as EXCEPT objects.

Because publicationForm is now read after the relation lookup,
ReleaseSysCache() moves to the end of BuildCreatePublicationStmt(); the
function has no early returns. This also fixes a pre-existing read of
publicationForm->pubviaroot after the tuple had been released.

All new code is gated on PG_VERSION_NUM >= PG_VERSION_19 and is a no-op
on older majors.

Validated by building against PG 17.10, 18.4 and 19 Beta2, by confirming
the isolated reproducer no longer crashes and records puballtables and
puballsequences on the workers, and by the pg19 regression test added
here, which asserts both CREATE reconstruction and ALTER propagation of
EXCEPT relations.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 373d1084-19bb-45e4-a85c-6567ac9b5725
Commit 3130044 fixed the SIGSEGV in ALTER PUBLICATION ... SET ALL TABLES
EXCEPT, but two PG19 EXCEPT-table paths outside the deparser were still
wrong.

1. GetPublicationRelationsDependencyList() called GetPublicationRelations()
   unconditionally. On PG19 that maps to GetIncludedPublicationRelations(),
   which asserts !GetPublication(pubid)->alltables. For a FOR ALL TABLES
   publication this trips the assert on assert-enabled builds, and on
   ordinary builds returns NIL, so the excluded tables are not recorded as
   dependencies. Metadata activation could then emit CREATE PUBLICATION
   before the tables its EXCEPT list names. Branch on alltables and collect
   the exclusions with GetExcludedPublicationTables() instead.

2. BuildCreatePublicationStmt() looked up excluded tables with
   PUBLICATION_PART_LEAF whenever publish_via_partition_root was off.
   GetPubPartitionOptionRelations() drops the partitioned root in that mode
   and returns only its current leaves, so an excluded partitioned root was
   either lost entirely (no partitions) or frozen to the partitions that
   existed at activation time. An EXCEPT list names relations explicitly and
   pg_publication_rel stores exactly those, so always request
   PUBLICATION_PART_ROOT when reconstructing the DDL.

Both paths are only reachable through catalog-based reconstruction, which
propagation does not use (it deparses the original parse tree), so the
regression test drives activate_node_snapshot() directly and asserts that
an excluded partitioned root survives as the root and that excluded tables
are ordered before the publication that references them.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 373d1084-19bb-45e4-a85c-6567ac9b5725
Two follow-up defects found in review of the PG19 ALL TABLES EXCEPT support.

1. Excluded local-table lifecycle gap. When a table is excluded from a
   FOR ALL TABLES publication while it is still a local table, the
   deparser correctly omits it from the propagated command, so workers
   receive a plain FOR ALL TABLES publication. If that table is later
   distributed, AddTableToPublications() only consulted
   GetRelationIncludedPublications(), which never reports EXCEPT
   membership, so the function returned early and the workers' publication
   kept publishing the newly created shell table while the coordinator
   still excluded it.

   EXCEPT membership cannot be amended with ALTER PUBLICATION ADD/DROP
   TABLE, so the fix rebuilds the full membership from the catalog and
   re-sends ALTER PUBLICATION ... SET ALL TABLES EXCEPT (...) to the
   worker nodes. The new GetAlterPublicationExcludedTablesDDLCommand()
   reuses BuildCreatePublicationStmt() so local excluded tables continue
   to be filtered out of the propagated command.

2. The metadata-activation ordering test was not discriminating. Its
   root-name predicate also matched the partition child, and because the
   publication was created after the root table was distributed, the
   snapshot ordered the table first regardless of the dependency edge.
   The child is now excluded explicitly and the publication is created
   while the excluded root is still local, so only the
   publication -> excluded-root dependency can produce the expected order.

All production changes are gated on PG_VERSION_NUM >= PG_VERSION_19, so
PG17 and PG18 behavior is unchanged.

Validation:
  PG19 Beta2: publication + pg19 regression tests pass, no diffs
  PG18.4:     publication + pg19 regression tests pass, no diffs
  PG17.10:    publication + pg19 regression tests pass, no diffs

Negative controls (each reverts one fix in isolation):
  metadata_sync.c excluded lookup disabled ->
      exclusion_restored_after_distribution t -> f
  dependency.c EXCEPT traversal disabled ->
      excluded_table_precedes_publication t -> f

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 373d1084-19bb-45e4-a85c-6567ac9b5725
Conversions that give a table a new OID lost its membership in a PG19
FOR ALL TABLES ... EXCEPT publication, because
GetAlterPublicationDDLCommandsForTable() only looked at the publications
that include a relation.

citus_add_local_table_to_metadata() was affected the worst.
DropTableFromPublications() also only considers included publications, so
the EXCEPT row survived, and ConvertLocalTableToShard() then renamed the
original relation. The exclusion followed the OID and ended up naming the
shard, while the new shell table under the original name was published
even though the user had excluded it. ReplaceTable(), used by
undistribute_table() and the other table type conversions, dropped the
EXCEPT row with the old table and restored only included memberships.

EXCEPT membership cannot be amended with ALTER PUBLICATION ADD TABLE, so
regenerate the complete membership instead. The deparsed commands name
the excluded tables rather than their OIDs, so capturing them before the
conversion and replaying them after the replacement table exists moves
the exclusion onto the replacement. Replacing the whole list at once also
evicts the exclusion that stayed behind on the relation that became a
shard, so no extra DROP command is needed.

GetAlterPublicationExcludedTablesDDLCommand() now takes includeLocalTables
so the local restore keeps excluding local tables, while the worker
propagation path in metadata_sync.c keeps filtering them out.

Add PG19 regression coverage for undistribute_table() and
citus_add_local_table_to_metadata(), asserting that the exclusion is
retained, moves to the shell table, leaves nothing behind on the shard
and is present on the workers.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 373d1084-19bb-45e4-a85c-6567ac9b5725
The check-style CI job failed on the PG19 EXCEPT-table publication work
because two files did not match the canonical citus_indent (uncrustify)
formatting:

- deparse_publication_stmts.c: missing blank line after the
  PG_VERSION_19 guard opening, missing the two blank lines before the
  matching #endif, and a one-column continuation-line misalignment in
  the "ALL SEQUENCES" ternary.
- dependency.c: one-column continuation-line misalignment in the
  GetExcludedPublicationTables() call.

This commit applies uncrustify's canonical output for those hunks only.
The change is whitespace-only: "git diff --ignore-all-space
--ignore-blank-lines" is empty, so there is no functional change.

Verified by reproducing the full check-style job against a clean clone
of the pushed branch: citus_indent --check now passes, and all other
CI style scripts (editorconfig, remove_useless_declarations,
sort_and_group_includes, normalize_expected, banned.h,
check_all_tests_are_run, check_all_ci_scripts_are_run,
check_gucs_are_alphabetically_sorted, check_migration_files) pass.
Also rebuilt against PostgreSQL 19 Beta2 with zero warnings and zero
errors.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 373d1084-19bb-45e4-a85c-6567ac9b5725
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants