Antalya 25.8 Backport of #89367: Crash in IN function where columns have different types and many columns are involved

[mkmkme]

Same as #1339, but for Antalya 25.8

Changelog category (leave one):

• Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Possible crash/undefined behavior in IN function where primary key column types are different from IN function right side column types (ClickHouse#89367 by @ilejn)

Documentation entry for user-facing changes

...

CI/CD Options

Exclude tests:

• Fast test
• Integration Tests
• Stateless tests
• Stateful tests
• Performance tests
• Aarch64 tests
• All with ASAN
• All with TSAN
• All with MSAN
• All with UBSAN
• All with Coverage
• All Regression
• Disable CI Cache

Regression jobs to run:

• Fast suites (mostly <1h)
• Aggregate Functions (2h)
• Alter (1.5h)
• Benchmark (30m)
• ClickHouse Keeper (1h)
• Iceberg (2h)
• LDAP (1h)
• OAuth (5m)
• Parquet (1.5h)
• RBAC (1.5h)
• SSL Server (1h)
• S3 (2h)
• S3 Export (2h)
• Swarms (30m)
• Tiered Storage (2h)

[mkmkme]

Tested locally, and the test fails on plain 25.8 Antalya and passes with this change. Will merge once CI finishes

[github-actions]

Workflow [PR], commit [d6f6e61]

[ilejn]

passes with this change

Could you explain, please.
Are changes against #1339 needed?

Update. Misha clarified that test from the PR fails without correspondent code from the PR and passes when PR fully applied.

[mkmkme]

@blau-ai

[blau-ai]

CI triage for #2164

Verdict: 0 failures are caused by this PR. All red checks are either pre-existing branch-wide regression failures or a one-off infra/runner death. The diff itself (a faithful backport of upstream ClickHouse#89367, touching only KeyCondition.cpp + sortBlock.cpp + one new stateless test) is unrelated to everything that failed.

Summary of the failing checks:

Check	Classification	PR-caused?
Regression x86 iceberg_1	pre-existing / branch-wide	❌ No
Regression x86 iceberg_2	pre-existing / branch-wide	❌ No
Regression x86 swarms	pre-existing / branch-wide	❌ No
Stateless tests (amd_debug, parallel)	flaky / infra (runner terminated)	❌ No
FinishCIReport, PR	aggregate gates (roll-up of the above)	❌ No

---

Regression: iceberg_1, iceberg_2, swarms — pre-existing on antalya-25.8

These are Altinity data-lake / distributed regression suites with no relationship to a KeyCondition set-index fix. Decisive log lines:

• iceberg_1 — [ Fail ] /iceberg/iceberg engine/rest catalog/position delete reads/read position deletes (AssertionError) and [ Error ] /iceberg/iceberg engine/rest catalog/sort key timezone → AttributeError: feature_supported (a test-harness error in clickhouse-regression, not a server bug).
• swarms — [ Fail ] /swarms/feature/node failure/check restart swarm node.

Evidence they're pre-existing: the two sibling backport PRs open against the same antalya-25.8 base — #2166 and #2168, both unrelated changes — show the identical three failures (iceberg_1 errored, iceberg_2 failed, swarms failed). This is a state of the base branch / the regression repo, not something #2164 introduced.

Next step: none required from this PR. These need a separate fix in the regression suite / base branch and shouldn't block the backport.

Stateless tests (amd_debug, parallel) — flaky infra (runner death)

The commit status reads "The test runner was terminated unexpectedly." The run got ~13 min in with every executed test green, then the clickhouse-test driver died and CI set the job to failure. Two things rule out this PR:

1. The new test 03635_in_function_different_types_many_columns never ran — it doesn't appear anywhere in the log before termination, so it can't be the cause.
2. The same Stateless tests (amd_debug, parallel) job passed on Antalya 25.8 Backport of #92250 - Reduce INSERT/merges memory usage with wide parts for very wide tables by enabling adaptive write buffers #2166 and Antalya 25.8 backport of #90243: Fix crash in StorageDistributed when parsing malformed shard directory names #2168 (7912 passed, 0 failed), so the driver isn't broken by anything on this branch.

There were also environment errors at setup (zramctl: /dev/zram0: failed to reset: Device or resource busy, mc: Unable to list folder ... clickminio not found, Connection refused (localhost:9000)), consistent with a flaky runner.

Next step: just re-run the Stateless tests (amd_debug, parallel) job. I'd expect it green on a clean runner. (FinishCIReport and PR will clear once it and the aggregate pass.)

---

PR health check

The backport looks clean and faithful to upstream ClickHouse#89367: it rewrites the null-map filtering in KeyCondition::tryPrepareSetIndex to accumulate a single shared filter across all set-index columns (fixing the row-count mismatch / crash when IN has many columns of differing types) and adds the #ifndef NDEBUG block.checkNumberOfRows() guard in sortBlock. The Fast test (6907 passed) and both Builds are green, so the change compiles and passes the fast suite. No PR-related failures remain — this is safe to re-run and, assuming stateless goes green, merge on the regression-suite exceptions above.

Note: I can't build or run ClickHouse in this environment — correctness is validated by CI on the PR, not locally.