Speed up TestHnswMergeAbort and make its assertion deterministic

[jeho-rpls]

Fixes #16397.

The tests asserted that rollback() returns within 10 seconds, which needed big segments (4 x 12k docs, beam width 250) to stay reliable, about 50s per test.

Changes:

• Instead of asserting on elapsed time, each test now asserts through InfoStream that the graph build never ran to completion. To make that deterministic at any size, the test InfoStream holds the merge thread at the build graph message until IndexWriter#abortMerges has marked the merge aborted, signaled by the now wait for N running merge/s to abort message.
• The abort exception is not asserted on: it is swallowed as expected control flow, and a small unchecked build can finish without throwing, since merge outputs are only checked for abort about once per written megabyte per file.
• MergingHnswGraphBuilder now emits a completion message like the other two builders (addVectors [...), merge completed: ...), so the graph join path has a completion signal to assert on.
• Each test asserts on the build graph start message to pin its intended merge path (full rebuild, graph join, concurrent with 2 workers).
• Segments shrink to 1k docs per segment, beam width to 100, with tinySegmentsThreshold=0 so a graph is always built.

Testing:

• With the abort check removed, all three tests fail on completion messages. The unchecked builds finished in 0.6-1.0s at this size, under the old 10s limit, so shrinking alone would have left the timing assertion passing.
• -Dtests.iters=20: 60/60 pass.
• Suite time: 63s before, 1.6s after on my machine (about 154s on the machine in Test suite TestHnswMergeAbort slows down build #16397).

[msokolov]

does this test rely on the fact that HNSW merging work is done on a background thread? If we had single-threaded HNSW merging (as we used to do) this would deadlock because the InfoStream would be waiting for the merging operation to be aborted, but it could not progress because it is the merging operation?

I guess that should be OK since all of our mergers do use their own thread pools now

[jeho-rpls]

Thanks for the review!

does this test rely on the fact that HNSW merging work is done on a background thread?

My intent was to depend only on rollback() being called from a different thread than the merge, not on where the merge runs. The test sets that up explicitly, with rollback() on the test thread and forceMerge on its own thread.

If we had single-threaded HNSW merging (as we used to do) this would deadlock because the InfoStream would be waiting for the merging operation to be aborted, but it could not progress because it is the merging operation?

I think that cycle is avoided because the release signal does not come from the merge side. It comes from the rollback thread, which marks every running merge aborted inside IndexWriter#abortMerges and emits the now wait for message before it ever starts waiting. The blocking branch in the test only fires for the HNSW build graph message, so the rollback thread is never held there. As a backstop the latch has a 2 minute timeout, so a missed release fails the releasedAfterAbort assertion instead of hanging the build.

I guess that should be OK since all of our mergers do use their own thread pools now

It holds even without that. I swapped ConcurrentMergeScheduler for SerialMergeScheduler locally, so the merge runs synchronously inside forceMerge, and all three tests still pass across repeated runs.

[msokolov]

thanks, that makes sense. I'll merge today