Parallelizing Isotonic Regression along sequence dimension

[nirmal129]

Description

This PR introduces sequence-level parallelization for the forward and backward passes of isotonic regression in torchsort (SoftSort_Parallel and SoftRank_Parallel), complementing the existing batch-wise parallelism.

Performance

• Sequence Length (Batch 4): Achieves $65\times$ to $100\times$ speedup over sequential implementation at 100K sequence length.
• Batch Size (Seq Length 512): Achieves $10\times$ to $14\times$ speedup over sequential implementation at 4096 batch size.

Correctness & Verification

• Passes all existing torchsort unit tests.
• Validated in a CIFAR-10 classification pipeline with top-K loss; parallel and sequential evaluation curves track identically.

Implementation & Fallback

• Forward Pass: Employs a Divide-and-Conquer Pool Adjacent Violators (PAV) algorithm with a parallel block-merge operation executed during a tree reduction.
• Backward Pass: Parallelized across the sequence dimension via segmented sum-averaging and scattering.
• Fallback Mechanism: Users can explicitly fall back to the original sequential implementation at any time if they encounter edge-case bugs in the parallel variants.

Full theoretical analysis, PRAM complexities, and future ROC-Convex-Hull optimization directions are detailed in the "DNC_PAV.pdf" file available in the repository report:
https://github.com/nirmal129/Parallel-torchsort