Background
Baum-Welch and Viterbi training iterate over obsLists sequentially. Each sequence contributes independently to the E-step accumulators (π numerator, transition counts, emission data). This is embarrassingly parallel — no sequence depends on another.
platform/thread_pool.h (ThreadPool) already exists in the codebase but is only used by diagnostic tools.
Intended design
Partition obsLists across threads in BasicBaumWelchTrainer<Obs>::train(). Each thread maintains its own local accumulator buffers; results are summed after the parallel section:
// Conceptually (exact API TBD):
std::vector<EStepBuffers> thread_bufs(n_threads, EStepBuffers{N});
ThreadPool::parallel_for(obsLists, [&](std::size_t tid, const SeqType &obs) {
accum_one_sequence(hmm, obs, N, logTransT, hasZeroTransitions, thread_bufs[tid]);
});
// Reduce: sum thread_bufs[*] into a single EStepBuffers
This adds no new public API — parallelism is transparent inside train(). A setNumThreads() or TrainingConfig field controls the thread count (default: hardware concurrency).
Same pattern applies to BasicViterbiTrainer (runIteration) and BasicSegmentalKMeansTrainer (learnEmis, optimizeCluster).
Acceptance criteria
- Parallel and serial
train() produce bit-identical log-probabilities (within floating-point reordering tolerance).
- Wall-clock speedup on ≥ 100 independent sequences scales linearly with thread count (up to hardware concurrency).
- New benchmark test: 1000-sequence training, measure speedup at 1/2/4/8 threads.
- Thread count configurable; defaults to 1 (no regression for existing single-threaded callers).
Background
Baum-Welch and Viterbi training iterate over
obsListssequentially. Each sequence contributes independently to the E-step accumulators (π numerator, transition counts, emission data). This is embarrassingly parallel — no sequence depends on another.platform/thread_pool.h(ThreadPool) already exists in the codebase but is only used by diagnostic tools.Intended design
Partition
obsListsacross threads inBasicBaumWelchTrainer<Obs>::train(). Each thread maintains its own local accumulator buffers; results are summed after the parallel section:This adds no new public API — parallelism is transparent inside
train(). AsetNumThreads()orTrainingConfigfield controls the thread count (default: hardware concurrency).Same pattern applies to
BasicViterbiTrainer(runIteration) andBasicSegmentalKMeansTrainer(learnEmis,optimizeCluster).Acceptance criteria
train()produce bit-identical log-probabilities (within floating-point reordering tolerance).