OpTop

OpTop provides principled, statistically grounded tools to select the optimal number of topics and to assess goodness-of-fit for probabilistic topic models. It implements fast parametric tests and discrepancy indices that make comparisons across different topic counts directly comparable. The methodology is developed on the multinomial structure of the Latent Dirichlet Allocation (LDA) family and applies to any fitted model that returns document-level word probabilities: supported engines include topicmodels (LDA with VEM or Gibbs, and CTM), seededlda, and NLPstudio, which can also be mixed within one model grid.

What It Does

• Optimal K selection
  Fast, parametric tests based on a Pearson-type statistic identify the topic count that best describes the corpus (optimal_topic()), with optional bootstrap or moment-matched calibration of the p-values under the fitted-model null (v0.10.0). The legacy redundant-topic diagnostics (topic_stability(), agg_topic_stability(), agg_document_stability()) are deprecated as of v0.9.8 and scheduled for removal.

• Model goodness-of-fit
  Regression-style indices for topic models (e.g., SE, Pearson-$\chi^2$, and deviance), summarized as micro/macro $R^2$ analogues (optop_index_se(), optop_index_chisq(), optop_index_deviance()), plus a convenience table across a grid (optop_index_table()).

• Held-out validation and residual diagnostics
  As of v0.14.0, optop_index_holdout() scores independent evaluation documents with confidence intervals for the average held-out fit, optop_gain_table() selects the smallest sufficient K from paired adjacent gains, and optop_moment_test() tests held-out residuals for structured bias along frequency or fit-based vocabulary strata.

• Harmonized comparisons across K
  Rare words are collapsed via a fixed, document-specific partition so all indices are evaluated on a common support (optop_make_partition()), with a fixed corpus baseline for fair normalization (optop_make_baseline()).

• Performance at scale
  Core routines are implemented in C/C++ for large vocabularies and model grids. As of v0.9.7, the optimal_topic() core works on blocked BLAS matrix products and pairs documents with the fitted models by identifier, so the dfm row order no longer matters. As of v0.12.0, both the Test 1 statistic and the bootstrap calibration are parallelized with OpenMP, controlled by the n_threads argument of optimal_topic(). Results are bit-identical for any thread count and reproducible under a seed; on toolchains without OpenMP support the routines run single-threaded. The bootstrap selects its sampling algorithm per document (alias-table token draws on wide envelopes, conditional-binomial draws otherwise) and envelope ties are resolved deterministically, so results are identical across platforms, including under the tied fitted probabilities that Gibbs estimators produce. On a 4-core machine the bootstrap-calibrated pass runs up to 45 times faster than in v0.11.0 (a corpus of 10,000 documents and 20,000 features drops from 1 h 47 m to 142 s); see the vignette section "A Note On Computational Efficiency" for the full simulation study.

• Current support and extensions
  As of v0.11.0, optimal_topic() and the discrepancy indices accept, through the internal adapter family (optop_as_theta_phi()), topicmodels fits (LDA with VEM or Gibbs, and CTM), all three seededlda models (textmodel_lda(), textmodel_seededlda(), textmodel_seqlda()), and NLPstudio fits (nlp_topic_fit). Engines can be mixed within one grid fitted on the same corpus: documents and features are aligned by identifier, per model. Supporting a new engine requires a single adapter method that returns the fitted document-topic and topic-word probabilities; WarpLDA fits from text2vec are supported through the optop_warplda() wrapper.

Authors

• Francesco Grossetti

  Assistant Professor of Accounting Analytics and Data Science
  Department of Accounting | Bocconi University
  Fellow at Bocconi Institute for Data Science and Analytics (BIDSA)
  Contact: francesco.grossetti@unibocconi.it

• Craig M. Lewis

  Madison S. Wigginton Professor of Finance, Emeritus
  Owen Graduate School of Management | Vanderbilt University
  Contact: craig.lewis@vanderbilt.edu

References

1. Lewis, C. M. and Grossetti, F. (2022). A statistical approach for optimal topic model identification. Journal of Machine Learning Research, 23(58), 1–20.
2. Grün, B. and Hornik, K. (2011). topicmodels: An R package for fitting topic models. Journal of Statistical Software, 40, 1-30.
3. Blei, D. M., Ng, A. Y., and Jordan, M. I. (2003). Latent Dirichlet Allocation. Journal of Machine Learning Research, 3(Jan):993–1022.
4. Grossetti F. (2026). NLPstudio: Tools for Social Science Text Analysis, Corpus Management, and Topic Modeling. R package version 1.1.1.

Installation

# From GitHub
# install.packages("pak")
pak::pkg_install("contefranz/OpTop")

The package vignette presents the complete workflow (optimal-K selection, the role of q, the discrepancy indices, and a simulation with known truth): vignette("OpTop").

Quick Start

library(OpTop)
library(quanteda)
library(topicmodels)

# 0) Get a quanteda corpus object
corpus_texts = data_corpus_inaugural

# 1) Build a weighted dfm (row-wise proportions) for optimal-K selection
dfm_counts   <- dfm(tokens(corpus_texts))                    # counts
weighted_dfm <- dfm_weight(dfm_counts, scheme = "prop")      # proportions

# 2) Fit a grid of topic models (e.g., K = 10:50 by 5); LDA via VEM is shown,
#    and any adapter-supported engine works (topicmodels, seededlda, NLPstudio)
set.seed(123)
K_grid    <- seq(10, 50, by = 5)
VEM_models <- lapply(
  K_grid, function(k) {
    lda = topicmodels::LDA(x = dfm_counts, k = k, method = "VEM")
  }
) 

# 3) Choose the optimal K (selection: "sequential" adequacy scan by default,
#    "min" for the paper's global-minimum rule; verbose = TRUE reports progress)
res_opt   <- optimal_topic(topic_models = VEM_models, 
                           weighted_dfm = weighted_dfm,
                           q = 0.95, 
                           alpha = 0.05, 
                           selection = "sequential",
                           do_plot = TRUE,
                           verbose = TRUE)

# 4) Goodness-of-fit across K (use counts here)
part      <- optop_make_partition(models = VEM_models, dtm = dfm_counts, c = 1)
base      <- optop_make_baseline(dtm = dfm_counts)
tab       <- optop_index_table(models = VEM_models, 
                               dtm = dfm_counts,
                               metrics = c("se","chisq","deviance"),
                               partition = part, 
                               baseline = base, 
                               macro = TRUE)
tab

Bug Reporting

Bugs and issues can be reported at https://github.com/contefranz/OpTop/issues.

---