mpi-sppy

Optimization under uncertainty for Pyomo and other models.

Documentation is available at readthedocs and there is a paper.

Status for internal tests

MPI

A recent version of MPI and a compatible version of mpi4py are needed.

Here are two methods that seem to work well for installation, at least when considering non-HPC platforms.

1. Install OpenMPI and mpi4py using conda. Keep the order.

   conda install openmpi
   conda install mpi4py
   

2. If you already have an existing version of MPI, it may be better compile mpi4py against it. This can be done by installing mpi4py though pip.

   pip install mpi4py
   

3. It is also possible to automate mpi4py installation through pip as an optional dependency when installing mpi-sppy from pip or from source by adding the [mpi] extras flag.

   pip install -e .[mpi]

   Run the line above after cloning and moving to the repo root directory. If you plan to run the tests or contribute, use pip install -e ".[mpi,dev]" instead to also get every optional feature package plus the test, documentation, and linting tools (the docs describe all of the extras).

To test your installation, give this command (once mpi-sppy is installed it works from any directory):

mpirun -n 2 mpi-sppy-one-sided-test

If you don't see any error messages, you might have an MPI installation that will work. Note that even if there is an error message, mpi-sppy may still execute and return correct results. Per the comment below, the run-times may just be unnecessarily inflated. If you're on an HPC, run this across two nodes, with one rank on each node.

Installing mpi-sppy

It is possible to pip install mpi-sppy; however, most users are better off getting the software from Github because it is under active development.

Command-line programs

Installing mpi-sppy — whether with pip install mpi-sppy or with pip install -e . from a clone (the recommended way to work from source) — puts a few console scripts on your PATH, so you can run the main drivers without locating the files in your environment:

• mpi-sppy-generic-cylinders — the general-purpose driver, equivalent to python -m mpisppy.generic_cylinders (see the generic_cylinders docs).
• mpi-sppy-mrp-generic — the sequential-sampling (MRP) driver, equivalent to python -m mpisppy.mrp_generic.
• mpi-sppy-one-sided-test — the MPI one-sided diagnostic shown above.

For multi-rank parallel runs, the python -m mpi4py module form is still recommended (e.g. mpiexec -np 3 python -m mpi4py -m mpisppy.generic_cylinders ...) so that mpi4py installs its abort-on-exception handler; a bare mpiexec -np 3 mpi-sppy-generic-cylinders ... runs, but a failure on one rank may leave the others hanging.

Citing mpi-sppy

If you find mpi-sppy useful in your work, we kindly request that you cite the following paper:

 @article{mpi-sppy,
   title={A Parallel Hub-and-Spoke System for Large-Scale Scenario-Based Optimization Under Uncertainty},
   author={Bernard Knueven and David Mildebrath and Christopher Muir and John D Siirola and Jean-Paul Watson and David L Woodruff},
   journal = {Math. Prog. Comp.},
   volume = {15}, 
   pages = {591-–619},
   year={2023}
 }

AN IMPORTANT NOTE FOR MPICH USERS ON HPC PLATFORMS

At least on some US Department of Energy (e.g., at Lawrence Livermore National Laboratory) compute clusters, users of mpi-sppy that are using an MPICH implementation of MPI may need to set the following in order for both (1) proper execution of the one-sided test referenced above and (2) rapid results when running any of the algorithms shipped with mpi-sppy:

export MPICH_ASYNC_PROGRESS=1

Without this setting, we have observed run-times increase by a factor of between 2 and 4, due to non-blocking point-to-point calls apparently being treated as blocking.

Further, without this setting and in situations with a large number of ranks (e.g., >> 10), we have observed mpi-sppy stalling once scenario instances are created.

2026 NOTICE: per-spoke solver-options now overlay the global dict

The per-spoke solver-options flags (--lagrangian-solver-options, --reduced-costs-solver-options, etc.) now overlay the global --solver-options dict for that spoke instead of replacing it. The spoke flag's keys win on the keys it names; the global flag's other keys survive. Previously the spoke flag wiped the global dict for that spoke.

In the unlikely event you relied on the spoke flag dropping a global key, re-spell every key you want in the spoke options or omit the global flag. See the solver-options section of the generic_cylinders docs for the worked example.

2022 NOTICE

There was a disruptive change on August 11, 2022 concerning how options are accessed. See the file disruptions.txt for more information. If you are a new user, this will not affect you, regardless of how you install. If you are an existing user, you should consider the disruption before updating to the latest mpi-sppy. The documentation on readthedocs probably refers to the newest version.