-
Notifications
You must be signed in to change notification settings - Fork 52
Naut combined #408
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
sblunt
wants to merge
60
commits into
sblunt:main
Choose a base branch
from
eshelDror:naut_combined
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Naut combined #408
Changes from all commits
Commits
Show all changes
60 commits
Select commit
Hold shift + click to select a range
d25b8f7
Implement initial Nautilus sampler
eshelDror eb63949
Add nautilus to requirements
eshelDror 8512213
Add multithreading for nautilus
eshelDror 66eca22
Add option to save nautilus progress
eshelDror 34aa0af
This is a pytest implemented using test_nested_sampler but implemente…
91cd179
Vectorize nautilus likelihood and prior transform
eshelDror 1e66635
Improve vectorization of ptform
eshelDror dd7824a
Merge remote-tracking branch 'eshel/nautilus_vectorized' into nautilu…
9e154b0
Allow weighted post in results
eshelDror 4fbd6d7
Save Nautilus weighted post to results
eshelDror e9ef51f
Make corner plot with weighted post if available
eshelDror d5c2d57
Print results with weighted post
eshelDror 250e544
Add saving and loading weighted results
eshelDror 38da696
Add end-to-end test
eshelDror bb5d3c1
Make Lux compatible
eshelDror 8ed95ff
Update GJ504 run parameters, fix Lux threads issue
eshelDror bc0cf83
Clean up Nautilus pool fix
eshelDror 4c45a90
This is a pytest for Nautilus Sampler!
5a5e55e
Fix Nautilus threading for different python versions
eshelDror 469610b
Add OFTI GJ504 fit
eshelDror 5476754
Fix loading unweighted results
eshelDror ea2e847
Undo changes to print_results for python version compatability
eshelDror 504be4d
Add downsampling for corner plotting
eshelDror 2d4aacf
add a basic nautilus tutorial with orbit generation and plotting
AnCh2008 9218abe
Add hr8799e nautilus end-to-end test
eshelDror d960159
Fix datadir
eshelDror 9e87bd4
add quick description and demonstration of weighted results in the tu…
AnCh2008 c76c982
Merge branch 'nautilus_initial' into nautilus
eshelDror 86a2913
Merge branch 'nautilus' into nautilus_weighted_results
eshelDror 66960e3
Remove visulizations.
376d058
Merge branch 'main' into nautilus
eshelDror 79c9381
Merge branch 'nautilus' into nautilus_weighted_results
eshelDror 29544de
Merge branch 'nautilus_weighted_results' into GJ504_naut
eshelDror 8373c2c
Merge branch 'GJ504_naut' into GJ504_OFTI
eshelDror b2ad8db
Merge branch 'GJ504_OFTI' into naut_better_results
eshelDror 7ce7a2b
Merge branch 'naut_better_results' into naut_hr8799e
eshelDror 99a8e51
Add nautilus to pyproject.toml
eshelDror 969a15b
Add BaseNestedSampler and document Nautilus
eshelDror 0e6e5c8
Document changes to results.py
eshelDror 9f611cc
Standardize load_results with array_not_none
eshelDror 963f719
Fix misconfigured ptform and initialization of Nautilus
eshelDror f071dac
Fix mp.Pool import
eshelDror d33e018
Merge remote-tracking branch 'quin/pytest_clean' into naut_combined
eshelDror 4698bc2
Merge remote-tracking branch 'upstream/main' into naut_combined
eshelDror 7195311
added more on weights, faster plot params and fixed language
AnCh2008 cb889f8
Merge remote-tracking branch 'eshel/naut_better_results' into naut_tu…
AnCh2008 960ae55
add improvements to the tutorial and trimmed it down
AnCh2008 180517a
Clean and add tests for downsampled plot_corner
eshelDror ac3de45
Remove extra end-to-end tests
eshelDror 50f6b88
Improve ptform and Nautilus sampler docstrings
eshelDror 58b135c
Clean Nautilus end-to-end test
eshelDror 7a96491
Fix downsampling for no lnlikes
eshelDror 4c2a18c
Add tests for downsampling and weighted results
eshelDror 6b7944f
Merge pull request #10 from AnCh2008/naut_tutorial
eshelDror 4410b0d
Clean up tutorial
eshelDror 2d7181d
Generalize nautilus tutorial thread count
eshelDror a9f2c8a
Merge branch 'main' into naut_combined
sblunt 0babfd0
adding three missing tutorials to toctree
sblunt 10f1bba
Clean tutorial
eshelDror c01e9ed
Merge pull request #11 from eshelDror/add-tutorials-to-toctree
eshelDror File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,287 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Nautilus Introduction #\n", | ||
| "\n", | ||
| "by Quinn Blackstone, Aniruddh Chalagulla, Eshel Dror and Niklas Naworal (2026)\n", | ||
| "\n", | ||
| "This is a tutorial for using the [Nautilus](https://arxiv.org/abs/2306.16923) sampler in `orbitize`!" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "import orbitize\n", | ||
| "import numpy as np\n", | ||
| "import multiprocessing as mp" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Basic Orbit Generating\n", | ||
| "\n", | ||
| "Before we get on with orbit generation, lets get some data ready. For this tutorial we generate some synthetic data with the `generate_synthetic_data()` function." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "tags": [] | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from orbitize.system import generate_synthetic_data\n", | ||
| "data_table, sma = generate_synthetic_data(\n", | ||
| " 95, # orbital fraction, how much of the orbit is covered (0-100%)\n", | ||
| " 1.2, # mass of the system\n", | ||
| " 60, # paralax\n", | ||
| " 0.1, # eccentricity (float)\n", | ||
| " np.pi/6, # inclination (float)\n", | ||
| " unc=2, # uncertainty (int)\n", | ||
| " num_obs=30, # number of observations to be made (int)\n", | ||
| ")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "tags": [] | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from orbitize import system\n", | ||
| "mySys = system.System(1, data_table, 1.2, 60)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Initialize the sampler" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "tags": [] | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from orbitize import sampler\n", | ||
| "mySampler = sampler.NautilusSampler(mySys)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Nice, now it's time to run the sampler\n", | ||
| "\n", | ||
| "A bit more on the hyperparameters\n", | ||
| "\n", | ||
| "#### Sampler arguments\n", | ||
| "`n_networks`: The number of neural networks trained to determine the iso-likelihood shells at each iteration. The likelihood scores prediceted by the neural networks are averaged to determine if a point is included in a shell. A higher value of `n_networks` leads to greater sampling efficiency at the cost of higher overhead per iteration. Neural network training is not parallelized.\n", | ||
| "`n_batch`: The number of orbits evaluated in each iteration, must be a multiple of num_threads. Each thread evaluates a set of `n_batch/num_threads` orbits. Higher `n_batch` takes advantage of the vectorized prior transform to improve computation speed, at the cost of some extra likelihood evaluations being performed in each shell (i.e. if `n_update` is 2500 samples and n_batch is 1000 then 3000 samples would be taken).\n", | ||
| "\n", | ||
| "#### Run arguments\n", | ||
| "`f_live`: The maximum fraction of the evidence left in the live set before switching from the exploration to the sampling phase. This can be used to act as an ending point for the exploration phase (when shells are created), larger values lead to faster convergence but may not converge fully.\n", | ||
| "`n_eff`: The total effective size of the sample. Larger values produce higher quality posterior estimations but take longer to run. Acts as the ending point of the sampling phase.\n", | ||
| "\n", | ||
| "`n_live`: Number of live points used in the sampling process. Greater `n_live` yields better evidence estimation and shell accuracy but increased run time and computational cost.\n", | ||
| "num_threads: Number of threads used in parallel, greater number increases speed.\n", | ||
| "`n_update`: Number of additions to the live set before creating a new shell. If None defaults to `n_live`. Greater `n_update` leads to better shell accuracy but increased run time.\n", | ||
| "`verbose`: Lets you choose wether or not to display live updates of the sampler.\n", | ||
| "`savefile`: The name for a file that you want to save or resume nautilus progress to or from.\n", | ||
| "\n", | ||
| "See more on the possible hyperparameters in the [Nautilus docs](https://nautilus-sampler.readthedocs.io/en/latest/api_full.html)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "sampler_arg = {\n", | ||
| " \"n_networks\": 4,\n", | ||
| " \"n_batch\": None, #(int) \n", | ||
| "}\n", | ||
| "run_arg = {\"f_live\": 0.01, \"n_eff\": 1000} \n", | ||
| "samples = mySampler.run_sampler(\n", | ||
| " n_live=500, #(int)\n", | ||
| " num_threads=mp.cpu_count(), #(int)\n", | ||
| " n_update=None, #(int)\n", | ||
| " verbose=True, #(bool) \n", | ||
| " savefile=None, #(str)\n", | ||
| " sampler_kwargs=sampler_arg,\n", | ||
| " run_kwargs=run_arg\n", | ||
| ")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Plotting and results ##\n", | ||
| "\n", | ||
| "For a more detailed guide on data visualization capabilities within orbitize, see the [Orbitize plotting tutorial](https://orbitize.readthedocs.io/en/latest/tutorials/Plotting_tutorial.html)." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "After generating the samples, the `run_sampler` method also creates a `Results` object that can be accessed\n", | ||
| "with `mySampler.results`:" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "myResults = mySampler.results" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "### Weighted results" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "What is the difference between a normal post and a weighted posterior?\n", | ||
| "Weighted results are all the points that have been sampled by Nautilus weighted by shell sampling density and likelihood. The main benefit of weighted results is that they contain all points instead of equal-weighted results which are downsampled from it with no duplicates. `plot_corner` is able to make use of the weighted results for better accuracy but `plot_orbits` is not and therefore defaults to the unweighted results.\n", | ||
| "\n", | ||
| "`myResults.weighted_post` returns weighted posteriors if it exists, and unweighted if it does not, so you can still get the posterior from Nautilus without weights.\n", | ||
| "`myResults.weighted_lnlike` functions similarly for getting the log-likelihoods\n", | ||
| "\n", | ||
| "`myResults.post` is always unweighted. The same goes for `myResults.lnlike`.\n", | ||
| "Weight can be acquired from `myResults.weight`" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "myResults.post.shape" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "myResults.weighted_post.shape" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "In case you want more points than that of the unweighted results but fewer points than the weighted, you can use the downsample function. Note that if you do not allow duplicates, you will not obtain a completely true posterior (especially if your `amount` approaches the weighted posterior size); if you want a true equal-weighted posterior without duplicates use myResults.post." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "# Samples from the posterior a number of times. If the posterior is weighted, samples proportionally to weight to get an equal-weighted posterior.\n", | ||
| "post, lnlike = myResults.downsample(\n", | ||
| " 150000, # amount\n", | ||
| " True # allow duplicates\n", | ||
| ")\n", | ||
| "post.shape, lnlike.shape" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "You might be wondering what kind of cases you might want `duplicates` to be `True`, it would be when you want more points that still have equal weight.\n", | ||
| "For example this would return an error:" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "try:\n", | ||
| " post, lnlike = myResults.downsample(250000, duplicates=False)\n", | ||
| " print(post.shape, lnlike.shape)\n", | ||
| "\n", | ||
| "except ValueError as error:\n", | ||
| " print(f\"Downsampling error: {error}\")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "In addition to our `orbits` array, Orbitize also creates a `Results` class that contains built-in plotting capabilities for two types of plots: corner plots and orbit plots. These cornerplots can also take in weighted results, in addition to downsizing them.\n", | ||
| "\n", | ||
| "### Corner Plot ###" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "We can now create a corner plot using the function `plot_corner` within the `Results` class. This function requires an input list of the parameters, in string format, that you wish to include in your corner plot. We can even plot all of the orbital parameters at once! You may wish to use the `downsample` keyword if you want a quick graph as the full weighted posteriors tend to take a while to plot. As shown below:" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "corner_figure = myResults.plot_corner(downsample=100000, param_list=['sma1', 'ecc1', 'inc1','tau1'])" | ||
| ] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": "Python 3 (ipykernel)", | ||
| "language": "python", | ||
| "name": "python3" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "name": "ipython", | ||
| "version": 3 | ||
| }, | ||
| "file_extension": ".py", | ||
| "mimetype": "text/x-python", | ||
| "name": "python", | ||
| "nbconvert_exporter": "python", | ||
| "pygments_lexer": "ipython3", | ||
| "version": "3.10.19" | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 4 | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.