Skip to content

Use memory efficiently, GNSS geometric heights - #795

Closed
jlmaurer wants to merge 3 commits into
dbekaert:devfrom
jlmaurer:update_xarray
Closed

Use memory efficiently, GNSS geometric heights#795
jlmaurer wants to merge 3 commits into
dbekaert:devfrom
jlmaurer:update_xarray

Conversation

@jlmaurer

@jlmaurer jlmaurer commented Apr 15, 2026

Copy link
Copy Markdown
Collaborator

Description

xarray makes available two methods for opening datasets, open_dataset and load_dataset. Using open_dataset allows for lazy loading and memory efficiency. This PR change all the locations were load_dataset was used to `open_dataset', resulting in much better memory performance for RAiDER. This PR also wraps netcdf dataset opening with try/except/finally logic that ensures the datasets are closed properly.

In addition, the CDS API call for ERA-5 model levels was doubling calls to the API because the variables were in two pairs. I think this was a holdover from a previous API version. Now all four variables (lnsp, z, t, and q) are called at once.

Finally, GNSS heights were not getting converted to orthometric heights prior to interpolation in RAiDER. This PR implements a conversion from ellipsoid heights to the geoid only for GNSS stations.

All unit tests pass locally.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

  • I have added an explanation of what your changes do and why you'd like us to include them.
  • I have written new tests for your core changes, as applicable.
  • I have successfully ran tests with your changes locally.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.

@jlmaurer jlmaurer changed the title Use memory efficiently when calling xarray Use memory efficiently, GNSS geometric heights Jun 5, 2026
@jlmaurer
jlmaurer marked this pull request as ready for review June 12, 2026 23:06
@jlmaurer

jlmaurer commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator Author

@sssangha I think this PR is still good to go if you are able to review it. The integration tests that we were talking about are in #805

@jlmaurer
jlmaurer marked this pull request as draft August 4, 2026 03:00
@jlmaurer

jlmaurer commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator Author

Closing this and replacing the memory-efficiency part with #808, still working on the geoid correction part.

@jlmaurer jlmaurer closed this Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant