At the 2025 ILAMB Workshop, one major topic of repeated discussion revolved around how we include notions of observational uncertainty in our benchmarking. While it is becoming more common for reference datasets to include some notion of uncertainty, there are still many that have none. A few long standing datasets (like those for soil carbon) vary highly among products suggesting a large amount of uncertainty yet with no explicit number assigned.
The group came up with several suggestions for what we could do in the absence of explicit information:
- We could ask experts/data providers to include a numeric assessment of the uncertainty along with their dataset. Briony reports that these discussions are happening and we may soon see that dataset providers will include this value.
- A long standing suggestion is that we craft composite products of all our datasets by finding a mean and standard deviation or range. This idea has failed to gain traction in terms of creating these composite products, but perhaps the range could be used as an external source of uncertainty.
- This could be used to create maps of uncertainty that are used directly to discount errors in the analysis.
- Alternatively, these maps could be used to develop expert uncertainty values if experts are unavailable.
- We could use the CARDAMOM ensemble stdev as a external source of uncertainty.
While not immediately clear which of these is best, it suggests to me that ilamb3 needs to enable any and all of it.
- A method for specifying an external source of uncertainty. This could be an expert global value (in units or as a percentage) as well as a dataset which gives the uncertainty that we give in the configure file:
Soil Carbon:
HWSD-2-0:
sources:
cSoil: HWSD-2-0/obs4MIPs_IIASA-FAO_HWSD-2-0_fx_cSoil_gn_v20250903.nc
uncertainty: 1 kg m-2
# OR
uncertainty: 5%
# OR
uncertainty: ILAMB_cSoil_range.nc
- It may be that we want to globally set uncertainty by a database of values
ilamb3.conf.set(uncertainty_database="values_we_collected.csv")
- Or we may want to flag that we use a series of external data products
ilamb3.conf.set(uncertainty_collection="CARDAMOM")
- With all of these ways to specify uncertainty information, the user would need to be able to set a priority of sources. It would look in the places and use the first source it finds. A default setting might look like this but the user could change it:
ilamb3.conf.set(uncertainty_priority=['reference_data','expert_scalar','external_dataset'])
At the 2025 ILAMB Workshop, one major topic of repeated discussion revolved around how we include notions of observational uncertainty in our benchmarking. While it is becoming more common for reference datasets to include some notion of uncertainty, there are still many that have none. A few long standing datasets (like those for soil carbon) vary highly among products suggesting a large amount of uncertainty yet with no explicit number assigned.
The group came up with several suggestions for what we could do in the absence of explicit information:
While not immediately clear which of these is best, it suggests to me that
ilamb3needs to enable any and all of it.