How to validate a single file¶
Here we document how to validate a single file with input4MIPs validation. This is the second step in preparing files for submission to the input4MIPs collection and, ultimately, publication in the ESGF index. This document assumes that you already have a file you want to submit. If you want to create the file from scratch, check out "How to write a single valid file".
Note: Before you submit your files, there are a few other steps you need to do too. See the instructions for data producers in the input4MIPs CVS repository. Don't forget to do those steps at some point too.
In [1]:
Copied!
import tempfile
from pathlib import Path
import iris
import ncdata.iris_xarray
import xarray as xr
import input4mips_validation.io
import tempfile
from pathlib import Path
import iris
import ncdata.iris_xarray
import xarray as xr
import input4mips_validation.io
In [2]:
Copied!
# Some iris config they recommend
iris.FUTURE.save_split_attrs = True
# Some iris config they recommend
iris.FUTURE.save_split_attrs = True
In [3]:
Copied!
starting_file = Path(
"CH4-em-biomassburning_input4MIPs_emissions_CMIP_CR-CMIP-0-2-0_gn_200001-201012.nc"
)
starting_file = Path(
"CH4-em-biomassburning_input4MIPs_emissions_CMIP_CR-CMIP-0-2-0_gn_200001-201012.nc"
)
Starting point¶
We assume that you are starting from an existing file. We have prepared one for this demo.
A word of advice here: start with the smallest file you possibly can. Get that passing validation first. Using a file with only a few years with data is a good approach, because this ensures that your grid information stays relevant and the time handling scales very easily to longer time axes (having said that, for particularly high resolution data, you may want to start with a coarser grid too). Once you've got your small/test files passing, then move onto validating your full data files. Applying the pattern to your full files is pretty straight forward, but figuring it out in the first place can be a bit fiddly, so you want the iteration time to be as short as possible. As bigger files are much slower to work with, your iteration time becomes much longer, which makes the whole process feel much more painful than it needs to.
In [5]:
Copied!
start = xr.open_dataset(starting_file)
start.data_vars
start = xr.open_dataset(starting_file)
start.data_vars
Out[5]:
Data variables:
CH4 (time, lat, lon) float64 152kB ...
time_bnds (time, bnds) datetime64[ns] 2kB ...
lat_bnds (lat, bnds) float64 192B ...
lon_bnds (lon, bnds) float64 192B ...
In [6]:
Copied!
start_iris = iris.load(starting_file)
start_iris
start_iris = iris.load(starting_file)
start_iris
/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/latest/lib/python3.9/site-packages/iris/fileformats/cf.py:877: IrisCfMissingVarWarning: Missing CF-netCDF measure variable 'gridcellarea', referenced by netCDF variable 'CH4' warnings.warn(
Out[6]:
| Biomass Burning Ch4 Flux (kg m-2 s-1) | time | latitude | longitude |
|---|---|---|---|
| Shape | 132 | 12 | 12 |
| Dimension coordinates | |||
| time | x | - | - |
| latitude | - | x | - |
| longitude | - | - | x |
| Cell methods | |||
| 0 | time: mean | ||
| Attributes | |||
| Conventions | 'CF-1.7' | ||
| activity_id | 'input4MIPs' | ||
| comment | 'Demo' | ||
| contact | 'zebedee.nicholls@climate-resource.com' | ||
| creation_date | '2024-07-25T18:23:47Z' | ||
| data_structure | 'grid' | ||
| dataset_category | 'emissions' | ||
| external_variables | 'gridcellarea,sources' | ||
| frequency | 'mon' | ||
| further_info_url | 'www.tbd.invalid' | ||
| grid_label | 'gn' | ||
| institution | 'Climate Resource' | ||
| institution_id | 'DRES' | ||
| license | 'The input4MIPs data linked to this entry is licensed under a Creative Commons ...' | ||
| mip_era | 'CMIP6Plus' | ||
| nominal_resolution | '25 km' | ||
| realm | 'atmos' | ||
| references | 'Reference to great paper' | ||
| source | 'Climate Resource demo' | ||
| source_id | 'CR-CMIP-0-2-0' | ||
| source_version | '0.2.0' | ||
| target_mip | 'CMIP' | ||
| title | 'Title here' | ||
| variable_id | 'CH4' | ||
The CVs¶
A note before we continue.
In the below, you will notice that there is an option, --cv-source.
This points to the source of the controlled vocabularies (CVs).
You can pick different sources for the CVs.
For example, you can load the CVs from local files,
or from the input4MIPs CVs GitHub
(or any other web source).
In this example, we're going to use a specific commit from the input4MIPs CVs GitHub to avoid anything breaking, even if we make further changes to the CVs. For your own work, you will probably want to use either:
- local files, e.g.
--cv-source path/to/local/files - the branch where you have added your information to the CVs,
--cv-source https://raw.githubusercontent.com/PCMDI/input4MIPs_CVs/branch_name/CVs/ - a tagged version of the input4MIPs CVs GitHub
--cv-source gh:tag-id - the main branch of the input4MIPs CVs GitHub
--cv-source gh:main - a specific commit in the input4MIPs CVs GitHub,
like we do below,
--cv-source gh:commit-hash
This is definitely not the best documented feature of the library, so if anything is unclear, please raise an issue.
Validate the file¶
As a first step, run the file through our validation. Below, we use our command-line interface. There is also a Python API, in case you want to do this directly from Python (note, the logging is setup slightly differently in the Python API so the default shown messages are different, but the behaviour is the same and you can always adjust the logging to suit your own preferences).
In [7]:
Copied!
# The full docs of this command can be accessed with
# !input4mips-validation validate-file --help
!input4mips-validation \
validate-file \
--cv-source "gh:v6.6.0" \
{starting_file}
# The full docs of this command can be accessed with
# !input4mips-validation validate-file --help
!input4mips-validation \
validate-file \
--cv-source "gh:v6.6.0" \
{starting_file}
4625 - 125283322107712 - 2025-10-01T14:41:54.918120+0000 - INFO_FILE - input4mips_validation.validation.file:file.py:105 - Creating validation results for file: docs/how-to-guides/CH4-em-biomassburning_input4MIPs_emissions_CMIP_CR-CMIP-0-2-0_gn_200001-201012.nc 4625 - 125283322107712 - 2025-10-01T14:41:54.927733+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Load controlled vocabularies to use during validation ran without error
4625 - 125283322107712 - 2025-10-01T14:41:54.988685+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Open data with `xr.open_dataset` ran without error /home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/latest/lib/python3.9/site-packages/iris/fileformats/cf.py:877: IrisCfMissingVarWarning: Missing CF-netCDF measure variable 'gridcellarea', referenced by netCDF variable 'CH4' warnings.warn( 4625 - 125283322107712 - 2025-10-01T14:41:55.001319+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Load data with `iris.load` ran without error 4625 - 125283322107712 - 2025-10-01T14:41:55.008437+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Load data with `iris.load_cube` ran without error 4625 - 125283322107712 - 2025-10-01T14:41:55.008545+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.file:file.py:150 - Using the cf-checker to check docs/how-to-guides/CH4-em-biomassburning_input4MIPs_emissions_CMIP_CR-CMIP-0-2-0_gn_200001-201012.nc
4625 - 125283322107712 - 2025-10-01T14:41:55.412231+0000 - INFO_INDIVIDUAL_CHECK_ERROR - input4mips_validation.validation.error_catching:error_catching.py:214 - Check data with cf-checker raised an error (ValueError)
4625 - 125283322107712 - 2025-10-01T14:41:55.451891+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'creation_date' attribute ran without error 4625 - 125283322107712 - 2025-10-01T14:41:55.452070+0000 - INFO_INDIVIDUAL_CHECK_ERROR - input4mips_validation.validation.error_catching:error_catching.py:214 - Validate the 'tracking_id' attribute raised an error (MissingAttributeError) 4625 - 125283322107712 - 2025-10-01T14:41:55.457219+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'frequency' attribute ran without error 4625 - 125283322107712 - 2025-10-01T14:41:55.457361+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'variable_id' attribute ran without error 4625 - 125283322107712 - 2025-10-01T14:41:55.457434+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'activity_id' attribute ran without error 4625 - 125283322107712 - 2025-10-01T14:41:55.457516+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'comment' attribute ran without error 4625 - 125283322107712 - 2025-10-01T14:41:55.457585+0000 - INFO_INDIVIDUAL_CHECK_ERROR - input4mips_validation.validation.error_catching:error_catching.py:214 - Validate the 'external_variables' attribute raised an error (ValueError) 4625 - 125283322107712 - 2025-10-01T14:41:55.457993+0000 - INFO_INDIVIDUAL_CHECK_ERROR - input4mips_validation.validation.error_catching:error_catching.py:214 - Validate the 'contact' attribute raised an error (ValueInconsistentWithCVsError) 4625 - 125283322107712 - 2025-10-01T14:41:55.458247+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'source_version' attribute ran without error 4625 - 125283322107712 - 2025-10-01T14:41:55.458308+0000 - INFO_FILE - input4mips_validation.validation.file:file.py:190 - Created validation results for file: docs/how-to-guides/CH4-em-biomassburning_input4MIPs_emissions_CMIP_CR-CMIP-0-2-0_gn_200001-201012.nc
╭───────────────────── Traceback (most recent call last) ──────────────────────╮ │ /home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda │ │ /latest/lib/python3.9/site-packages/input4mips_validation/cli/__init__.py:29 │ │ 1 in validate_file_command │ │ │ │ 288 │ # TODO: allow this to be passed from CLI │ │ 289 │ bounds_info = None │ │ 290 │ │ │ ❱ 291 │ validate_file( │ │ 292 │ │ file=file, │ │ 293 │ │ cv_source=cv_source, │ │ 294 │ │ write_in_drs=write_in_drs, │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ allow_cf_checker_warnings = False │ │ │ │ bnds_coord_indicators = 'bnds;bounds' │ │ │ │ bounds_info = None │ │ │ │ cv_source = 'gh:v6.6.0' │ │ │ │ file = PosixPath('docs/how-to-guides/CH4-em-biomas… │ │ │ │ frequency_metadata_key = 'frequency' │ │ │ │ frequency_metadata_keys = FrequencyMetadataKeys( │ │ │ │ │ frequency_metadata_key='frequency', │ │ │ │ │ no_time_axis_frequency='fx' │ │ │ │ ) │ │ │ │ no_time_axis_frequency = 'fx' │ │ │ │ time_dimension = 'time' │ │ │ │ write_in_drs = None │ │ │ │ xr_variable_processor = XRVariableHelper( │ │ │ │ │ bounds_coord_indicators=( │ │ │ │ │ │ 'bnds', │ │ │ │ │ │ 'bounds' │ │ │ │ │ ), │ │ │ │ │ climatology_bounds_coord_indicators=( │ │ │ │ │ │ 'climatology', │ │ │ │ │ ) │ │ │ │ ) │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ │ │ │ /home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda │ │ /latest/lib/python3.9/site-packages/input4mips_validation/cli/__init__.py:18 │ │ 7 in validate_file │ │ │ │ 184 │ allow_cf_checker_warnings │ │ 185 │ │ Allow validation to pass, even if the CF-checker raises warnin │ │ 186 │ """ │ │ ❱ 187 │ get_validate_file_result( │ │ 188 │ │ file, │ │ 189 │ │ cv_source=cv_source, │ │ 190 │ │ xr_variable_processor=xr_variable_processor, │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ allow_cf_checker_warnings = False │ │ │ │ bounds_info = None │ │ │ │ cv_source = 'gh:v6.6.0' │ │ │ │ file = PosixPath('docs/how-to-guides/CH4-em-biomas… │ │ │ │ frequency_metadata_keys = FrequencyMetadataKeys( │ │ │ │ │ frequency_metadata_key='frequency', │ │ │ │ │ no_time_axis_frequency='fx' │ │ │ │ ) │ │ │ │ time_dimension = 'time' │ │ │ │ write_in_drs = None │ │ │ │ xr_variable_processor = XRVariableHelper( │ │ │ │ │ bounds_coord_indicators=( │ │ │ │ │ │ 'bnds', │ │ │ │ │ │ 'bounds' │ │ │ │ │ ), │ │ │ │ │ climatology_bounds_coord_indicators=( │ │ │ │ │ │ 'climatology', │ │ │ │ │ ) │ │ │ │ ) │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ │ │ │ /home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda │ │ /latest/lib/python3.9/site-packages/input4mips_validation/validation/error_c │ │ atching.py:241 in raise_if_errors │ │ │ │ 238 │ │ │ One of the validation steps in `self.validation_results` f │ │ 239 │ │ """ │ │ 240 │ │ if not self.all_passed: │ │ ❱ 241 │ │ │ raise ValidationResultsStoreError(self) │ │ 242 │ │ │ 243 │ def checks_summary_str(self, passing: bool) -> str: │ │ 244 │ │ """ │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ self = ValidationResultsStore( │ │ │ │ │ validation_results=[ │ │ │ │ │ │ ValidationResult( │ │ │ │ │ │ │ description='Load controlled vocabularies to use │ │ │ │ during validation', │ │ │ │ │ │ │ passed=True, │ │ │ │ │ │ │ result=Input4MIPsCVs( │ │ │ │ │ │ │ │ raw_loader=RawCVLoaderKnownRemoteRegistry( │ │ │ │ │ │ │ │ │ registry=<pooch.core.Pooch object at │ │ │ │ 0x71f1ae55c6a0>, │ │ │ │ │ │ │ │ │ force_download=False │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ DRS=DataReferenceSyntax( │ │ │ │ │ │ │ │ │ │ │ │ │ directory_path_template='<activity_id>/<mip_era>/<target_mip>/<i… │ │ │ │ │ │ │ │ │ │ │ │ │ directory_path_example='input4MIPs/CMIP6Plus/CMIP/PCMDI/PCMDI-AM… │ │ │ │ │ │ │ │ │ │ │ │ │ filename_template='<variable_id>_<activity_id>_<dataset_category… │ │ │ │ │ │ │ │ │ │ │ │ │ filename_example='tos_input4MIPs_SSTsAndSeaIce_CMIP_PCMDI-AMIP-1… │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ activity_id_entries=ActivityIDEntries( │ │ │ │ │ │ │ │ │ entries=( │ │ │ │ │ │ │ │ │ │ ActivityIDEntry( │ │ │ │ │ │ │ │ │ │ │ activity_id='input4MIPs', │ │ │ │ │ │ │ │ │ │ │ values=ActivityIDValues( │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ URL='https://pcmdi.llnl.gov/mips/input4MIPs/', │ │ │ │ │ │ │ │ │ │ │ │ long_name='input forcing datasets │ │ │ │ for Model Intercomparison Projects' │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ institution_ids=( │ │ │ │ │ │ │ │ │ 'CCCma', │ │ │ │ │ │ │ │ │ 'CNRM-Cerfacs', │ │ │ │ │ │ │ │ │ 'CR', │ │ │ │ │ │ │ │ │ 'DRES', │ │ │ │ │ │ │ │ │ 'IACETH', │ │ │ │ │ │ │ │ │ 'IAMC', │ │ │ │ │ │ │ │ │ 'ImperialCollege', │ │ │ │ │ │ │ │ │ 'MOHC', │ │ │ │ │ │ │ │ │ 'MPI-B', │ │ │ │ │ │ │ │ │ 'MPI-M', │ │ │ │ │ │ │ │ │ ... +13 │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ license_entries=LicenseEntries( │ │ │ │ │ │ │ │ │ entries=( │ │ │ │ │ │ │ │ │ │ LicenseEntry( │ │ │ │ │ │ │ │ │ │ │ license_id='CC BY 4.0', │ │ │ │ │ │ │ │ │ │ │ values=LicenseValues( │ │ │ │ │ │ │ │ │ │ │ │ conditions='The input4MIPs data │ │ │ │ linked to this entry is licensed under a Creative Commons │ │ │ │ At'+569, │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ license_url='https://creativecommons.org/licenses/by/4.0/', │ │ │ │ │ │ │ │ │ │ │ │ long_name='Creative Commons │ │ │ │ Attribution 4.0 International' │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ source_id_entries=SourceIDEntries( │ │ │ │ │ │ │ │ │ entries=( │ │ │ │ │ │ │ │ │ │ SourceIDEntry( │ │ │ │ │ │ │ │ │ │ │ source_id='CEDS-CMIP-2024-07-08', │ │ │ │ │ │ │ │ │ │ │ values=SourceIDValues( │ │ │ │ │ │ │ │ │ │ │ │ contact='Rachel Hoesly │ │ │ │ (rachel.hoesly@pnnl.gov); Steve Smith (ssmith@pnnl.gov)', │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ further_info_url='https://www.pnnl.gov/projects/ceds', │ │ │ │ │ │ │ │ │ │ │ │ institution_id='PNNL-JGCRI', │ │ │ │ │ │ │ │ │ │ │ │ mip_era='CMIP6Plus', │ │ │ │ │ │ │ │ │ │ │ │ source_version='2024-07-08', │ │ │ │ │ │ │ │ │ │ │ │ license_id='CC BY 4.0', │ │ │ │ │ │ │ │ │ │ │ │ authors=None │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ │ SourceIDEntry( │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ source_id='CEDS-CMIP-2024-07-08-supplemental', │ │ │ │ │ │ │ │ │ │ │ values=SourceIDValues( │ │ │ │ │ │ │ │ │ │ │ │ contact='Rachel Hoesly │ │ │ │ (rachel.hoesly@pnnl.gov); Steve Smith (ssmith@pnnl.gov)', │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ further_info_url='https://www.pnnl.gov/projects/ceds', │ │ │ │ │ │ │ │ │ │ │ │ institution_id='PNNL-JGCRI', │ │ │ │ │ │ │ │ │ │ │ │ mip_era='CMIP6Plus', │ │ │ │ │ │ │ │ │ │ │ │ source_version='2024-07-08', │ │ │ │ │ │ │ │ │ │ │ │ license_id='CC BY 4.0', │ │ │ │ │ │ │ │ │ │ │ │ authors=None │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ │ SourceIDEntry( │ │ │ │ │ │ │ │ │ │ │ source_id='CEDS-CMIP-2024-10-21', │ │ │ │ │ │ │ │ │ │ │ values=SourceIDValues( │ │ │ │ │ │ │ │ │ │ │ │ contact='Rachel Hoesly │ │ │ │ (rachel.hoesly@pnnl.gov); Steve Smith (ssmith@pnnl.gov)', │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ further_info_url='https://www.pnnl.gov/projects/ceds', │ │ │ │ │ │ │ │ │ │ │ │ institution_id='PNNL-JGCRI', │ │ │ │ │ │ │ │ │ │ │ │ mip_era='CMIP6Plus', │ │ │ │ │ │ │ │ │ │ │ │ source_version='2024-10-21', │ │ │ │ │ │ │ │ │ │ │ │ license_id='CC BY 4.0', │ │ │ │ │ │ │ │ │ │ │ │ authors=None │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ │ SourceIDEntry( │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ source_id='CEDS-CMIP-2024-10-21-supplemental', │ │ │ │ │ │ │ │ │ │ │ values=SourceIDValues( │ │ │ │ │ │ │ │ │ │ │ │ contact='Rachel Hoesly │ │ │ │ (rachel.hoesly@pnnl.gov); Steve Smith (ssmith@pnnl.gov)', │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ further_info_url='https://www.pnnl.gov/projects/ceds', │ │ │ │ │ │ │ │ │ │ │ │ institution_id='PNNL-JGCRI', │ │ │ │ │ │ │ │ │ │ │ │ mip_era='CMIP6Plus', │ │ │ │ │ │ │ │ │ │ │ │ source_version='2024-10-21', │ │ │ │ │ │ │ │ │ │ │ │ license_id='CC BY 4.0', │ │ │ │ │ │ │ │ │ │ │ │ authors=None │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ │ SourceIDEntry( │ │ │ │ │ │ │ │ │ │ │ source_id='CR-CMIP-0-2-0', │ │ │ │ │ │ │ │ │ │ │ values=SourceIDValues( │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ contact='zebedee.nicholls@climate-resource.com;malte.meinshausen… │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ further_info_url='http://www.tbd.invalid', │ │ │ │ │ │ │ │ │ │ │ │ institution_id='CR', │ │ │ │ │ │ │ │ │ │ │ │ mip_era='CMIP6Plus', │ │ │ │ │ │ │ │ │ │ │ │ source_version='0.2.0', │ │ │ │ │ │ │ │ │ │ │ │ license_id='CC BY 4.0', │ │ │ │ │ │ │ │ │ │ │ │ authors=None │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ │ SourceIDEntry( │ │ │ │ │ │ │ │ │ │ │ source_id='CR-CMIP-0-3-0', │ │ │ │ │ │ │ │ │ │ │ values=SourceIDValues( │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ contact='zebedee.nicholls@climate-resource.com;malte.meinshausen… │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ further_info_url='http://www.tbd.invalid', │ │ │ │ │ │ │ │ │ │ │ │ institution_id='CR', │ │ │ │ │ │ │ │ │ │ │ │ mip_era='CMIP6Plus', │ │ │ │ │ │ │ │ │ │ │ │ source_version='0.3.0', │ │ │ │ │ │ │ │ │ │ │ │ license_id='CC BY 4.0', │ │ │ │ │ │ │ │ │ │ │ │ authors=None │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ │ SourceIDEntry( │ │ │ │ │ │ │ │ │ │ │ source_id='DRES-CMIP-BB4CMIP7-1-0', │ │ │ │ │ │ │ │ │ │ │ values=SourceIDValues( │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ contact='Margreet.vanMarle@Deltares.nl;Guido.vanderWerf@wur.nl', │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ further_info_url='http://www.globalfiredata.org', │ │ │ │ │ │ │ │ │ │ │ │ institution_id='DRES', │ │ │ │ │ │ │ │ │ │ │ │ mip_era='CMIP6Plus', │ │ │ │ │ │ │ │ │ │ │ │ source_version='1.0', │ │ │ │ │ │ │ │ │ │ │ │ license_id='CC BY 4.0', │ │ │ │ │ │ │ │ │ │ │ │ authors=None │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ │ SourceIDEntry( │ │ │ │ │ │ │ │ │ │ │ source_id='MRI-JRA55-do-1-6-0', │ │ │ │ │ │ │ │ │ │ │ values=SourceIDValues( │ │ │ │ │ │ │ │ │ │ │ │ contact='Hiroyuki Tsujino │ │ │ │ (htsujino@mri-jma.go.jp)', │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ further_info_url='https://climate.mri-jma.go.jp/pub/ocean/JRA55-… │ │ │ │ │ │ │ │ │ │ │ │ institution_id='MRI', │ │ │ │ │ │ │ │ │ │ │ │ mip_era='CMIP6Plus', │ │ │ │ │ │ │ │ │ │ │ │ source_version='1.6.0', │ │ │ │ │ │ │ │ │ │ │ │ license_id=None, │ │ │ │ │ │ │ │ │ │ │ │ authors=None │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ │ SourceIDEntry( │ │ │ │ │ │ │ │ │ │ │ source_id='PCMDI-AMIP-1-1-9', │ │ │ │ │ │ │ │ │ │ │ values=SourceIDValues( │ │ │ │ │ │ │ │ │ │ │ │ contact='PCMDI │ │ │ │ (pcmdi-cmip@llnl.gov)', │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ further_info_url='https://pcmdi.llnl.gov/mips/amip', │ │ │ │ │ │ │ │ │ │ │ │ institution_id='PCMDI', │ │ │ │ │ │ │ │ │ │ │ │ mip_era='CMIP6Plus', │ │ │ │ │ │ │ │ │ │ │ │ source_version='1.1.9', │ │ │ │ │ │ │ │ │ │ │ │ license_id='CC BY 4.0', │ │ │ │ │ │ │ │ │ │ │ │ authors=None │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ │ SourceIDEntry( │ │ │ │ │ │ │ │ │ │ │ source_id='PCMDI-AMIP-ERSST5-1-0', │ │ │ │ │ │ │ │ │ │ │ values=SourceIDValues( │ │ │ │ │ │ │ │ │ │ │ │ contact='zelinka1@llnl.gov; │ │ │ │ durack1@llnl.gov', │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ further_info_url='https://pcmdi.llnl.gov/mips/amip', │ │ │ │ │ │ │ │ │ │ │ │ institution_id='PCMDI', │ │ │ │ │ │ │ │ │ │ │ │ mip_era='CMIP6Plus', │ │ │ │ │ │ │ │ │ │ │ │ source_version='1.0', │ │ │ │ │ │ │ │ │ │ │ │ license_id='CC BY 4.0', │ │ │ │ │ │ │ │ │ │ │ │ authors=None │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ │ ... +10 │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ exception=None, │ │ │ │ │ │ │ exception_info=None │ │ │ │ │ │ ), │ │ │ │ │ │ ValidationResult( │ │ │ │ │ │ │ description='Open data with `xr.open_dataset`', │ │ │ │ │ │ │ passed=True, │ │ │ │ │ │ │ result=<xarray.Dataset> Size: 156kB │ │ │ │ Dimensions: (time: 132, lat: 12, lon: 12, bnds: 2) │ │ │ │ Coordinates: │ │ │ │ * time (time) object 1kB 2000-01-01 00:00:00 ... │ │ │ │ 2010-12-01 00:00:00 │ │ │ │ * lat (lat) float64 96B -82.5 -67.5 -52.5 -37.5 ... 37.5 │ │ │ │ 52.5 67.5 82.5 │ │ │ │ * lon (lon) float64 96B -165.0 -135.0 -105.0 ... 105.0 │ │ │ │ 135.0 165.0 │ │ │ │ Dimensions without coordinates: bnds │ │ │ │ Data variables: │ │ │ │ │ CH4 (time, lat, lon) float64 152kB ... │ │ │ │ │ time_bnds (time, bnds) object 2kB ... │ │ │ │ │ lat_bnds (lat, bnds) float64 192B ... │ │ │ │ │ lon_bnds (lon, bnds) float64 192B ... │ │ │ │ Attributes: (12/24) │ │ │ │ │ Conventions: CF-1.7 │ │ │ │ │ activity_id: input4MIPs │ │ │ │ │ comment: Demo │ │ │ │ │ contact: zebedee.nicholls@climate-resource.com │ │ │ │ │ creation_date: 2024-07-25T18:23:47Z │ │ │ │ │ data_structure: grid │ │ │ │ │ ... ... │ │ │ │ │ source: Climate Resource demo │ │ │ │ │ source_id: CR-CMIP-0-2-0 │ │ │ │ │ source_version: 0.2.0 │ │ │ │ │ target_mip: CMIP │ │ │ │ │ title: Title here │ │ │ │ │ variable_id: CH4, │ │ │ │ │ │ │ exception=None, │ │ │ │ │ │ │ exception_info=None │ │ │ │ │ │ ), │ │ │ │ │ │ ValidationResult( │ │ │ │ │ │ │ description='Load data with `iris.load`', │ │ │ │ │ │ │ passed=True, │ │ │ │ │ │ │ result=[<iris 'Cube' of biomass_burning_CH4_flux / │ │ │ │ (kg m-2 s-1) (time: 132; latitude: 12; longitude: 12)>], │ │ │ │ │ │ │ exception=None, │ │ │ │ │ │ │ exception_info=None │ │ │ │ │ │ ), │ │ │ │ │ │ ValidationResult( │ │ │ │ │ │ │ description='Load data with `iris.load_cube`', │ │ │ │ │ │ │ passed=True, │ │ │ │ │ │ │ result=<iris 'Cube' of biomass_burning_CH4_flux / (kg │ │ │ │ m-2 s-1) (time: 132; latitude: 12; longitude: 12)>, │ │ │ │ │ │ │ exception=None, │ │ │ │ │ │ │ exception_info=None │ │ │ │ │ │ ), │ │ │ │ │ │ ValidationResult( │ │ │ │ │ │ │ description='Check data with cf-checker', │ │ │ │ │ │ │ passed=False, │ │ │ │ │ │ │ result=None, │ │ │ │ │ │ │ exception=ValueError('cf-checker validation failed. │ │ │ │ cfchecks output:\n\nCHECKING NetCDF FILE: │ │ │ │ docs/how-to-guides/CH4-em-biomassburning_input4MIPs_emissions_CM… │ │ │ │ CF Checker Version 4.1.0\nChecking against CF Version │ │ │ │ CF-1.7\nUsing Standard Name Table Version 92 │ │ │ │ (2025-07-24T14:20:46Z)\nUsing Area Type Table Version 13 (20 │ │ │ │ March 2025)\nUsing Standardized Region Name Table Version 5 (12 │ │ │ │ November 2024)\n\nERROR: (2.6.3): external_variables attribute │ │ │ │ must be a blank separated list of variable │ │ │ │ names\n\n------------------\nChecking variable: │ │ │ │ CH4\n------------------\nERROR: (3.3): Invalid standard_name: │ │ │ │ biomass_burning_CH4_flux\nERROR: (7.2): cell_measures variable │ │ │ │ gridcellarea must either exist in this netCDF file or be named by │ │ │ │ the external_variables attribute\n\n------------------\nChecking │ │ │ │ variable: │ │ │ │ time\n------------------\n\n------------------\nChecking │ │ │ │ variable: │ │ │ │ time_bnds\n------------------\n\n------------------\nChecking │ │ │ │ variable: lat\n------------------\n\n------------------\nChecking │ │ │ │ variable: │ │ │ │ lat_bnds\n------------------\n\n------------------\nChecking │ │ │ │ variable: lon\n------------------\n\n------------------\nChecking │ │ │ │ variable: lon_bnds\n------------------\n\nERRORS detected: │ │ │ │ 3\nWARNINGS given: 0\nINFORMATION messages: 0\n'), │ │ │ │ │ │ │ exception_info='Traceback (most recent call last):\n │ │ │ │ File "/home/docs/checkouts/readthedocs.org/'+2733 │ │ │ │ │ │ ), │ │ │ │ │ │ ValidationResult( │ │ │ │ │ │ │ description="Validate the 'creation_date' attribute", │ │ │ │ │ │ │ passed=True, │ │ │ │ │ │ │ result=None, │ │ │ │ │ │ │ exception=None, │ │ │ │ │ │ │ exception_info=None │ │ │ │ │ │ ), │ │ │ │ │ │ ValidationResult( │ │ │ │ │ │ │ description="Validate the 'tracking_id' attribute", │ │ │ │ │ │ │ passed=False, │ │ │ │ │ │ │ result=None, │ │ │ │ │ │ │ exception=MissingAttributeError('tracking_id'), │ │ │ │ │ │ │ exception_info='Traceback (most recent call last):\n │ │ │ │ File "/home/docs/checkouts/readthedocs.org/'+536 │ │ │ │ │ │ ), │ │ │ │ │ │ ValidationResult( │ │ │ │ │ │ │ description="Validate the 'frequency' attribute", │ │ │ │ │ │ │ passed=True, │ │ │ │ │ │ │ result=None, │ │ │ │ │ │ │ exception=None, │ │ │ │ │ │ │ exception_info=None │ │ │ │ │ │ ), │ │ │ │ │ │ ValidationResult( │ │ │ │ │ │ │ description="Validate the 'variable_id' attribute", │ │ │ │ │ │ │ passed=True, │ │ │ │ │ │ │ result=None, │ │ │ │ │ │ │ exception=None, │ │ │ │ │ │ │ exception_info=None │ │ │ │ │ │ ), │ │ │ │ │ │ ValidationResult( │ │ │ │ │ │ │ description="Validate the 'activity_id' attribute", │ │ │ │ │ │ │ passed=True, │ │ │ │ │ │ │ result=None, │ │ │ │ │ │ │ exception=None, │ │ │ │ │ │ │ exception_info=None │ │ │ │ │ │ ), │ │ │ │ │ │ ... +4 │ │ │ │ │ ] │ │ │ │ ) │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ ╰──────────────────────────────────────────────────────────────────────────────╯ ValidationResultsStoreError: Checks passing: 71.43% (10 / 14) Checks failing: 28.57% (4 / 14) Failing checks details Check data with cf-checker (ValueError) Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/cf_checker.py" , line 125, in check_with_cf_checker subprocess.check_output( File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/subprocess.py", line 424, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/subprocess.py", line 528, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/ latest/bin/cfchecks', '-v', '1.7', 'docs/how-to-guides/CH4-em-biomassburning_input4MIPs_emissions_CMIP_CR-CMIP-0-2- 0_gn_200001-201012.nc']' returned non-zero exit status 3. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/error_catching .py", line 202, in decorated res_func = func_to_call(*args, **kwargs) File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/cf_checker.py" , line 141, in check_with_cf_checker raise ValueError(error_msg) from exc ValueError: cf-checker validation failed. cfchecks output: CHECKING NetCDF FILE: docs/how-to-guides/CH4-em-biomassburning_input4MIPs_emissions_CMIP_CR-CMIP-0-2-0 _gn_200001-201012.nc ===================== Using CF Checker Version 4.1.0 Checking against CF Version CF-1.7 Using Standard Name Table Version 92 (2025-07-24T14:20:46Z) Using Area Type Table Version 13 (20 March 2025) Using Standardized Region Name Table Version 5 (12 November 2024) ERROR: (2.6.3): external_variables attribute must be a blank separated list of variable names ------------------ Checking variable: CH4 ------------------ ERROR: (3.3): Invalid standard_name: biomass_burning_CH4_flux ERROR: (7.2): cell_measures variable gridcellarea must either exist in this netCDF file or be named by the external_variables attribute ------------------ Checking variable: time ------------------ ------------------ Checking variable: time_bnds ------------------ ------------------ Checking variable: lat ------------------ ------------------ Checking variable: lat_bnds ------------------ ------------------ Checking variable: lon ------------------ ------------------ Checking variable: lon_bnds ------------------ ERRORS detected: 3 WARNINGS given: 0 INFORMATION messages: 0 Validate the 'tracking_id' attribute (MissingAttributeError) Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/error_catching .py", line 202, in decorated res_func = func_to_call(*args, **kwargs) File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/datasets_to_wr ite_to_disk.py", line 104, in validate_attribute raise MissingAttributeError(attribute) input4mips_validation.validation.exceptions.MissingAttributeError: 'tracking_id' Validate the 'external_variables' attribute (ValueError) Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/external_varia bles.py", line 28, in validate_external_variables check_only_allowed_characters( File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/allowed_charac ters.py", line 74, in check_only_allowed_characters raise ValueError(msg) ValueError: Only alphanumeric characters, underscores and whitespaces are allowed. value='gridcellarea,sources'. invalid_chars={','}. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/error_catching .py", line 202, in decorated res_func = func_to_call(*args, **kwargs) File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/datasets_to_wr ite_to_disk.py", line 107, in validate_attribute validation_function(attribute_value) File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/external_varia bles.py", line 39, in validate_external_variables raise ValueError(msg) from exc ValueError: The `external_variables` attribute contains invalid characters. If you are providing multiple variables, they should be whitespace-separated. Only alphanumeric characters, underscores and whitespaces are allowed. value='gridcellarea,sources'. invalid_chars={','}. Validate the 'contact' attribute (ValueInconsistentWithCVsError) Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/error_catching .py", line 202, in decorated res_func = func_to_call(*args, **kwargs) File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/datasets_to_wr ite_to_disk.py", line 150, in validate_attribute_that_depends_on_other_attribute validation_function(attribute_value, attribute_dependent_on_value) File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/cvs/cvs.py", line 130, in validate_contact raise ValueInconsistentWithCVsError( input4mips_validation.cvs.exceptions.ValueInconsistentWithCVsError: The value provided for contact was 'zebedee.nicholls@climate-resource.com'. According to the CVs, contact depends on the value of source_id. As a result, contact must be 'zebedee.nicholls@climate-resource.com;malte.meinshausen@climate-resource.com'. If helpful, the full CV entry for source_id is: SourceIDEntry(source_id='CR-CMIP-0-2-0', values=SourceIDValues(contact='zebedee.nicholls@climate-resource.com;malte.meins hausen@climate-resource.com', further_info_url='http://www.tbd.invalid', institution_id='CR', mip_era='CMIP6Plus', source_version='0.2.0', license_id='CC BY 4.0', authors=None)).
Getting more detail¶
In this case, our file has failed validation. We can see that there are three checks which failed:
- the check with the cf-checker
- the check of the "contact" attribute
- the check of the "tracking_id" attribute
To find out exactly why this failed, we can re-run the validation with a more detailed log level.
Note: This produces more output.
In [8]:
Copied!
!input4mips-validation --logging-level "DEBUG" \
validate-file \
--cv-source "gh:v6.6.0" \
{starting_file}
!input4mips-validation --logging-level "DEBUG" \
validate-file \
--cv-source "gh:v6.6.0" \
{starting_file}
4632 - 132830095095616 - 2025-10-01T14:41:57.044269+0000 - INFO_FILE - input4mips_validation.validation.file:file.py:105 - Creating validation results for file: docs/how-to-guides/CH4-em-biomassburning_input4MIPs_emissions_CMIP_CR-CMIP-0-2-0_gn_200001-201012.nc 4632 - 132830095095616 - 2025-10-01T14:41:57.044398+0000 - DEBUG - input4mips_validation.validation.file:file.py:110 - Instantiating a new `ValidationResultsStore` 4632 - 132830095095616 - 2025-10-01T14:41:57.049378+0000 - DEBUG - input4mips_validation.cvs.loading:loading.py:46 - raw_cvs_loader=RawCVLoaderKnownRemoteRegistry(registry=<pooch.core.Pooch object at 0x78cecd0f5940>, force_download=False) 4632 - 132830095095616 - 2025-10-01T14:41:57.053660+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Load controlled vocabularies to use during validation ran without error
4632 - 132830095095616 - 2025-10-01T14:41:57.112475+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Open data with `xr.open_dataset` ran without error
/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/latest/lib/python3.9/site-packages/iris/fileformats/cf.py:877: IrisCfMissingVarWarning: Missing CF-netCDF measure variable 'gridcellarea', referenced by netCDF variable 'CH4' warnings.warn( 4632 - 132830095095616 - 2025-10-01T14:41:57.125542+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Load data with `iris.load` ran without error 4632 - 132830095095616 - 2025-10-01T14:41:57.132483+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Load data with `iris.load_cube` ran without error 4632 - 132830095095616 - 2025-10-01T14:41:57.132575+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.file:file.py:150 - Using the cf-checker to check docs/how-to-guides/CH4-em-biomassburning_input4MIPs_emissions_CMIP_CR-CMIP-0-2-0_gn_200001-201012.nc 4632 - 132830095095616 - 2025-10-01T14:41:57.132612+0000 - DEBUG - input4mips_validation.validation.file:file.py:154 - allow_cf_checker_warnings=False
4632 - 132830095095616 - 2025-10-01T14:41:57.409154+0000 - INFO_INDIVIDUAL_CHECK_ERROR - input4mips_validation.validation.error_catching:error_catching.py:214 - Check data with cf-checker raised an error (ValueError)
4632 - 132830095095616 - 2025-10-01T14:41:57.447693+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'creation_date' attribute ran without error 4632 - 132830095095616 - 2025-10-01T14:41:57.447862+0000 - INFO_INDIVIDUAL_CHECK_ERROR - input4mips_validation.validation.error_catching:error_catching.py:214 - Validate the 'tracking_id' attribute raised an error (MissingAttributeError) 4632 - 132830095095616 - 2025-10-01T14:41:57.452914+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'frequency' attribute ran without error 4632 - 132830095095616 - 2025-10-01T14:41:57.453047+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'variable_id' attribute ran without error 4632 - 132830095095616 - 2025-10-01T14:41:57.453122+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'activity_id' attribute ran without error 4632 - 132830095095616 - 2025-10-01T14:41:57.453215+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'comment' attribute ran without error 4632 - 132830095095616 - 2025-10-01T14:41:57.453274+0000 - INFO_INDIVIDUAL_CHECK_ERROR - input4mips_validation.validation.error_catching:error_catching.py:214 - Validate the 'external_variables' attribute raised an error (ValueError) 4632 - 132830095095616 - 2025-10-01T14:41:57.453668+0000 - INFO_INDIVIDUAL_CHECK_ERROR - input4mips_validation.validation.error_catching:error_catching.py:214 - Validate the 'contact' attribute raised an error (ValueInconsistentWithCVsError) 4632 - 132830095095616 - 2025-10-01T14:41:57.453880+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'source_version' attribute ran without error 4632 - 132830095095616 - 2025-10-01T14:41:57.453972+0000 - INFO_FILE - input4mips_validation.validation.file:file.py:190 - Created validation results for file: docs/how-to-guides/CH4-em-biomassburning_input4MIPs_emissions_CMIP_CR-CMIP-0-2-0_gn_200001-201012.nc
╭───────────────────── Traceback (most recent call last) ──────────────────────╮ │ /home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda │ │ /latest/lib/python3.9/site-packages/input4mips_validation/cli/__init__.py:29 │ │ 1 in validate_file_command │ │ │ │ 288 │ # TODO: allow this to be passed from CLI │ │ 289 │ bounds_info = None │ │ 290 │ │ │ ❱ 291 │ validate_file( │ │ 292 │ │ file=file, │ │ 293 │ │ cv_source=cv_source, │ │ 294 │ │ write_in_drs=write_in_drs, │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ allow_cf_checker_warnings = False │ │ │ │ bnds_coord_indicators = 'bnds;bounds' │ │ │ │ bounds_info = None │ │ │ │ cv_source = 'gh:v6.6.0' │ │ │ │ file = PosixPath('docs/how-to-guides/CH4-em-biomas… │ │ │ │ frequency_metadata_key = 'frequency' │ │ │ │ frequency_metadata_keys = FrequencyMetadataKeys( │ │ │ │ │ frequency_metadata_key='frequency', │ │ │ │ │ no_time_axis_frequency='fx' │ │ │ │ ) │ │ │ │ no_time_axis_frequency = 'fx' │ │ │ │ time_dimension = 'time' │ │ │ │ write_in_drs = None │ │ │ │ xr_variable_processor = XRVariableHelper( │ │ │ │ │ bounds_coord_indicators=( │ │ │ │ │ │ 'bnds', │ │ │ │ │ │ 'bounds' │ │ │ │ │ ), │ │ │ │ │ climatology_bounds_coord_indicators=( │ │ │ │ │ │ 'climatology', │ │ │ │ │ ) │ │ │ │ ) │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ │ │ │ /home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda │ │ /latest/lib/python3.9/site-packages/input4mips_validation/cli/__init__.py:18 │ │ 7 in validate_file │ │ │ │ 184 │ allow_cf_checker_warnings │ │ 185 │ │ Allow validation to pass, even if the CF-checker raises warnin │ │ 186 │ """ │ │ ❱ 187 │ get_validate_file_result( │ │ 188 │ │ file, │ │ 189 │ │ cv_source=cv_source, │ │ 190 │ │ xr_variable_processor=xr_variable_processor, │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ allow_cf_checker_warnings = False │ │ │ │ bounds_info = None │ │ │ │ cv_source = 'gh:v6.6.0' │ │ │ │ file = PosixPath('docs/how-to-guides/CH4-em-biomas… │ │ │ │ frequency_metadata_keys = FrequencyMetadataKeys( │ │ │ │ │ frequency_metadata_key='frequency', │ │ │ │ │ no_time_axis_frequency='fx' │ │ │ │ ) │ │ │ │ time_dimension = 'time' │ │ │ │ write_in_drs = None │ │ │ │ xr_variable_processor = XRVariableHelper( │ │ │ │ │ bounds_coord_indicators=( │ │ │ │ │ │ 'bnds', │ │ │ │ │ │ 'bounds' │ │ │ │ │ ), │ │ │ │ │ climatology_bounds_coord_indicators=( │ │ │ │ │ │ 'climatology', │ │ │ │ │ ) │ │ │ │ ) │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ │ │ │ /home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda │ │ /latest/lib/python3.9/site-packages/input4mips_validation/validation/error_c │ │ atching.py:241 in raise_if_errors │ │ │ │ 238 │ │ │ One of the validation steps in `self.validation_results` f │ │ 239 │ │ """ │ │ 240 │ │ if not self.all_passed: │ │ ❱ 241 │ │ │ raise ValidationResultsStoreError(self) │ │ 242 │ │ │ 243 │ def checks_summary_str(self, passing: bool) -> str: │ │ 244 │ │ """ │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ self = ValidationResultsStore( │ │ │ │ │ validation_results=[ │ │ │ │ │ │ ValidationResult( │ │ │ │ │ │ │ description='Load controlled vocabularies to use │ │ │ │ during validation', │ │ │ │ │ │ │ passed=True, │ │ │ │ │ │ │ result=Input4MIPsCVs( │ │ │ │ │ │ │ │ raw_loader=RawCVLoaderKnownRemoteRegistry( │ │ │ │ │ │ │ │ │ registry=<pooch.core.Pooch object at │ │ │ │ 0x78cecd0f5940>, │ │ │ │ │ │ │ │ │ force_download=False │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ DRS=DataReferenceSyntax( │ │ │ │ │ │ │ │ │ │ │ │ │ directory_path_template='<activity_id>/<mip_era>/<target_mip>/<i… │ │ │ │ │ │ │ │ │ │ │ │ │ directory_path_example='input4MIPs/CMIP6Plus/CMIP/PCMDI/PCMDI-AM… │ │ │ │ │ │ │ │ │ │ │ │ │ filename_template='<variable_id>_<activity_id>_<dataset_category… │ │ │ │ │ │ │ │ │ │ │ │ │ filename_example='tos_input4MIPs_SSTsAndSeaIce_CMIP_PCMDI-AMIP-1… │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ activity_id_entries=ActivityIDEntries( │ │ │ │ │ │ │ │ │ entries=( │ │ │ │ │ │ │ │ │ │ ActivityIDEntry( │ │ │ │ │ │ │ │ │ │ │ activity_id='input4MIPs', │ │ │ │ │ │ │ │ │ │ │ values=ActivityIDValues( │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ URL='https://pcmdi.llnl.gov/mips/input4MIPs/', │ │ │ │ │ │ │ │ │ │ │ │ long_name='input forcing datasets │ │ │ │ for Model Intercomparison Projects' │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ institution_ids=( │ │ │ │ │ │ │ │ │ 'CCCma', │ │ │ │ │ │ │ │ │ 'CNRM-Cerfacs', │ │ │ │ │ │ │ │ │ 'CR', │ │ │ │ │ │ │ │ │ 'DRES', │ │ │ │ │ │ │ │ │ 'IACETH', │ │ │ │ │ │ │ │ │ 'IAMC', │ │ │ │ │ │ │ │ │ 'ImperialCollege', │ │ │ │ │ │ │ │ │ 'MOHC', │ │ │ │ │ │ │ │ │ 'MPI-B', │ │ │ │ │ │ │ │ │ 'MPI-M', │ │ │ │ │ │ │ │ │ ... +13 │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ license_entries=LicenseEntries( │ │ │ │ │ │ │ │ │ entries=( │ │ │ │ │ │ │ │ │ │ LicenseEntry( │ │ │ │ │ │ │ │ │ │ │ license_id='CC BY 4.0', │ │ │ │ │ │ │ │ │ │ │ values=LicenseValues( │ │ │ │ │ │ │ │ │ │ │ │ conditions='The input4MIPs data │ │ │ │ linked to this entry is licensed under a Creative Commons │ │ │ │ At'+569, │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ license_url='https://creativecommons.org/licenses/by/4.0/', │ │ │ │ │ │ │ │ │ │ │ │ long_name='Creative Commons │ │ │ │ Attribution 4.0 International' │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ source_id_entries=SourceIDEntries( │ │ │ │ │ │ │ │ │ entries=( │ │ │ │ │ │ │ │ │ │ SourceIDEntry( │ │ │ │ │ │ │ │ │ │ │ source_id='CEDS-CMIP-2024-07-08', │ │ │ │ │ │ │ │ │ │ │ values=SourceIDValues( │ │ │ │ │ │ │ │ │ │ │ │ contact='Rachel Hoesly │ │ │ │ (rachel.hoesly@pnnl.gov); Steve Smith (ssmith@pnnl.gov)', │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ further_info_url='https://www.pnnl.gov/projects/ceds', │ │ │ │ │ │ │ │ │ │ │ │ institution_id='PNNL-JGCRI', │ │ │ │ │ │ │ │ │ │ │ │ mip_era='CMIP6Plus', │ │ │ │ │ │ │ │ │ │ │ │ source_version='2024-07-08', │ │ │ │ │ │ │ │ │ │ │ │ license_id='CC BY 4.0', │ │ │ │ │ │ │ │ │ │ │ │ authors=None │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ │ SourceIDEntry( │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ source_id='CEDS-CMIP-2024-07-08-supplemental', │ │ │ │ │ │ │ │ │ │ │ values=SourceIDValues( │ │ │ │ │ │ │ │ │ │ │ │ contact='Rachel Hoesly │ │ │ │ (rachel.hoesly@pnnl.gov); Steve Smith (ssmith@pnnl.gov)', │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ further_info_url='https://www.pnnl.gov/projects/ceds', │ │ │ │ │ │ │ │ │ │ │ │ institution_id='PNNL-JGCRI', │ │ │ │ │ │ │ │ │ │ │ │ mip_era='CMIP6Plus', │ │ │ │ │ │ │ │ │ │ │ │ source_version='2024-07-08', │ │ │ │ │ │ │ │ │ │ │ │ license_id='CC BY 4.0', │ │ │ │ │ │ │ │ │ │ │ │ authors=None │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ │ SourceIDEntry( │ │ │ │ │ │ │ │ │ │ │ source_id='CEDS-CMIP-2024-10-21', │ │ │ │ │ │ │ │ │ │ │ values=SourceIDValues( │ │ │ │ │ │ │ │ │ │ │ │ contact='Rachel Hoesly │ │ │ │ (rachel.hoesly@pnnl.gov); Steve Smith (ssmith@pnnl.gov)', │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ further_info_url='https://www.pnnl.gov/projects/ceds', │ │ │ │ │ │ │ │ │ │ │ │ institution_id='PNNL-JGCRI', │ │ │ │ │ │ │ │ │ │ │ │ mip_era='CMIP6Plus', │ │ │ │ │ │ │ │ │ │ │ │ source_version='2024-10-21', │ │ │ │ │ │ │ │ │ │ │ │ license_id='CC BY 4.0', │ │ │ │ │ │ │ │ │ │ │ │ authors=None │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ │ SourceIDEntry( │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ source_id='CEDS-CMIP-2024-10-21-supplemental', │ │ │ │ │ │ │ │ │ │ │ values=SourceIDValues( │ │ │ │ │ │ │ │ │ │ │ │ contact='Rachel Hoesly │ │ │ │ (rachel.hoesly@pnnl.gov); Steve Smith (ssmith@pnnl.gov)', │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ further_info_url='https://www.pnnl.gov/projects/ceds', │ │ │ │ │ │ │ │ │ │ │ │ institution_id='PNNL-JGCRI', │ │ │ │ │ │ │ │ │ │ │ │ mip_era='CMIP6Plus', │ │ │ │ │ │ │ │ │ │ │ │ source_version='2024-10-21', │ │ │ │ │ │ │ │ │ │ │ │ license_id='CC BY 4.0', │ │ │ │ │ │ │ │ │ │ │ │ authors=None │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ │ SourceIDEntry( │ │ │ │ │ │ │ │ │ │ │ source_id='CR-CMIP-0-2-0', │ │ │ │ │ │ │ │ │ │ │ values=SourceIDValues( │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ contact='zebedee.nicholls@climate-resource.com;malte.meinshausen… │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ further_info_url='http://www.tbd.invalid', │ │ │ │ │ │ │ │ │ │ │ │ institution_id='CR', │ │ │ │ │ │ │ │ │ │ │ │ mip_era='CMIP6Plus', │ │ │ │ │ │ │ │ │ │ │ │ source_version='0.2.0', │ │ │ │ │ │ │ │ │ │ │ │ license_id='CC BY 4.0', │ │ │ │ │ │ │ │ │ │ │ │ authors=None │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ │ SourceIDEntry( │ │ │ │ │ │ │ │ │ │ │ source_id='CR-CMIP-0-3-0', │ │ │ │ │ │ │ │ │ │ │ values=SourceIDValues( │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ contact='zebedee.nicholls@climate-resource.com;malte.meinshausen… │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ further_info_url='http://www.tbd.invalid', │ │ │ │ │ │ │ │ │ │ │ │ institution_id='CR', │ │ │ │ │ │ │ │ │ │ │ │ mip_era='CMIP6Plus', │ │ │ │ │ │ │ │ │ │ │ │ source_version='0.3.0', │ │ │ │ │ │ │ │ │ │ │ │ license_id='CC BY 4.0', │ │ │ │ │ │ │ │ │ │ │ │ authors=None │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ │ SourceIDEntry( │ │ │ │ │ │ │ │ │ │ │ source_id='DRES-CMIP-BB4CMIP7-1-0', │ │ │ │ │ │ │ │ │ │ │ values=SourceIDValues( │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ contact='Margreet.vanMarle@Deltares.nl;Guido.vanderWerf@wur.nl', │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ further_info_url='http://www.globalfiredata.org', │ │ │ │ │ │ │ │ │ │ │ │ institution_id='DRES', │ │ │ │ │ │ │ │ │ │ │ │ mip_era='CMIP6Plus', │ │ │ │ │ │ │ │ │ │ │ │ source_version='1.0', │ │ │ │ │ │ │ │ │ │ │ │ license_id='CC BY 4.0', │ │ │ │ │ │ │ │ │ │ │ │ authors=None │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ │ SourceIDEntry( │ │ │ │ │ │ │ │ │ │ │ source_id='MRI-JRA55-do-1-6-0', │ │ │ │ │ │ │ │ │ │ │ values=SourceIDValues( │ │ │ │ │ │ │ │ │ │ │ │ contact='Hiroyuki Tsujino │ │ │ │ (htsujino@mri-jma.go.jp)', │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ further_info_url='https://climate.mri-jma.go.jp/pub/ocean/JRA55-… │ │ │ │ │ │ │ │ │ │ │ │ institution_id='MRI', │ │ │ │ │ │ │ │ │ │ │ │ mip_era='CMIP6Plus', │ │ │ │ │ │ │ │ │ │ │ │ source_version='1.6.0', │ │ │ │ │ │ │ │ │ │ │ │ license_id=None, │ │ │ │ │ │ │ │ │ │ │ │ authors=None │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ │ SourceIDEntry( │ │ │ │ │ │ │ │ │ │ │ source_id='PCMDI-AMIP-1-1-9', │ │ │ │ │ │ │ │ │ │ │ values=SourceIDValues( │ │ │ │ │ │ │ │ │ │ │ │ contact='PCMDI │ │ │ │ (pcmdi-cmip@llnl.gov)', │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ further_info_url='https://pcmdi.llnl.gov/mips/amip', │ │ │ │ │ │ │ │ │ │ │ │ institution_id='PCMDI', │ │ │ │ │ │ │ │ │ │ │ │ mip_era='CMIP6Plus', │ │ │ │ │ │ │ │ │ │ │ │ source_version='1.1.9', │ │ │ │ │ │ │ │ │ │ │ │ license_id='CC BY 4.0', │ │ │ │ │ │ │ │ │ │ │ │ authors=None │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ │ SourceIDEntry( │ │ │ │ │ │ │ │ │ │ │ source_id='PCMDI-AMIP-ERSST5-1-0', │ │ │ │ │ │ │ │ │ │ │ values=SourceIDValues( │ │ │ │ │ │ │ │ │ │ │ │ contact='zelinka1@llnl.gov; │ │ │ │ durack1@llnl.gov', │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ further_info_url='https://pcmdi.llnl.gov/mips/amip', │ │ │ │ │ │ │ │ │ │ │ │ institution_id='PCMDI', │ │ │ │ │ │ │ │ │ │ │ │ mip_era='CMIP6Plus', │ │ │ │ │ │ │ │ │ │ │ │ source_version='1.0', │ │ │ │ │ │ │ │ │ │ │ │ license_id='CC BY 4.0', │ │ │ │ │ │ │ │ │ │ │ │ authors=None │ │ │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ │ │ │ ... +10 │ │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ │ ) │ │ │ │ │ │ │ ), │ │ │ │ │ │ │ exception=None, │ │ │ │ │ │ │ exception_info=None │ │ │ │ │ │ ), │ │ │ │ │ │ ValidationResult( │ │ │ │ │ │ │ description='Open data with `xr.open_dataset`', │ │ │ │ │ │ │ passed=True, │ │ │ │ │ │ │ result=<xarray.Dataset> Size: 156kB │ │ │ │ Dimensions: (time: 132, lat: 12, lon: 12, bnds: 2) │ │ │ │ Coordinates: │ │ │ │ * time (time) object 1kB 2000-01-01 00:00:00 ... │ │ │ │ 2010-12-01 00:00:00 │ │ │ │ * lat (lat) float64 96B -82.5 -67.5 -52.5 -37.5 ... 37.5 │ │ │ │ 52.5 67.5 82.5 │ │ │ │ * lon (lon) float64 96B -165.0 -135.0 -105.0 ... 105.0 │ │ │ │ 135.0 165.0 │ │ │ │ Dimensions without coordinates: bnds │ │ │ │ Data variables: │ │ │ │ │ CH4 (time, lat, lon) float64 152kB ... │ │ │ │ │ time_bnds (time, bnds) object 2kB ... │ │ │ │ │ lat_bnds (lat, bnds) float64 192B ... │ │ │ │ │ lon_bnds (lon, bnds) float64 192B ... │ │ │ │ Attributes: (12/24) │ │ │ │ │ Conventions: CF-1.7 │ │ │ │ │ activity_id: input4MIPs │ │ │ │ │ comment: Demo │ │ │ │ │ contact: zebedee.nicholls@climate-resource.com │ │ │ │ │ creation_date: 2024-07-25T18:23:47Z │ │ │ │ │ data_structure: grid │ │ │ │ │ ... ... │ │ │ │ │ source: Climate Resource demo │ │ │ │ │ source_id: CR-CMIP-0-2-0 │ │ │ │ │ source_version: 0.2.0 │ │ │ │ │ target_mip: CMIP │ │ │ │ │ title: Title here │ │ │ │ │ variable_id: CH4, │ │ │ │ │ │ │ exception=None, │ │ │ │ │ │ │ exception_info=None │ │ │ │ │ │ ), │ │ │ │ │ │ ValidationResult( │ │ │ │ │ │ │ description='Load data with `iris.load`', │ │ │ │ │ │ │ passed=True, │ │ │ │ │ │ │ result=[<iris 'Cube' of biomass_burning_CH4_flux / │ │ │ │ (kg m-2 s-1) (time: 132; latitude: 12; longitude: 12)>], │ │ │ │ │ │ │ exception=None, │ │ │ │ │ │ │ exception_info=None │ │ │ │ │ │ ), │ │ │ │ │ │ ValidationResult( │ │ │ │ │ │ │ description='Load data with `iris.load_cube`', │ │ │ │ │ │ │ passed=True, │ │ │ │ │ │ │ result=<iris 'Cube' of biomass_burning_CH4_flux / (kg │ │ │ │ m-2 s-1) (time: 132; latitude: 12; longitude: 12)>, │ │ │ │ │ │ │ exception=None, │ │ │ │ │ │ │ exception_info=None │ │ │ │ │ │ ), │ │ │ │ │ │ ValidationResult( │ │ │ │ │ │ │ description='Check data with cf-checker', │ │ │ │ │ │ │ passed=False, │ │ │ │ │ │ │ result=None, │ │ │ │ │ │ │ exception=ValueError('cf-checker validation failed. │ │ │ │ cfchecks output:\n\nCHECKING NetCDF FILE: │ │ │ │ docs/how-to-guides/CH4-em-biomassburning_input4MIPs_emissions_CM… │ │ │ │ CF Checker Version 4.1.0\nChecking against CF Version │ │ │ │ CF-1.7\nUsing Standard Name Table Version 92 │ │ │ │ (2025-07-24T14:20:46Z)\nUsing Area Type Table Version 13 (20 │ │ │ │ March 2025)\nUsing Standardized Region Name Table Version 5 (12 │ │ │ │ November 2024)\n\nERROR: (2.6.3): external_variables attribute │ │ │ │ must be a blank separated list of variable │ │ │ │ names\n\n------------------\nChecking variable: │ │ │ │ CH4\n------------------\nERROR: (3.3): Invalid standard_name: │ │ │ │ biomass_burning_CH4_flux\nERROR: (7.2): cell_measures variable │ │ │ │ gridcellarea must either exist in this netCDF file or be named by │ │ │ │ the external_variables attribute\n\n------------------\nChecking │ │ │ │ variable: │ │ │ │ time\n------------------\n\n------------------\nChecking │ │ │ │ variable: │ │ │ │ time_bnds\n------------------\n\n------------------\nChecking │ │ │ │ variable: lat\n------------------\n\n------------------\nChecking │ │ │ │ variable: │ │ │ │ lat_bnds\n------------------\n\n------------------\nChecking │ │ │ │ variable: lon\n------------------\n\n------------------\nChecking │ │ │ │ variable: lon_bnds\n------------------\n\nERRORS detected: │ │ │ │ 3\nWARNINGS given: 0\nINFORMATION messages: 0\n'), │ │ │ │ │ │ │ exception_info='Traceback (most recent call last):\n │ │ │ │ File "/home/docs/checkouts/readthedocs.org/'+2733 │ │ │ │ │ │ ), │ │ │ │ │ │ ValidationResult( │ │ │ │ │ │ │ description="Validate the 'creation_date' attribute", │ │ │ │ │ │ │ passed=True, │ │ │ │ │ │ │ result=None, │ │ │ │ │ │ │ exception=None, │ │ │ │ │ │ │ exception_info=None │ │ │ │ │ │ ), │ │ │ │ │ │ ValidationResult( │ │ │ │ │ │ │ description="Validate the 'tracking_id' attribute", │ │ │ │ │ │ │ passed=False, │ │ │ │ │ │ │ result=None, │ │ │ │ │ │ │ exception=MissingAttributeError('tracking_id'), │ │ │ │ │ │ │ exception_info='Traceback (most recent call last):\n │ │ │ │ File "/home/docs/checkouts/readthedocs.org/'+536 │ │ │ │ │ │ ), │ │ │ │ │ │ ValidationResult( │ │ │ │ │ │ │ description="Validate the 'frequency' attribute", │ │ │ │ │ │ │ passed=True, │ │ │ │ │ │ │ result=None, │ │ │ │ │ │ │ exception=None, │ │ │ │ │ │ │ exception_info=None │ │ │ │ │ │ ), │ │ │ │ │ │ ValidationResult( │ │ │ │ │ │ │ description="Validate the 'variable_id' attribute", │ │ │ │ │ │ │ passed=True, │ │ │ │ │ │ │ result=None, │ │ │ │ │ │ │ exception=None, │ │ │ │ │ │ │ exception_info=None │ │ │ │ │ │ ), │ │ │ │ │ │ ValidationResult( │ │ │ │ │ │ │ description="Validate the 'activity_id' attribute", │ │ │ │ │ │ │ passed=True, │ │ │ │ │ │ │ result=None, │ │ │ │ │ │ │ exception=None, │ │ │ │ │ │ │ exception_info=None │ │ │ │ │ │ ), │ │ │ │ │ │ ... +4 │ │ │ │ │ ] │ │ │ │ ) │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ ╰──────────────────────────────────────────────────────────────────────────────╯ ValidationResultsStoreError: Checks passing: 71.43% (10 / 14) Checks failing: 28.57% (4 / 14) Failing checks details Check data with cf-checker (ValueError) Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/cf_checker.py" , line 125, in check_with_cf_checker subprocess.check_output( File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/subprocess.py", line 424, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/subprocess.py", line 528, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/ latest/bin/cfchecks', '-v', '1.7', 'docs/how-to-guides/CH4-em-biomassburning_input4MIPs_emissions_CMIP_CR-CMIP-0-2- 0_gn_200001-201012.nc']' returned non-zero exit status 3. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/error_catching .py", line 202, in decorated res_func = func_to_call(*args, **kwargs) File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/cf_checker.py" , line 141, in check_with_cf_checker raise ValueError(error_msg) from exc ValueError: cf-checker validation failed. cfchecks output: CHECKING NetCDF FILE: docs/how-to-guides/CH4-em-biomassburning_input4MIPs_emissions_CMIP_CR-CMIP-0-2-0 _gn_200001-201012.nc ===================== Using CF Checker Version 4.1.0 Checking against CF Version CF-1.7 Using Standard Name Table Version 92 (2025-07-24T14:20:46Z) Using Area Type Table Version 13 (20 March 2025) Using Standardized Region Name Table Version 5 (12 November 2024) ERROR: (2.6.3): external_variables attribute must be a blank separated list of variable names ------------------ Checking variable: CH4 ------------------ ERROR: (3.3): Invalid standard_name: biomass_burning_CH4_flux ERROR: (7.2): cell_measures variable gridcellarea must either exist in this netCDF file or be named by the external_variables attribute ------------------ Checking variable: time ------------------ ------------------ Checking variable: time_bnds ------------------ ------------------ Checking variable: lat ------------------ ------------------ Checking variable: lat_bnds ------------------ ------------------ Checking variable: lon ------------------ ------------------ Checking variable: lon_bnds ------------------ ERRORS detected: 3 WARNINGS given: 0 INFORMATION messages: 0 Validate the 'tracking_id' attribute (MissingAttributeError) Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/error_catching .py", line 202, in decorated res_func = func_to_call(*args, **kwargs) File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/datasets_to_wr ite_to_disk.py", line 104, in validate_attribute raise MissingAttributeError(attribute) input4mips_validation.validation.exceptions.MissingAttributeError: 'tracking_id' Validate the 'external_variables' attribute (ValueError) Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/external_varia bles.py", line 28, in validate_external_variables check_only_allowed_characters( File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/allowed_charac ters.py", line 74, in check_only_allowed_characters raise ValueError(msg) ValueError: Only alphanumeric characters, underscores and whitespaces are allowed. value='gridcellarea,sources'. invalid_chars={','}. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/error_catching .py", line 202, in decorated res_func = func_to_call(*args, **kwargs) File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/datasets_to_wr ite_to_disk.py", line 107, in validate_attribute validation_function(attribute_value) File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/external_varia bles.py", line 39, in validate_external_variables raise ValueError(msg) from exc ValueError: The `external_variables` attribute contains invalid characters. If you are providing multiple variables, they should be whitespace-separated. Only alphanumeric characters, underscores and whitespaces are allowed. value='gridcellarea,sources'. invalid_chars={','}. Validate the 'contact' attribute (ValueInconsistentWithCVsError) Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/error_catching .py", line 202, in decorated res_func = func_to_call(*args, **kwargs) File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/validation/datasets_to_wr ite_to_disk.py", line 150, in validate_attribute_that_depends_on_other_attribute validation_function(attribute_value, attribute_dependent_on_value) File "/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/la test/lib/python3.9/site-packages/input4mips_validation/cvs/cvs.py", line 130, in validate_contact raise ValueInconsistentWithCVsError( input4mips_validation.cvs.exceptions.ValueInconsistentWithCVsError: The value provided for contact was 'zebedee.nicholls@climate-resource.com'. According to the CVs, contact depends on the value of source_id. As a result, contact must be 'zebedee.nicholls@climate-resource.com;malte.meinshausen@climate-resource.com'. If helpful, the full CV entry for source_id is: SourceIDEntry(source_id='CR-CMIP-0-2-0', values=SourceIDValues(contact='zebedee.nicholls@climate-resource.com;malte.meins hausen@climate-resource.com', further_info_url='http://www.tbd.invalid', institution_id='CR', mip_era='CMIP6Plus', source_version='0.2.0', license_id='CC BY 4.0', authors=None)).
Understanding the issue¶
From the above we can see four issues:
- the "external_variables" attribute is formatted incorrectly
- the "standard_name" assigned to the variable doesn't exist
- the "cell_measures" variable appears to be missing
- the value of the "contact" attribute is wrong
- the "tracking_id" attribute is missing
We will go through what these mean in the next paragraph. However, in general, there can be quite some mystery surrounding these errors. If something doesn't make sense, please make an issue and tag "@znichollscr" and "@durack1" so we can help you. Please don't spend lots of time banging your head against a wall.
In this particular case, the errors are the following:
- the supplied value of "external_variables" is "gridcellarea,sources". This is meant to be a set of whitespace separated values, i.e. "gridcellarea sources". The CF-checker error code, 2.6.3, can help us find the right part of the CF conventions docs, although you would be forgiven for not finding them completely obvious. This is an easy fix, which we perform below
- the standard_name assigned to the variable does not exist in the list of CF standard names. We're not sure where this is documented, but the fix here is to simply use "long_name" instead of "standard_name" for any variables which aren't in the CF standard name table.
- because "external_variables" isn't formatted correctly, the CF-checker gets confused and things that cell_measures refers to variables which aren't properly documented. As we will see, fixing the issue with "external_variables" will also fix this issue.
- all datasets need to have a "tracking_id" attribute.
This has a particular form,
which can be generated with
input4mips_validation.io.generate_tracking_id
Fixing the file¶
In this case, the fixes are relatively trivial. You could do them with any tool you like. Here, we do them with a combination of xarray, ncdata and iris. We use this combination because we find xarray easiest to work with, but only iris writes the files correctly, and ncdata is the best to translate between the two (yes, you can imagine how fun it was figuring all of this out).
In [9]:
Copied!
fixed = xr.open_dataset(starting_file)
# Fix the contact information
fixed.attrs[
"contact"
] = "zebedee.nicholls@climate-resource.com;malte.meinshausen@climate-resource.com"
# Fix the whitespace issue in external_variables
fixed.attrs["external_variables"] = fixed.attrs["external_variables"].replace(",", " ")
# Convert long_name to standard_name
# (iris would actually do this for us, but for completeness...)
fixed["CH4"].attrs["long_name"] = fixed["CH4"].attrs.pop("standard_name")
# Add the tracking_id
fixed.attrs["tracking_id"] = input4mips_validation.io.generate_tracking_id()
# The eagle eyed will notice that this file name is definitely not correct.
# We will soon show you why this doesn't matter in this particular case.
TMP_DIR = Path(tempfile.mkdtemp())
fixed_file = (
TMP_DIR
/ "fixed_CH4-em-biomassburning_input4MIPs_emissions_CMIP_CR-CMIP-0-2-0_gn_200001-201012.nc" # noqa: E501
)
cubes = ncdata.iris_xarray.cubes_from_xarray(fixed)
iris.save(cubes, fixed_file)
# Check the updated attributes, could also be done with e.g. ncdump
print(f"New external_variables: {fixed.attrs['external_variables']!r}")
print(f"New variable attributes: {fixed['CH4'].attrs!r}")
print(f"New tracking_id: {fixed.attrs['tracking_id']!r}")
fixed = xr.open_dataset(starting_file)
# Fix the contact information
fixed.attrs[
"contact"
] = "zebedee.nicholls@climate-resource.com;malte.meinshausen@climate-resource.com"
# Fix the whitespace issue in external_variables
fixed.attrs["external_variables"] = fixed.attrs["external_variables"].replace(",", " ")
# Convert long_name to standard_name
# (iris would actually do this for us, but for completeness...)
fixed["CH4"].attrs["long_name"] = fixed["CH4"].attrs.pop("standard_name")
# Add the tracking_id
fixed.attrs["tracking_id"] = input4mips_validation.io.generate_tracking_id()
# The eagle eyed will notice that this file name is definitely not correct.
# We will soon show you why this doesn't matter in this particular case.
TMP_DIR = Path(tempfile.mkdtemp())
fixed_file = (
TMP_DIR
/ "fixed_CH4-em-biomassburning_input4MIPs_emissions_CMIP_CR-CMIP-0-2-0_gn_200001-201012.nc" # noqa: E501
)
cubes = ncdata.iris_xarray.cubes_from_xarray(fixed)
iris.save(cubes, fixed_file)
# Check the updated attributes, could also be done with e.g. ncdump
print(f"New external_variables: {fixed.attrs['external_variables']!r}")
print(f"New variable attributes: {fixed['CH4'].attrs!r}")
print(f"New tracking_id: {fixed.attrs['tracking_id']!r}")
New external_variables: 'gridcellarea sources'
New variable attributes: {'units': 'kg m-2 s-1', 'cell_methods': 'time: mean', 'cell_measures': 'area: gridcellarea', 'long_name': 'biomass_burning_CH4_flux'}
New tracking_id: 'hdl:21.14100/44ff2e59-9ade-4661-b7ae-b434ee365293'
/home/docs/checkouts/readthedocs.org/user_builds/input4mips-validation/conda/latest/lib/python3.9/site-packages/iris/fileformats/cf.py:877: IrisCfMissingVarWarning: Missing CF-netCDF measure variable 'gridcellarea', referenced by netCDF variable 'CH4' warnings.warn(
Validate again¶
To make sure we have actually fixed the issues, let's run the validation again.
In [10]:
Copied!
!input4mips-validation validate-file --cv-source "gh:v6.6.0" {fixed_file}
!input4mips-validation validate-file --cv-source "gh:v6.6.0" {fixed_file}
4641 - 129591773886272 - 2025-10-01T14:41:59.091309+0000 - INFO_FILE - input4mips_validation.validation.file:file.py:105 - Creating validation results for file: /tmp/tmpvysaa35u/fixed_CH4-em-biomassburning_input4MIPs_emissions_CMIP_CR-CMIP-0-2-0_gn_200001-201012.nc
4641 - 129591773886272 - 2025-10-01T14:41:59.102340+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Load controlled vocabularies to use during validation ran without error
4641 - 129591773886272 - 2025-10-01T14:41:59.163281+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Open data with `xr.open_dataset` ran without error 4641 - 129591773886272 - 2025-10-01T14:41:59.175470+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Load data with `iris.load` ran without error 4641 - 129591773886272 - 2025-10-01T14:41:59.182449+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Load data with `iris.load_cube` ran without error 4641 - 129591773886272 - 2025-10-01T14:41:59.182585+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.file:file.py:150 - Using the cf-checker to check /tmp/tmpvysaa35u/fixed_CH4-em-biomassburning_input4MIPs_emissions_CMIP_CR-CMIP-0-2-0_gn_200001-201012.nc
4641 - 129591773886272 - 2025-10-01T14:41:59.507891+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Check data with cf-checker ran without error 4641 - 129591773886272 - 2025-10-01T14:41:59.544873+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'creation_date' attribute ran without error 4641 - 129591773886272 - 2025-10-01T14:41:59.545229+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'tracking_id' attribute ran without error 4641 - 129591773886272 - 2025-10-01T14:41:59.550090+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'frequency' attribute ran without error 4641 - 129591773886272 - 2025-10-01T14:41:59.550218+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'variable_id' attribute ran without error 4641 - 129591773886272 - 2025-10-01T14:41:59.550289+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'activity_id' attribute ran without error 4641 - 129591773886272 - 2025-10-01T14:41:59.550377+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'comment' attribute ran without error 4641 - 129591773886272 - 2025-10-01T14:41:59.550431+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'external_variables' attribute ran without error 4641 - 129591773886272 - 2025-10-01T14:41:59.550500+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'contact' attribute ran without error 4641 - 129591773886272 - 2025-10-01T14:41:59.550587+0000 - INFO_INDIVIDUAL_CHECK - input4mips_validation.validation.error_catching:error_catching.py:208 - Validate the 'source_version' attribute ran without error 4641 - 129591773886272 - 2025-10-01T14:41:59.550633+0000 - INFO_FILE - input4mips_validation.validation.file:file.py:190 - Created validation results for file: /tmp/tmpvysaa35u/fixed_CH4-em-biomassburning_input4MIPs_emissions_CMIP_CR-CMIP-0-2-0_gn_200001-201012.nc 4641 - 129591773886272 - 2025-10-01T14:41:59.551518+0000 - SUCCESS - input4mips_validation.cli:__init__.py:196 - File passed validation: /tmp/tmpvysaa35u/fixed_CH4-em-biomassburning_input4MIPs_emissions_CMIP_CR-CMIP-0-2-0_gn_200001-201012.nc
Now the file passes all of the validation.
Next steps¶
This procedure can obviously be repeated over a number of files with loops etc. We currently don't have a tool that repeats this procedure over numerous files, but are happy to receive requests for one in our issues. Having said that, if you've got this far, we assume you can write a loop in Python or bash :)
Once you have written your files, you have two options for what to do next:
- If you want to really understand how your data is handled, you can write it according to the input4MIPs data reference syntax (DRS). If you want to go this path, please see "How to write a file in the DRS".
- If you just want to get your data in the publishing queue, you can upload it as is to LLNL's FTP server. If you want to go this path, please see "How to upload to an FTP server".