eradiate.test_tools.regression.ZTest

class eradiate.test_tools.regression.ZTest(name, value, reference=None, *, variable='brf_srf', threshold, archive_dir, plot)[source]

Bases: RegressionTest

Z-Test with Šidák correction factor

Implement a Z-test, testing the significance of paired differences between a set of observations and a set of references. It considers the observations variance.

Paired tests are aggregated into one p-value using a Šidák correction. The test passes if the null hypothesis is accepted for at least 99.75% of the paired Z-tests

This paired Z-test requires an equal degree of freedom of the two groups.

Parameters:

  • name (str) – Test case name.

  • value (xarray.Dataset) – Simulation result. Must be specified as a dataset.

  • reference (xarray.Dataset or path-like, optional, default: None) – Reference data. Can be specified as an xarray dataset, a path to a NetCDF file or a path to a resource.

  • variable (str, default: brf_srf) – Tested variable

  • threshold (float) – Test metric threshold

  • archive_dir (path-like) – Path to output artefact storage directory. Relative paths are interpreted with respect to the current working directory.

  • plot (bool) – Enable pyplot charts

Fields:

  • name (str) – Test case name.

  • value (xarray.Dataset) – Simulation result.

  • reference (xarray.Dataset or None) – Reference data.

  • variable (str) – Tested variable.

  • threshold (float) – Test metric threshold.

  • archive_dir (pathlib.Path) – Path to output artefact storage directory.

  • plot (bool) – Enable pyplot charts.

run(diagnostic=False)

This method controls the execution steps of the regression test:

  • handle missing reference data

  • catch errors during text evaluation

  • create the appropriate plots and data archives

Returns:

bool – Result of the test criterion comparison.