eradiate.test_tools.regression.Chi2Test

eradiate.test_tools.regression.Chi2Test#

class eradiate.test_tools.regression.Chi2Test(name, value, reference=None, *, threshold, archive_dir)[source]#

Bases: RegressionTest

This class implements a statistical test for the regression testing campaign, based on Pearson’s Chi-squared test. https://en.wikipedia.org/wiki/Pearson%27s_chi-squared_test

It determines the probability for the reference and the test result following the same distribution.

This test will pass if the computed p-value is strictly larger than the given threshold.

Parameters:

  • name (str) – Test case name.

  • value (xarray.Dataset) – Simulation result. Must be specified as a dataset.

  • reference (xarray.Dataset or path-like, optional, default: None) – Reference data. Can be specified as an xarray dataset, a path to a NetCDF file or a path to a resource served by the data store.

  • threshold (float) – Threshold for test evaluation

  • archive_dir (path-like) – Path to output artefact storage directory. Relative paths are interpreted with respect to the current working directory.

Fields:

  • name (str) – Test case name.

  • value (xarray.Dataset) – Simulation result.

  • reference (xarray.Dataset or None) – Reference data.

  • threshold (float) – Threshold for test evaluation.

  • archive_dir (pathlib.Path) – Path to output artefact storage directory.

run()#

This method controls the execution steps of the regression test:

  • handle missing reference data

  • catch errors during text evaluation

  • create the appropriate plots and data archives

Returns:

bool – Result of the test criterion comparison.

Contents