eradiate.test_tools.regression.Chi2Test

class eradiate.test_tools.regression.Chi2Test(name, value, reference=None, *, variable='brf_srf', threshold, archive_dir, plot)[source]

Bases: RegressionTest

This class implements a statistical test for the regression testing campaign, based on Pearson’s Chi-squared test. https://en.wikipedia.org/wiki/Pearson%27s_chi-squared_test

It determines the probability for the reference and the test result following the same distribution.

This test will pass if the computed p-value is strictly larger than the given threshold.

Parameters:

  • name (str) – Test case name.

  • value (xarray.Dataset) – Simulation result. Must be specified as a dataset.

  • reference (xarray.Dataset or path-like, optional, default: None) – Reference data. Can be specified as an xarray dataset, a path to a NetCDF file or a path to a resource.

  • variable (str, default: brf_srf) – Tested variable

  • threshold (float) – Test metric threshold

  • archive_dir (path-like) – Path to output artefact storage directory. Relative paths are interpreted with respect to the current working directory.

  • plot (bool) – Enable pyplot charts

Fields:

  • name (str) – Test case name.

  • value (xarray.Dataset) – Simulation result.

  • reference (xarray.Dataset or None) – Reference data.

  • variable (str) – Tested variable.

  • threshold (float) – Test metric threshold.

  • archive_dir (pathlib.Path) – Path to output artefact storage directory.

  • plot (bool) – Enable pyplot charts.

run(diagnostic=False)

This method controls the execution steps of the regression test:

  • handle missing reference data

  • catch errors during text evaluation

  • create the appropriate plots and data archives

Returns:

bool – Result of the test criterion comparison.