eradiate.radprops.AbsorptionDatabase

eradiate.radprops.AbsorptionDatabase#

class eradiate.radprops.AbsorptionDatabase(dir_path, index, spectral_coverage, metadata=NOTHING, lazy=False, cache=NOTHING)[source]#

Bases: object

Common parent type for absorption coefficient databases.

This class implements most of the data indexing logic common to all absorption coefficient databases. A database is composed of a set of NetCDF files compliant with the absorption coefficient database format specification and placed in the same directory. A database instance is initialized by specifying the path to the directory where the files are stored.

If it exists, a metadata.json file is loaded into the metadata attribute.

Databases are usually not initialized using the constructor, but rather using the class method constructors from_directory(), from_name() and from_dict().

Parameters:

  • dir_path (path-like) – Path to database root directory.

  • index (DataFrame) – File index, assumed sorted by ascending wavelengths.

  • spectral_coverage (DataFrame) – Dataframe that unrolls the spectral information contained in all data files in the database.

  • metadata (dict) – Dictionary that contains the database metadata.

  • lazy (bool) – Access mode switch: if True, load data lazily; else, load data eagerly.

  • cache (cachetools.LRUCache) – A mapping that implements an LRU caching policy.

Fields:

lazy (bool) – Access mode switch: if True, load data lazily; else, load data eagerly.

Notes

A file index, stored as the _index private attribute, associates to each file the spectral region it covers. The index is preferably loaded from a CSV file that contains all this information; if it is not found, the table is built upon database initialization and saved to the database directory. The indexing step requires to access all files and may take a while. The file index table is used during queries to select efficiently the file where data will be read. For convenience, information about bounds contained in the index is assembled into a spectral mesh suitable for query using numpy.digitize() and stored in the _chunks dictionary.

A spectral coverage table, stored as the _spectral_coverage private attribute, merges the spectral coordinates of all files into a consistent index. This table is used to provide spectral coverage information to higher-level components that drive the simulation. Table contents are preferably loaded from a CSV file; if it is not found, the table is build upon database initialization and saved to the database directory. This indexing step also requires to access all files and may take a while.

Database access and memory usage can be controlled through two parameters:

  • File queries are stored in an LRU cache. The initial size is set to a low value (8) and should be appropriate for most situations. If more cache control is needed, the cache_clear(), cache_close() and cache_reset() methods can be used.

  • Datasets can be open with an eager or lazy approach. This behaviour is controlled using the lazy constructor parameter. In eager mode, the entire file used for a query is loaded into memory. This can bring significant access overhead when using large files. If desired, datasets can instead be open lazily, triggering disk access only for the specific data that are used.

cache_clear()[source]#

Clear the cache.

cache_close()[source]#

Close all cached datasets.

cache_reset(maxsize)[source]#

Reset the cache with the specified maximum size.

static convert(value)[source]#

Attempt conversion of a value to an absorption database.

Parameters:

value – The value for which conversion is attempted.

Returns:

MonoAbsorptionDatabase or CKDAbsorptionDatabase

Notes

Conversion rules are as follows:

  • If value is a string, try converting using the from_name() constructor. Do not raise if this fails.

  • If value is a string or a path, try converting using the from_directory() constructor. The returned type is consistent with the active mode.

  • If value is a dict, try converting using the from_dict() constructor. The returned type is consistent with the active mode.

  • Otherwise, do not convert.

static default()[source]#

Return a default database, depending on the active mode.

Defaults are as follows:

  • Monochromatic: "komodo"

  • CKD: "monotropa"

Returns:

AbsorptionDatabase

eval_sigma_a_ckd(w, g, thermoprops, error_handling_config=None)[source]#

Compute the absorption coefficient given spectral coordinates and a thermophysical profile (CKD variant).

Parameters:

  • w (quantity) – The wavelength for which the absorption coefficient is evaluated.

  • g (float) – The g-point for which the absorption coefficient is evaluated.

  • thermoprops (Dataset) – The thermophysical profile for which the absorption coefficient is evaluated.

  • error_handling_config (ErrorHandlingConfiguration, optional) – The error handling policy applied if corrdinates are missing, do not have the appropriate dimension or are out of the dataset’s bounds. If unset, the default policy specified by the absorption_dataset.error_handling setting is applied.

Returns:

DataArray – A data array containing the evaluated absorption coefficient as a function of the spectral coordinate and altitude.

See also

eradiate.config

eval_sigma_a_mono(w, thermoprops, error_handling_config=None)[source]#

Compute the absorption coefficient given spectral coordinates and a thermophysical profile (mono variant).

Parameters:

  • w (quantity) – The wavelength for which the absorption coefficient is evaluated.

  • thermoprops (Dataset) – The thermophysical profile for which the absorption coefficient is evaluated.

  • error_handling_config (ErrorHandlingConfiguration, optional) – The error handling policy applied if corrdinates are missing, do not have the appropriate dimension or are out of the dataset’s bounds. If unset, the default policy specified by the absorption_dataset.error_handling setting is applied.

Returns:

DataArray – A data array containing the evaluated absorption coefficient as a function of the spectral coordinate and altitude.

See also

eradiate.config

classmethod from_dict(value)[source]#

Construct from a dictionary. The dictionary has a required entry "construct" that specifies the constructor that will be used to instantiate the database. Additional entries are keyword arguments passed to the selected constructor.

Parameters:

value (dict) – Converted value.

Returns:

AbsorptionDatabase

classmethod from_directory(dir_path, lazy=False, fix=True)[source]#

Initialize a CKD database from a directory that contains one or several datasets.

Parameters:

  • dir_path (path-like) – Path where the CKD database is located.

  • lazy (bool) – Access mode switch: if True, load data lazily; else, load data eagerly.

  • fix (bool) – If True, attempt generating missing index files upon initialization. Otherwise, raise if they are missing.

Returns:

AbsorptionDatabase

Raises:

FileNotFoundError – If an index file is missing and fix is False.

static from_name(name, **kwargs)[source]#

Initialize a database from a name.

Parameters:

  • name (path-like) – Name of the requested CKD database.

  • kwargs – Additional keyword arguments are forwarded to the from_directory() constructor.

Returns:

AbsorptionDatabase

load_dataset(fname)[source]#

Convenience method to load a dataset. This method is decorated with functools.lru_cache() with maxsize=1, which limits the number of reload events when repeatedly querying the same file.

The behaviour of this method is also affected by the lazy parameter: if lazy is False, files are loaded eagerly with xarray.load_dataset(); if lazy is True, files are loaded lazily with xarray.open_dataset().

Parameters:

fname (str) – Name of the file that is to be loaded.

Returns:

Dataset

lookup_datasets(**kwargs)[source]#

Perform a dataset lookup based on the requested spectral coordinate. See lookup_filenames() for the accepted arguments.

lookup_filenames(**kwargs)[source]#

Look up a filename in the index table from the coordinate values passed as keyword arguments.

Parameters:

  • wl (quantity or array-like, optional) – Wavelength (scalar or array, quantity or unitless). If passed as a unitless value, it is interpreted using the units of the wavelength chunk bounds.

  • wn (quantity or array-like, optional) – Wavenumber (scalar or array, quantity or unitless). If passed as a unitless value, it is interpreted using the units of the wavenumber chunk bounds.

Returns:

filenames (list of str) – Names of the successfully looked up files, relative to the database root directory.

Raises:

ValueError – If the requested spectral coordinate is out of bounds.

Notes

Depending on the specified keyword argument (wl or wn), the lookup will be performed in wavelength or wavenumber mode. Both are equivalent.

property dir_path#
Returns:

Path – Database root path.

property metadata#
Returns:

dict – Database metadata.

property spectral_coverage#
Returns:

DataFrame – Spectral coverage table.