eradiate.pipelines

`eradiate.pipelines`#

This module contains definitions for all post-processing pipeline operations. The implementation complies with the Hamilton dataflow framework’s specifications: the post-processing pipeline is modelled as a directed acyclic graph (DAG), and all post-processing operations are a node in that DAG.

Eradiate uses advanced Hamilton features to mutate the pipeline based on configuration input. For example:

Node names reflect the processed physical quantity.
Viewing angles are only computed if the measure is distant.
Post-processing operations vary depending on the active mode. For example, SRF weighted averaging is only performed in CKD modes.

For clarity, the implementation is split as follows:

eradiate.pipelines.logic provides the fundamental logic powering each step of post-processing;
eradiate.pipelines.definitions contains the pipeline definition and is intended to be consumed by the Hamilton driver constructor for pipeline initialization;
eradiate.pipelines.core provides convenience entry points to facilitate post-processing pipeline initialization.

Pipeline configuration#

Post-processing pipelines are configured using a dictionary specifying parameters listed in the table below. This dictionary is used as the input of the eradiate.pipelines.driver() function. It should be noted that eradiate.pipelines.config() can be used to generate a configuration dictionary easily.

Pipeline configuration variables#
Parameter	Type	Description
`add_viewing_angles`	`bool`	Whether the measure provides viewing angles that can be mapped to film pixels.
`apply_spectral_response`	`bool`	Whether we should apply SRF weighting (a.k.a convolution) to spectral variables.
`measure_distant`	`bool`	Whether we are processing the results of a distant measure.
`mode_id`	`str`	The ID of the Eradiate mode for which the pipeline is configured.
`var_name`, `var_metadata`	`str`	The name and metadata for the physical variable that is being processed.

Many nodes of the DAG defining the pipeline are tagged for filtering upon query. Tags explicitly used in the pipeline setup and execution are listed in the table below. The eradiate.pipelines.outputs() function leverages these tags to generate a list of default output nodes when running the post-processing pipeline.

Pipeline node tags#
Tag	Value	Description
`"final"`	`"true"`	The node’s output will be selected to be part of the default pipeline output.
`"kind"`		Defines the type of variable the node’s output corresponds to. Data with the `"data"` tag will be selected to be part of the default result dataset.
	`"data"`	The output is a data variable and can be assembled into a dataset.
	`"coord"`	The output is a coordinate variable.

Hamilton driver creation and manipulation#

Note

The following entry points are defined in the eradiate.pipelines.core module.

`config`(measure[, mode, integrator])	Generate a pipeline configuration for a specific scene setup.
`driver`(config)	Create a Hamilton `Driver` instance using the specified configuration.
`list_variables`(drv[, as_table])	List variables available in a pipeline.
`outputs`(drv)	Generate a list of default outputs for a given pipeline.

Pipeline logic (`eradiate.pipelines.logic`)#

Functions:

`aggregate_ckd_quad`(mode_id, raw_data, ...)	Compute CKD quadrature.
`apply_spectral_response`(spectral_data, srf)	Apply SRF weighting (a.k.a.
`compute_albedo`(radiosity_data, irradiance_data)	Compute the bi-hemispherical reflectance (a.k.a surface albedo) from radiosity and irradiance records.
`compute_bidirectional_reflectance`(...)	Compute the spectral bidirectional reflectance distribution function (BRDF) and bidirectional reflectance factor (BRF) from radiance and irradiance records.
`degree_of_linear_polarization`(stokes_vec)	Calculate the degree of linear polarization from a Stokes vector.
`extract_irradiance`(mode_id, illumination, ...)	Derive an irradiance dataset from the irradiance spectrum of an illuminant, as well as solar angles.
`gather_bitmaps`(mode_id, var_name, ...)	Gather a collection of Mitsuba bitmaps into xarray data arrays.
`moment2_to_variance`(expectation, m2, spp)	Calculate the variance (central 2nd moment) from the raw 2nd moment and expected value.
`radiosity`(sector_radiosity)	Aggregate sector radiosity into a full-hemisphere radiosity dataset.
`spectral_response`(srf)	Evaluate a spectral response function as a data array.
`viewing_angles`(angles)	Collect viewing angles associated with each film pixel from the measure, if relevant.

eradiate.pipelines.logic.aggregate_ckd_quad(mode_id, raw_data, spectral_grid, ckd_quads, is_variance)[source]#

Compute CKD quadrature.

In CKD modes, this pipeline step aggregates spectral data and evaluates the selected quadrature rule. The following updates to the input data are expected:

the g dimension is dropped;
a bin_wmin (resp. bin_wmax) coordinate is created and contains the lower (resp. upper) spectral bound of each bin;
the dataset is reordered by ascending w values.

Parameters:

mode_id (str) – Eradiate mode for which this pipeline step is configured. If set to “ckd”, CKD quadrature computation is performed; otherwise, it is a no-op.
raw_data (DataArray) – A data array holding raw bitmap data indexed against pixel indices.
spectral_grid (CKDSpectralGrid) – Spectral set for which the CKD quadrature is computed.
ckd_quads (list of Quad) – List of quadrature rules matching the spectral bins held by spectral_grid.
is_variance (bool) – Flag that specifies whether the raw_data is a variance value.

Returns:

DataArray – A data array indexed against spectral coordinates associated with each spectral bin.

Notes

In non-CKD modes, this step is a no-op.
During pipeline assembly, this node expands as a single node named <var>.
The spp variable is averaged on the index dimension.

eradiate.pipelines.logic.apply_spectral_response(spectral_data, srf)[source]#

Apply SRF weighting (a.k.a. convolution) to spectral data and turn it into a band aggregate.

Parameters:

spectral_data (DataArray) – Spectral data to process.
srf (SpectralResponseFunction) – Spectral response function to apply.

Returns:

DataArray or None – A data array where the spectral dimension is removed after applying SRF weighting, or None if the SRF is a DeltaSRF.

eradiate.pipelines.logic.compute_albedo(radiosity_data, irradiance_data)[source]#

Compute the bi-hemispherical reflectance (a.k.a surface albedo) from radiosity and irradiance records.

Parameters:

radiosity_data (DataArray) – Radiosity record.
irradiance_data (DataArray) – Irradiance record.

Returns:

dict – A dictionary with the following content:

albedoxarray.DataArray: An albedo data array.

eradiate.pipelines.logic.compute_bidirectional_reflectance(radiance_data, irradiance_data, calculate_stokes)[source]#

Compute the spectral bidirectional reflectance distribution function (BRDF) and bidirectional reflectance factor (BRF) from radiance and irradiance records.

Parameters:

radiance_data (DataArray) – Radiance record.
irradiance_data (DataArray) – Irradiance record.
calculate_stokes (bool) – Specifies whether the radiance is a Stokes vector

Returns:

dict – A dictionary with the following content:

brdfxarray.DataArray: A BRDF data array.
brfxarray.DataArray: A BRF data array.

Notes

This function is expanded to brdf and brf nodes during pipeline assembly.

eradiate.pipelines.logic.degree_of_linear_polarization(stokes_vec)[source]#

Calculate the degree of linear polarization from a Stokes vector.

Parameters:: stokes_vec (DataArray) – The data’s Stokes vector ([I,Q,U,V]).
Returns:: DataArray – Degree of linear polarization (\(\sqrt{Q^2+U^2}/I\)).

eradiate.pipelines.logic.extract_irradiance(mode_id, illumination, spectral_grid)[source]#

Derive an irradiance dataset from the irradiance spectrum of an illuminant, as well as solar angles.

Parameters:

mode_id (str) – Eradiate mode for which this pipeline step is to be configured.
illumination (Illumination) – The illuminant whose irradiance is to be evaluated.
spectral_grid (SpectralGrid) – Spectral grid driving the simulation.

Returns:

dict – A dictionary with the following content:

irradiancexarray.DataArray: A spectral irradiance data array compatible with processed spectral radiance records and suitable for deriving BRDF data.
solar_anglesxarray.Dataset or None: If relevant, a dataset holding solar angles.

Notes

This function is expanded into irradiance and solar_angles nodes during pipeline assembly.

eradiate.pipelines.logic.gather_bitmaps(mode_id, var_name, var_metadata, gather_variance, calculate_stokes, bitmaps, viewing_angles, solar_angles)[source]#

Gather a collection of Mitsuba bitmaps into xarray data arrays.

Parameters:

mode_id (str) – Eradiate mode from which this pipeline step is configured.
var_name (str) – Name of the processed physical variable.
var_metadata (dict) – A metadata dictionary to be attached to the data array holding the processed physical variable.
calculate_stokes (bool) – Flag that specifies whether the variable is calculated as a Stokes vector or not.
gather_variance (bool) – Flag that specifies whether the variance bitmaps should be gathered.

bitmaps (dict) – A dictionary mapping spectral loop indexes to the corresponding bitmap. Dictionary structure is as follows:

{
    <loop_index_0>: {
        {"bitmap": <bitmap_0>},
        {"spp": <sample_count_0>},
    },
    <loop_index_1>: {
        {"bitmap": <bitmap_1>},
        {"spp": <sample_count_1>},
    },
    ...
}

viewing_angles (Dataset, optional) – A dataset holding the viewing angles associated with each pixel in the processed bitmaps.
solar_angles (Dataset, optional) – A dataset holding the solar angles associated with the processed observation data.

Returns:

data_vars (dict[str, DataArray]) – A dictionary mapping data variable names to a corresponding data array. These can easily be aggregated into an xarray dataset, or scattered around other nodes of the post-processing pipeline.

eradiate.pipelines.logic.moment2_to_variance(expectation, m2, spp)[source]#

Calculate the variance (central 2nd moment) from the raw 2nd moment and expected value.

Parameters:

expectation (DataArray) – The data expected value. For Monte Carlo integration, this is the output of the estimator.
m2 (DataArray) – The data’s raw 2nd moment.
spp (DataArray) – The number of samples per pixels used during the Monte Carlo integration.

Returns:

DataArray – The variance (central 2nd moment) of the data. The DataArray has the same shape as the expectation or m2.

Notes

This function calculates the Monte-Carlo variance, which aggregates the variance of each sample. It thus needs to be divided by the spp.

eradiate.pipelines.logic.radiosity(sector_radiosity)[source]#

Aggregate sector radiosity into a full-hemisphere radiosity dataset.

Parameters:: sector_radiosity (DataArray) – Radiosity values for the hemisphere sector corresponding to each film pixel. This quantity is expressed in spectral flux units (typically W/m²/nm) and, when summed over the entire film, aggregates into a spectral radiosity (W/m²/nm).
Returns:: DataArray – A global radiosity record, with no film pixel / viewing angle indexing left.

eradiate.pipelines.logic.spectral_response(srf)[source]#

Evaluate a spectral response function as a data array.

Parameters:: srf (Spectrum) – The spectral function to be evaluated.
Returns:: DataArray – A data array mapping SRF values against the wavelength. To avoid confusion with the spectral coordinate, the wavelength dimension is here named srf_w.

eradiate.pipelines.logic.viewing_angles(angles)[source]#

Collect viewing angles associated with each film pixel from the measure, if relevant.

Parameters:: angles (ndarray) – An array with shape (m, n, 2) containing the processed measure’s viewing angles in degree. Typically obtained by calling DistantMeasure.viewing_angles().
Returns:: Dataset – An xarray dataset holding viewing angle values indexed by film coordinates.

eradiate.pipelines

Contents

eradiate.pipelines#

Pipeline configuration#

Hamilton driver creation and manipulation#

Pipeline logic (eradiate.pipelines.logic)#

`eradiate.pipelines`#

Pipeline logic (`eradiate.pipelines.logic`)#