rail.evaluation.metrics.pit module

class rail.evaluation.metrics.pit.PIT(*args, **kwargs)

Bases: MetricEvaluator

Probability Integral Transform

evaluate(eval_grid=array([0., 0.01010101, 0.02020202, 0.03030303, 0.04040404, 0.05050505, 0.06060606, 0.07070707, 0.08080808, 0.09090909, 0.1010101, 0.11111111, 0.12121212, 0.13131313, 0.14141414, 0.15151515, 0.16161616, 0.17171717, 0.18181818, 0.19191919, 0.2020202, 0.21212121, 0.22222222, 0.23232323, 0.24242424, 0.25252525, 0.26262626, 0.27272727, 0.28282828, 0.29292929, 0.3030303, 0.31313131, 0.32323232, 0.33333333, 0.34343434, 0.35353535, 0.36363636, 0.37373737, 0.38383838, 0.39393939, 0.4040404, 0.41414141, 0.42424242, 0.43434343, 0.44444444, 0.45454545, 0.46464646, 0.47474747, 0.48484848, 0.49494949, 0.50505051, 0.51515152, 0.52525253, 0.53535354, 0.54545455, 0.55555556, 0.56565657, 0.57575758, 0.58585859, 0.5959596, 0.60606061, 0.61616162, 0.62626263, 0.63636364, 0.64646465, 0.65656566, 0.66666667, 0.67676768, 0.68686869, 0.6969697, 0.70707071, 0.71717172, 0.72727273, 0.73737374, 0.74747475, 0.75757576, 0.76767677, 0.77777778, 0.78787879, 0.7979798, 0.80808081, 0.81818182, 0.82828283, 0.83838384, 0.84848485, 0.85858586, 0.86868687, 0.87878788, 0.88888889, 0.8989899, 0.90909091, 0.91919192, 0.92929293, 0.93939394, 0.94949495, 0.95959596, 0.96969697, 0.97979798, 0.98989899, 1.        ]), meta_options={<class 'rail.evaluation.metrics.pit.PITOutRate'>: {'default': {'pit_min': 0.0001, 'pit_max': 0.9999}}, <class 'rail.evaluation.metrics.pit.PITAD'>: {'default': {'pit_min': 0.0, 'pit_max': 1.0}}})

Compute PIT array using qp.Ensemble class .. rubric:: Notes

We will create a quantile Ensemble to store the PIT distribution, but also store the full set of PIT values as ancillary data of the (single PDF) ensemble. I think the current metrics do not actually need the distribution, but we’ll keep it here in case future PIT metrics need to make use of it.

property pit_samps

Return the samples used to compute the PIT

class rail.evaluation.metrics.pit.PITAD(*args, **kwargs)

Bases: PITMeta

Anderson-Darling statistic

evaluate(pit_min=0.0, pit_max=1.0)

Use scipy.stats.anderson_ksamp to compute the Anderson-Darling statistic for the PIT values by comparing with a uniform distribution between 0 and 1. Up to the current version (1.6.2), scipy.stats.anderson does not support uniform distributions as reference for 1-sample test.

Parameters:

• pit_min (float, optional) – PIT values below this are discarded

• pit_max (float, optional) – PIT values greater than this are discarded

class rail.evaluation.metrics.pit.PITCvM(*args, **kwargs)

Bases: PITMeta

Cramer-von Mises statistic

evaluate()

Use scipy.stats.cramervonmises to compute the Cramer-von Mises statistic for the PIT values by comparing with a uniform distribution between 0 and 1.

class rail.evaluation.metrics.pit.PITKS(*args, **kwargs)

Bases: PITMeta

Kolmogorov-Smirnov test statistic

evaluate()

Use scipy.stats.kstest to compute the Kolmogorov-Smirnov test statistic for the PIT values by comparing with a uniform distribution between 0 and 1.

class rail.evaluation.metrics.pit.PITMeta(*args, **kwargs)

Bases: object

A superclass for metrics of the PIT

evaluate()

Evaluates the metric a function of the truth and prediction

Returns:

metric – value of the metric and statistics thereof

Return type:

dictionary

rail.evaluation.metrics.pit.PITMetaMetric(cls)

Decorator function to attach metrics to a class

class rail.evaluation.metrics.pit.PITOutRate(*args, **kwargs)

Bases: PITMeta

Fraction of PIT outliers

evaluate(pit_min=0.0001, pit_max=0.9999)

Compute fraction of PIT outliers