:py:mod:`bluecast.conformal_prediction.evaluation`
==================================================

.. py:module:: bluecast.conformal_prediction.evaluation


Module Contents
---------------


Functions
~~~~~~~~~

.. autoapisummary::

   bluecast.conformal_prediction.evaluation.prediction_set_coverage
   bluecast.conformal_prediction.evaluation.prediction_interval_coverage



.. py:function:: prediction_set_coverage(y_true: Union[numpy.ndarray, pandas.Series], prediction_sets: Union[pandas.Series, pandas.DataFrame]) -> float

   Calculate the percentyge of prediction sets that include the true label.

   This check can be used to validate that the model covers the true labels
   according to the alpha set during the prediction process.
   :param y_true: Ground truth labels.
   :param prediction_sets: Predicted probabilities of shape (n_samples, 1) where each row is a set of classes.


.. py:function:: prediction_interval_coverage(y_true: Union[numpy.ndarray, pandas.Series], prediction_intervals: pandas.DataFrame, alphas: List[float]) -> Dict[float, float]

   Calculate the percentage of prediction intervals that cover the true value.

   This check can be used to validate that the model covers the true values
   according to the alpha set during the prediction process.
   :param y_true: Ground truth labels.
   :param prediction_intervals: DataFrame with predicted bands according to provided confidence levels. This must
       contain columns of format f"{alpha}_low" and f"{1-alpha}_high" for each format.
   :param alphas: List of alphas indicating which confidence levels to check