bluecast.evaluation.eval_metrics

Module for evaluation metrics.

This is called as part of the fit_eval function.

Module Contents

Classes

ClassificationEvalWrapper

Wrapper to evaluate classification metrics.

RegressionEvalWrapper

Wrapper to evaluate regression metrics.

Functions

plot_lift_chart(→ None)

plot_roc_auc(→ None)

Plot the ROC curve and calculate the AUC (Area Under the Curve).

plot_probability_distribution(probs, y_classes[, opacity])

Plots the probability distribution of each class in an individual color.

balanced_log_loss(y_true, y_pred)

eval_classifier(→ Dict[str, Any])

mean_squared_error_diff_sklearn_versions(y_true, y_preds)

root_mean_squared_error_diff_sklearn_versions(y_true, ...)

eval_regressor(→ Dict[str, Any])

bluecast.evaluation.eval_metrics.plot_lift_chart(y_probs: numpy.array, y_true: numpy.array, num_bins: int = 20) → None
bluecast.evaluation.eval_metrics.plot_roc_auc(y_true: numpy.array, predicted_probabilities: numpy.array, title='ROC Curve') → None

Plot the ROC curve and calculate the AUC (Area Under the Curve).

Parameters:

  • y_true – True labels (0 or 1) for the binary classification problem.

  • predicted_probabilities – Predicted probabilities for the positive class.

  • title – Title for the ROC curve plot.

bluecast.evaluation.eval_metrics.plot_probability_distribution(probs: numpy.ndarray, y_classes: numpy.ndarray, opacity: float = 0.5)

Plots the probability distribution of each class in an individual color.

Parameters:

  • probs – A 2D array of shape (n_samples, n_classes) containing probability distributions.

  • y_classes – An array of shape (n_samples,) containing the class labels for each sample.

  • opacity – Opacity level for the plots. Default is 0.5.

bluecast.evaluation.eval_metrics.balanced_log_loss(y_true, y_pred)
bluecast.evaluation.eval_metrics.eval_classifier(y_true: numpy.ndarray, y_probs: numpy.ndarray, y_classes: numpy.ndarray) → Dict[str, Any]
bluecast.evaluation.eval_metrics.mean_squared_error_diff_sklearn_versions(y_true, y_preds)
bluecast.evaluation.eval_metrics.root_mean_squared_error_diff_sklearn_versions(y_true, y_preds)
bluecast.evaluation.eval_metrics.eval_regressor(y_true: numpy.ndarray, y_preds: numpy.ndarray) → Dict[str, Any]
class bluecast.evaluation.eval_metrics.ClassificationEvalWrapper(higher_is_better: bool = True, eval_against: Literal[probas_all_classes, probas_target_class, classes] = 'classes', metric_func=matthews_corrcoef, metric_name: str = 'Matthews Correlation Coefficient', **metric_func_args)

Wrapper to evaluate classification metrics.

Parameters:

  • metric_func – Function object to calculate the metric.

  • higher_is_better – Boolean indicating if higher metric values are better.

  • eval_against – String indicating if the metric should be evaluated against probabilities or classes. Can be ‘probas_all_classes’, ‘probas_target_class’ or ‘classes’. For ‘probas_all_classes’, the metric is calculated against the predicted probabilities for all classes. For ‘probas_target_class’, the metric is calculated against the predicted probabilities for the best class. For ‘classes’, the metric is calculated against the predicted classes. This parameter decides how the predictions arrive in the metric function.

Returns:

Float value of the metric score.

classification_eval_func_wrapper(y_true: numpy.ndarray | pandas.Series | list, y_probs: numpy.ndarray | pandas.Series | list) → float | int

Wrapper function to evaluate classification metrics.

Parameters:

  • y_true – Numpy array of true labels.

  • y_probs – NumPy array of predicted probabilities.

Returns:

Float value of the metric score.

class bluecast.evaluation.eval_metrics.RegressionEvalWrapper(higher_is_better: bool = False, metric_func=None, metric_name: str = 'Mean squared error', **metric_func_args)

Wrapper to evaluate regression metrics.

Parameters:

  • metric_func – Function object to calculate the metric.

  • higher_is_better – Boolean indicating if higher metric values are better.

Returns:

Float value of the metric score.

regression_eval_func_wrapper(y_true: numpy.ndarray | pandas.Series | list, y_hat: numpy.ndarray | pandas.Series | list) → float | int

Wrapper function to evaluate classification metrics.

Parameters:

  • y_true – Numpy array of true targets.

  • y_hat – NumPy array of predicted targets.

Returns:

Float value of the metric score.