:py:mod:`bluecast.experimentation.tracking`
===========================================

.. py:module:: bluecast.experimentation.tracking


Module Contents
---------------

Classes
~~~~~~~

.. autoapisummary::

   bluecast.experimentation.tracking.ExperimentTracker




.. py:class:: ExperimentTracker(db_path: Optional[str] = None)


   Bases: :py:obj:`bluecast.config.base_classes.BaseClassExperimentTracker`

   DuckDB-based implementation of ExperimentTracker used in BlueCast
   and BlueCastCV pipelines. This triggers automatically as long
   as the default Xgboost model is used. For custom ml models
   users need to create an own Tracker. The base class from
   bluecast.config.base_classes can be used as an inspiration.

   Uses separate tables for hyperparameter tuning and evaluation data
   for better organization and query performance.

   .. py:property:: experiment_id
      :type: List[int]

      Legacy compatibility property.

   .. py:method:: _init_database() -> None

      Initialize database schema with separate tables for different data types.


   .. py:method:: add_results(experiment_id: int, score_category: Literal[simple_train_test_score, cv_score, oof_score], training_config: bluecast.config.training_config.TrainingConfig, model_parameters: Dict[Any, Any], eval_scores: Union[float, int, None], metric_used: str, metric_higher_is_better: bool) -> None

      Add an individual experiment result into the tracker.

      :param experiment_id: Sequential id. Make sure add an increment.
      :param score_category: Chose one of ["simple_train_test_score", "cv_score", "oof_score"].
          "simple_train_test_score" is the default where a simple train-test split is done. "cv_score" is called
          when cross validation has been enabled in the TrainingConfig.
      :param training_config: TrainingConfig instance from bluecast.config.training_config.
      :param model_parameters: Dictionary with parameters of ml model (i.e. learning rate)
      :param eval_scores: The actual score of the experiment.
      :param metric_used: The name of the eval metric.
      :param metric_higher_is_better: True or False.


   .. py:method:: retrieve_results_as_df(table: Optional[str] = None) -> pandas.DataFrame

      Convert ExperimentTracker information into a Pandas DataFrame.

      :param table: Which table to query ("hyperparameter_experiments", "evaluation_experiments", or None for both)
      :return: Pandas DataFrame with experiment results


   .. py:method:: get_best_score(target_metric: str, table: Optional[str] = None) -> Union[int, float]

      Get the best score for a target metric.

      :param target_metric: The metric to find the best score for
      :param table: Which table to query ("hyperparameter_experiments", "evaluation_experiments", or None for both)
      :return: Best score value


   .. py:method:: get_hyperparameter_results() -> pandas.DataFrame

      Get only hyperparameter tuning results.


   .. py:method:: get_evaluation_results() -> pandas.DataFrame

      Get only evaluation results.


   .. py:method:: get_experiment_summary() -> Dict[str, Any]

      Get a summary of experiments stored in the tracker.

      :return: Dictionary with experiment statistics


   .. py:method:: close() -> None

      Close database connection and cleanup temporary files if created.


   .. py:method:: __del__()

      Cleanup on deletion.