dashi.supervised_characterization package
Submodules
dashi.supervised_characterization.arrange_metrics module
Functions for arranging metrics in a data frame from the original metrics dictionary.
- arrange_performance_metrics(*, metrics, metric_name)[source]
Organizes and formats a subset of metrics from a dictionary into a pandas DataFrame.
This function filters the metrics based on the provided metric name, selects only those relevant to the ‘test’ subset, and formats the result into a DataFrame with a corrected index.
- Parameters:
metrics (dict of {str: float}) – A dictionary containing the calculated metrics for each batch and model combination.
metric_name (str) –
The name of the metric to be selected from the metrics dictionary. Regression metric names, when applicable:
’MEAN_ABSOLUTE_ERROR’
’MEAN_SQUARED_ERROR’
’ROOT_MEAN_SQUARED_ERROR’
’R_SQUARED’
- Classification metric names, when applicable:
’AUC_{class_identifier}’
’AUC_MACRO’
’LOGLOSS’
’RECALL_{class_identifier}’
’PRECISION_{class_identifier}’
’F1-SCORE_{class_identifier}’
’ACCURACY’
’RECALL_MACRO’
’RECALL_MICRO’
’RECALL_WEIGHTED’
’PRECISION_MACRO’
’PRECISION_MICRO’
’PRECISION_WEIGHTED’
’F1-SCORE_MACRO’
’F1-SCORE_MICRO’
’F1-SCORE_WEIGHTED’
- Returns:
A DataFrame where the rows represent the combinations and the columns represent the metric values, with the index corrected for cumulative learning strategies.
- Return type:
pandas.DataFrame
- Raises:
TypeError – If metrics is not a dictionary or if metric_name is not a string.
Notes
The function assumes that the metrics dictionary contains a third element ‘test’ for filtering the relevant metrics.
dashi.supervised_characterization.estimate_models module
Main function for estimating models over multiple temporal or multi-source batches.
- estimate_multibatch_models(*, data, inputs_numerical_column_names=None, inputs_categorical_column_names=None, output_regression_column_name=None, output_classification_column_name=None, date_column_name=None, period=None, source_column_name=None, learning_strategy='from_scratch')[source]
Estimates models across multiple batches, based on either time (temporal) or source. Requires specifying one target variable (regression or classification) and at least one numerical or categorical input feature within the input DataFrame. At the same time, it is necessary to provide either a date variable (indicating the period with the corresponding argument) or a source variable. The date variable must be a valid date, and the source variable categories need to be specified as strings. Additionally, it is recommended that the amount of data in each batching group be sufficient for statistical representativeness.
- Parameters:
data (DataFrame) – The input data containing numerical and/or categorical features, as well as the target variable (either a classification or regression target).
inputs_numerical_column_names (Optional[List[str]], default=None) – List of column names representing numerical input features. If there are no numerical input features, set this to None.
inputs_categorical_column_names (Optional[List[str]], default=None) – List of column names representing categorical input features. If there are no categorical input features, set this to None.
output_regression_column_name (Optional[str], default=None) – Column name for the regression target variable. If there is no regression target, set this to None.
output_classification_column_name (Optional[str], default=None) – Column name for the classification target variable. If there is no classification target, set this to None.
date_column_name (Optional[str], default=None) – Column name containing date or time information for temporal batching. If performing source-based analysis instead of temporal batching, set this to None.
period (Optional[str], default=None) – Period for batching the data (‘month’ or ‘year’) when using temporal batching. If not using temporal batching, set this to None.
source_column_name (Optional[str], default=None) – Column name representing the source of the data (for multi-source batching). If performing temporal batching, set this to None.
learning_strategy (Optional[str], default='from_scratch') – Defines the learning strategy: either ‘from_scratch’ or ‘cumulative’. Note that the ‘cumulative’ strategy can only be applied to temporal analyses, not multi-source analyses.
- Returns:
A dictionary containing the calculated metrics for each batch and model combination. Regression metrics, if applicable:
’MEAN_ABSOLUTE_ERROR’
’MEAN_SQUARED_ERROR’
’ROOT_MEAN_SQUARED_ERROR’
’R_SQUARED’
- Classification metrics, if applicable:
’AUC_{class_identifier}’
’AUC_MACRO’
’LOGLOSS’
’RECALL_{class_identifier}’
’PRECISION_{class_identifier}’
’F1-SCORE_{class_identifier}’
’ACCURACY’
’RECALL_MACRO’
’RECALL_MICRO’
’RECALL_WEIGHTED’
’PRECISION_MACRO’
’PRECISION_MICRO’
’PRECISION_WEIGHTED’
’F1-SCORE_MACRO’
’F1-SCORE_MICRO’
’F1-SCORE_WEIGHTED’
- Return type:
Dict[str, float]
dashi.supervised_characterization.plot_performance module
Main function for multi-batch metrics exploration.
- plot_multibatch_performance(*, metrics, metric_name)[source]
Plots a heatmap visualizing the specified metric for multiple batches of training and test models.
The function takes a dictionary of metrics and filters them based on the metric identifier. It then generates a heatmap where the x-axis represents the test batches, the y-axis represents the training batches, and the color scale indicates the values of the specified metric.
The plot is interactive and can be explored (zoomed, hovered, etc.) using Plotly.
- Parameters:
metrics (dict) – A dictionary where keys are tuples of (training_batch, test_batch, dataset_type), and values are the metric values for the corresponding combination. The dataset_type should be ‘test’ to include the metric in the heatmap.
metric_name (str) –
The name of the metric to visualize. The function will filter metrics based on this identifier and only plot those for the ‘test’ set. Regression metric names, when applicable:
’MEAN_ABSOLUTE_ERROR’
’MEAN_SQUARED_ERROR’
’ROOT_MEAN_SQUARED_ERROR’
’R_SQUARED’
- Classification metric names, when applicable:
’AUC_{class_identifier}’
’AUC_MACRO’
’LOGLOSS’
’RECALL_{class_identifier}’
’PRECISION_{class_identifier}’
’F1-SCORE_{class_identifier}’
’ACCURACY’
’RECALL_MACRO’
’RECALL_MICRO’
’RECALL_WEIGHTED’
’PRECISION_MACRO’
’PRECISION_MICRO’
’PRECISION_WEIGHTED’
’F1-SCORE_MACRO’
’F1-SCORE_MICRO’
’F1-SCORE_WEIGHTED’
- Returns:
This function generates and displays an interactive heatmap using Plotly, and does not return any value. The heatmap is displayed directly in the output environment (e.g., Jupyter notebook, web browser).
- Return type:
None
- Raises:
TypeError – If the metrics parameter is not a dictionary or if metric_identifier is not a string.