logml.report.controllers.baseline_modeling

Functions

_get_averaged_curve(metrics)	For a given list of per-fold curves returns averaged curve.
_get_baseline_curves(curve_type, problems, ...)	Returns baseline ROC/PR curves.
_map_metrics_to_coordinates(metrics, curve_cfg)	Extracts required metrics corresponding to X- and Y- axes.

Classes

ModelSelectionController(cfg, global_params)

Implements data handling and plotting API for ModelSearch results.

class logml.report.controllers.baseline_modeling.ModelSelectionController(cfg: GlobalConfig, global_params: dict, setup_id: str = '')

Bases: object

Implements data handling and plotting API for ModelSearch results.

list_available_models()

Get names of all available models.

get_dataset_name() → str

Returns a dataset that was used to produce HPO results.

get_hpo_losses() → Tuple[Dict[str, List], list]

Collects trial losses from available models.

plot_hpo_losses()

For each model plots HPO trace - (loss, iteration) in form of scatter.

get_dataset() → ModelingDataset

Returns a dataset that was used for MS.

evaluate_tuned_models(metric=None) → Dict[str, logml.models.base.CVMetricsResult]

Returns metrics from all folds (validation) for all tuned model. In case only a specific metric is required - the param ‘metric’ is used.

get_models_performance_summary() → pandas.core.frame.DataFrame

Summarizes models performance across CV folds.

display_modeling_info()

Render modeling info.

show_performance_metrics()

Displays a table that summarizes models performance across folds.

show_classification_curves(curve_type: str)

Plots averaged ROC/PR curves per model.

get_metrics_per_problem(curve_cfg, model_metrics)

show_classification_predictions()

Plots confusion matrix averaged across CV folds for each model.

show_regression_predictions()

Plots true vs predicted for all CV folds for each model.

show_prediction_details()

Produces task-specific plots: confusion matrix for classification and pred/true scatterplot for regression.

plot_metrics_distributions() → None

Prot vioilin plot for loss distribution for baseline and all trained models.

get_metrics_distribution(metric_name=None)

Returns dataframe with raw metrics, nullified outside percentiles.