Machine Learning

fyron.ml provides tabular classification workflows for clinical data science. It is designed for patient-level feature matrices where the target is a binary or multiclass endpoint, such as treatment response, readmission, toxicity, or progression status.

The module covers the full lightweight workflow:

1. split data reproducibly,
2. handle class imbalance,
3. train Random Forest or XGBoost classifiers,
4. compute clinically meaningful metrics,
5. cross-validate and tune,
6. inspect feature importance,
7. generate publication-ready evaluation plots,
8. save fitted estimators.

What This Module Is For

Use fyron.ml when you have a patient-level feature matrix and a classification endpoint. The module is intentionally focused on tabular clinical prediction workflows. It does not decide which variables are clinically valid, whether leakage is present, or whether a model is appropriate for deployment; those remain study-design decisions.

For transparent feature derivation from ICD-10, OPS, delimited code strings, or date columns, use DataFrame Operations before ML preprocessing and model fitting.

Core Concepts

Install

uv add "fyron[ml]"

Optional Boruta feature selection:

uv add "fyron[ml,boruta]"

Import Pattern

from fyron import ml

Using the ml namespace keeps notebooks readable and makes it clear which helpers come from Fyron.

Expected Data

API Contract

Clinical ML usually benefits from explicit preprocessing before modeling:

• encode categorical variables,
• impute missing values,
• avoid leakage from post-outcome variables,
• split by patient, not row, when repeated measurements exist,
• keep a held-out validation set when possible.

Fyron does not hide those choices because they are study-design decisions.

Research Decision: Model Development Versus Validation

How To Interpret Outputs

Fyron ML functions return dictionaries instead of a single score so you can inspect the model, predictions, probabilities, metrics, and figures independently. In clinical settings, accuracy is rarely enough. Review sensitivity, specificity, AUC, confusion counts, and calibration together, then decide whether threshold tuning or subgroup analysis is needed before reporting performance.

Use feature importance as a model-inspection tool, not as causal evidence. If a feature is clinically surprising, check leakage, coding artifacts, missingness patterns, and whether the train/test split respects patient-level independence.

Parameter Notes

API Summary

Before training, use fyron.preprocessing.feature_matrix_report to flag missingness, constants, rare binary variables, and leakage-like column names.

Function Parameter Reference

Train/Test Split

X_train, X_test, y_train, y_test = ml.train_test_split_data(
    X,
    y,
    test_size=0.2,
    stratify=True,
    random_state=42,
)

Use stratify=True for binary clinical endpoints unless there is a specific study-design reason not to.

Class Imbalance

For Random Forest:

class_weight = ml.compute_class_weights(y_train)

For XGBoost:

scale_pos_weight = ml.compute_scale_pos_weight(y_train)

These helpers are intentionally separate because sklearn and XGBoost use different imbalance APIs.

Random Forest

rf = ml.train_random_forest(
    X_train,
    y_train,
    X_test,
    y_test,
    class_weight=class_weight,
    n_estimators=500,
    max_depth=None,
    random_state=42,
    plot=True,
)

rf["metrics"]
rf["figures"].keys()

Returned keys include:

For clinical reporting, preserve metrics, y_pred, y_prob, and the exact train/test split so validation can be audited later.

XGBoost

xgb = ml.train_xgboost(
    X_train,
    y_train,
    X_test,
    y_test,
    scale_pos_weight=scale_pos_weight,
    n_estimators=500,
    max_depth=3,
    learning_rate=0.05,
    random_state=42,
    plot=True,
)

For binary targets, Fyron defaults eval_metric to logloss. For multiclass targets, it defaults to mlogloss and multi:softprob.

Metrics

metrics = ml.calculate_classification_metrics(
    y_test,
    rf["y_pred"],
    rf["y_prob"],
)

Binary outputs include:

• accuracy
• balanced_accuracy
• f1
• precision
• recall
• sensitivity
• specificity
• auc
• tn, fp, fn, tp

For clinical models, inspect sensitivity and specificity directly. Accuracy can be misleading when events are rare.

Cross-Validation

from sklearn.ensemble import RandomForestClassifier

cv = ml.cross_validate_model(
    RandomForestClassifier(n_estimators=300, random_state=42),
    X,
    y,
    cv=5,
    return_oof_predictions=True,
)

cv["fold_metrics"]
cv["mean_metrics"]
cv["std_metrics"]

Set return_oof_predictions=True when you want out-of-fold predictions for calibration or error analysis.

Grid Search

from sklearn.ensemble import RandomForestClassifier

grid = ml.get_default_param_grid("random_forest")

search = ml.grid_search_model(
    RandomForestClassifier(random_state=42),
    grid,
    X_train,
    y_train,
    X_val=X_test,
    y_val=y_test,
    scoring="roc_auc",
    cv=5,
    n_jobs=1,
)

search["best_params"]
search["val_metrics"]

n_jobs=1 is useful in restricted notebook, CI, or sandbox environments. Use higher parallelism on local machines when appropriate.

End-To-End Pipeline

result = ml.run_classification_pipeline(
    X,
    y,
    model="xgboost",
    test_size=0.2,
    handle_imbalance=True,
    use_grid_search=False,
    random_state=42,
    n_estimators=300,
    learning_rate=0.05,
    plot=True,
)

Use the pipeline helper for fast baselines. For final analyses, consider calling the lower-level functions explicitly so every preprocessing and modeling step is visible in your methods.

Feature Importance

importance = ml.get_feature_importance(
    rf["model"],
    feature_names=list(X.columns),
)

fig, ax = ml.plot_feature_importance(
    rf["model"],
    feature_names=list(X.columns),
    top_n=20,
)

For XGBoost, Fyron extracts booster importance scores and normalizes them when possible.

Boruta Feature Selection

boruta = ml.run_boruta_feature_selection(
    X_train,
    y_train,
    max_iter=100,
    random_state=42,
)

selected = boruta["selected_features"]
ranking = boruta["ranking_df"]

Boruta is an all-relevant feature-selection method. It can be useful in biomarker discovery, but it should still be validated with external or held-out data.

External Validation And Calibration

Use these helpers when a model is already trained and the next question is reliability on locked or external data.

nested = ml.nested_cross_validate_model(
    estimator,
    {"C": [0.1, 1.0, 10.0]},
    X,
    y,
    outer_cv=5,
    inner_cv=3,
)

external = ml.evaluate_external_validation(
    fitted_model,
    X_external,
    y_external,
    threshold=0.35,
)

calibrator = ml.fit_probability_calibrator(
    fitted_model,
    X_calibration,
    y_calibration,
)

card = ml.model_card_summary(
    fitted_model,
    X=X_train,
    y=y_train,
    metrics=external["metrics"],
    intended_use="Research endpoint prediction",
)

Clinical Plotting

Model evaluation plots:

fig_roc, _ = ml.plot_roc_curve(y_test, rf["y_prob"])
fig_pr, _ = ml.plot_precision_recall_curve(y_test, rf["y_prob"])
fig_cal, _ = ml.plot_calibration_curve(y_test, rf["y_prob"])
fig_cm, _ = ml.plot_confusion_matrix(y_test, rf["y_pred"])

Exploratory feature plots:

fig_corr, _, corr = ml.plot_correlation_heatmap(
    X,
    method="spearman",
    mask_upper=True,
)

Model comparison:

fig_metrics, _ = ml.plot_metric_bars(
    {
        "Random Forest": rf["metrics"],
        "XGBoost": xgb["metrics"],
    },
    metrics_to_plot=["auc", "sensitivity", "specificity", "f1"],
)

Grouped clinical summaries:

fig_bar, _ = ml.plot_patterned_bar(
    summary_df,
    x="endpoint",
    y="rate",
    group="cohort",
    error="ci_half_width",
    title="Endpoint rates by cohort",
    annotate=True,
)

See Clinical Plotting for detailed plotting guidance.

Persistence

ml.save_estimator(rf["model"], "models/rf.joblib")
loaded = ml.load_estimator("models/rf.joblib")

Persist only the fitted estimator. Keep feature-generation code and schema validation separately versioned so the model can be reproduced.

Practical Checklist

• Confirm rows represent independent patients or split by patient ID.
• Define the endpoint before feature engineering.
• Remove leakage variables.
• Preserve feature names with DataFrames.
• Use stratified splits for imbalanced endpoints.
• Report sensitivity, specificity, AUC, and calibration when probabilities are used clinically.
• Save both figures and metric tables for reproducibility.