Bits & Flames bitsandflames/fyron

Cohort Tables

fyron.cohort helps create patient-level tables suitable for survival analysis and machine learning. It sits between data acquisition and modeling: once FHIR, SQL, imaging, or document features are extracted, cohort helpers validate and combine them into one analysis table.

The cohort table is the contract between extraction and analysis. It should contain one row per analysis unit, stable identifiers, explicit index/follow-up dates, endpoint columns, and covariates that are available at the intended prediction or analysis time.

What This Module Is For

Use fyron.cohort when you already have feature and outcome tables and need to create a reviewable analysis table. It does not decide your inclusion criteria or endpoint definition for you; it helps encode those decisions into explicit joins, survival columns, and validation checks.

Use DataFrame Operations before this step when raw ICD-10, OPS, local code, or date columns need to become explicit feature columns.

When To Use It

SituationUse
Feature and outcome tables need to become one analysis tablejoin_cohort_tables
You need time and event columns for survival analysisbuild_survival_columns
You have separate index, event, and censor datesbuild_time_to_event_endpoint
You need a compact cohort audit summarycohort_profile
You need to fail early on missing columns or duplicate IDsvalidate_cohort_table
You are preparing a cohort for Table 1, KM curves, Cox models, or MLall three helpers

Why Cohort Tables Matter

Most downstream functions assume one row per patient or one row per analysis unit. A good cohort table should make these choices explicit:

  • unique patient identifier,
  • baseline/index date,
  • outcome/event date or last follow-up,
  • event indicator,
  • covariates measured before the outcome,
  • no duplicated patient rows unless intentionally modeled.

Cohort Design Checklist

  • Define the analysis unit before joining tables.
  • Keep raw source identifiers so joins can be audited.
  • Separate baseline covariates from outcome-derived columns.
  • Build time and event from documented date fields.
  • Validate required columns before running survival or ML functions.

API Contract

TopicContract
Input shapePatient-level or analysis-unit-level pandas.DataFrame objects.
Required columnsStable ID column for joins/QC; date and event columns for endpoint builders; explicit required columns for validation.
Return shapeDataFrame copies or compact QC/profile DataFrames; validation helpers raise on invalid inputs.
Saved artifactsCaller exports cohort CSVs, endpoint tables, profiles, and flow tables as study artifacts.
Failure modesMissing columns, duplicate IDs, invalid date order, negative/zero follow-up, missing censoring, or invalid event coding.

Imports

python
from fyron.cohort import (
    build_time_to_event_endpoint,
    build_survival_columns,
    cohort_profile,
    join_cohort_tables,
    validate_cohort_table,
)

Join Features And Outcomes

python
cohort = join_cohort_tables(
    features_df,
    outcomes_df,
    on="patient_id",
    how="inner",
)

Required and Optional Parameters

ParameterRequiredTypeDefaultDescription
featuresyesDataFramenoneFeature table, usually one row per patient.
outcomesyesDataFramenoneOutcome/follow-up table.
onnostr"patient_id"Join key present in both tables.
hownostr"inner"Pandas merge strategy.
validatenostr"one_to_one"Pandas merge validation mode.

Returns a merged DataFrame.

Typical inputs:

TableExample columns
features_dfpatient_id, age, stage, risk_score, tumor_volume
outcomes_dfpatient_id, event, last_followup_or_event_date
cohortmerged feature/outcome table

Use how="inner" when final analysis requires both features and outcomes. Use left joins during quality control to inspect missing outcomes.

Validate Required Columns

python
validate_cohort_table(
    cohort,
    required_columns=["patient_id", "age", "stage", "event"],
    id_column="patient_id",
)

Required and Optional Parameters

ParameterRequiredTypeDefaultDescription
dfyesDataFramenoneCohort table to validate.
required_columnsyesSequence[str]noneColumns that must be present.
id_columnno`strNone`"patient_id"ID column checked for duplicates. Pass None to skip ID checks.
allow_duplicate_idsnoboolFalsePermit duplicated IDs.

Returns a validated copy/reference of the DataFrame or raises a clear error.

Validation should be early and loud. It is better to fail at cohort construction than to fit a model on a silently malformed table.

Build Survival Columns

python
cohort = build_survival_columns(
    cohort,
    start_col="diagnosis_date",
    end_col="last_followup_or_event_date",
    event_col="death_event",
    time_col="time",
)

Required and Optional Parameters

ParameterRequiredTypeDefaultDescription
dfyesDataFramenoneInput table.
start_colyesstrnoneStart/index date column.
end_colyesstrnoneEvent/follow-up date column.
event_colyesstrnoneExisting event indicator column.
time_colnostr"time"Name of output duration column in days.
event_out_colnostr"event"Name of normalized output event column.

Returns a DataFrame with time_col and event_out_col.

The resulting time and event columns are ready for fyron.survival.

Build Endpoint From Event And Censor Dates

Use build_time_to_event_endpoint when event and censor dates are stored separately. It records both the final endpoint date and whether that date came from an event or censoring.

python
cohort = build_time_to_event_endpoint(
    cohort,
    index_date_col="baseline_ct_date",
    event_date_col="death_date",
    censor_date_col="last_followup_date",
    time_unit="days",
)

Returned columns include time, event, endpoint_date, and endpoint_source. If an event happens after the censor date, the row is treated as censored at the censor date.

Cohort Profile

python
profile = cohort_profile(
    cohort,
    id_col="patient_id",
    endpoint_cols=["event"],
    group_col="stage",
)

Use this table before modeling. It summarizes row counts, unique IDs, missing IDs, duplicate ID rows, endpoint counts/rates, and group sizes.

Full Cohort Pattern

python
from fyron.cohort import (
    build_survival_columns,
    join_cohort_tables,
    validate_cohort_table,
)

cohort = join_cohort_tables(
    features_df,
    outcomes_df,
    on="patient_id",
    how="inner",
)

cohort = build_survival_columns(
    cohort,
    start_col="index_date",
    end_col="last_followup_or_event_date",
    event_col="event",
)

validate_cohort_table(
    cohort,
    required_columns=["patient_id", "time", "event", "age", "risk_score"],
    id_column="patient_id",
)

Quality-Control Checklist

  • Confirm patient_id uniqueness.
  • Confirm index dates precede outcome/follow-up dates.
  • Check missingness before dropping rows.
  • Confirm event coding: 1 means event and 0 means censored for survival.
  • Keep feature measurement windows before the index/outcome.
  • Save cohort-building code with the analysis, not only the final CSV.

Common Downstream Uses

python
from fyron.survival import plot_kaplan_meier
from fyron import ml

plot_kaplan_meier(cohort, "time", "event", group_col="stage")

result = ml.run_classification_pipeline(
    cohort[["age", "risk_score"]],
    cohort["binary_endpoint"],
)

Return Values

FunctionReturn shapeNotes
join_cohort_tablesmerged DataFrameUses pandas merge semantics and optional merge validation.
validate_cohort_tablevalidated DataFrameRaises clear errors for missing columns or duplicate IDs.
build_survival_columnsDataFrame with duration/event columnsDuration is computed from date columns; event is normalized into the requested output column.
build_time_to_event_endpointDataFrame with endpoint columnsUses index, event, and censor dates to create time, event, endpoint_date, and endpoint_source.
cohort_profileaudit DataFrameCompact cohort-level counts for review.