Bits & Flames bitsandflames/fyron

FHIR REST

fyron.fhir.rest is the API-facing FHIR client for bounded searches, pagination, row-wise cohort lookups, FHIRPath-style extraction, and server write-back. Use it when your data source is a live FHIR endpoint and you want inspectable pandas.DataFrame outputs rather than opaque client objects.

When To Use It

  • query resources such as Patient, Observation, Condition, Procedure, or ImagingStudy,
  • page through search results with explicit limits,
  • run row-wise queries from a cohort table,
  • flatten or project returned resources into analysis columns,
  • create/update resources or submit reviewed transaction bundles.

Use FHIR SQL when the same data is already available in a relational warehouse.

Core Concepts

ConceptMeaning in Fyron
resource_typeThe FHIR resource name, such as Patient, Observation, Condition, or ImagingStudy.
paramsOrdinary FHIR search parameters sent to the server, for example _count, code, subject, or date.
max_pagesSafety limit for Bundle pagination. Use a small value during exploration and -1 only for deliberate full extracts.
fhir_pathsExplicit field projection from nested FHIR JSON into DataFrame columns.
FHIRPath presetReusable list of (column_name, fhir_path) pairs for common resources.
process_functionCustom Python function for resource-to-row transformation when simple paths are not enough.
column_mapMapping from FHIR search parameter names to columns in an input cohort table.
deduplicate_byColumn or columns used to remove repeated resources after partitioned extraction.

Most read functions return pandas.DataFrame outputs. Write functions return FHIRWriteResult objects with request status and server response information.

API Contract

TopicContract
Input shapeFHIR resource type plus explicit search parameters; row-wise helpers also accept a cohort DataFrame.
Required columnssearch_df_from and related helpers require the columns referenced by column_map.
Return shapeRead helpers return pandas.DataFrame; lower-level fetch helpers return FHIR bundles/resources; write helpers return FHIRWriteResult.
Saved artifactsOptional CSV/JSON outputs are caller-managed; CLI workflows can write result tables and logs.
Failure modesMissing mapped columns, authentication errors, server errors, invalid FHIRPath extraction, pagination limits, or write-back rejection.

Research Decision: Extraction Safety

DecisionPractical guidance
PaginationKeep max_pages low during exploration; remove or expand only for planned extracts.
PartitioningUse search_df_partitioned for broad date ranges or servers with result caps.
DeduplicationSet deduplicate_by when overlapping date windows can return the same resource.
FHIRPathUse fhir_paths for explicit columns instead of ad hoc nested JSON parsing.
Write-backTreat create/update/transaction calls as reviewed outputs, not exploration steps.

Required Inputs

InputNeeded forNotes
base_urlall REST queriesMay come from FHIR_BASE_URL.
resource_typesearchesFHIR resource name, e.g. Observation.
query parametersfiltered searchesUse ordinary FHIR search params.

Optional Inputs

InputUse
auth, basic_auth, bearer_tokenauthenticated servers
max_pagessafe development limits
fhir_pathsexplicit field projection
get_fhir_path_preset(...)reusable default projections for common resources
process_functioncustom resource-to-row logic
cache_ttl_secondsrepeat query caching

Minimal Example

python
from fyron import FHIRRestClient

client = FHIRRestClient("https://hapi.fhir.org/baseR4")
patients = client.search_df(
    "Patient",
    params={"_count": 25, "_sort": "_id"},
    max_pages=1,
)

search_df is the right starting point when one FHIR query should become one DataFrame. Keep _count and max_pages explicit so the extract is reproducible.

Clinical Example

python
observations = client.search_df_from(
    cohort,
    resource_type="Observation",
    column_map={"subject": "patient_id"},
    params={"code": "http://loinc.org|718-7"},
    fhir_paths=[
        ("observation_id", "id"),
        ("patient_ref", "subject.reference"),
        ("value", "valueQuantity.value"),
        ("unit", "valueQuantity.unit"),
    ],
    max_pages=1,
)

search_df_from is for row-wise cohort lookups. In this example, each cohort row contributes a subject search parameter from its patient_id column. This is useful when you already have a reviewed cohort and want to retrieve related resources.

If extracted Condition or Procedure resources contain ICD-10, OPS, medication, or local code strings, use DataFrame Operations to derive broad code indicators or date intervals before cohort modeling.

FHIRPath Extraction

FHIRPath extraction is the most readable way to turn nested FHIR JSON into a stable analysis table when the desired values are direct resource fields. In Fyron, fhir_paths accepts either plain path strings or named tuples:

python
fhir_paths=[
    ("condition_id", "id"),
    ("patient_reference", "subject.reference"),
    ("diagnosis_date", "recordedDate"),
    ("icd_code", "code.coding.code"),
]

Named tuples are preferred because the first value becomes the DataFrame column name. This keeps downstream cohort joins, Table 1 creation, and model feature tables readable.

FHIRPath results follow a simple table contract:

FHIRPath resultDataFrame value
no valuemissing value
one valuescalar
multiple valueslist

Use FHIRPath for direct projections. Use process_function when you need logic such as choosing one coding system from many, exploding components into separate rows, unit conversion, extension parsing, or local phenotype rules.

FHIRPath Extraction Presets

Fyron includes copyable defaults for common FHIR R4 resources. They are intended as starting points, not as hidden clinical assumptions. Review the resulting columns against your server profile and remove or add paths before freezing an analysis.

python
from fyron import FHIRRestClient, get_fhir_path_preset

client = FHIRRestClient("https://fhir.example.org/r4")

paths = get_fhir_path_preset("Observation")
paths.append(("local_lab_flag", "extension.where(url = 'https://example.org/fhir/lab-flag').valueCode"))

labs = client.search_df(
    "Observation",
    params={"code": "http://loinc.org|718-7", "_count": 500},
    max_pages=2,
    fhir_paths=paths,
)

Available presets:

ResourceTypical useImportant columns
Patientdemographics and patient identifierspatient_id, identifier_value, gender, birth_date, death fields
Encountervisit timelines and encounter contextencounter_id, patient_reference, class_code, period_start, period_end
Conditiondiagnoses and clinical phenotypescondition_id, patient_reference, code, recorded_date, onset_datetime
Observationlabs, vitals, scores, componentsobservation_id, patient_reference, code, effective_datetime, value, unit
Procedureoperations and interventionsprocedure_id, patient_reference, code, performed date/period
DiagnosticReportreports and linked observationsreport_id, patient_reference, code, issued, result_reference, conclusion
ImagingStudyimaging cohorts and DICOM joinsstudy_instance_uid, series_instance_uid, series_modality, series_description
MedicationStatementmedication exposure tablesmedication code/display, effective period, assertion date
ServiceRequestorders and requested proceduresrequest id, patient, code, authored date, requester
Specimenbiospecimen timelinesspecimen id, patient, collection date/period, body site
DocumentReferenceclinical notes and document artifactsdocument id, patient, type, date, content URL/title

The raw preset dictionary is also available when you want to inspect or document it:

python
from fyron.fhir import FHIR_PATH_PRESETS

FHIR_PATH_PRESETS["ImagingStudy"]

Preset Examples By Resource

Use Patient for the first cohort identity table:

python
patients = client.search_df(
    "Patient",
    params={"_count": 500, "_sort": "_id"},
    max_pages=1,
    fhir_paths=get_fhir_path_preset("Patient"),
)

Use Condition to build diagnosis cohorts:

python
conditions = client.search_df_partitioned(
    "Condition",
    partition_param="recorded-date",
    start="2020-01-01",
    end="2026-01-01",
    params={"code": "http://hl7.org/fhir/sid/icd-10|C34", "_count": 500},
    fhir_paths=get_fhir_path_preset("Condition"),
    deduplicate_by="condition_id",
)

Use ImagingStudy when you need StudyInstanceUID and SeriesInstanceUID for DICOM download or BOA merges:

python
imaging = client.search_df_from(
    cohort,
    resource_type="ImagingStudy",
    column_map={"subject": "patient_id"},
    param_prefix={"subject": "Patient/"},
    fhir_paths=get_fhir_path_preset("ImagingStudy"),
    include_column_map_columns=True,
    max_pages=1,
)

For high-stakes analyses, save the exact paths beside the extracted CSV as provenance. This makes it possible to reproduce why a column exists and which FHIR field populated it.

Function Guide

FunctionUse it whenOutput
search_dfOne FHIR search should become one DataFrame.DataFrame or dict of DataFrames when a processing function splits outputs.
search_df_fromA cohort table should drive one query per row.Combined result DataFrame with optional source-row metadata.
search_df_partitionedA broad search may hit server result caps.Merged DataFrame with partition trace columns.
get_fhir_path_presetYou want a reviewed starting projection for a common resource.list of (column_name, fhir_path) pairs.
query_df / query_df_fromYou need compatibility aliases for older code.Same as search aliases.
build_partition_paramsYou want to preview date windows before running.list of parameter dictionaries.
create_resource / update_resourceYou need single-resource write-back.FHIRWriteResult.
submit_bundleYou need transaction-bundle write-back.FHIRWriteResult.

Parameter Notes

ParameterPractical guidance
paramsValues are passed as FHIR search params. Use server-supported names from the CapabilityStatement.
_countControls server page size, not total rows. Combine with max_pages during development.
max_pagesUse 1 for smoke tests; use -1 only when you intentionally want all pages.
fhir_pathsPrefer named tuples like ("diagnosis_date", "recordedDate") so output columns are readable.
process_functionUse when values require custom logic, multiple rows per resource, or more robust coding extraction.
include_source_columnsUse row-wise queries to preserve cohort identifiers beside extracted FHIR data.

Sequenced Search For Large Result Sets

Some FHIR servers enforce hard result caps, commonly around 10,000 resources. When a query reaches that cap, ordinary Bundle next links may stop even though more matching resources exist. This is a server-side search behavior; the remedy is to split the search into smaller, non-overlapping windows and run those windows one after another.

Fyron calls this partitioned search in the API. Conceptually, it is a sequenced FHIR REST extraction: build date windows, query each window, merge the rows, and optionally de-duplicate by the resource ID.

Use search_df_partitioned when a resource supports a date-like search parameter such as recorded-date, date, authored, issued, started, or _lastUpdated.

Minimal Sequenced Example

python
from fyron import FHIRRestClient

client = FHIRRestClient("https://fhir.example.org/r4")

conditions = client.search_df_partitioned(
    "Condition",
    partition_param="recorded-date",
    start="2020-01-01",
    end="2024-01-01",
    frequency="YS",
    params={"_count": 500, "code": "C34,C34.9"},
    max_pages_per_partition=-1,
)

This creates yearly windows:

text
recorded-date=ge2020-01-01&recorded-date=lt2021-01-01
recorded-date=ge2021-01-01&recorded-date=lt2022-01-01
recorded-date=ge2022-01-01&recorded-date=lt2023-01-01
recorded-date=ge2023-01-01&recorded-date=lt2024-01-01

Clinical Condition Extraction

python
conditions = client.search_df_partitioned(
    "Condition",
    partition_param="recorded-date",
    start="2018-01-01",
    end="2026-01-01",
    frequency="YS",
    params={
        "_count": 500,
        "identifier": "https://example.org/system|",
        "code": "C34,C34.0,C34.1,C34.2,C34.3,C34.4,C34.5,C34.8,C34.9",
    },
    fhir_paths=[
        ("condition_id", "id"),
        ("patient_reference", "subject.reference"),
        ("encounter_reference", "encounter.reference"),
        ("diagnosis_date", "recordedDate"),
        ("icd_code", "code.coding[0].code"),
    ],
    deduplicate_by="condition_id",
    show_progress=True,
)

Partitioned search adds _sort=_id by default when no sort is provided, uses valid repeated FHIR date bounds like recorded-date=ge2020-01-01&recorded-date=lt2021-01-01, and adds _fyron_partition_start / _fyron_partition_end columns for traceability.

Monthly Or Fixed-Day Windows

Use monthly windows when yearly windows still hit the server cap:

python
monthly_observations = client.search_df_partitioned(
    "Observation",
    partition_param="date",
    start="2023-01-01",
    end="2024-01-01",
    frequency="MS",
    params={"_count": 500, "code": "http://loinc.org|718-7"},
    deduplicate_by="id",
)

Use fixed-day windows when you want predictable partition size:

python
weekly_reports = client.search_df_partitioned(
    "DiagnosticReport",
    partition_param="issued",
    start="2023-01-01",
    end="2024-01-01",
    window_days=7,
    params={"_count": 200},
)

De-Duplication

Use deduplicate_by when partitions may overlap or when a server returns the same resource in adjacent windows because of date precision:

python
conditions = client.search_df_partitioned(
    "Condition",
    partition_param="recorded-date",
    start="2018-01-01",
    end="2026-01-01",
    frequency="YS",
    params={"code": "C34,C34.9"},
    fhir_paths=[
        ("condition_id", "id"),
        ("patient_reference", "subject.reference"),
        ("diagnosis_date", "recordedDate"),
    ],
    deduplicate_by="condition_id",
)

Parameter Reference

ParameterRequiredMeaning
resource_typeyesFHIR resource type, e.g. Condition, Observation, DiagnosticReport.
partition_paramyesDate-like FHIR search parameter used to split the query.
start, endyesInclusive lower bound and exclusive upper bound for the overall search.
paramsnoOrdinary FHIR search parameters reused for every partition.
frequencynoPandas-style frequency such as YS for yearly or MS for monthly windows.
window_daysnoFixed-size windows in days; overrides frequency when set.
max_pages_per_partitionnoPage limit for each partition. Use -1 for all pages within every window.
deduplicate_bynoColumn name or columns used to drop duplicate rows after merging.
include_partition_columnsnoAdds _fyron_partition_start and _fyron_partition_end; default is True.
parallel_fetchnoFetch partition windows in parallel when the client has multiple processes and suitable auth.
auto_sortnoAdds _sort=_id by default for stable pagination.

Preview Windows Before Running

Preview the query windows before running a large extraction:

python
from fyron.fhir import build_partition_params

windows = build_partition_params(
    {"_count": 500, "code": "C34,C34.9"},
    partition_param="recorded-date",
    start="2018-01-01",
    end="2026-01-01",
    frequency="YS",
)

The output is a list of ordinary parameter dictionaries. The partition bounds are represented internally as repeated FHIR parameters so Fyron sends a valid query string instead of a comma-joined date expression.

Choosing The Right Partition Parameter

ResourceTypical parameterNotes
Conditionrecorded-dateGood for diagnosis/cohort extraction when supported by the server.
ObservationdateOften maps to effective time.
DiagnosticReportissued or dateServer support varies; inspect the CapabilityStatement.
ProceduredateUseful for operation/procedure cohorts.
ImagingStudystartedUseful for imaging cohort discovery.
Any resource_lastUpdatedGood for incremental synchronization, not always equivalent to clinical event time.

When Not To Use It

Do not use partitioning as a substitute for correct clinical filters. It is a transport strategy for large result sets. If every monthly or weekly partition still reaches the server cap, narrow the clinical query, use a smaller window_days, or ask the FHIR server administrator whether the endpoint supports asynchronous bulk export.

CLI Workflow

bash
fyron fhir-rest \
  --base-url "$FHIR_BASE_URL" \
  --resource Observation \
  --input cohort.csv \
  --column-map subject=patient_id \
  --param 'code=http://loinc.org|718-7' \
  --max-pages 1 \
  --output observations.csv

Function Table

FunctionRequiredOptionalReturns
FHIRRestClient(...)base_url or FHIR_BASE_URLauth, timeout, cache, loggingclient
search_df(resource_type, ...)resource typeparams, max pagesDataFrame
search_df_partitioned(...)resource type, partition param, start, endfrequency, window days, de-duplicationDataFrame
search_df_from(df, ...)input table, resource type, column mapparams, paths, metadataDataFrame
query_df(...) / query_df_from(...)same as search aliasescompatibility names for existing code
build_partition_params(...)params, partition param, start, endfrequency, window dayslist of parameter dicts
create_resource(resource, ...)FHIR resource dictresource type overrideFHIRWriteResult
update_resource(resource, ...)FHIR resource dictid/type overrideFHIRWriteResult
submit_bundle(bundle)transaction bundlenoneFHIRWriteResult

Return Values

OutputMeaning
DataFrame rowsUsually one row per returned resource, unless a custom processor emits multiple rows.
FHIR path columnsValues projected from nested FHIR JSON; missing paths become missing table values.
_fyron_partition_start, _fyron_partition_endTrace columns added by partitioned search when enabled.
FHIRWriteResultWrite-back status, response payload, and request context for review/logging.

Common Pitfalls

  • Keep max_pages during development so exploratory queries do not pull a whole server.
  • If a server caps broad searches, partition by date instead of relying on endless pagination.
  • FHIR references often need Patient/<id> rather than a bare ID; inspect your server behavior.
  • collection bundles are for local review; submit transaction bundles to servers.
  • Fyron validates structure lightly. Production write-back should still use server/profile validation.