--- title: Sync a daily patient list with the huli CLI description: Pull each day's patients and appointments with the huli CLI on a cron schedule — cursor-based pagination that resumes mid-walk on restart, plus 429 Retry-After backoff and id-level dedup at load. nav: Recipes order: 40 version: v1 source: handwritten updated: 2026-08-13 --- # Sync a daily patient list with the huli CLI Keep a daily mirror of your patients and appointments: schedule a nightly pull of every `Patient` and `Appointment` for one organization, write each page to durable storage, and resume from the exact page you left off when the host reboots mid-run. The loop is cursor-based, so a restart re-reads the last unfinished page and continues — it commits each page before advancing the checkpoint, and an id-level dedup at load time collapses any re-read page into a single row per resource. ## Audience You run server-to-server integrations and own a cron host. You have synced a paginated API before, you know what a checkpoint file buys you, and you want a patient-and-appointment mirror that survives a reboot — backed by an id-level dedup at load time, not a hope that the walk never repeats a page. ## You'll need - The huli CLI on the cron host. `huli version` should print a `1.x` build; confirm it speaks the v1 public API with `huli auth status` (below), which fails loudly against an older contract. - A SMART Backend Services client registered in HuliPractice (**Practice Settings → Integrations → API Keys**), holding these scopes: - — read plus search on Patient. - — read plus search on Appointment. - The client's RS384 private key on the cron host (file mode `0600`), plus the `client_id` issued at registration. - A writable checkpoint directory the cron user owns — this recipe uses `/var/lib/huli-sync`. SMART Backend Services issues a 5-minute access token from POST https://api.huli.ai/auth/token (`client_credentials` + `private_key_jwt`, RS384). The token endpoint is host-rooted — it sits at `/auth/token`, not under `/fhir`. The CLI mints and refreshes that token for you once `huli auth setup` has stored the client credentials; you never hand-build the assertion in the loop. This recipe uses huli auth setup (machine-to-machine, shipping today). Interactive browser login — huli auth login (Auth-Code + PKCE) — is forthcoming and not in the current CLI. Do not script against it yet. ## End state A cron job runs nightly at 02:15 America/Mexico_City. Each run walks every page of the `Patient` searchset, then every page of the `Appointment` searchset for Doctora María Fernández's organization at Clínica San Rafael, appending each page to a dated NDJSON file under `/var/lib/huli-sync`. A checkpoint file records the cursor of the last page committed. If the host reboots mid-run, the next start reads the checkpoint and resumes from that cursor — at worst re-reading the one page that was in flight. The step-5 load dedups on resource `id`, so the final mirror carries one row per `Patient` and `Appointment`. ## Steps ### 1. Store the machine-to-machine credentials once Register the client once. `huli auth setup` reads the private key, records the `client_id` and the token endpoint, and persists an encrypted profile under `~/.config/huli/`. Run it once as the cron user, not inside the cron job. huli auth setup --profile roster-sync --client-id $HULI_CLIENT_ID --private-key /etc/huli/roster-sync.pem --token-url https://api.huli.ai/auth/token Confirm the profile authenticates. `huli auth status` mints a throwaway access token against `/auth/token` and prints the resolved org and granted scopes without writing any data. huli auth status --profile roster-sync ```text profile: roster-sync client: roster-sync@clinica-san-rafael org: Clínica San Rafael (01965e2a-8c4d-7000-9001-0000000000b0) scopes: system/Patient.rs system/Appointment.rs token: valid, expires in 4m51s ``` If the scope line is missing `system/Appointment.rs`, the key was minted without it — re-mint in Practice Settings before scheduling. A scope gap surfaces at runtime as `HPB-00104`, not at setup. ### 2. Write the sync script The script walks one resource at a time. It reads the page cursor from a checkpoint file, calls the CLI for that page, appends the entries, then advances the checkpoint to the `next` cursor the page returned. Committing the output **before** advancing the checkpoint is what makes a restart safe: a crash between append and checkpoint-write re-reads the same page, and the loader in step 5 dedupes on resource `id`. Save this as `/usr/local/bin/huli-roster-sync.sh`. ```bash #!/usr/bin/env bash set -euo pipefail PROFILE="roster-sync" STATE_DIR="/var/lib/huli-sync" RUN_DATE="$(date +%F)" # e.g. 2026-06-02 OUT_DIR="${STATE_DIR}/${RUN_DATE}" mkdir -p "${OUT_DIR}" # Walk one FHIR resource type to exhaustion, resuming from a per-resource checkpoint. sync_resource() { local resource="$1" local ckpt="${STATE_DIR}/${resource}.cursor" local out="${OUT_DIR}/${resource}.ndjson" # Resume: a non-empty checkpoint means a prior run stopped mid-roster. local cursor="" if [[ -s "${ckpt}" ]]; then cursor="$(cat "${ckpt}")" echo "[$(date -Iseconds)] ${resource}: resuming from cursor ${cursor:0:16}…" else echo "[$(date -Iseconds)] ${resource}: starting fresh" fi while :; do # `huli fhir search` emits one JSON object per line: # {"page":[...entries...], "next":"|"} # --cursor "" requests the first page. The CLI handles token refresh, # 429 Retry-After backoff, and 5xx retries internally. local page_json page_json="$(huli fhir search "${resource}" \ --profile "${PROFILE}" \ --count 100 \ --cursor "${cursor}" \ --output ndjson-page)" # Append this page's entries, then advance the checkpoint. Output is # committed BEFORE the cursor moves, so a crash here re-reads this page. jq -c '.page[]' <<<"${page_json}" >>"${out}" cursor="$(jq -r '.next' <<<"${page_json}")" if [[ -z "${cursor}" || "${cursor}" == "null" ]]; then : >"${ckpt}" # roster exhausted: clear checkpoint echo "[$(date -Iseconds)] ${resource}: complete" break fi printf '%s' "${cursor}" >"${ckpt}" # durable resume point done } sync_resource Patient sync_resource Appointment ``` Make it executable. chmod 0755 /usr/local/bin/huli-roster-sync.sh The CLI absorbs rate-limit and transient-server retries so the loop stays linear. On `HPB-00105` it reads the `Retry-After` header, sleeps that many seconds, and re-issues the same page — the cursor does not advance, so no entries are skipped or doubled. On `5xx` it retries with exponential backoff. It surfaces (`HPB-00104` insufficient scope) and (`HPB-00106` auth failed) as a non-zero exit immediately, because those will not clear on retry. ### 3. Pin the cursor semantics Three invariants make the resume correct: - **The cursor is opaque and stable.** Treat `next` as a base64 token — never parse, truncate, or regenerate it. Passing yesterday's cursor into today's run resumes exactly where that token points; passing an empty string starts a fresh full walk. - **Append, then advance.** The script writes the page to NDJSON before it overwrites the checkpoint. A power loss between those two lines costs you one re-read of a single page on restart, never a gap. Reversing the order — advancing the checkpoint first — is the one change that can drop a page; do not do it. - **The raw NDJSON can repeat a page.** A resume re-reads the in-flight page, and because the cursor orders by a server-side sort key, rows created mid-walk can re-surface on a later page. The walk itself does not promise a duplicate-free file — the step-5 id-level dedup is what makes the final mirror one row per resource. Treat the NDJSON as an at-least-once stream, not exactly-once. ### 4. Schedule the cron entry Run nightly at 02:15 in the clinic's timezone. The `CRON_TZ` prefix pins the schedule to America/Mexico_City regardless of the host clock. A flock guard stops a long run from overlapping the next night's trigger. Install it as the `huli-sync` user's per-user crontab. A per-user crontab line has **no username field** — the schedule goes straight to the command: ```cron CRON_TZ=America/Mexico_City 15 2 * * * /usr/bin/flock -n /var/lib/huli-sync/.lock /usr/local/bin/huli-roster-sync.sh >> /var/log/huli-roster-sync.log 2>&1 ``` Save that as `/etc/huli/roster-sync.crontab` (owned by the cron user) and install it for the `huli-sync` service user: crontab -u huli-sync /etc/huli/roster-sync.crontab A per-user crontab and `/etc/cron.d/` are two different mechanisms — do not mix their syntax. A `/etc/cron.d/` system-crontab line carries a username field (`15 2 * * * huli-sync /usr/bin/flock …`) and you install it by dropping the file into `/etc/cron.d/` with no `crontab` command. A per-user crontab (the form above) omits the username field and is installed with `crontab -u`. Add a username field to a per-user crontab and `crontab` misparses `huli-sync` as the command. `flock -n` makes the job idempotent against overlap: if a run is still going when the next 02:15 fires, the second invocation exits without starting a parallel walk. Combined with the per-resource checkpoint, an overrun simply continues on the following night from where it stopped. ### 5. Load with id-level dedup A resume re-reads at most the in-flight page, and a mid-walk write can re-surface a row on a later page. Dedup on the FHIR resource `id` at load time so any repeat collapses to a single row. This `sort -u`-then-upsert pattern keeps the loader idempotent no matter how many times a page was re-read. ```bash for resource in Patient Appointment; do jq -r '[.id, (.|tojson)] | @tsv' \ "/var/lib/huli-sync/$(date +%F)/${resource}.ndjson" \ | sort -u -k1,1 \ | your-loader upsert --table "fhir_${resource,,}" --key id done ``` `sort -u -k1,1` keeps the first row per `id`; `your-loader upsert --key id` makes the database write a no-op when the row already exists. Either layer alone makes the load idempotent — running both is deliberate redundancy for a cron job you will not be watching. ## What to verify - `huli auth status --profile roster-sync` resolves the org and lists both `system/Patient.rs` and `system/Appointment.rs`. - After a full run, both `Patient.cursor` and `Appointment.cursor` are empty — a non-empty checkpoint means the walk stopped partway and will resume next start. - If the first page's searchset Bundle reports `total`, the NDJSON line count for each resource should approximate it, allowing for one duplicated page from a resume. Treat it as a sanity check, not an exact reconciliation — a searchset `total` is an estimate the server may omit or revise across pages. - After the step-5 load, row counts in `fhir_patient` / `fhir_appointment` equal the distinct `id` count in the NDJSON — no duplicates survived the dedup. - Kill the script mid-run (`Ctrl-C` during a page), re-run it, and confirm the output resumes from the saved cursor rather than restarting from page one. ## What can go wrong Every error is a FHIR `OperationOutcome`. Branch on the HTTP status and `issue[0].code` (the FHIR IssueType); the Huli code (`HPB-…`) is the prefix of `issue[0].diagnostics` — split on `": "` to read it. There is no `details` object and no `coding`. `HPB-00105` — rate limited. The CLI handles this for you: it reads `Retry-After`, sleeps, and re-issues the same page without advancing the cursor. If you ever drive the loop with raw `curl` instead, you must replicate that — honor `Retry-After` and retry the **same** cursor, never the next one. ```json { "resourceType": "OperationOutcome", "issue": [ { "severity": "error", "code": "throttled", "diagnostics": "HPB-00105: Rate limit exceeded" } ] } ``` `HPB-00104` — insufficient scope. The token authenticated but lacks `system/Appointment.rs` (or `system/Patient.rs`). The CLI exits non-zero immediately rather than retrying. Re-mint the key in Practice Settings with both `.rs` scopes selected. `HPB-00106` — auth failed. The `client_assertion` was rejected — usually a `client_id`/private-key mismatch or a clock skew that pushed the assertion's `iat`/`exp` out of tolerance. Re-run `huli auth status` to isolate setup from the loop, and verify the cron host's clock is NTP-synced. `HPB-00107` — auth expired. The 5-minute access token lapsed mid-walk on a slow page. The CLI refreshes automatically before each request; you only see this if you pinned a stale token by hand. Let the CLI manage the token — never cache the bearer across pages yourself. `HPB-00101` — validation error. A search parameter or the `_cursor` is malformed — most often a checkpoint file that was hand-edited or truncated. Treat the cursor as opaque: if a checkpoint is corrupt, clear it (`: > Patient.cursor`) to restart that resource's walk from page one rather than patching the token. A corrupt or partially-written checkpoint produces `HPB-00101`, not a silent wrong-page resume. If a crash truncates the checkpoint mid-write, the next run rejects the malformed cursor loudly. Recover by clearing the file and re-walking from the start — the step-5 id-level dedup absorbs the full re-read without leaving duplicate rows in the mirror. ## Next recipes - **Authenticate as a SMART Backend Service** — the `client_credentials` + `private_key_jwt` (RS384) handshake the CLI performs under `huli auth setup`, end to end, for when you need to drive `/auth/token` yourself. - **Incremental sync with `_lastUpdated`** — narrow the nightly walk to records changed since the last run instead of a full pull. - **Paginate a large patient list** — the cursor / `link[rel=next]` mechanics this loop rides on, walked by hand against a single searchset. - **Mirror Encounters and Observations** — extend the same checkpointed loop to the clinical resources, adding `system/Encounter.rs` and `system/Observation.rs`.