# Refinement Phase 7 — Unattended operation: scheduler, locking & multi-instance readiness — Report (2026-07-13)

**Track:** backend (infra) · **Depends on:** phase 6 (its settlement reconciliation is a future job here) ·
**Gate:** `dotnet build` 0 new warnings · `dotnet test` **402 pass** (396 prior + 6 new scheduling tests).

## The headline (7.1) — the platform now runs itself

Before this phase only two hand-written `PeriodicTimer` hosted services existed; the credential-expiry scan, EVV
no-show sweep, and **weekly payout-batch generation** were admin-click-only while their seeded cadence keys sat
unread — so **nurses were paid only when an operator clicked**. Now a single in-process scheduler drives every job
on its own cadence.

**`RecurringJobSchedulerHostedService`** (`Persistence/Services/Scheduling/`) + the **`IRecurringJob`** seam:
- The scheduler owns one independent loop per job (their cadences don't couple; a crash in one never stops the
  others), the per-tick DI scope, error isolation (a throwing tick logs and the next tick retries on schedule),
  and a per-tick `IDistributedLock("scheduler:{name}")`. A job says only *how often* (usually a `platform_configs`
  cadence key, re-read each tick so an admin change applies without a restart) and *what one idempotent run does*.
- **No new infrastructure.** SQL Server stays the only external dependency — a single-instance MVP needs neither
  Hangfire/Quartz (durable/cross-restart scheduling is the only thing they add for idempotent periodic sweeps) nor
  Redis. Adding a cron = implement `IRecurringJob` + one `AddSingleton<IRecurringJob, …>()`.

Jobs registered (`Services/Scheduling/Jobs/`), each dispatching the **same idempotent command the admin trigger
sends** (the admin endpoints are unchanged and remain overrides):

| Job (`Name`) | Cadence source | Re-homed / new |
| --- | --- | --- |
| `booking_request_expiry` | 1 min const | re-homed from `BookingRequestExpiryHostedService` (deleted) |
| `notification_retention` | 24 h const | re-homed from `NotificationRetentionHostedService` (deleted) |
| `verification_expiry_scan` | `verification_expiry_scan_cadence_hours` (24) | **new** → `ScanExpiringCredentialsCommand` |
| `no_show_sweep` | `no_show_scan_cadence_hours` (1) | **new** → `DetectNoShowSessionsCommand` |
| `weekly_payout_generation` | `nurse_payout_interval_days` (7) | **new** → `GeneratePayoutBatchCommand` |

## Money movement stays human-approved (critical rule)

The payout job schedules **generation only** — it opens a `draft` batch over the trailing window; the irreversible
`process` (money-moving) step remains an explicit admin action until trust is earned. To let an unattended run
record a batch with no human initiator, `NursePayoutBatch.InitiatedByAdminId` is now **nullable** (`null` =
system-initiated) — migration `RefinementPhase7SystemPayoutBatch` (alters the column + FK to nullable; the FK, the
`PayoutBatchDto` projection, and `swagger.v1.json` were updated to match). The command's `SystemInitiated` flag is
**scheduler-only**: `AdminPayoutsController.Generate` neutralizes any request-supplied value (`command with {
SystemInitiated = false }`), so an API caller can never bypass the authenticated-admin requirement. A quiet week
(no eligible bookings) is a benign no-op; a re-run over an overlapping window is safe — the
`nurse_payout_booking_links.booking_id` UNIQUE prevents re-selecting an already-paid booking.

## 7.2 — Redis is the scale-out gate, NOT added

Per the phase's "don't add Redis because", the in-process `ICacheService`/`IDistributedLock` stay. They are the
documented **>1-instance scale-out gate**: the moment a second API instance runs, swap the lock seam to Redis and
the scheduler's per-tick lock serializes ticks across nodes (idempotency + the DB uniques cover a double-run
either way). Nothing speaks Redis today; no package added. (Registry rows `ICacheService`/`IDistributedLock`
updated with the framing.)

## 7.3 — Migrations split from boot

`dotnet run -- migrate` is a deploy-time one-shot: it applies EF migrations + the idempotent seeders, then exits —
so concurrent multi-instance start-ups never race on DDL and the runtime login needs no permanent DDL rights.
**Development** boot still migrates + seeds (incl. the Development-only sandbox gateway + demo world) for
convenience; **deployed** boot only *checks* the schema is current (`EnsureSchemaUpToDateAsync` — fail-fast on a
pending migration) and seeds roles/break-glass admin. Env-gated in `Program.cs`.

## What is now testable and exactly how

- **6 new Foundation tests** (`Tests/Baya.Test.Foundation/Scheduling/`): each cadence job reads the right config
  key and dispatches the right command (incl. the payout job asserting `SystemInitiated=true` + the trailing
  window); the scheduler runs a job at startup under `scheduler:{name}`, keeps siblings alive when one throws, and
  stays **dormant under the `Testing` environment**.
- **Live cadence check:** set a short `no_show_scan_cadence_hours` / `verification_expiry_scan_cadence_hours` (or
  a short interval) in `platform_configs`, run the API (Development), and watch the job fire on schedule in the
  logs, producing the same result as the admin manual trigger.
- **Migration path:** `dotnet run -- migrate` applies + seeds and exits; a deployed-env boot with a pending
  migration fails fast with the list of pending migrations.

## What is mocked / deferred (follow-ups)

- **Moadian reconciliation + refund-settlement poll** are **Phase 8's** jobs — they have no command/cadence key
  today and Phase 8 explicitly owns registering them. They slot in as new `IRecurringJob`s with one `AddSingleton`
  — no scheduler change. Documented in the mocks-registry row.
- **Redis** — the scale-out gate above (only when >1 instance).

## Contracts produced/consumed

- `PayoutBatchDto.initiatedByAdminId` is now nullable (`null` = system/scheduled batch). Updated
  `dev/contracts/domains/payouts.md` + `dev/contracts/openapi/swagger.v1.json`. No other wire change.

## Files

New: `Services/Scheduling/{IRecurringJob, RecurringJobSchedulerHostedService}.cs` +
`Services/Scheduling/Jobs/{BookingRequestExpiry, NotificationRetention, CredentialExpiryScan, NoShowSweep,
WeeklyPayoutGeneration}Job.cs`; migration `RefinementPhase7SystemPayoutBatch`; 2 test files. Deleted: the two old
hosted services. Changed: `AddPersistenceServices` (registration) + `EnsureSchemaUpToDateAsync`; `Program.cs`
(migrate one-shot + env-gated boot); `NursePayoutBatch`/`NursePayoutBatchConfig`/`PayoutBatchDto` (nullable
initiator); `GeneratePayoutBatchCommand`(+Handler)/`AdminPayoutsController` (system-initiated path).
