cleanup phases 6

2026-08-02 18:48:32 +03:30
parent e2db97392a
commit 51e86a1e5f
239 changed files with 118 additions and 70 deletions
@@ -0,0 +1,306 @@
+# Runtime services — deployment topology
+
+**Audit date:** 2026-07-10 · **Derivation:** every entry below is justified from the code — the DI seam
+that depends on it, the config key that names it, or the package/startup wiring that talks to it. Nothing
+is invented; "not needed" claims are backed by the absence of the package/code. Make-it-real steps live in
+`dev/shared-working-context/reports/mocks-registry.md` (row references below).
+
+**The shape in one sentence:** today the API binary talks to exactly **one external system — SQL Server**
+(app DB + log DB); everything else (18 seams) is an in-process mock, so "deployment" today is one container
+ one database — and the table below is the roadmap of what must exist as each seam goes real.
+
+> **Refinement-phase-8 update (2026-07-13):** services **8–14** and **16** below now have a **real HTTP adapter
+> shipped behind their seam**, config-selected by a per-rail `Seams:*:Provider` selector (mock stays the default).
+> "Depends via" now points at a real client, not just a planned one — provisioning the vendor account + credential
+> and flipping the selector turns each on, no code change. What is still genuinely absent (no adapter): **Redis**
+> (5), **Elasticsearch** (17), and the LLM **review-moderation classifier** (15, optional). MoH/INO/eNamad (16)
+> stay **manual by design**. See the mocks-registry refinement-phase-8 banner for the provider tokens per rail.
+>
+> **Refinement-phase-9 update (2026-07-13) — observability (service 4):** the two overlapping metric stacks were
+> consolidated onto **one OpenTelemetry stack** — prometheus-net was removed; metrics are scraped at `/metrics` via
+> the OTel Prometheus exporter, and **distributed tracing** (ASP.NET Core + EF Core) was added, exporting **OTLP
+> only when `OpenTelemetry:Otlp:Endpoint` is configured** (an MVP with Prometheus alone is unchanged). A request's
+> `ApiResult.requestId` is its W3C trace id (support ↔ trace 1:1). Health checks split into `/healthz/live`
+> (process) vs `/healthz/ready` (app DB + log DB [deployed] + an object-storage write probe); `/HealthCheck` stays
+> as the aggregate. An **OTLP collector** (Grafana Tempo / Jaeger / OTEL Collector) becomes the optional new
+> observability service when trace export is turned on. Elasticsearch (17) is still deferred.
+
+## Service inventory
+
+| # | Service | Purpose | Depends via (seam / config) | MVP? | Registry row |
+| --- | --- | --- | --- | --- | --- |
+| 1 | **SQL Server** (app DB `Baya`) | System of record — 12 schemas (`usr ops geo catalog verif search booking payments payouts reviews messaging partner`) | EF Core; `ConnectionStrings:SqlServer` | **Required now** | — |
+| 2 | **SQL Server** (log DB `Baya_Logs`) | Serilog sink in deployed envs (Warning+, auto-created `log.LogEvents`) | `ConnectionStrings:logDb` | **Required now** (deployed) | — |
+| 3 | **Reverse proxy / TLS** (nginx·caddy·traefik) | TLS termination, HTTP/1.1+2, forwarded headers | Kestrel config; JWE bearer | **Required now** | — |
+| 4 | **Prometheus** (+ Grafana) · optional **OTLP collector** | Scrapes `/metrics` (one OTel stack, refinement-phase-9); traces export OTLP when configured | OTel metrics + tracing; `OpenTelemetry:Otlp:Endpoint` | **Recommended now** (collector optional) | — |
+| 5 | **Redis** | `ICacheService` + `IDistributedLock` (money-path mutex) | `Seams:*` (keys TBD; none today) | Before >1 API instance | rows 14, 42 |
+| 6 | **MinIO / S3 / ArvanCloud** | `IObjectStorage` — verification docs, avatars (REQ-006), invoice PDFs | `Seams:ObjectStorage:*` | Before real verification | row 13 |
+| 7 | **Job scheduler** — in-process, SQL only (refinement-phase-7 **done**) | The recurring crons: booking-expiry, notification-retention, credential-expiry scan, no-show sweep, weekly payout-batch generation (Moadian/refund-settlement poll = Phase 8) | `RecurringJobSchedulerHostedService` + `IRecurringJob`s | **Done for single instance** (no new infra) | row 26 |
+| 8 | **SMS gateway** (Kavenegar·Ghasedak·SMS.ir) | `ISmsSender` — OTP delivery (login is impossible without it) | `Seams:Sms:*` (to be added) | **Launch-critical** | row 12 |
+| 9 | **PSP / IPG + Shaparak** (ZarinPal·Sadad·Vandar·Jibit) | `IPaymentProvider` + `IWebhookVerifier` + `ISettlementSplitProvider` (تسهیم) | encrypted `payment_gateways.config_json` | Real payments | rows 39–41 |
+| 10 | **BNPL providers** (SnappPay·Digipay) | `IBnplProvider` / `IBnplProviderResolver` / `ICurrencyNormalizer` | `Seams:Bnpl:*`, `Seams:Currency:*`, gateway config | Optional at launch | rows 46–47 |
+| 11 | **Bank-transfer rail** (Jibit·Vandar·Sadad payout API → PAYA/SATNA) | `IBankTransferProvider` — weekly nurse payouts | `Seams:BankTransfer:*` | Real payouts | row 23 |
+| 12 | **سامانه مودیان** (tax e-invoicing) | `IMoadianClient` — legal invoice registration | `Seams:Moadian:*` + signing cert | Legal — soon after launch | row 45 |
+| 13 | **KYC bridge vendor** (Finnotech-class) | `IShahkarVerifier` + `IIdentityKycProvider` + `IBankAccountOwnershipVerifier` | `Seams:Shahkar:*`, `Seams:IdentityKyc:*`, `Seams:BankOwnership:*` | Real verification + payout gate | rows 27, 28, 30 |
+| 14 | **Geocoding** (Neshan) | `IGeocoder` — address → coordinates (EVV distance) | `Seams:Geocoding:*` | With real EVV | row 31 |
+| 15 | **Review-moderation classifier** (LLM/API) | `IReviewModerationService` — AI pre-screen | `Seams:ReviewModeration:*` | Optional (human queue is default) | row 33 |
+| 16 | **MoH / INO / eNamad** | `ICredentialVerifier` + `ILicenseVerificationService` — **manual admin process; no public B2B API exists** | `Seams:LicenseVerification:*` | Manual = the MVP design | rows 29, 50 |
+| 17 | **Elasticsearch** | `INurseSearch` alt backend + outbox feeder | `Search:Backend` (non-`sql` throws today) | **Not MVP** — SQL search is real | rows 38, 43 |
+
+**Explicitly not needed** (verified absent from `server/Directory.Packages.props` and code): message
+broker (no RabbitMQ/Kafka), Redis today (no `StackExchange.Redis`), Hangfire/Quartz today, Elasticsearch
+client (`Elastic.Clients.*`), any cloud SDK, any payment/SMS vendor SDK. The only externally-pointing
+package beyond SQL Server is `Serilog.Sinks.Elasticsearch` — its wiring is commented out
+(`Baya.Infrastructure.CrossCutting/Logging/LoggingConfiguration.cs:58-70`).
+
+---
+
+## Dependency graph
+
+Solid edges are live today; dashed edges are behind a mocked seam (the arrow exists in code, the wire does
+not). One line per node below the graph.
+
+```mermaid
+flowchart LR
+  subgraph fe[Frontend]
+    WEB["Next.js client"]
+  end
+  subgraph host["API host — Baya.Web.Api (:5002)"]
+    API["ASP.NET Core API\nREST /api/v1 · /metrics · /HealthCheck"]
+    JOBS["In-proc interval jobs\n(→ Hangfire/Quartz later)"]
+  end
+  subgraph data[Data & platform infra]
+    SQL[("SQL Server 'Baya'\n12 schemas · migrations on boot")]
+    LOG[("SQL Server 'Baya_Logs'\nSerilog sink, Warning+")]
+    REDIS[("Redis — cache + dist. lock\n(in-proc today)")]
+    S3[("MinIO / S3 — object storage\n(local disk today)")]
+  end
+  subgraph money["Money rails (all mocked today)"]
+    PSP["PSP / IPG + Shaparak\ncapture · webhook · تسهیم"]
+    BNPL["BNPL — SnappPay / Digipay"]
+    BANK["PAYA / SATNA payout rail"]
+    MOAD["سامانه مودیان e-invoicing"]
+  end
+  subgraph trust["Trust & identity rails (all mocked today)"]
+    SMS["SMS gateway (OTP)"]
+    KYC["Shahkar · e-KYC · استعلام شبا"]
+    GEOC["Neshan geocoding"]
+    MODAI["Review-moderation classifier"]
+    MANUAL["MoH / INO / eNamad\n(manual admin review)"]
+  end
+  subgraph obs[Observability]
+    PROM["Prometheus (+ Grafana)"]
+  end
+  WEB -->|HTTPS/JSON| API
+  API --> SQL
+  API --> LOG
+  JOBS --- API
+  API -.-> REDIS
+  API -.-> S3
+  API -.-> SMS
+  API <-.-> PSP
+  API <-.-> BNPL
+  API -.-> BANK
+  API -.-> MOAD
+  API -.-> KYC
+  API -.-> GEOC
+  API -.-> MODAI
+  MANUAL -.- API
+  PROM -->|scrape /metrics| API
+```
+
+- **Next.js client** — the only API consumer; reads `NEXT_PUBLIC_API_URL` (root `CLAUDE.md`).
+- **API** — single ASP.NET Core host; all seams resolve in-process today.
+- **In-proc jobs** — the two `BackgroundService` sweeps (booking-request expiry 1 min, notification
+  retention 24 h); the scheduler upgrade re-homes them (plan §4.1).
+- **SQL Server `Baya`** — system of record; migrations + seeding run on every non-Testing boot.
+- **SQL Server `Baya_Logs`** — deployed-env Serilog sink (auto-creates DB/table).
+- **Redis** — target for `ICacheService`/`IDistributedLock`; nothing speaks Redis yet.
+- **MinIO/S3** — target for `IObjectStorage`; local disk + `file://` URLs today.
+- **PSP/IPG** — card capture, callback signatures, تسهیم settlement split.
+- **BNPL** — provider-financed installments; settle/revert callbacks.
+- **PAYA/SATNA** — weekly nurse payout batches + async reconciliation.
+- **مودیان** — legal e-invoice registration (pending→registered poll).
+- **KYC vendor** — Shahkar phone↔NID, identity+liveness, Sheba ownership (payout gate).
+- **Neshan** — geocoding for address coordinates / EVV distance.
+- **Moderation classifier** — optional AI pre-screen; human moderation is the default gate.
+- **MoH/INO/eNamad** — human verification workflows, by design (no API exists).
+- **Prometheus** — scrapes `/metrics`; health check results forwarded as gauges.
+
+---
+
+## Per-service notes
+
+### 1–2 · SQL Server (`Baya` + `Baya_Logs`)
+
+- **Evidence:** `UseSqlServer` at
+  `server/src/Infrastructure/Baya.Infrastructure.Persistence/ServiceConfiguration/ServiceCollectionExtensions.cs:41`;
+  sink at `…CrossCutting/Logging/LoggingConfiguration.cs:44-45` (schema `log`, auto-create); the 12 schemas
+  via per-entity `ToTable(name, schema)` (e.g. `PaymentsConfig/LedgerEntryConfig.cs:18`,
+  `PayoutsConfig/NursePayoutConfig.cs:19`).
+- **Default:** `mcr.microsoft.com/mssql/server:2022-latest` (Developer for dev; licensed edition in prod).
+  Both DBs fit one instance; `Baya_Logs` can move later.
+- **Config:** `ConnectionStrings:SqlServer`, `ConnectionStrings:logDb` — **rotate + externalize first**
+  (plan §1.1; live `sa` credentials are committed today).
+- **Health/readiness (refinement-phase-9 — §7.2 landed):** split `/healthz/live` (process) vs `/healthz/ready`
+  (app DB + `logDb` [deployed] + object-storage write probe); `/HealthCheck` stays as the aggregate
+  (`Monitoring/Configurations/HealthCheckConfigurations.cs`). Boot runs
+  `MigrateAsync` + 3 seeders (`Program.cs:99-104`) → the login needs DDL rights and concurrent multi-node
+  boot races (plan §4.3).
+
+### 3 · Reverse proxy / TLS
+
+- **Evidence of need:** JWE bearer auth (`RequireHttpsMetadata` must be true in prod —
+  `Identity/ServiceConfiguration/ServiceCollectionExtension.cs:139`); Kestrel `EndpointDefaults=Http2`
+  breaks non-TLS HTTP/1.1 (`appsettings.json:29-33`, plan §1.5); the rate limiter partitions on
+  `RemoteIpAddress` with **no ForwardedHeaders middleware**
+  (`WebFramework/ServiceConfiguration/RateLimitingServiceExtension.cs:69`, plan §1.6) — the proxy must pass
+  `X-Forwarded-For` *and* the app must be taught to honor it.
+- **Default:** caddy 2 / nginx 1.27; terminate TLS, h2 to clients, HTTP/1.1 (or h2c) upstream once §1.5
+  lands.
+
+### 4 · Prometheus (+ Grafana) · optional OTLP collector
+
+- **Evidence (refinement-phase-9 — §7.1 landed):** **one** OpenTelemetry stack — metrics scraped at `/metrics` via
+  `UseOpenTelemetryPrometheusScrapingEndpoint()` (`Monitoring/Configurations/PrometheusMetricsConfigurations.cs`),
+  plus `WithTracing` (ASP.NET Core + EF Core) exporting **OTLP only when `OpenTelemetry:Otlp:Endpoint` is set**
+  (`OpenTelemetryConfigurations.cs`). The duplicate prometheus-net stack (`UseMetricServer`/`UseHttpMetrics`/
+  `ForwardToPrometheus` + packages) was removed. A request's `ApiResult.requestId` is its W3C trace id.
+- **Default:** `prom/prometheus:v2.53` + `grafana/grafana:11`. A **tracing backend / OTLP collector** (Grafana
+  Tempo · Jaeger · OpenTelemetry Collector) is the optional new service — set `OpenTelemetry:Otlp:Endpoint` at it to
+  turn trace + metric export on; Prometheus-scrape-only is an acceptable MVP.
+
+### 5 · Redis
+
+- **Evidence of the gap:** `MemoryCacheService` and `InProcessDistributedLock`
+  (`CrossCutting/Seams/MemoryCacheService.cs:11`, `InProcessDistributedLock.cs:14`) — single-process only;
+  the money-path lock convention `booking:{id}:payment|refund` is already in the handlers.
+- **Default:** `redis:7-alpine` (AOF on). **Required the moment a second API instance runs** (shared cache
+  invalidation generation-tokens + cross-instance money mutex). Config keys to be introduced with the swap
+  (plan §4.2); none exist today.
+
+### 6 · MinIO / S3 / ArvanCloud
+
+- **Evidence:** `LocalDiskObjectStorage` writes under a temp root and returns `file://` URLs
+  (`CrossCutting/Seams/LocalDiskObjectStorage.cs:20,56`); consumers: b6 verification documents
+  (signed-URL upload flow), future avatars (REQ-006), invoice `PdfStorageKey`
+  (`InvoicesConfig/InvoiceConfig.cs` — `pdf_storage_key` column).
+- **Default:** `minio/minio:latest` (S3-compatible; ArvanCloud object storage is the Iran-hosted option).
+- **Config:** `Seams:ObjectStorage:RootPath` today → bucket/endpoint/keys with the swap. Presigned PUT/GET
+  with expiry is the contract the frontend already codes against.
+
+### 7 · Job scheduler
+
+- **Evidence:** two `PeriodicTimer` hosted services only
+  (`Persistence/ServiceConfiguration/ServiceCollectionExtensions.cs:63,67`); the payout/expiry/no-show/
+  Moadian crons are admin-manual with seeded-but-unread cadence keys (plan §4.1). **No `IJobScheduler`
+  interface exists** — the registry name is aspirational.
+- **Default:** Hangfire on the existing SQL Server (no new container) — dashboard behind admin auth; or
+  Quartz with SQL persistence. Not a separate service to "spin up", but it changes the SQL footprint
+  (schema) and ops (dashboard, retries).
+
+### 8 · SMS gateway
+
+- **Evidence:** `LoggingSmsSender` logs OTPs instead of sending (`CrossCutting/Seams/LoggingSmsSender.cs:16`)
+  — **no real user can log in**; per-phone resend window + `otp` rate policy already enforced upstream.
+- **Default:** Kavenegar / Ghasedak / SMS.ir (SaaS — API key, no container). Keys to add:
+  `Seams:Sms:{ApiKey,SenderLine,BaseUrl}` (registry row 12). Template/pattern OTP send for deliverability.
+
+### 9 · PSP / IPG + Shaparak (تسهیم)
+
+- **Evidence:** `MockPaymentProvider` (instant success, `VerifyAsync` echoes the expected amount —
+  `CrossCutting/Seams/MockPaymentProvider.cs:24`), `MockWebhookVerifier` (marker-based "signature" —
+  `MockWebhookVerifier.cs:21`), `MockSettlementSplitProvider` (`MockSettlementSplitProvider.cs:13`). Real
+  merchant credentials belong in the **encrypted** `payment_gateways.config_json` (seeded sandbox row:
+  `Persistence/ServiceCollectionExtensions.cs:109-117` — environment-gate it, plan §1.4).
+- **Default:** SaaS (ZarinPal / Sadad / Vandar / Jibit) — needs merchant + terminal registration and
+  تسهیم (settlement-split) setup to registered IBANs; webhook endpoint is already public
+  (`POST webhooks/payments/{provider}`) with upsert-first idempotency in place.
+
+### 10 · BNPL providers
+
+- **Evidence:** `MockBnplProvider` drives the full verb set + state machine
+  (`CrossCutting/Seams/MockBnplProvider.cs:17-53`); callback endpoint `WebhooksBnplController` (signed,
+  rate-limited). `Seams:Bnpl:*` + `Seams:Currency:TomanToIrrMultiplier` (`SeamOptions.cs:98-120,79`).
+- **Default:** SnappPay and/or Digipay (SaaS, OAuth). **Do plan §2.1 first** — the revert-clearing path is
+  currently unreachable, so real BNPL refunds would strand ledger state.
+
+### 11 · Bank-transfer rail (PAYA/SATNA)
+
+- **Evidence:** `MockBankTransferProvider` settles every instruction instantly
+  (`CrossCutting/Seams/MockBankTransferProvider.cs:18-31`); the handler already chooses PAYA vs SATNA by
+  `payout_satna_threshold_irr` (`platform_configs` seed row 22) and the irreversibility backstops
+  (unconditional `UNIQUE(booking_id)` link, forward-only payout machine) are in place.
+- **Default:** Jibit / Vandar / Sadad payout API (SaaS). Needs the source settlement account + the async
+  `submitted → paid/failed` reconciliation callback the mock collapses (registry row 23, steps 3–4).
+
+### 12 · سامانه مودیان
+
+- **Evidence:** `MockMoadianClient` leaves invoices `pending` forever (`CrossCutting/Seams/MockMoadianClient.cs:21`);
+  no reconciliation job exists (plan §6.5). VAT-on-commission + sequential invoice numbers are already
+  correct server-side.
+- **Default:** government SaaS — enrollment (memory/economic code) + a signing certificate; the certificate
+  is a deploy-time secret.
+
+### 13 · KYC bridge (Shahkar / e-KYC / استعلام شبا)
+
+- **Evidence:** three deterministic mocks with magic-value failure cases
+  (`MockShahkarVerifier.cs:26-37`, `MockIdentityKycProvider.cs:25`,
+  `MockBankAccountOwnershipVerifier.cs:26`); handlers already persist `external_response_json` and treat
+  shared-SIM as a handled state. The Sheba-ownership result gates first payouts (`matched_national_id`).
+- **Default:** one Finnotech-class bridge covers all three استعلام‌ها (SaaS; API keys under
+  `Seams:{Shahkar,IdentityKyc,BankOwnership}:*`).
+
+### 14 · Geocoding (Neshan)
+
+- **Evidence:** `MockGeocoder` jitters ±5 km around 8 hardcoded centroids (`MockGeocoder.cs:52`) — EVV
+  distance checks are noise until this (and/or REQ-008's user pin) is real. `Seams:Geocoding:*` is one of
+  only three seam sections present in `appsettings.json` (`:22-26`).
+- **Default:** Neshan (Iran coverage; SaaS API key), rate-limit/retry per registry row 31.
+
+### 15 · Review moderation
+
+- **Evidence:** keyword-list mock; clean text stays in the human queue by default
+  (`MockReviewModerationService.cs:23`; `Seams:ReviewModeration:*`). Human `ModerateReviewCommand` retains
+  decision authority — so this can stay mocked indefinitely at low volume.
+- **Default:** any text-moderation API / LLM endpoint when review volume outgrows the human queue.
+
+### 16 · MoH / INO / eNamad — a process, not a service
+
+- **Evidence:** `MockCredentialVerifier` always returns `RequiresManualReview`
+  (`MockCredentialVerifier.cs:18`); `MockLicenseVerificationService` likewise
+  (`MockLicenseVerificationService.cs:26`). The registry itself records **no public B2B API exists** —
+  the admin review queue *is* the real implementation. Provision: admin staffing + the f15 console.
+
+### 17 · Elasticsearch — deliberately later
+
+- **Evidence:** `SqlNurseSearch` is the real MVP backend; `Search:Backend` ≠ `sql` throws at startup
+  (`Persistence/ServiceConfiguration/ServiceCollectionExtensions.cs:74-78`); no ES client package.
+- **Default when needed:** `elasticsearch:8.x` + the outbox feeder (registry rows 38/43; plan §8.1).
+
+---
+
+## Deployment notes (from the code, not aspiration)
+
+1. **Boot ≠ migrate (refinement-phase-7).** DDL is a separate deploy step — `dotnet run -- migrate` applies
+   migrations + idempotent seeders then exits. **Development** boot still migrates + seeds (incl. the
+   Development-only sandbox gateway + demo world) for convenience; **deployed** boot only *checks* the schema is
+   current (`EnsureSchemaUpToDateAsync`, fail-fast on a pending migration) and seeds roles/break-glass admin. So
+   multi-instance boots no longer race on DDL and the runtime login needs no permanent DDL rights (plan §1.3/§1.4
+   also landed — no committed `admin`/`qw123321`; sandbox gateway is Development-only).
+2. **Environment files:** `appsettings.json` ≡ `appsettings.Development.json` (byte-identical); **no
+   Production/Staging file exists.** All non-secret env differences ride on ~14 `Seams:*` groups whose
+   defaults live in code (`SeamOptions.cs`), not in config files.
+3. **HTTP posture:** mixed `Http1AndHttp2` Kestrel (refinement-phase-5), so gRPC shares the listener via ALPN;
+   gRPC **reflection is Development-only** (refinement-phase-9 §7.5); TLS required for JWE sanity.
+4. **Single-instance constraints today:** in-memory cache, in-proc money lock, the in-proc recurring-job
+   scheduler (refinement-phase-7 — its per-tick lock is that same in-proc seam), per-instance rate-limit buckets.
+   §4.3 (migrations split from boot) **landed**; scaling past one instance still requires §4.2 (Redis for the
+   shared cache + the cross-instance lock the scheduler/money path use) first — the DB uniques keep money
+   *correct* either way, but locks/cache/limits/scheduler-de-dup silently degrade.
+5. **Logs (refinement-phase-9 — §7.3 landed):** deployed envs write **Information+** to `Baya_Logs` (framework
+   categories held at Warning); **no PII/secrets** (the OTP code is no longer logged in any env). The dead
+   Elasticsearch sink + package were removed; log-table retention is an ops/DBA task (or ship logs to the OTLP
+   collector). Dev writes console + `logs/log.json`.
+6. **Client:** the Next.js app needs `NEXT_PUBLIC_API_URL` pointing at the proxy; wire casing camelCase;
+   snake_case routes.