Arthur Providers — Phase 1

What the two companion PRs change across the backend service and the backoffice, with database, API and application-interaction diagrams.

Backend · bgs-merlin-service PR #61 Frontend · bgs-backoffice PR #1130 branch feat/arthur-providers targets develop / merlin
In one sentence. A Provider is a reusable integration › domain › action leaf with its own pre-defined flow; a new Arthur backoffice app manages providers, and the Merlin flow editor gains a “Call Provider” action that materialises a per-partner internal clone of the provider’s flow at save time and keeps it in sync.

1 · Database changes

One new collection (providers) and new opt-in fields on the shared flow schema. The pre-defined-flows and partner-flows collections share one schema factory (buildFlowSchema); the PR adds a kind marker to the pre-defined side only and internal/provider fields to the partner side only.

providers (NEW collection) _id integration slug · req · trim domain slug · req · trim action slug · req · trim name req description default '' preDefinedFlowId → pre-defined flow UNIQUE (integration, domain, action) pre-defined-flows _id, name, actionName … groups / gateways / actionRefs + kind: 'provider' | absent (NEW · pre-defined only — tags a flow as a provider’s backing flow) provider flows are filtered OUT of Settings / onboarding / builder lists partner-flows _id, partnerId, name, actionName … groups / gateways / actionRefs + internal: boolean + provider: { providerId, integration, domain, action, contentHash } (NEW · partner-flows only — the per-partner internal clone) sparse idx provider.providerId UNIQUE (partnerId, provider.providerId) partial: internal = true preDefinedFlowId clone → provider.providerId (one internal flow per partner × provider)
Solid arrow = foreign-key-style reference; dashed = a cloned copy written at save time.

New indexes & migrations

MigrationCollectionIndexWhy
20261016-providers-indexesprovidersunique (integration, domain, action)One provider per leaf; a duplicate insert surfaces as Mongo E11000 → mapped to 409.
20261019-provider-internal-flow-indexespartner-flowssparse provider.providerIdCheap lookup of every internal flow backing a provider (delete-guard, sync).
unique (partnerId, provider.providerId) · partial internal:trueAt most one internal clone per partner × provider — closes the concurrent-save race (loser gets E11000 and reuses the winner).

Both migrations are idempotent (the schema also declares the indexes for autoIndex environments); migrations run on boot so a fresh DB gets them deterministically.

2 · Backend endpoints (bgs-merlin-service)

A new NestJS providers module exposes CRUD + sync under /api/v1/providers. Coordinates are immutable; actionName is derived as provider__<integration>__<domain>__<action>.

MethodRouteBehaviour
GET/api/v1/providersList all providers, sorted by integration, domain, action.
GET/api/v1/providers/:idGet one; ?withFlow=true attaches the backing flow (same shape as GET /pre-defined-flow/:id).
POST/api/v1/providersCreate a provider — does not create a flow. Blank name → 400; duplicate leaf → 409.
PUT/api/v1/providers/:idUpdate name/description and/or save the backing flow in one call (cleans up the flow if the second write fails). Coordinates are stripped by the whitelist pipe.
DELETE/api/v1/providers/:id204 on success; 409 while partners use it (message names them). Provider removed before its flow.
POST/api/v1/providers/:id/sync-partnersRe-sync every partner internal flow to the provider’s current flow. ?dryRun is a strict boolean — anything but true/false → 400.

What happens when a partner flow with a Call Provider item is saved

Partner flow save ProviderSnapshotService partner-flows (DB) 1 · validate parent flow (before any internal write) 2 · for each «call-provider» item 3 · contentHash(provider flow) (excludes name / actionName) 4 · create / reuse / overwrite internal clone internal:true · provider:{…contentHash} race loser → E11000 → reuse winner (unique partner × provider) 5 · point item.subFlow.flowKey at the clone 6 · delete internal flows nothing references internal flows are hidden from all listings
The clone’s flowKey / actionName is deterministic: provider__<integration>__<domain>__<action> — byte-identical on both the service and backoffice sides.

How a provider edit reaches partners already using it

Editing a provider’s flow does not touch partners immediately — each partner holds its own cloned internal flow, tagged with the contentHash of the provider flow it was copied from. A provider edit propagates to those clones in one of two ways; drift is always detected by comparing hashes.

Provider flow edited PUT /providers/:id (new flow body) → new contentHash H2 (was H1) Partners are now STALE each internal clone still carries H1 ≠ H2; parent partner flows are never auto-rewritten Path A — PUSH on demand (sync-partners) POST /providers/:id/sync-partners?dryRun=… 1 · 400 if the provider has no backing flow 2 · hash provider flow once → H2 3 · findByProviderId → every internal clone 4 · per clone, in batches of 4 (bounded): hash differs → overwrite clone + push to engine → updated++ hash equal → skip → unchanged++ write fails → failed[] (partnerId, flowId, error) dryRun=true: counts would-be updates, writes nothing · parent flows never rewritten Path B — LAZY refresh on next partner save partner re-saves its flow (any reason) the save pipeline’s step 4 re-hashes the provider flow and compares to the clone: H1 ≠ H2 → overwrite the clone with H2 H1 = H2 → left untouched No explicit sync needed — the partner picks up the latest provider flow the next time it is saved (same create/reuse/overwrite logic). A provider rename alone does not change the hash → no sync required.
The contentHash excludes name/actionName, so cosmetic edits never trigger a rewrite. Engine push on Path A is per-clone; order (internal before parent) is fire-and-forget.

Partner export / import of “Call Provider” partners updated

Provider _ids differ between environments, so an exported bundle cannot carry them. On import, every Call Provider action is re-matched to the target environment’s provider by coordinates and its internal flow is re-materialised locally.

Import bundle source-env provider ids + internal flow definitions (both discarded) 1 · resolveImportedProviders match each action by coordinates: configurations.provider → fallback the provider__ flowKey read-only · runs on dry-run too 400 — nothing written provider missing, or has no saved flow (names the coordinates) 2 · materializeImportedActions partner exists → one internal flow per provider; re-point each action rolled back with the rest on failure 3 · flow definitions phase drop bundle’s provider__ internal defs, then after the engine clear push the freshly materialized ones all resolved
Replace imports also delete the partner’s old internal flows first. The whole write is atomic — an unresolvable reference aborts before any change.

Key backend files

FileRole
newsrc/providers/providers.controller.tsThe 6 routes above.
newsrc/providers/providers.service.tsCRUD, 404/409/400 mapping, orchestrates the flow save.
newsrc/providers/provider-snapshot.service.tsCreate/reuse/overwrite/GC of per-partner internal clones.
newsrc/providers/provider-flow-cloner.tsDeep copy of a flow (groups, gateways, conditions, deps) with reassigned ids.
newsrc/providers/provider-content-hash.tsStable content hash used to detect drift.
newsrc/providers/providers.schema.tsThe providers collection + unique index.
modsrc/flow/schemas/flow-base.schema.tsOpt-in kind (pre-defined) and internal/provider (partner) fields + indexes.
modsrc/flow/flow.service.ts / flow.repository.tsSave hooks, findByProviderId, kind filtering.
modsrc/partner-import-export/writers/merlin-config.writer.tsResolves providers by coordinates and re-materialises internal flows on import; replace also deletes old internal flows.
modsrc/partner-import-export/components/flow-definitions.component.tsDrops the bundle’s provider__ internal definitions and pushes the freshly materialised ones after the engine clear.

3 · Backoffice application & interactions (bgs-backoffice)

A brand-new Nx app apps/arthur (basePath /arthur, port 3019) is mounted into the shell, plus a “Call Provider” action added to the existing Merlin flow editor.

Browser bolttech backoffice apps/shell (modified) • home card “Arthur” (gated) • /arthur/* rewrite → ARTHUR_BACKOFFICE_URL • route permissions (ARTHUR_*) • gated by ARTHUR_ENABLED + BUILD_ARTHUR pipeline option apps/arthur (NEW · :3019) /arthur/providers — list (Table, StatusTag) /arthur/providers/create — single form /arthur/providers/:id/edit — FlowTimelineView · read-only coordinates · “Sync partners” (dryRun preview → apply) · delete shows 409 partner names read-only for users without MANAGE apps/merlin (flow editor, modified) “Call Provider (Arthur)” palette entry next to “Call Carrier”; only when a provider has a saved flow (Integration→Domain→Action) read-only sub-flow target bgs-merlin-service /api/v1/providers · /pre-defined-flow reached via the /api/merlin/* proxy rewrite /api/merlin/* GET /providers (build cascade)
The shell proxies /arthur/* to the new app and /api/merlin/* to the service. The Merlin editor reads the provider list to build the Call Provider cascade.

Permissions (temporary Phase-1 aliases)

Dedicated ARTHUR.PROVIDER.* Keycloak roles do not exist yet (PR 10), so the three Arthur roles alias Merlin Settings roles 1:1. Only this one file changes when the real roles land.

Arthur roleAliasesGrants
ARTHURMERLIN. (prefix)Shell card + any Arthur access.
ARTHUR_PROVIDER_VIEWMERLIN.SETTINGS.VIEWList, read, open edit page read-only.
ARTHUR_PROVIDER_MANAGEMERLIN.SETTINGS.MANAGECreate, edit, delete, sync.

User journeys

Create / edit a provider (Arthur)
  • Create: pick existing-or-new integration / domain / action slug + name/description → POST /providers.
  • Edit: build the flow in the shared FlowTimelineView; coordinates are read-only → PUT /providers/:id saves provider + flow together.
  • Delete: blocked with 409 and the partner names if any partner uses it.
  • Sync partners: dryRun preview of would-be updates, then apply.
Use a provider in a partner flow (Merlin)
  • Editor offers “Call Provider (Arthur)” only when at least one provider has a saved flow.
  • Integration → Domain → Action cascade selects the leaf; sub-flow target is read-only.
  • On save, the service clones the provider flow into a hidden per-partner internal flow and points the item at it.
  • “Call Carrier” and manual sub-flows are unchanged; the sub-flow target is only cleared when switching to/from Call Provider.
Rollout / known limits. Ship ARTHUR_ENABLED=false per environment until rollout and set ARTHUR_BACKOFFICE_URL. The frontend PR needs the companion service PR. Export/import of partners using Call Provider is now handled (resolved by coordinates on import — see §2). Remaining follow-ups: engine push order (internal before parent) is fire-and-forget, not awaited; ARTHUR.* Keycloak roles come in PR 10 (backoffice aliases MERLIN.SETTINGS for now).

Generated from the two PRs’ descriptions and the feat/arthur-providers source (service rev f742446, import-by-coordinates). Service suite: 1675 passed / 9 skipped, plus a 25/25 import E2E. Diagrams are schematic, not exhaustive.