Architecture

PLACEHOLDER Cloud has four moving parts: a React UI, a FastAPI backend (api/), an ARQ worker (worker/), and a Postgres + Redis pair behind them. The contributor-facing deep-dive is in AGENTS.md §1 — including Mermaid diagrams that render in GitHub. This page is the user-facing overview.

At a glance

[ ui/ ]  ──HTTPS─▶  [ api/ ]  ──asyncpg──▶  [ Postgres 16 ]
                       │                      │
                       └──Redis (queue + cache)
                              │
                       [ worker/ ]
                              │
                              ├──▶ customer datasource (direct mode)
                              ├──▶ DQ Agent (agent mode) ──▶ customer datasource
                              └──▶ LLM provider (Anthropic / OpenAI / Ollama)

The full Mermaid versions — including the per-tenant schema fan-out and the auth-flow sequence — live in AGENTS.md. Read them there; we don't duplicate them here, partly to avoid drift and partly because they're the same diagrams.

Multi-tenancy

One Postgres database; one schema per organisation. The API resolves the tenant per request and sets search_path = tenant_<org_id>, public. The public schema holds shared rows (orgs, users, tokens, LLM configs); the tenant schemas hold the data-quality data (datasources, assets, suites, results, anomalies, suggestions). See concepts/multi-tenancy.md.

Job orchestration

The API enqueues jobs into Redis using ARQ. The worker polls and executes:

Job	When it fires
checkpoint_run	A checkpoint is run manually or by a schedule.
profile_run	An asset is profiled (manual or daily 3 AM UTC cron).
generate_ai_suggestion	A profile completes; the LLM is asked for rule suggestions.
dispatch_alert	A validation result with success=false matches an alert rule.

Job timeouts and concurrency are tunable via DQ_EXECUTOR_TIMEOUT, DQ_WORKER_CONCURRENCY — see Configuration.

Network model

The API speaks to Postgres, Redis, and (on the auth path) verifies tokens internally. The worker is the only component that talks to customer datasources and LLM providers — keeping those network boundaries small is intentional.

Datasource access is gated by an SSRF allowlist (DATASOURCE_NETWORK_ALLOWLIST) so a tenant admin can't aim PLACEHOLDER Cloud at 169.254.169.254 to read cloud-metadata credentials.

Tokens, briefly

Three token types — full table in AGENTS.md §5 and reference/api.md:

• JWT for user sessions (15-min lifetime, refreshed via dqr_… cookies for 30 days).
• dqk_… for programmatic users (CI scripts, the planned Python / TS SDKs).
• dqa_… for on-prem DQ Agents — scoped to one org + (optionally) one workspace.

All three are looked up against a SHA256 hash in Postgres, cached in Redis for 5 seconds.

Where the source lives

Component	Path
FastAPI app	api/src/dq_cloud_api/
ARQ worker	worker/src/dq_cloud_worker/
Shared Pydantic models + LLM client	shared/src/dq_shared/
Alembic migrations	alembic/
React UI	ui/src/
Local infra (Docker Compose)	infra/
One-off scripts (seeders, etc.)	scripts/

Deep-dive

For the diagrams + the line-by-line flow, jump to AGENTS.md. That file is the contributor's source of truth; this one is a pointer.