Multi-tenancy

PLACEHOLDER Cloud is schema-per-tenant multi-tenant. One Postgres database hosts every org's metadata; each org gets its own schema for its tenant-scoped data (datasources, assets, suites, results, anomalies, suggestions).

Why schema-per-tenant

Option	Trade-off
One DB per org	Cleanest isolation; expensive to operate.
Schema per org	Strong isolation via search_path; one DB to operate. ← PLACEHOLDER Cloud chose this.
Shared tables with org_id filter	Cheapest; one missing WHERE org_id = … is a data leak.

The boundary lives at the request layer — every authenticated request resolves a Principal, and the DB session is opened with SET search_path TO tenant_<org_id>, public. Tenant-scoped queries hit the org's schema; cross-org references (orgs, users, tokens, LLM configs) live in public and remain readable.

Defense in depth: the runtime guarantee is reinforced by HARDEN-10 (Postgres RLS + org_id column) and HARDEN-11 (reset search_path on connection checkout). These are tracked, not yet shipped.

Workspace isolation: dual-layer enforcement

Within a tenant schema, workspaces carve the org's data into logical slices. PLACEHOLDER Cloud enforces workspace isolation at two layers, both active on every request:

1. Application layer (TENANCY-1). A SQLAlchemy do_orm_execute listener splices a workspace_id IN (<accessible>) filter into every SELECT / UPDATE / DELETE against a workspace-bearing tenant table. A before_flush listener does the equivalent for ORM unit-of-work writes (session.delete(obj) / dirty-attribute UPDATEs). The accessible set is derived from the principal's direct workspace_memberships plus team grants; org owners and admins bypass filtering.

2. Database layer (TENANCY-2). A Postgres Row-Level Security policy (workspace_isolation) keyed on the app.accessible_workspaces GUC is attached to every workspace-bearing tenant table. The GUC has four recognised states:

   GUC value	Effect
   unset (NULL)	unrestricted — admin / CLI / unscoped session path
   "*"	unrestricted — explicit admin sentinel
   "" (empty string)	no concrete-workspace rows visible (org-wide workspace_id IS NULL rows still pass)
   "<uuid>,<uuid>,…"	only those workspaces are visible (plus org-wide rows)

   The policy itself is permissive for the unset state, so the fail-closed default for request-scoped sessions is enforced one layer up: a pool-checkin listener in api/src/dq_cloud_api/db/workspace_isolation.py resets the GUC to "" every time a connection is returned to the pool. A request handler that forgets to call set_config therefore lands in the empty-list state and sees zero concrete-workspace rows — not the unset/admin state. Admins and the CLI bypass this either by running outside the listener path (raw connections, alembic) or by explicitly setting the GUC to "*". The policy is declared AS RESTRICTIVE, so it composes with the HARDEN-10 org_isolation policy via AND: a row is visible iff both the org check and the workspace check pass.

The DB-layer policy is the load-bearing guarantee. Even if a router emits raw text("DELETE FROM datasources WHERE id = ...") — which sidesteps the ORM and therefore the application listener — the RLS policy still hides rows whose workspace_id isn't in the caller's access set. A buggy or forgotten filter at the application layer becomes a no-op rather than a cross-workspace data-loss event.

Rows with workspace_id IS NULL are org-wide: every workspace member of the org can read them, and an org-wide asset survives the empty-access-set deny path (a workspace-less org member can still see them). This matches the HARDEN-10 convention.

Restricted-asset filtering (third dimension)

Stacked on top of the workspace (and team) predicates, the same application-layer listener enforces the restricted-asset dimension: data_assets.is_restricted = true rows are hidden from viewers whose viewer_access_level is restricted on their membership row. The scope is resolved per request from the principal's memberships — per workspace, with org-wide flagged rows hidden fail-closed whenever the principal is a restricted viewer anywhere — and a hidden asset reads as a 404, never a 403, so the flag doesn't become an existence oracle. Unlike the workspace dimension there is no Postgres RLS policy for this filter yet; the session listener is the enforcement point. Machine principals (agent / SCIM tokens), the worker, and the CLI are never restricted — the flag only narrows human viewers. See Data assets → Restricted assets for the user-facing semantics and the admin-only flag-change rule.

Public vs. tenant data

Schema	What's in it
public	organisations, users, org_memberships, workspaces, workspace_memberships, agent_tokens, personal_api_keys, refresh_tokens, llm_configs, audit_events, global_audit_events, identity_providers, invites, custom_expectations. Per-org data_docs_config and catalog_sync_config knobs live as columns on organisations.

global_audit_events growth bound. This table captures auth events that have no org context (failed logins for unknown emails, rate-limit hits where no user could be resolved). Because it accepts traffic from anonymous probes it grows in proportion to public-internet attempts, not active customers. The worker's prune_global_audit_events cron runs daily at 03:30 UTC and deletes rows older than GLOBAL_AUDIT_EVENT_RETENTION_DAYS (default 90 days, see Configuration). Set the env var to 0 to opt out — the table will then grow unbounded on an internet-exposed deployment.

Audit tables are append-only. Both audit_events and global_audit_events carry a BEFORE UPDATE OR DELETE trigger (migration 030) that raises on any in-place edit or unguarded delete, so the single application DB role can't silently rewrite or erase the trail after a compromise. The one sanctioned mutation is the retention prune above — it opts in by setting the session-local dq_cloud.audit_prune GUC, which the trigger checks. A least-privilege Cloud SQL role split (read-only app role, privileged prune role) is the complementary infra-side control and remains tracked separately. | tenant_<org_id> | datasources, data_assets, asset_dependencies, batch_definitions, rule_suites, validation_configs, checkpoints, validation_results, result_comments, scheduled_runs, run_queue, alert_rules, alert_events, profile_runs, anomaly_events, ai_suggestions, metric_runs, context_config, lakehouse_snapshot_history. |

Orgs, workspaces, and roles

Organisation
└── Workspace (one or many)
    └── Datasources, assets, rules, …

The Phase-1 (Measured Cloud) model has six roles across three scopes. A user is assigned a role and a corresponding workspace or team — except the two org-wide roles, which span the whole org:

Scope	Role	What you get
Org	super_admin	Full authority across the whole org (the new "owner"). Manages members, tokens, billing, RBAC.
Org	technical_admin	Settings / integrations / tokens only — no data access, no user management, no billing.
Workspace	workspace_admin	Manages teams and every data mutation within one workspace.
Team	team_admin	Manages members within one team — no data access.
Team / Workspace	editor	Create / edit rules, schedules and assets. (Reuses the editor string.)
Team / Workspace	viewer	Read-only access + alerts, with an unrestricted / restricted access level. (Reuses the viewer string.)

The legacy org owner / admin strings stay accepted for a one-release compat window: auth.roles.coerce_* maps an org owner/admin onto super_admin, and a workspace-level admin onto workspace_admin, so an in-flight JWT keeps resolving cleanly. The canonical matrix lives in api/src/dq_cloud_api/auth/roles.py.

Legacy "member" role string. Early seed scripts (pre-RBAC-1) stamped some org_memberships / workspace_memberships rows with role = "member" as a synonym for what's now "editor". Migration 028 (issue #274) normalised every such row to "editor" and added a CHECK constraint on both tables that rejects any value outside the canonical bounded set above. A short-lived legacy alias in auth.roles.coerce_role continues to map an incoming "member" string to Role.EDITOR for one release, so a JWT minted in the 15-minute window before the migration ran keeps resolving cleanly until it expires. New code (seed scripts, SCIM role mappings, hand-rolled SQL) must use "editor".

Effective role

A user can reach a workspace by more than one path: a direct workspace_memberships row (hand-managed) or membership in a team that belongs to that workspace. A team is a subdivision of a single workspace (teams.workspace_id); a member's role on the team lives on team_memberships.role. The effective role on a workspace is the maximum role across every such path, on the ladder owner > admin > editor > viewer. The API resolves it in a single indexed query against workspace_memberships and team_memberships JOIN teams ON teams.workspace_id, caches the result per request, and surfaces it as WorkspacePrincipal.workspace_role. A team membership can only raise a user's access — it never downgrades a stronger direct membership. Org owners and org admins bypass the lookup and are admitted to every workspace by definition. (billing is org-level only and never sits on a team membership.) An orphan team — one whose workspace_id is NULL (it had no workspace to re-parent onto, or was provisioned by SCIM/SSO before a workspace was assigned) — grants no workspace access until an admin assigns it a workspace.

User invites + a UI to manage members ship in FEATURE-38: an admin mints a dqi_ invite token via Settings → Members → Invite teammate; the operator forwards the resulting invite_url to the recipient out-of-band (SMTP wiring is tracked separately in PARITY-7). The recipient opens the URL, sets a password + display name, and is signed in at the invite's role.

The invite modal exposes the full six-role model (#669): pick the role, and the form asks for the scope that role needs — a workspace for workspace_admin / editor / viewer, a team for team_admin, or nothing for the org-wide super_admin / technical_admin. The role options are gated to what you may grant (a workspace admin can grant workspace + team roles but not a super admin), mirroring the server's grant ceiling. A team-scoped invite seeds a team_memberships row (added_via='invite') and an editor org-membership floor; a workspace_admin invite seeds a workspace_memberships row plus the same floor (workspace_admin is not a valid org role). Workspace membership rows (role change, remove) are managed inline on the same Members page; team membership is managed on Settings → Teams. SAML / OIDC SSO is configured per-org — see Configure SSO. Setting an IdP to enforced disables password login for the org; members sign in through the IdP and are JIT-provisioned with the role you choose on the provider.

Team-scoped resources

Within a workspace, five resource kinds can additionally be pinned to one of the workspace's teams: rule suites, validation configs, checkpoints, schedules, and alert rules. Each carries an optional team_id:

• team_id = null (the default) means workspace-shared — visible to every member of the workspace, exactly as before teams.
• A concrete team_id means team-scoped — visible only to that team's members, plus anyone whose effective role on the workspace is workspace-admin-or-higher (and org owners/admins, who are unrestricted). The same do_orm_execute listener that enforces workspace isolation stacks the team predicate on top — and, like the workspace dimension, the team boundary is dual-layer: a team_isolation Postgres RLS policy keyed on the app.accessible_teams GUC (same four states as app.accessible_workspaces — unset, "*", "", CSV-of-UUIDs; declared AS RESTRICTIVE so the three policies AND together) backstops the listener at the database, so even raw SQL that sidesteps the ORM can neither read a sibling team's rows nor re-pin a reachable row to a team outside the caller's set.

Assigning a team. Every create/update surface for the five resources accepts an optional team_id in the request body, validated by one shared server-side guard:

• The team must exist in your org, not be deleted, and belong to the same workspace as the resource — otherwise the API returns 422 "Invalid team for this workspace" (one deliberately uniform message, so team UUIDs can't be probed across orgs or workspaces). Teams are workspace-pinned, so a team assignment on a resource with no workspace anchor at all (an org-scoped URL with nothing to derive a workspace from) is a 422 "Assigning a team requires a workspace-scoped resource".
• You must be able to reach the team: a member of it, or workspace-admin-or-higher in its workspace — otherwise 403.
• An explicit team_id: null on update un-scopes the resource back to workspace-shared. Un-scoping widens the resource's visibility from one team to the whole workspace — the inverse of the create-time widen rule below — so it requires workspace-admin-or-higher (or an org owner/admin); members of the resource's current team do not qualify on their own. Un-scoping keeps the resource's workspace pin: the resource becomes workspace-shared, never org-wide. (For alert rules, restoring the legacy org-wide shape is a deliberately separate operation that team_id semantics never perform implicitly — it is not currently exposed.)
• More generally, on update any transition away from the resource's current team — re-pinning it to a different team just as much as un-scoping it — and any move of the resource to a different workspace (whether it is team-scoped or workspace-shared: relocating a row hides it from its current workspace's members either way) requires workspace-admin-or-higher on the resource's current workspace (or an org owner/admin). Authority is always evaluated against where the resource is, never against the destination of the request: being a member of both teams, or an admin of the destination workspace, is not enough to take a resource away from the team — or the workspace — that currently holds it. The workspace-move rules apply to data assets too, even though they carry no team_id: relocating an asset (or capturing an org-wide one into a workspace) is the same visibility change, gated by the same matrix. Update requests also lock the target row for the duration of the transaction, so two concurrent updates can't interleave into a combined scope transition neither was authorized for.
• On the org-scoped URLs (no {workspace_id} in the path) an update body may also carry workspace_id. A concrete value is a workspace move, gated as above, and the destination must be a live workspace of your org — anything else (a typo'd UUID, another org's workspace) is a 422 "Invalid workspace for this organization", one deliberately uniform message checked for every caller including unrestricted owners, so a row can never be orphaned onto a workspace that doesn't exist. An explicit workspace_id: null is a transition to org-wide: it is rejected with 422 whenever a team is involved (a team-scoped resource can never go org-wide in one request — that includes sending team_id: null and workspace_id: null together, because un-scoping keeps the workspace pin), and for a team-less resource it requires workspace-admin-or-higher on the resource's current workspace, org-wide visibility being the maximal widening. Note that only PUT actually applies a permitted workspace_id: null; PATCH skips null fields, so the request is validated identically but the pin is kept.
• Narrowing a previously shared resource onto a team (null → team) is deliberately split: pinning a workspace-shared resource onto a team you belong to, inside its own workspace, stays member-level — the ergonomic default for editors organizing their own workspace's assets. Pinning an org-wide resource (a legacy alert rule with no workspace) onto a team hides it from every other workspace and stamps a workspace onto it as a side effect, so it requires workspace-admin-or-higher on the destination workspace.
• Omitting team_id on update leaves the pin unchanged.

Inheritance on create. When you create a child resource on a team-scoped parent and don't say otherwise, the child follows the parent's team, so a team's pipeline stays coherently scoped without repeating team_id on every call:

You create…	…referencing	Result when body omits team_id
a checkpoint	a team-scoped rule suite (checkpoint_config.rule_suite_id)	checkpoint inherits the suite's team
a validation config	a team-scoped rule suite (rule_suite_id)	validation config inherits the suite's team
a schedule	a team-scoped checkpoint	schedule inherits the checkpoint's team
an alert rule	a team-scoped checkpoint	alert rule inherits the checkpoint's team (and is pinned to its workspace)

If the parent's pinned team has since been deleted, the create fails with a self-explanatory 422 ("the referenced parent's team no longer exists") — pass team_id explicitly or re-scope the parent.

A body-supplied team_id (including an explicit null) wins over inheritance, but must still cohere with the parent:

• a team_id that names a different team than the parent's is rejected with 422 — two teams' assets can't be mixed in one chain, even by an admin;
• an explicit null child under a team-scoped parent widens the parent's visibility to the whole workspace, so it requires workspace-admin-or-higher — and the child is pinned to the parent's workspace ("widen" means workspace-shared, never org-wide);
• a workspace-shared (team_id: null) parent accepts any child — team-scoped or shared.

A parent reference that doesn't resolve under your own visibility (for example a rule_suite_id belonging to a sibling team) triggers neither inheritance nor coherence; the child simply lands workspace-shared, matching how those references behaved before team scoping. A team-scoped parent in a different workspace than the child is likewise treated as no parent — its team never propagates across workspaces (and on the workspace-scoped alert-rule URL, a checkpoint_id pinned to a different workspace than the URL's is a 404 outright).

Derived data follows the parent's team. The rows a team-scoped checkpoint produces are team-scoped too, enforced by the same session listener through the parent linkage (a correlated EXISTS against the parent's team_id):

• Validation results are visible only when every parent they link — the checkpoint and the rule suite — is workspace-shared or one of your teams'. A result detached from both parents (for example after the parent checkpoint was deleted; run history is preserved with a nulled pointer) is workspace-shared.
• Incidents follow their checkpoint_id the same way. A manual incident with no checkpoint linkage is workspace-shared — its body is operator-authored triage text, not run output — even if it references a specific validation result.
• Deleting a team-scoped parent declassifies its derived history. Deleting a checkpoint (or rule suite) preserves its validation results — and, for a checkpoint, its incidents — by nulling the parent pointer, and a detached row's visibility is no longer derived from that team: the surviving results and incidents become workspace-shared, failure payloads included (a result still follows its other parent if one remains linked). The alternatives were considered and rejected: deleting the derived rows with the parent would destroy the run/triage audit trail, and snapshotting the team onto the detached rows would pin history to a team that may itself be deleted — preserving history wins, and the widening is the documented behaviour. Delete a team's checkpoints only when you accept its run history becoming visible to the whole workspace.
• A hidden derived row reads as a 404 on get-by-id and is absent from lists, search, exports, and the dashboard aggregates — the same no-existence-oracle posture as the workspace dimension.
• Metric runs are the documented exception: the metric_runs table carries no checkpoint, suite, or asset linkage at all (rows are minted by the generic metrics endpoints with a free-form payload), so there is nothing to derive a team from. Metric runs remain visible to every member of their workspace until a future schema change adds a linkage.
• Profile runs, anomaly events, and AI suggestions derive from data assets, which teams do not scope (datasources and assets are workspace-shared by design) — so they carry no team boundary either.

The team filter in the UI. The workspace switcher carries a per-workspace team filter ("All teams" by default). With a team active, the four team-bearing list pages (rule suites, checkpoints, schedules, alert rules) narrow to that team's rows plus the workspace-shared ones, and the dashboard narrows the one aggregate built on a team-bearing resource — the checkpoint count. The dashboard's other widgets (pass rate, recent runs, the 30-day trend, anomalies) are derived from validation results, which the server already narrows to the caller's enforced team visibility; the client-side team filter does not further narrow them — the page says so in an explicit scope hint rather than showing numbers a team filter can't actually narrow. Data sources are workspace-shared by design and never team-owned.

Incident visibility. Incidents are checkpoint-attached, and the incident list/detail responses surface the linked checkpoint's team_id as a derived, read-only field (joined at read time — incidents have no team_id column and no write path accepts one; an incident with no checkpoint link, or whose checkpoint is workspace-shared or was deleted, reads team_id: null). The Incidents page narrows on it client-side under the team filter, with the same shared-rows-stay-visible semantics. The derived field is also enforced: the session listener team-scopes validation results and incidents through their parent linkage (see “Derived data follows the parent's team” above), so a team-tier user cannot read a sibling team's incidents through the API either. Each team's detail page under Settings → Teams also shows a read-only "Team resources" summary — counts of the rule suites / checkpoints / schedules / alert rules the team owns, with links into the filtered list pages — so an admin can see what a team owns before re-parenting or deleting it.

Switching workspaces

The active workspace scopes everything you see — data sources, rule sets, check results — so it's surfaced at the top of the sidebar with a workspace switcher. The switcher shows the current workspace name prominently and, on click, lists every workspace you can reach (owners and admins see all of the org's workspaces; everyone else sees only the ones they're a member of, via direct membership or a team that belongs to the workspace). Selecting a workspace re-scopes the whole app to it and remembers the choice across reloads. Users who can manage workspaces (owner / org-admin) also get a + Create workspace action in the switcher; the control is visible-but-disabled, with a "Requires …" tooltip, for anyone without that permission. The list comes from GET /api/v1/organizations/{org_id}/workspaces.

Provisioning a new tenant

You — as a self-host operator — run:

dqc provision-tenant <org_id>

That command creates the tenant_<org_id> schema and stamps the current tenant tables into it. See reference/cli.md. Incremental migrations against existing tenant schemas are HARDEN-9 — not yet shipped.

Agent-mode datasources

For customers whose data sits behind a firewall, a DQ Agent runs inside that network and registers datasources with PLACEHOLDER Cloud using a dqa_… token. The agent token is scoped to one org + (optionally) one workspace; revoking the token disables every datasource it registered. The agent also executes checkpoints — full Great Expectations runs against the customer DB happen on-prem, and only redacted metadata (counts, success flags, column names) crosses back. See Agent mode for the dispatch + metadata cut, and Deploy the agent for the operator runbook.

Where does my data live? (data residency)

PLACEHOLDER Cloud is self-host-first, so the honest answer is "wherever you, the operator, deploy it" — there is no central control plane and no managed multi-region replication. What that means in practice, network call by network call:

Surface	Where the data sits
Customer datasource data (rows, samples, query results)	Read by the worker process only. Never persisted in PLACEHOLDER Cloud — the worker executes Great Expectations in a sandboxed subprocess and only retains the summary metrics + per-expectation pass/fail counts.
Metadata + result snapshots (orgs, users, datasources, assets, suites, validation results, anomalies, AI suggestions)	The operator's Postgres, in the schema-per-tenant layout described above.
Agent-mode datasources	The DQ Agent runs inside the customer's network and proxies validation / profile work; row data never crosses the network boundary back to the API / worker. See agent-mode datasources above.
Encrypted secrets at rest (datasource credentials, LLM API keys, IdP client secrets)	Stored in the operator's Postgres, AES-256-GCM-encrypted under a per-tenant DEK that's wrapped by the KEK (env-fallback or AWS KMS — see HARDEN-16).
AI calls (suggestions, explanations, autoresolve, NL→SQL)	Outbound to the configured LLM provider — Anthropic (api.anthropic.com), OpenAI (api.openai.com), an Ollama instance you supply, or a custom OpenAI-compatible base URL. The provider, model, and API key are configured per-org via llm_configs; the fallback knobs (DEFAULT_LLM_*) live in Configuration. Per-org cost / token budgeting is tracked in HARDEN-25.
Outbound alerts	To the per-alert-rule destination (Slack webhook URL, Microsoft Teams URL, PagerDuty Events API, generic webhook URL, SMTP server).
Telemetry	Optional. Prometheus is scrape-pull (the API exposes /metrics); OpenTelemetry is configurable via OTEL_EXPORTER_OTLP_ENDPOINT and disabled by default.

PLACEHOLDER Cloud surfaces a region claim so customers and auditors can confirm at a glance where a given deployment is running. The operator declares it via the DEPLOYMENT_REGION env var (suggested values: us-east-1 / eu-west-1 / apac-northeast-1 / self-hosted); it is rendered in the UI footer next to the version and returned by the public, unauthenticated GET /api/v1/deployment endpoint:

{
  "data": {
    "region": "eu-west-1",
    "version": "0.1.0",
    "sso_enabled": true,
    "llm_provider": "anthropic"
  }
}

Multi-region replication is out of scope today — pin a single region in the operator's hosting platform and the answer to "where is my data?" is that region.

Step-up auth for sensitive actions

Some endpoints mutate credentials or bearer secrets — minting or revoking an agent token, rotating an LLM API key, creating or updating a datasource, creating or updating an alert rule. A captured access token shouldn't be able to do those things without the user re-proving they hold the password, even though the session itself remains valid for routine read / write. PLACEHOLDER Cloud enforces a recent password proof on those routes (HARDEN-24).

Every access token carries an auth_fresh_at claim. /auth/login and /auth/elevate set it to now; /auth/refresh mints with auth_fresh_at = 0 ("never fresh"). The fresh_auth server dependency rejects requests whose auth_fresh_at is older than AUTH_STEP_UP_WINDOW_SECONDS (default 300 = 5 min) with 401 detail=step_up_required and the header WWW-Authenticate: StepUp. The UI catches that signal, prompts for the password inline, calls POST /auth/elevate to mint a refreshed token, and retries the original request.

Protected endpoint	Reason
POST /api/v1/.../agent-tokens	Minting a long-lived bearer secret.
DELETE /api/v1/.../agent-tokens/{id}	Revoking a token a session shouldn't be able to kick offline silently.
POST / PATCH /api/v1/.../llm-configs	Rotates the LLM API key (encrypted at rest, but the call carries plaintext).
POST / PUT /api/v2/.../datasources	Rotates datasource credentials.
POST / PATCH /api/v1/.../alert-rules	Rotates the Slack webhook / PagerDuty key / generic-webhook URL.

Personal API keys (dqk_…) and agent tokens (dqa_…) carry auth_fresh_at = 0 by construction — they have no associated password, so they can never satisfy the gate. A token automation can read and write routine resources, but it cannot use itself to mint a sibling agent token or rotate the org's LLM key. That separation is deliberate.