Data assets

A data asset is one table, one view, or one parquet dataset that you want PLACEHOLDER Cloud to look after. Assets belong to a datasource; rule suites and profile runs belong to assets.

Data model

DataAsset lives in the per-tenant schema.

Column	What it stores
`name`	The qualified name as it appears in the source (`analytics.orders`, `s3://bucket/orders/`).
`type`	`table` / `view` / `query` / `parquet_dataset`.
`datasource_id`	FK to the parent `Datasource`.
`config`	Open-ended JSONB. For SQL assets this is the schema + table; for object-store assets, the prefix and partitioning hint.
`description`	Free-text blurb. Rendered under the asset name in the list view. Nullable.
`owner_email`	Free-text email of the human who owns the asset. Not FK-bound — owners can be people outside the org. Nullable.
`tags`	JSONB list of short strings (e.g. `["pii", "revenue"]`).
`is_restricted`	Boolean governance flag for sensitive (PII / PHI / PCI) assets, default `false`. When `true`, the asset is invisible to restricted viewers — see Restricted assets below.

Restricted assets

Mark an asset is_restricted: true when it holds sensitive data (PII, PHI, PCI) that not every read-only user should even know exists. The flag drives a server-side visibility filter keyed on the viewer's membership:

Every viewer-role membership row (workspace_memberships, team_memberships, and legacy org-level viewer rows) carries a viewer_access_level of unrestricted (the default) or restricted.
A viewer whose effective access to a workspace is restricted does not see is_restricted assets there — they vanish from lists, search, and lineage, and fetching one by id returns a plain 404 (indistinguishable from a nonexistent id). On the asset endpoints the flag therefore never leaks the asset's existence; for data derived from the asset, see the caveat below.
Everyone else — editors, workspace admins, org admins, and unrestricted viewers — sees restricted assets exactly like any other asset. A legacy viewer row with a NULL access level reads as unrestricted.

The filter is enforced at the database-session layer (the same listener that enforces workspace and team isolation — see Multi-tenancy), so every endpoint that returns assets respects it without per-route code.

Semantics worth knowing:

Scoping is per workspace. A user who is a restricted viewer in workspace A but an editor in workspace B keeps full visibility of B's restricted assets; only A's are hidden.
Org-wide assets fail closed. An asset with no workspace (workspace_id IS NULL) is visible in every workspace's view, so if a user is a restricted viewer anywhere, org-wide restricted assets are hidden from them everywhere.
Changing the flag is workspace-admin-or-higher. Editors can create and edit assets but cannot set or clear is_restricted (the API returns 403) — both directions are a data-governance decision: clearing it widens visibility for every restricted viewer, setting it hides the asset from them.
Derived data is gated per endpoint, not at the session layer. Profile runs, anomalies, suggestions, and validation results produced from a restricted asset carry no flag of their own, so the session-level filter doesn't cover them. The profile-run, anomaly-event, AI-suggestion, and RCA endpoints gate by parent-asset visibility instead: a restricted viewer gets the same empty list / 404 a nonexistent id would produce. Validation results are the residual gap — a restricted viewer can still read a flagged asset's validation results on the flat results endpoints. Session-level correlated filtering of derived rows is tracked as a follow-up alongside the equivalent team-scoping gap.

PATCH /api/v1/.../data-assets/{id} with {"data": {"is_restricted": true}} flips the flag (subject to the role gate above).

Tags, owners, descriptions

Each asset carries three optional annotations on top of the qualified name. They exist so a workspace with hundreds of assets stays navigable.

Description — a short human-readable blurb. Shows up under the asset name in the list view and is included in the asset detail.
Owner — a free-text email. The list view renders the email as-is; there is no autocomplete against public.users because an asset's owner is often a person outside the org (a customer DBA, a vendor on-call).
Tags — a JSONB list of short strings. Lower-case, free-form (pii, revenue, marketing-export). The UI editor commits a tag on Enter or comma; backspace on an empty buffer pops the last chip.

Editing

PATCH /api/v1/.../data-assets/{id} with a partial body — only the fields you want to change need to be present.

PATCH /api/v1/organizations/{org}/workspaces/{ws}/data-assets/{id}
Content-Type: application/json

{"data": {"tags": ["pii", "revenue"], "owner_email": "data-team@example.com"}}

Sending "tags": [] empties the tag list. Sending "description": null clears the description.

Filtering by tag

List endpoints accept ?tag=<value>:

GET /api/v1/organizations/{org}/workspaces/{ws}/data-assets?tag=pii

The server returns assets whose tags JSONB array contains the supplied tag string (exact match, case-sensitive — tags are normalised to lower-case at write time by the UI). Combining ?tag= with ?name= is supported; both filters AND together.

Operations you can do

Create manually by typing the qualified name on the asset-new form.
Auto-discover from the parent datasource — much faster, see Auto-discover assets. Discover surfaces tables and views; PLACEHOLDER Cloud filters out system schemas (pg_catalog, information_schema, etc.).
Promote a discovered table to a first-class asset (#456). In the discover modal, every listed table has a Promote button (and the multi-select Add selected registers the whole batch in one atomic call), each turning a catalog row into a DataAsset keyed by datasource_id + name. A promoted asset is immediately a profile / checkpoint target — identical in shape to a manually-created or dbt-ingested asset. Re-promoting a table that already exists is a no-op (skipped, not duplicated).
Profile the asset to compute column statistics (row count, null %, distinct count, min/max/mean for numerics). See Profile runs.
Author rule suites that target the asset. See Rule suites.
View the change history of every rule that targets this asset via the History button on the asset detail page, backed by GET /api/v1/organizations/{org_id}/audit-events?resource_type=data_asset&resource_id={asset_id}. Shipped in FEATURE-8.
Define batches / partitions when the asset is partitioned and you want to validate one slice at a time. See Batch definitions below.

Batch definitions

A batch definition slices an asset into a partition or interval for validation. The asset still maps to a single table — what changes is the WHERE clause the worker applies when running an expectation. The same rule suite can then be re-pointed at successive batches without authoring a copy of the asset.

Three kinds are supported:

Kind	Config	What the worker does at run time
`whole_asset`	`{}`	Validates the asset in full. The default; identical to the pre-batch behaviour.
`column_partition`	`{"column": "<col>", "value": "<value>"}`	Wraps the asset reference in `(SELECT * FROM <table> WHERE <col> = <value>)`. Values can be strings or ints; strings are quoted and escaped.
`interval`	`{"column": "<col>", "granularity": "day"\|"hour"\|"month"}`	Validates the latest interval: `date_trunc(granularity, <col>) = (SELECT date_trunc(granularity, MAX(<col>)) FROM <table>)`. Snowflake / Redshift share the Postgres spelling; BigQuery uses `DATE_TRUNC` / `TIMESTAMP_TRUNC`; MySQL uses `DATE(col)` / `DATE_FORMAT`.

The CRUD surface hangs off the asset:

GET    /api/v1/.../data-assets/{asset_id}/batch-definitions
POST   /api/v1/.../data-assets/{asset_id}/batch-definitions
GET    /api/v1/.../batch-definitions/{id}
DELETE /api/v1/.../batch-definitions/{id}

POST bodies follow the standard {"data": ...} envelope:

{"data": {"name": "daily", "kind": "column_partition",
          "config": {"column": "order_date", "value": "2025-01-01"}}}

A name must be unique within an asset (the migration enforces this with a unique constraint). Editors can view and list; only owners and admins can delete a definition — the same authority gradient as the rest of the catalog mutations.

A ValidationConfig can carry a batch_definition_id pointing at one of these rows; the worker reads it when the linked checkpoint runs and applies the matching WHERE. validation_configs.batch_definition_id is nullable on disk so legacy configs (which existed before this migration) keep working — a NULL is interpreted as "whole asset" by the worker, matching the migration's backfill default of one whole-asset definition per existing asset.

Per-partition backfills ("re-run this checkpoint against every daily partition for the last 30 days") are layered on top of this surface via POST /api/v1/.../checkpoints/{id}/run-subset with a body of {"batch_definition_ids": [...]}. Shipped in FEATURE-37; see also Checkpoints.

Lineage

Every data asset has an associated lineage graph that surfaces both PLACEHOLDER Cloud's own producer/consumer relationships (DataAsset → ValidationConfig → Checkpoint → ValidationResult) and the manual edges you record to upstream systems outside PLACEHOLDER Cloud (dbt models, Airflow DAGs, anything else that feeds or reads your asset).

The graph lives behind the Lineage tab on the asset detail page. Open an asset from the data-source detail row and the tab renders a left-to-right columnar graph: producers sit on the left, the asset you're looking at is highlighted, and its downstream consumers fan out to the right. You can zoom and pan (drag the canvas, scroll to zoom, or use the on-canvas controls) to follow a larger graph, and a depth selector beside the tab title walks 1–5 hops out so you can expand or tighten the view without leaving the page. Asset nodes are clickable (jump to that asset's lineage) and labelled by name — a blank-named asset falls back to a short id rather than rendering empty; external nodes show their system glyph and, when you give them a URL, an outbound link.

Impact analysis. Every asset reachable downstream of the one you're viewing is highlighted, and the path that connects them is drawn heavier, so "what does this asset feed?" reads at a glance — exactly what you want when a check fails and you need the blast radius. A small legend distinguishes this asset, the downstream-impacted set, and external systems. The highlight is computed in the browser from the lineage the graph already fetched, so there's no extra request and it tracks the depth selector.

Adding an upstream

Open the asset's Lineage tab and use the Add upstream form. Pick one of:

Asset in this workspace — choose another asset in the same org. Useful for marts → fact tables or whatever rollup chain you maintain inside PLACEHOLDER Cloud.
External system — pick dbt, airflow, or external; enter the upstream model/DAG name and (optionally) a URL. The link is stored as JSONB on the dependency row; PLACEHOLDER Cloud renders it but doesn't follow it.

The API mirror is POST /api/v1/.../data-assets/{id}/dependencies with a body of either {"upstream_asset_id": "..."} or {"upstream_external": {"system": "dbt", "model": "stg_orders"}}. Exactly one of the two fields must be set; the server rejects bodies with both or neither.

Walking the graph

GET /api/v1/.../data-assets/{id}/lineage?depth=N walks both directions from the asset. The default depth is 3 hops; the hard cap is 5. The walker is cycle-safe — a graph with A → B → A returns the two nodes and two edges and terminates. The response is the shape react-flow expects:

{
  "data": {
    "nodes": [
      {"id": "<uuid>", "kind": "asset", "name": "...", "type": "..."},
      {"id": "ext:dbt:stg_orders", "kind": "external", "system": "dbt", "model": "stg_orders"}
    ],
    "edges": [
      {"id": "<uuid>", "source": "...", "target": "...", "source_kind": "asset", "origin": "manual"}
    ]
  }
}

Deleting an edge

DELETE /api/v1/.../asset-dependencies/{dep_id} removes one edge. Workspace editors can view lineage but only owners and admins can tear edges down — the same authority gradient applies as for other catalog mutations.

Manual edges carry source = "manual". Edges auto-ingested from dbt manifests (FEATURE-27 follow-up) will carry source = "dbt" so an ingest re-run can purge its own rows without touching anything a human typed in. Today every edge is manual.

Column-level lineage is explicitly out of scope. Auto-extraction from SQL is tracked under FEATURE-27 (dbt artifact ingestion).

Relationship to checkpoints

A ValidationConfig row binds one asset + one rule suite. A Checkpoint references one or more validation configs. When the checkpoint runs, every config inside it runs the bound suite against the bound asset.

In practice you'll see this stack from the UI:

Datasource (analytics-snowflake)
└─ Asset (analytics.orders)
   ├─ Rule suite (orders-completeness)
   ├─ Rule suite (orders-volume)
   └─ Checkpoint (orders-daily)  ─ runs both suites every morning

Deleting an asset

An asset that is referenced by a rule group (one of the group's member suites is scoped to it) is not deletable: the request is blocked with 409 Conflict and a message naming how many rule groups still reference it. Remove the asset from those rule groups first (drop the relevant members), then delete the asset. This referential-integrity guard (#722) keeps a group from being left pointing at a vanished asset. An asset with no rule-group reference deletes cleanly, taking its own subtree (profiling history, anomaly events, AI suggestions, validation configs, batch definitions, lineage edges) with it and returning 204.

Data model​

Restricted assets​

Tags, owners, descriptions​

Editing​

Filtering by tag​

Operations you can do​

Batch definitions​

Lineage​

Adding an upstream​

Walking the graph​

Deleting an edge​

Relationship to checkpoints​

Deleting an asset​