Skip to main content

Auto-discover assets

Goal​

You'll let PLACEHOLDER Cloud list every table and view in a datasource, pick the ones you care about, and create the corresponding DataAsset rows in one click.

Discover works for Postgres (PARITY-2) and the warehouse / lakehouse connectors shipped under FEATURE-1 (Snowflake), FEATURE-2 (BigQuery), FEATURE-3 (Redshift), FEATURE-4 (Databricks SQL), FEATURE-5 (Microsoft SQL Server), and FEATURE-6 (Trino / Presto).

Prereqs​

  • A working datasource that passes Test connection.
  • Workspace editor role or higher.

Steps​

  1. Open the datasource detail page.

  2. Click "Discover assets". A modal opens. PLACEHOLDER Cloud queries the source's catalog (e.g. pg_class for Postgres, INFORMATION_SCHEMA.TABLES for warehouses).

  3. Filter the list — by schema (dropdown) or by free-text on the table name.

  4. Tick the tables and views you want PLACEHOLDER Cloud to manage.

  5. Click "Add selected". PLACEHOLDER Cloud promotes every ticked entry to a first-class DataAsset in one atomic call (#456) — the whole batch lands together or not at all, so a transient failure can't leave a half-registered partial state. Names that already exist are skipped (matched on datasource_id + name), so re-running "Add selected" over the same tables is safe.

    Prefer to promote a single table? Each row has a "Promote" button — one click registers just that table as a DataAsset without touching the rest. A toast confirms the outcome (N registered, M already existed) and the modal stays open so you can keep promoting.

Verify​

The new assets appear on the datasource detail page. Each one is now a target for rule suites and profile runs.

Caveats​

  • Discover excludes system schemas: pg_catalog, information_schema, pg_toast (Postgres); the analogous schemas in other warehouses. If you legitimately need to validate information_schema, create the asset manually.
  • Results are cached in Redis for 15 minutes per datasource. If you create a table in Postgres after discovery and don't see it, that's the cache; wait or click the refresh action.
  • row_count_estimate shown in the list is the warehouse's stats estimate, not a fresh COUNT(*). It's there to help you decide which tables are worth validating, not as an exact figure.

Bulk profile-on-add​

After Add selected, PLACEHOLDER Cloud does not automatically profile the new assets — you decide when. The daily 3 AM UTC profile sweep will pick them up; or click Run profile now on each.