Skip to main content

Create and run a validation

Goal​

You'll create a validation that covers one or more data assets, run it, watch it fill in asset by asset, and open the per-rule breakdown. End state: a validation with a health score and a run in Results.

Prereqs​

  • At least one data asset with rules — see Add rules to a data asset. Assets without rules can't be added (the picker shows them disabled).
  • The editor role or higher in the workspace.

Steps​

  1. Create the validation. Go to Validations and click New validation. Give it a name that says what it covers — Nightly orders, Customer 360 — and optionally a description and tags.

  2. Pick its assets. In the same dialog, select the assets whose rules it should run. The asset list is a table: sort and filter it from its column headers (the asset's name, its team, or whether it has rules yet). Each asset's current rules are added. Optionally choose Start from a template to apply a rule template to each selected asset as it's added. Click Create.

  3. Choose the rules. Open the validation. The Rules card lists every rule a run executes, grouped by asset, with each rule's severity. Add rules picks rules from any asset: all of its current rules, or just the ones you tick. Take a rule out with its red ✕. To change a rule itself, click it to open it on its asset.

    A validation holds the rules you picked, not the assets: a rule added to an asset later joins its asset's default validation and the validations you choose when you create it, but not this validation automatically. Add it here, or tick this validation on the rule (see Which validations a rule is in).

  4. Run it. Click ▶ Run on the validation page (or on its row in the Validations list, or use Run validation… in the command palette). The button becomes a spinning Stop while the run is in flight. To run just some rules this once, or on specific partitions of a partitioned asset, use Run with options next to Run.

  5. Watch it. Run history shows the new run expanding automatically and filling in as each asset finishes. Assets on an agent-mode data source finish when your Privacy Agent next polls.

  6. Read the outcome. Click the run to open its Validation run page, with failing counts, sample failing rows, root-cause analysis, and an optional AI explanation. Or expand it (the chevron) for its per-rule breakdown in place, grouped by asset. Each failing rule links back to its asset with the rule focused, so you can fix it where it's defined.

Verify​

  • The validation's Failures card charts how many rules failed in each run over the last 30 days (a passing run is a short teal mark on the baseline); click a bar to open that run. The validations list shows the 30-day Health score. See Failures on the detail page.
  • Results lists the run; filter by the validation to see only its runs.

Next​

Via the API​

BASE=https://dq.example.com/api/v2/organizations/<org-id>/workspaces/<ws-id>
AUTH="Authorization: Bearer dqk_…"

# 1. Create the validation.
VAL=$(curl -s -X POST -H "$AUTH" -H "Content-Type: application/json" "$BASE/validations" \
  -d '{"data": {"name": "Nightly orders"}}' | jq -r .data.id)

# 2. Add all of an asset's current rules (rules added to it later don't join)...
curl -s -X PUT -H "$AUTH" -H "Content-Type: application/json" \
  "$BASE/validations/$VAL/assets/<asset-id>/rules" -d '{"data": {"mode": "all"}}'
#    ...or just some of another asset's rules.
curl -s -X PUT -H "$AUTH" -H "Content-Type: application/json" \
  "$BASE/validations/$VAL/assets/<other-asset-id>/rules" \
  -d '{"data": {"mode": "selected", "rule_ids": ["<rule-id>", "<rule-id>"]}}'

#    A new rule can join the validation as it's created:
curl -s -X POST -H "$AUTH" -H "Content-Type: application/json" \
  "$BASE/data-assets/<asset-id>/rules" \
  -d "{\"data\": {\"rule_type\": \"expect_column_values_to_not_be_null\",
                \"kwargs\": {\"column\": \"order_id\"}, \"validation_ids\": [\"$VAL\"]}}"

# 3. Run it (202 Accepted), then list its execution runs. An optional body runs
#    a subset: {"data": {"rule_ids": ["<rule-id>"]}}.
curl -s -X POST -H "$AUTH" "$BASE/validations/$VAL/run"
curl -s -H "$AUTH" "$BASE/validations/$VAL/execution-runs"

See Validations and Execution runs for the full model.