Skip to main content

Integrate PLACEHOLDER Cloud with your data catalog

Have a DataHub, Atlan, Data.world, or Alation catalog? Use the first-party connector for it instead of this generic webhook — it speaks the catalog's native API so you write zero middleware. See DataHub, Atlan, Data.world, and Alation. This page is for any other HTTP-shaped catalog (OpenMetadata, Collibra, a homegrown one).

Goal​

You'll wire PLACEHOLDER Cloud's validation-result stream into your enterprise data catalog (OpenMetadata, Collibra, or any other) so stakeholders see data-quality scores in the catalog they already use. PLACEHOLDER Cloud POSTs a stable JSON envelope to a webhook you control; you transform that envelope into your catalog's authoring contract.

Prereqs​

  • You're an owner of the organisation. The Catalog sync settings page is gated to the owner role, same as Data docs and SSO.
  • A webhook endpoint on your side that can receive HTTPS POST. This is typically a serverless function (Lambda, Cloud Run, Azure Function) or a small service in your own VPC that has credentials to call your catalog's API.

Steps​

  1. Navigate to Settings → Catalog sync.

  2. Enter a webhook URL. PLACEHOLDER Cloud will POST a JSON envelope to this URL after every validation result. Use https://. PLACEHOLDER Cloud refuses URLs that resolve to internal / metadata IPs at write time (SSRF guard).

  3. Optionally set a signing secret. When set, PLACEHOLDER Cloud adds an X-DQ-Signature: sha256=<hex> header where the signature is HMAC-SHA256 over the exact body bytes. Verify it on your side before trusting the payload — without verification, anyone who knows your webhook URL can forge events. The secret must be at least 32 characters (a shorter one is rejected with a 400); generate one with openssl rand -hex 32.

  4. Click "Send test event" to fire a synthetic payload at the webhook so you can verify the receiver is wired up correctly without waiting for a real checkpoint. The test event carries "event": "validation_result.test" so you can distinguish a probe from a real fire and skip writing it through to the catalog if you want.

  5. From then on, PLACEHOLDER Cloud POSTs a validation_result envelope after every checkpoint commit. The Catalog sync settings page shows the timestamp of the last successful fire.

The payload shape​

{
  "event": "validation_result",
  "schema_version": "1",
  "org_id": "11111111-1111-1111-1111-111111111111",
  "result": {
    "id": "22222222-2222-2222-2222-222222222222",
    "checkpoint": {
      "id": "33333333-3333-3333-3333-333333333333",
      "name": "nightly_orders_checkpoint"
    },
    "asset": {
      "id": "44444444-4444-4444-4444-444444444444",
      "name": "orders",
      "datasource_id": "55555555-5555-5555-5555-555555555555"
    },
    "rule_suite": {
      "id": "66666666-6666-6666-6666-666666666666",
      "name": "orders_v2"
    },
    "success": false,
    "summary": {
      "total": 12,
      "passed": 11,
      "failed": 1,
      "severity_max": "warning"
    },
    "run_at": "2026-05-14T12:00:00+00:00",
    "deep_link": "https://app.example.com/results/22222222-2222-2222-2222-222222222222"
  }
}

Field notes:

  • schema_version is the contract version of the envelope. PLACEHOLDER Cloud bumps it when fields are renamed or removed; additions of new optional fields don't bump the version. Pin your adapter to a specific version and update on your own cadence.
  • summary.severity_max is the worst per-expectation severity that failed in this run: info (all-pass), warning, error, or critical. Catalogs usually surface one badge per asset rather than 200 — this is the right value to drive the badge.
  • asset / checkpoint / rule_suite may be null if the referenced row was deleted between the run and the webhook fire. Defensive adapters write null-tolerant code.
  • deep_link is present only when PUBLIC_BASE_URL is configured on the PLACEHOLDER Cloud deployment. It points at the validation-result page in the PLACEHOLDER Cloud UI so the catalog can offer a "open in PLACEHOLDER Cloud" button.

Verifying the signature​

import hashlib
import hmac

def verify(body_bytes: bytes, header: str, secret: str) -> bool:
    """body_bytes is the raw HTTP body. header is the `X-DQ-Signature` value."""
    if not header.startswith("sha256="):
        return False
    received = header.split("=", 1)[1]
    expected = hmac.new(secret.encode(), body_bytes, hashlib.sha256).hexdigest()
    return hmac.compare_digest(received, expected)

Use hmac.compare_digest (not ==) to avoid timing attacks. The signature is computed over the bytes PLACEHOLDER Cloud sent, so verify it before deserialising — don't re-serialise the parsed JSON and hash that, or whitespace differences will break verification.

Example transformers​

Atlan​

Atlan exposes a GraphQL endpoint that accepts a setMetadata mutation against an existing asset. Map PLACEHOLDER Cloud's asset.name (or asset.id, kept as a stable custom attribute on the Atlan side) to the catalog's qualifiedName, then write dqStatus and dqScore as custom string / number fields:

def to_atlan(envelope):
    r = envelope["result"]
    return {
        "query": "mutation($attrs: AssetAttributes!) { updateAsset(input: $attrs) { guid } }",
        "variables": {
            "attrs": {
                "qualifiedName": r["asset"]["name"],  # or look up by id
                "customAttributes": {
                    "dqStatus": "passed" if r["success"] else "failed",
                    "dqSeverity": r["summary"]["severity_max"],
                    "dqFailedCount": r["summary"]["failed"],
                    "dqLastRunAt": r["run_at"],
                    "dqDeepLink": r["deep_link"],
                },
            }
        },
    }

DataHub​

DataHub's REST API takes a dataQuality aspect on a Dataset URN. Build the URN from your datasource metadata and emit an MCP (MetadataChangeProposal):

def to_datahub_mcp(envelope, *, platform: str):
    r = envelope["result"]
    urn = f"urn:li:dataset:(urn:li:dataPlatform:{platform},{r['asset']['name']},PROD)"
    return {
        "entityType": "dataset",
        "entityUrn": urn,
        "changeType": "UPSERT",
        "aspectName": "dataQuality",
        "aspect": {
            "value": {
                "passed": r["summary"]["passed"],
                "failed": r["summary"]["failed"],
                "severity": r["summary"]["severity_max"],
                "lastRunAt": r["run_at"],
                "externalUrl": r["deep_link"],
            }
        },
    }

OpenMetadata​

OpenMetadata's /api/v1/dataQuality/testCases/{name}/testCaseResult endpoint accepts a TestCaseResult payload. Map one PLACEHOLDER Cloud rule suite to one OpenMetadata test case:

def to_openmetadata_result(envelope):
    r = envelope["result"]
    return {
        "timestamp": int(__import__("time").time() * 1000),
        "testCaseStatus": "Success" if r["success"] else "Failed",
        "result": (
            f"{r['summary']['passed']}/{r['summary']['total']} passed. "
            f"Worst severity: {r['summary']['severity_max']}."
        ),
        "sampleData": r["deep_link"],
    }

Notes​

  • The webhook URL is stored as plain text in public.organisations.catalog_sync_webhook_url. The URL alone isn't a credential against the receiver — knowing the URL just lets you POST events at it, which is what the signing secret protects against (set one if your receiver doesn't otherwise authenticate the caller).
  • The signing secret is HARDEN-16 envelope-encrypted at rest under the tenant DEK. The plaintext only exists in memory during a dispatch; rotating it is a single PATCH against /api/v1/organizations/{org_id}/catalog-sync-config.
  • Retries: PLACEHOLDER Cloud retries up to 3 times with exponential backoff on transport errors and 5xx responses. A persistently-broken receiver drops the event after the final attempt rather than dead-lettering forever — fix the receiver and the next checkpoint commit re-arms the feed.
  • Out of scope: bi-directional sync (pulling glossary terms back from the catalog into PLACEHOLDER Cloud) and per-asset routing rules. The single org-level webhook is the minimum surface that closes the catalog-feed gap; the broader sync work tracks separately.
  • See FEATURE-25 for the tracking ticket.