Skip to main content

Quickstart

Five minutes from git clone to a green validation result against a local PLACEHOLDER Cloud. Pick the path that matches what you're doing — for the longer walk-throughs, see Self-host and Run your first checkpoint.

Two paths​

PathWhenWhat you need
A. Docker-onlyYou just want to try it.Docker. That's it.
B. Local devYou'll edit Python or UI code and want hot-reload.Docker + Python 3.12 + uv + Node 22.

A. Docker-only​

The all-in-one compose brings up Postgres + Redis + API + worker + UI behind one command, runs the public Alembic migrations, and seeds a dev org for you.

git clone https://github.com/your-org/dq-cloud.git && cd dq-cloud
make compose-up

That's it. Open http://localhost:3000 and log in as admin@dqcloud.dev / devpassword. The seed script created org 00000000-0000-0000-0000-000000000001, workspace …002, a personal API key, and an agent token.

Tear down:

make compose-down                                   # stop, keep data
docker volume rm dqcloud-local_dq_postgres_data     # wipe the DB

For production deploys (managed Postgres + Redis, registry-pulled images), see Deploy with Docker. The same three images can also be pulled from GHCR (ghcr.io/<owner>/dq-cloud-{api,worker,ui}) — the CI workflow publishes them on every push to main and every v* tag.

B. Local dev — hot-reload on Python + UI​

This is the path used by contributors. The Python services run on your host so a save reloads them; only Postgres + Redis come from Docker.

1. Boot the stack​

git clone https://github.com/your-org/dq-cloud.git && cd dq-cloud
cp .env.example .env

# Generate the two secrets the API refuses to start without.
python3 - <<'PY' >> .env
import base64, secrets
print(f"SECRET_KEY={secrets.token_hex(32)}")
print(f"ENCRYPTION_KEY={base64.urlsafe_b64encode(secrets.token_bytes(32)).decode().rstrip('=')}")
PY

make dev          # docker compose up: Postgres on :5432, Redis on :6379
make migrate      # public-schema alembic upgrade head

If you already have a Postgres on :5432, either stop it or set DB_URL in .env to a free port. The Docker container shares the same default port and host-side Postgres wins.

2. Seed a dev org​

uv run python scripts/seed_dev.py

This creates org 00000000-0000-0000-0000-000000000001, workspace …002, user admin@dqcloud.dev / devpassword, and a personal API key + agent token. Safe to re-run.

3. Start the API, worker, and UI​

Three terminals (or background them):

# Terminal 1 — API on :8000
uv run uvicorn dq_cloud_api.main:app --reload --port 8000

# Terminal 2 — ARQ worker
uv run arq dq_cloud_worker.main.WorkerSettings

# Terminal 3 — Vite dev server on :3000 (proxies /api → :8000)
cd ui && npm install && npm run dev

Run a checkpoint end-to-end​

The fastest path is the UI: open http://localhost:3000, log in with admin@dqcloud.dev / devpassword, then click through Datasources → New → Discover assets → Rule suites → New → Create checkpoint → Run now. Detailed flow in Run your first checkpoint.

If you'd rather drive it from a script, here's the same path against the seeded org via curl:

ORG=00000000-0000-0000-0000-000000000001
WS=00000000-0000-0000-0000-000000000002
JWT=$(curl -s -X POST localhost:8000/auth/login \
    -H "Content-Type: application/json" \
    -d '{"email":"admin@dqcloud.dev","password":"devpassword"}' \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["access_token"])')

DS=$(curl -s -X POST -H "Authorization: Bearer $JWT" -H "Content-Type: application/json" \
    localhost:8000/api/v2/organizations/$ORG/workspaces/$WS/datasources \
    -d '{"name":"local-pg","type":"postgres","execution_mode":"direct",
         "config_encrypted":{"connection_string":"postgresql://dqcloud:dqcloud@localhost:5432/dqcloud"}}' \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["data"]["id"])')

ASSET=$(curl -s -X POST -H "Authorization: Bearer $JWT" -H "Content-Type: application/json" \
    localhost:8000/api/v1/organizations/$ORG/workspaces/$WS/data-assets \
    -d "{\"data\":{\"datasource_id\":\"$DS\",\"name\":\"public.organisations\",
         \"type\":\"table\",\"config\":{\"schema\":\"public\",\"table\":\"organisations\"}}}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["data"]["id"])')

SUITE=$(curl -s -X POST -H "Authorization: Bearer $JWT" -H "Content-Type: application/json" \
    localhost:8000/api/v1/organizations/$ORG/workspaces/$WS/rule-suites \
    -d '{"data":{"name":"orgs-baseline","suite":{"rules":[
         {"rule_type":"unexpected_rows_expectation",
          "kwargs":{"query":"SELECT * FROM {batch} WHERE id IS NULL",
                    "description":"id required","severity":"error"}}]}}}' \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["data"]["id"])')

curl -s -X POST -H "Authorization: Bearer $JWT" -H "Content-Type: application/json" \
    localhost:8000/api/v1/organizations/$ORG/workspaces/$WS/validation-configs \
    -d "{\"data\":{\"data_asset_id\":\"$ASSET\",\"rule_suite_id\":\"$SUITE\",\"name\":\"vc\"}}" > /dev/null

CP=$(curl -s -X POST -H "Authorization: Bearer $JWT" -H "Content-Type: application/json" \
    localhost:8000/api/v1/organizations/$ORG/workspaces/$WS/checkpoints \
    -d "{\"data\":{\"name\":\"orgs-cp\",
         \"checkpoint_config\":{\"data_asset_id\":\"$ASSET\",\"rule_suite_id\":\"$SUITE\",\"name\":\"orgs-cp\"}}}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["data"]["id"])')

curl -s -X POST -H "Authorization: Bearer $JWT" \
    localhost:8000/api/v1/organizations/$ORG/workspaces/$WS/checkpoints/$CP/run

Within a couple of seconds the worker picks the job off Redis, hydrates the asset + suite, spawns a sandboxed GX subprocess against your DSN, and writes a validation_results row. Refresh the UI and you'll see one passing run.

Verify​

curl -s -H "Authorization: Bearer $JWT" \
    localhost:8000/api/v1/organizations/$ORG/workspaces/$WS/validation-results \
  | python3 -m json.tool | head -20

You should see "success": true on the first result.

Next​