Skip to main content

Accept an AI rule suggestion

Goal​

You'll review the rules PLACEHOLDER Cloud's AI profiler proposed for an asset and append the good ones to a rule suite — without typing the expectation kwargs by hand.

Status: partial (PARITY-8) — single-row accept / dismiss UI is in flight. Bulk accept / dismiss (FEATURE-39) is shipped.

Prereqs​

  • The org has an LLMConfig set up (org settings → AI). The default fallback is the env-configured DEFAULT_LLM_* values; per-org overrides take precedence.
  • The asset has been profiled at least once. The worker enqueues rule suggestions automatically after every profile run.
  • Workspace editor role or higher.

Steps​

  1. Open the Suggestions route. The list shows every pending AISuggestion for the workspace, newest first.

  2. Scan the row. Each row shows:

    • Target asset.
    • Proposed expectation in human form (email should never be null).
    • Model rationale (a one-line explanation of why the LLM thinks this matters).
    • Confidence score (0.0 – 1.0).

  3. Click "Accept". A small modal opens with the resulting expectation pre-filled (column, kwargs).

  4. Pick a rule suite — either an existing one ("Add to users-validity") or "Create new suite" with a name.

  5. Click "Confirm". PLACEHOLDER Cloud appends the rule to the chosen suite (writes a new RuleSuite.suite.rules entry, preserving order) and sets the suggestion's status = accepted.

  6. Click "Run now" on the suite to validate the new rule immediately against the asset. If the rule fails the very first time, the LLM hallucinated; click Edit on the suite, refine or remove, and dismiss the suggestion with a note explaining what went wrong (it feeds back into HARDEN-25's feedback signal eventually).

Verify​

  • The suggestion drops out of the pending list and appears under Accepted.
  • The chosen rule suite now contains the new expectation.

Dismissing​

Click Dismiss instead of Accept. PLACEHOLDER Cloud asks for an optional reason and sets status = dismissed; the reason is stored on AISuggestion.flags for later review and (eventually) prompt tuning.

Bulk accept / dismiss (FEATURE-39)​

When a profile run on a 60-column table produces dozens of suggestions, single-row clicks add up. The Suggestions list shows a checkbox on each eligible row (status completed, not yet reviewed) and a Select all on this page checkbox in the toolbar. Once one or more rows are selected:

  • Accept N selected — pick a target rule suite once; the selected suggestions' rules are appended in a single call. Already-reviewed rows are skipped and reported back in the summary. Per-row failures (missing suggestion, incompatible kind, suite in another workspace) are reported individually — they don't block the rest of the batch.
  • Dismiss N selected — optional reason textarea, then confirm. Every selected row's review_status flips to rejected and the reason lands on each row's review_notes.

Both bulk endpoints require a recent password proof (step-up auth) — same bar as the single-row accept. Each affected suggestion produces its own audit_events row.

Cost control​

Every accepted / dismissed suggestion bills against the org's LLM cost budget when HARDEN-25 lands. Until then, the budget is the LLM provider's own.