Skip to main content

Manage incidents

Status: new. The triage/assign/resolve lifecycle and timeline ship now. Notifications (assignee alert on open, resolution notice, daily digest of open incidents), resolution auto-link to the first passing run, and external-ticket (Jira/ServiceNow) bidirectional sync are planned under #411.

Goal​

You'll open an incident for a data-quality failure, triage it (acknowledge + assign), discuss it on the timeline, and resolve it with a note — all inside PLACEHOLDER Cloud. See Incidents for the data model.

Prereqs​

  • A role that can manage incidents: editor, admin, or owner. A viewer can read incidents but every mutate control is disabled.

How incidents start​

  • Automatically. When a checkpoint run fails, the worker opens an incident (source = auto) alongside the usual alert fan-out. Only one open auto-incident exists per checkpoint at a time, so a checkpoint failing on every run does not flood you — re-runs land on the one open incident.
  • Manually. Open one yourself for a failure you want to track.

Steps​

  1. Navigate to Incidents. The list shows each incident's title, severity, status, assignee, age (time since it opened), and source. It refreshes every 30 seconds.

  2. Open a new incident (optional). Click New incident, give it a title and severity, optionally a root-cause note, then Create incident. It starts in open.

  3. Open the detail panel. Click any row. You'll see the severity/status badges, the assignee, the root-cause and resolution notes, the lifecycle controls, and the timeline.

  4. Acknowledge. While the incident is open, click Acknowledge to move it to acknowledged. This stamps the acknowledged time and logs a status_change entry on the timeline.

  5. Comment. Type into the timeline's comment box and click Comment. Your email (from your authenticated session — never something you type) is recorded as the author.

  6. Resolve. Add a resolution note (optional) and click Mark resolved. This stamps the resolved time, records the note, and logs the final status_change. A resolved incident is read-only in the lifecycle panel.

  7. Delete (optional). The Delete control (red, in the detail footer) removes the incident and its timeline. It's gated on the same manage permission and stays red even when disabled for a viewer.

What's logged​

Every action writes an audit row (resource_type = "incident" / "incident_event") and an IncidentEvent on the timeline, so the full triage history is reconstructable.

What's not here yet​

The pieces below are tracked under #411 and are not in this release:

  • Notifications — no email to the assignee when an incident opens, no resolution notice, no daily digest of open incidents.
  • Resolution auto-link — you record a resolution note by hand; the incident is not automatically linked to (or closed by) the first passing run after resolution.
  • External ticket sync — incident status is not synced back to a linked Jira/ServiceNow ticket. (The outbound ticketing alert channels are a separate feature; see Alerts.)
  • SLA breach alerting — the age shown in the list is a display nicety; there is no SLA timer that fires when an incident ages out.