# Alerts and channels

> Route Falak alerts — failed deployments, offline agents, crash loops, backups, certificates — to e-mail, Slack, Discord, Telegram or signed webhooks.

Source: https://falak.sh/docs/observability/alerts/

Falak raises an **alert** when something needs attention. **Rules** decide which alerts go to which **channels**. Every alert also lands in the in-app notification center (the bell).

## 1. Create a channel

1. Open **Settings → Alert channels** → **Add** (`alerting.manage`, owners and admins).
2. Pick the type and fill in its settings.
3. Save, then send a test.

  
    | Setting | Rule |
    |---|---|
    | Recipients | 1–20 e-mail addresses |

    Needs working mail on the control plane (`MAIL_*`). See [Configuration](/docs/operations/configuration/#mail).
  
  
    | Setting | Rule |
    |---|---|
    | Webhook URL | `https://hooks.slack.com/services/…`, `/workflows/…` or `/triggers/…` |
  
  
    | Setting | Rule |
    |---|---|
    | Webhook URL | `https://discord.com/api/webhooks/…` (also `discordapp.com`, `canary.`, `ptb.`) |
  
  
    | Setting | Rule |
    |---|---|
    | Bot token | `123456:ABC…` from @BotFather |
    | Chat id | A numeric id (`-100…` for groups) or `@channelname` |
  
  
    | Setting | Rule |
    |---|---|
    | URL | `http(s)://…`, public hosts only (see below) |
    | Secret | 16–255 characters, used to sign deliveries |

    Each delivery carries these headers:

    | Header | Value |
    |---|---|
    | `X-Falak-Event` | The alert type, e.g. `deployments.failed` |
    | `X-Falak-Delivery` | Unique delivery id |
    | `X-Falak-Timestamp` | Unix timestamp |
    | `X-Falak-Signature` | `sha256=` + HMAC-SHA256(secret, `<timestamp>.<raw body>`) |

    ```php title="Verify a delivery (PHP)"
    $expected = 'sha256='.hash_hmac('sha256', $timestamp.'.'.$rawBody, $secret);
    abort_unless(hash_equals($expected, $request->header('X-Falak-Signature')), 401);
    ```

    Webhooks to loopback, private and link-local addresses are refused unless the operator sets `FALAK_ALERTING_ALLOW_PRIVATE_WEBHOOKS=true`.
  

![Settings → Alert channels: configured channels with their type and status.](./_images/settings-alert-channels.png)

## 2. Create a rule

**Settings → Alert rules** → **New rule**:

| Field | Rule |
|---|---|
| Name | Up to 100 characters |
| Event types | 1–50 patterns: an exact type (`deployments.failed`), a prefix wildcard (`deployments.*`) or `*` for everything |
| Minimum severity | `info`, `warning` or `critical` |
| Channels | Up to 20 channels |
| Rate limit | Optional: at most N deliveries per hour (1–1000) |
| Quiet hours | Optional start/end (`HH:MM`), timezone, days (1–7), and whether critical alerts still go through |

![Settings → Alert rules: rules with their event types, minimum severity and channels.](./_images/settings-alert-rules.png)

## Alert types

| Type | Label | Severity |
|---|---|---|
| `deployments.failed` | Deployment failed | critical |
| `deployments.rolled_back` | Site rolled back | warning |
| `builds.failed` | Build failed | warning |
| `fleet.agent_offline` | Server agent offline | critical |
| `fleet.agent_online` | Server agent back online | info |
| `fleet.agent_revoked` | Server agent revoked | warning |
| `fleet.agent_upgrade_failed` | Agent upgrade failed | warning |
| `fleet.agent_upgraded` | Agent upgraded after a failure | info |
| `servers.provisioned` | Server provisioned | info |
| `processes.crash_loop` | Process keeps crashing | critical |
| `processes.recovered` | Process running again | info |
| `network.firewall_failed` | Firewall apply failed | critical |
| `network.firewall_recovered` | Firewall applied again | info |
| `edge.certificate_failed` | Certificate install failed | critical |
| `edge.certificate_installed` | Certificate installed | info |
| `databases.backup_failed` | Database backup failed | critical |
| `databases.backup_recovered` | Database backups succeed again | info |
| `databases.restore_failed` | Database restore failed | critical |
| `databases.restore_succeeded` | Database restore finished | info |
| `insights.issue_opened` | New issue | warning |
| `insights.issue_regressed` | Issue regressed | warning |
| `insights.issue_resolved` | Issue resolved | info |
| `insights.threshold_breached` | Performance threshold breached | warning |
| `insights.heartbeat_missed` | Scheduled task missed | critical |

## History and notifications

- **Observability → Alerts** (or **Alert history**) lists every alert and its deliveries.
- The bell in the top bar shows in-app notifications for members with `alerting.view`.
- History older than 90 days is pruned (`FALAK_ALERTING_RETENTION_DAYS`).

## Limits

- Delivery to real Slack, Discord and Telegram is covered by unit and feature tests, not yet by the end-to-end suite.
- The webhook SSRF guard checks the host but does not resolve DNS, so DNS rebinding is not blocked.

## Next steps