Alerts and notification rules
The hub raises alerts on its own from what the agents report. One alert exists per machine and problem; it resolves itself when the condition clears, or by hand. Nobody is notified until a company has a notification rule.
What raises an alert
| kind | severity | opens when | resolves when |
|---|---|---|---|
| offline | critical | no contact past the grace period (OFFLINE_GRACE_MIN, default 5 minutes) | the machine connects again |
| check | the check's own severity | a custom check fails | the check recovers |
| disk | warning; critical at 97% | a volume or ZFS pool crosses 90% | a heartbeat shows under 88% |
| zpool health | critical | a ZFS pool leaves ONLINE | a heartbeat shows ONLINE |
| reboot | notice | a machine reboots | auto-resolved |
| network path | warning | the external address or, on routers, the active WAN changes (failover to 4G) | the earlier address or path comes back |
Lifecycle
open → acked → resolved. Re-occurrence bumps a count on the same alert; only an escalation (warning to critical) re-announces. The first notification waits ALERT_NOTIFY_DELAY_SEC (default 60) so BE AI's assessment from the automatic investigation can ride along.
Notification rules
Per company, under Companies → manage → alerts. A rule is: which machines (by tag; empty means all), which kinds (empty means all), a minimum severity, and contacts. Several rules can match; each contact is told once. A test button sends a sample through the rule.
Channels
| channel | setup on the hub | notes |
|---|---|---|
| webhook | none | JSON POST to any URL: the alert, the machine and the assessment |
EMAIL_API_KEY, EMAIL_FROM; EMAIL_API_URL for a different provider | Emailit by default; any API taking {from, to, subject, text} | |
| SMS | TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN, TWILIO_FROM | one-way; replies are not read |
| phone call | same Twilio settings | reads the alert out and repeats it once |
Unconfigured channels are skipped and say so. Every delivery, sent, failed or skipped, is a row under the company's notifications.
Acknowledging and discussing
Ack an alert to stop re-announcements while you work on it. Discuss opens the machine's conversation with BE AI with the alert as context.
Try it on one machine first.
The install line takes a minute. Uninstall the agent and you are out.