Push Notifications for Everything: ntfy as the Homelab Alert Bus

A monitoring stack that only shows dashboards is a monitoring stack you check when something's already broken. The real value of observability is knowing something went wrong before you notice it yourself, ideally via a push notification on your phone while you're away from the keyboard.

Here's how I built a complete alerting pipeline for my homelab using ntfy as the notification hub. Two independent alert paths (Prometheus Alertmanager and Uptime Kuma) both converge on ntfy, which relays notifications to my phone in seconds. The whole thing is declarative, self-hosted, and costs nothing.

Why ntfy?

ntfy (pronounced "notify") is a self-hosted push notification server. You publish messages to topics via HTTP, and subscribers receive them instantly on web, mobile, or CLI. It's dead simple: curl -d "Server is down" ntfy.sh/my-topic sends a push notification.

Why self-host it instead of using PagerDuty, Slack, or OpsGenie?

• No SaaS dependency. If my cluster can reach the internet, it can send notifications. No API keys to rotate, no pricing tiers, no vendor lock-in.
• Works from inside the cluster. Alertmanager can reach ntfy via a Kubernetes Service (ntfy.ntfy.svc.cluster.local). No external network path required for the initial delivery.
• Free. The self-hosted version is fully featured. The upstream relay to ntfy.sh for FCM/APNs push delivery is also free.
• Universal. Any tool that can send an HTTP POST can send an ntfy notification. No protocol adapters needed.

The ntfy Deployment

ntfy runs as a single lightweight pod in my cluster:

containers:
  - name: ntfy
    image: binwiederhier/ntfy:v2.17.0
    args: ["serve"]
    env:
      - name: NTFY_BASE_URL
        value: "https://ntfy.naviauxlab.com"
      - name: NTFY_BEHIND_PROXY
        value: "true"
      - name: NTFY_UPSTREAM_BASE_URL
        value: "https://ntfy.sh"
      - name: NTFY_CACHE_FILE
        value: "/var/lib/ntfy/cache.db"
    resources:
      requests:
        cpu: 10m
        memory: 32Mi
      limits:
        cpu: 100m
        memory: 128Mi

The key configuration is NTFY_UPSTREAM_BASE_URL: https://ntfy.sh. This tells my self-hosted ntfy instance to relay messages through the public ntfy.sh server, which handles Firebase Cloud Messaging (FCM) and Apple Push Notification service (APNs) delivery. Without this, mobile push notifications wouldn't work: your phone's ntfy app needs the upstream relay to receive notifications when the app isn't in the foreground.

Resource usage is minimal: 10m CPU, 32 Mi RAM. It's one of the lightest workloads in the cluster.

Alert Path 1: Prometheus Alertmanager → ntfy

The first alert path handles infrastructure-level alerts from Prometheus. The chain looks like this:

Prometheus (detects condition)
  → Alertmanager (groups + routes alerts)
    → ntfy-alertmanager bridge (translates webhook → ntfy format)
      → ntfy (stores + relays to devices)

Custom Alert Rules

I define PrometheusRules for the conditions I actually care about:

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: homelab-alerts
  namespace: monitoring
spec:
  groups:
    - name: flux.rules
      rules:
        - alert: FluxKustomizationNotReady
          expr: gotk_reconcile_condition{type="Ready",status="False",kind="Kustomization"} == 1
          for: 15m
          labels:
            severity: warning
          annotations:
            summary: "Flux Kustomization {{ $labels.name }} not ready"

    - name: longhorn.rules
      rules:
        - alert: LonghornVolumeFaulted
          expr: longhorn_volume_robustness == 3
          for: 1m
          labels:
            severity: critical
          annotations:
            summary: "Longhorn volume {{ $labels.volume }} is FAULTED"

    - name: certmanager.rules
      rules:
        - alert: CertificateExpiringSoon
          expr: certmanager_certificate_expiration_timestamp_seconds - time() < 14 * 24 * 3600
          for: 1h
          labels:
            severity: warning
          annotations:
            summary: "Certificate {{ $labels.name }} expiring within 14 days"

Three categories: Flux health (GitOps reconciliation), Longhorn storage (volume degradation, capacity), and cert-manager (certificate expiry). The for duration prevents transient conditions from firing alerts: a Flux Kustomization that's not-ready for 15 minutes is a problem; one that blips for 30 seconds during reconciliation is normal.

AlertmanagerConfig

Alertmanager routes alerts to the ntfy bridge via a webhook:

apiVersion: monitoring.coreos.com/v1alpha1
kind: AlertmanagerConfig
metadata:
  name: homelab-notifications
  namespace: monitoring
spec:
  route:
    receiver: ntfy
    groupBy: ["alertname", "namespace"]
    groupWait: 30s
    groupInterval: 5m
    repeatInterval: 4h
    matchers:
      - matchType: "!="
        name: alertname
        value: Watchdog
      - matchType: "!="
        name: alertname
        value: InfoInhibitor
  receivers:
    - name: ntfy
      webhookConfigs:
        - url: "http://ntfy-alertmanager.ntfy.svc.cluster.local:80"
          sendResolved: true

The configuration:

• groupBy: Groups alerts by name and namespace so you don't get 10 separate notifications for the same problem
• groupWait: 30s: Waits 30 seconds after the first alert before sending, to batch related alerts
• repeatInterval: 4h: Repeats unresolved alerts every 4 hours, frequent enough to keep attention, not so frequent it becomes noise
• sendResolved: true: Sends a notification when the alert clears, so you know the problem is fixed
• matchers: Filters out Watchdog (a synthetic "I'm alive" alert) and InfoInhibitor (a meta-alert); neither is actionable

The ntfy-alertmanager Bridge

Alertmanager speaks a webhook format that ntfy doesn't natively understand. The ntfy-alertmanager bridge translates between them:

containers:
  - name: ntfy-alertmanager
    image: xenrox/ntfy-alertmanager:0.4.0
    ports:
      - containerPort: 8080
    resources:
      requests:
        cpu: 5m
        memory: 10Mi
      limits:
        cpu: 50m
        memory: 32Mi

Its configuration maps Alertmanager severity labels to ntfy priority levels and emoji tags:

config.scfg: |
  http-address :8080
  log-level info
  alert-mode multi
  ntfy {
    topic alerts
    server http://ntfy.ntfy.svc.cluster.local
  }
  labels {
    severity "critical" {
      priority 5
      tags "rotating_light"
    }
    severity "warning" {
      priority 3
      tags "warning"
    }
  }
  resolved {
    tags "resolved,white_check_mark"
    priority 1
  }

Critical alerts get maximum priority (your phone will break through Do Not Disturb) with a red rotating light emoji. Warnings get normal priority. Resolved alerts get minimum priority with a green checkmark. The visual distinction makes it easy to triage from a phone lock screen.

Alert Path 2: Uptime Kuma → ntfy

The second alert path handles availability monitoring. Uptime Kuma checks whether services are actually responding to HTTP requests, and sends notifications through ntfy when they go down.

What makes this interesting is AutoKuma, a Kubernetes controller that syncs monitor definitions from CRDs to Uptime Kuma. Instead of clicking through the Uptime Kuma web UI to create monitors, I declare them as Kubernetes resources:

apiVersion: autokuma.bigboot.dev/v1
kind: KumaEntity
metadata:
  name: grafana
  namespace: uptime-kuma
spec:
  config:
    type: http
    name: "Grafana"
    url: "http://kube-prometheus-stack-grafana.monitoring.svc.cluster.local:80"
    interval: "60"
    maxretries: "3"
---
apiVersion: autokuma.bigboot.dev/v1
kind: KumaEntity
metadata:
  name: ntfy
  namespace: uptime-kuma
spec:
  config:
    type: http
    name: "ntfy"
    url: "http://ntfy.ntfy.svc.cluster.local:80/v1/health"
    interval: "60"
    maxretries: "3"
---
apiVersion: autokuma.bigboot.dev/v1
kind: KumaEntity
metadata:
  name: public-site
  namespace: uptime-kuma
spec:
  config:
    type: http
    name: "Public Site (naviauxlab.com)"
    url: "https://steven.naviauxlab.com"
    interval: "120"
    maxretries: "3"

This is GitOps for monitoring targets. Adding a new monitor means adding a YAML resource and pushing to Git. Deleting a monitor means removing the resource. AutoKuma watches these CRDs and synchronizes them to Uptime Kuma every 30 seconds.

The Monitor Coverage

I have 26 monitors across three categories:

Infrastructure services: Grafana, Prometheus, Alertmanager, Traefik dashboard, Longhorn UI, Authentik, cert-manager, Speedtest exporter

Application services: Ghost (this blog), Miniflux, Mealie, Navidrome, Linkding, Audiobookshelf, IT-Tools, Stirling-PDF, Vaultwarden, Actual Budget, Homepage, ntfy

External endpoints: Public site via Cloudflare Tunnel, external Authentik, external Grafana, verifying that Cloudflare Tunnel is forwarding traffic correctly

Internal monitors use cluster-internal Service URLs (http://service.namespace.svc.cluster.local:port), which means they test the full Kubernetes networking path: DNS resolution, Service routing, and pod health. External monitors use the public URLs, testing the full Cloudflare Tunnel → Traefik → Service → Pod path.

AutoKuma Architecture

AutoKuma runs as a sidecar-style controller alongside Uptime Kuma:

containers:
  - name: autokuma
    image: ghcr.io/bigboot/autokuma:2.0.0
    env:
      - name: AUTOKUMA__KUMA__URL
        value: http://uptime-kuma.uptime-kuma.svc.cluster.local:80
      - name: AUTOKUMA__KUBERNETES__ENABLED
        value: "true"
      - name: AUTOKUMA__DOCKER__ENABLED
        value: "false"
      - name: AUTOKUMA__TAG_NAME
        value: AutoKuma
      - name: AUTOKUMA__TAG_COLOR
        value: "#42C0FB"
      - name: AUTOKUMA__ON_DELETE
        value: delete
      - name: AUTOKUMA__SYNC_INTERVAL
        value: "30"

Key behaviors:

• Kubernetes-native: Watches KumaEntity CRDs instead of Docker labels
• Tagged management: All AutoKuma-created monitors get a distinctive blue "AutoKuma" tag, so you can tell which monitors are managed vs manually created
• Delete on remove: When you delete a KumaEntity from Git, AutoKuma removes the corresponding monitor from Uptime Kuma
• Init container: Waits for Uptime Kuma to be healthy before starting sync (prevents race conditions on pod startup)

Uptime Kuma is configured to send notifications via ntfy when a monitor goes down. The notification path is direct: Uptime Kuma → ntfy → phone. No bridge needed since Uptime Kuma has native ntfy support.

The Complete Notification Chain

Putting it all together, here are the two alert paths:

Path 1: Infrastructure Alerts
  Prometheus (evaluates rules every 60s)
    → Alertmanager (groups by alertname + namespace, waits 30s)
      → ntfy-alertmanager bridge (maps severity → priority + emoji)
        → ntfy pod (stores message, relays to upstream)
          → ntfy.sh (FCM/APNs relay)
            → Phone notification

Path 2: Availability Monitoring
  Uptime Kuma (checks 26 endpoints every 60-120s)
    → ntfy pod (on status change: up → down or down → up)
      → ntfy.sh (FCM/APNs relay)
        → Phone notification

From the moment a condition is detected to the notification arriving on my phone: typically 30-90 seconds for Path 1 (Alertmanager groupWait + relay), and 10-30 seconds for Path 2 (Uptime Kuma's faster path).

The Ingress Split

ntfy's ingress configuration is more nuanced than most apps because it serves two audiences: the web UI (which should be authenticated) and the API (which mobile apps and internal services need unauthenticated access to).

I use Traefik IngressRoute with priority-based routing:

# API + topic paths: NO forward-auth (priority 100)
routes:
  - match: Host(`ntfy.naviauxlab.com`) && (PathPrefix(`/v1/`) || PathPrefix(`/alerts`) || PathPrefix(`/homelab`))
    kind: Rule
    services:
      - name: ntfy
        port: 80

# Web UI: WITH forward-auth (priority 10)
routes:
  - match: Host(`ntfy.naviauxlab.com`)
    kind: Rule
    services:
      - name: ntfy
        port: 80
    middlewares:
      - name: authentik-forwardauth
        namespace: auth

Higher priority routes match first. API paths (/v1/, /alerts, /homelab) get served without authentication: the ntfy mobile app needs /v1/info to discover upstream relay capabilities and register for push delivery. Everything else falls through to the lower-priority route, which requires Authentik SSO.

Network Policies for the Alert Chain

The notification chain spans three namespaces (monitoring, ntfy, uptime-kuma), which means network policies need to explicitly allow the cross-namespace traffic:

# Alertmanager → ntfy-alertmanager bridge (monitoring → ntfy namespace)
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-alertmanager-egress
  namespace: monitoring
spec:
  endpointSelector:
    matchLabels:
      app.kubernetes.io/name: alertmanager
  egress:
    - toEndpoints:
        - matchLabels:
            "k8s:io.kubernetes.pod.namespace": ntfy
            app: ntfy-alertmanager
      toPorts:
        - ports:
            - port: "8080"
              protocol: TCP

# ntfy-alertmanager bridge ← Alertmanager (ntfy namespace allows from monitoring)
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-bridge-ingress-from-alertmanager
  namespace: ntfy
spec:
  podSelector:
    matchLabels:
      app: ntfy-alertmanager
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: monitoring
      ports:
        - protocol: TCP
          port: 8080

Every hop in the notification chain has a matching pair of policies: an egress rule on the sender and an ingress rule on the receiver. Miss either side and the traffic gets dropped silently.

What I'd Do Differently

If I were starting over, I'd add a few things:

• Alert deduplication across paths. Right now, if Grafana goes down, I get a Prometheus alert (service health) AND an Uptime Kuma notification (HTTP check failed). Two notifications for one problem. Solving this would require either shared state between Alertmanager and Uptime Kuma or a dedup layer in ntfy.
• Escalation. Critical alerts should repeat more aggressively than warning alerts. The current 4-hour repeat interval is a compromise.
• Runbook links. Alert annotations could include links to troubleshooting runbooks, surfaced in the ntfy notification body.

But for a homelab running on a single laptop, getting a phone notification within a minute of any service going down (with clear severity, actionable context, and automatic resolution notifications) is more than enough.

The Full Stack

Stepping back, the complete observability and alerting stack in this cluster is:

• Metrics: Prometheus (60s scrape, 30d retention, 768 Mi limit)
• Visualization: Grafana (dashboards, OIDC auth via Authentik)
• Alerting: Alertmanager → ntfy-alertmanager bridge → ntfy → phone
• Availability: Uptime Kuma (26 monitors via AutoKuma CRDs) → ntfy → phone
• Push delivery: Self-hosted ntfy with ntfy.sh upstream relay for FCM/APNs

Every component is declared in Git, deployed by Flux, hardened with network policies, and monitored by the other components. Prometheus monitors ntfy. Uptime Kuma monitors Prometheus. If either alert path dies, the other one catches it.

That's the kind of operational resilience that makes a homelab worth the complexity.

This post is part of a series documenting my homelab build on Talos Linux. Previous posts cover the platform layer, GitOps with Flux, observability, Cloudflare Tunnel, Authentik SSO, production hardening, network policies, and Renovate. You can find the full series on the blog index.