Skip to main content

Architecture

DNS APUs — architecture

Two PC Engines APU2 boards in a 1U dual-slot 19" chassis are the household recursive DNS. Clients do not pick a box. DHCP (and the core intercept of rogue DNS) hands out 10.53.53.53. Each APU puts that address on lo and FRR advertises it as an OSPF stub. The CCR2116 core installs an ECMP pair and hashes each source/destination pair onto one arm.

This page is the map. The self-heal that withdraws a sick arm is Anycast self-heal. What still listens where, after HOST-APU-01, is Hardening and dashboard scrape.

Rack photos

Hardware photos are the original 2019 gallery shots from racked the apu2s. The software on the boards has moved on (Unbound + FRR anycast, not Pi-hole/Traefik); the metal is the same.

These are my apu2 DNServers running pihole on docker with traefik 2.0

These are my apu2 DNServers running pihole on docker with traefik 2.0

Dual APU2 19" rack unit in the cabinet (yellow uplinks on ETH0)

Dual APU2 19" rack unit in the cabinet (yellow uplinks on ETH0)

Case parts

  • the bare case (326744)
  • the power supplies (326747)
  • two fans (SUN-HA40201V4-1)
  • two fan mounts (311857)

The boards were remounted from the standard APU case; the original aluminium heat spreader does not reuse — use APUCOOL. The 326744 kit includes the power socket that still needs to be soldered to the APU2; polarity matters.

Anycast architecture

DNS APUs — anycast 10.53.53.53

Addressing

Box vlan89 (OOB) vlan53 (OSPF / Unbound) Other Grafana
apu01 10.9.8.206 192.168.53.3 wg0 172.16.75.1 (home WireGuard) http://10.9.8.206:3000/
apu02 10.9.8.207 192.168.53.2 — http://10.9.8.207:3000/
anycast — 10.53.53.53 on lo of each APU advertised via FRR OSPF —
CCR2116 core 10.9.8.252 192.168.53.1 ECMP for 10.53.53.53/32 —
hp02 dashboard 10.9.8.253 — also 172.16.62.253; UI :8787 —

OSPF/BFD peers on vlan53: CCR2116 192.168.53.1 ↔ apu01 192.168.53.3 and apu02 192.168.53.2. Both APUs run Ubuntu 24.04, Unbound (DNSSEC hardened), FRR, and BFD.

How a query lands

flowchart LR
  C[Clients / DHCP stubs] -->|"UDP/TCP 53 → 10.53.53.53"| CORE[CCR2116 core]
  CORE -->|"ECMP 10.53.53.53/32"| A1[apu01 Unbound]
  CORE -->|"ECMP 10.53.53.53/32"| A2[apu02 Unbound]
  A1 --- LO1["lo 10.53.53.53"]
  A2 --- LO2["lo 10.53.53.53"]
  CORE -->|"vlan53 OSPF + BFD"| A1
  CORE -->|"vlan53 OSPF + BFD"| A2

RouterOS hashes ECMP per source/destination pair. One source address samples one arm. That is why /dns probes the anycast from more than one hp02 address (172.16.62.253 and 10.9.8.253) — a single probe cannot prove both APUs.

The core also intercepts rogue DNS (dst-nat of UDP/TCP 53 not already aimed at 10.53.53.53, from sources not on DNS-SERVERS) and sends it to the anycast. Production DHCP DNS is 10.53.53.53, not a vlan89 Grafana address.

What each box runs

  • Unbound — recursive resolver, DNSSEC (harden-glue, harden-dnssec-stripped). After HOST-APU-01 it is not bound to 0.0.0.0; it listens on the fabric addresses (vlan89, vlan53, the anycast on lo, and on apu01 also wg0). ACL is fabric prefixes, not all of RFC1918.
  • FRR OSPF — advertises 10.53.53.53/32 as a stub of lo. Removing the address from lo withdraws it from OSPF within about a second; putting it back re-advertises. Both boxes are hardened with no zebra nexthop kernel enable after the Sep 2026 kernel-NHG blackholes.
  • Grafana :3000 — vlan89 only (10.9.8.206 / .207). Not on vlan53, not on wg0.
  • Prometheus :9090, node_exporter :9100, unbound-exporter :9167 — 127.0.0.1 only. The hp02 dashboard scrapes them over SSH as bodo@vlan89 to 127.0.0.1 (see the hardening page). LAN :9090 / :9100 connection-refuses on purpose.

Operator surfaces

Surface URL What it is
Grafana apu01 http://10.9.8.206:3000/ On-box dashboards, vlan89 only
Grafana apu02 http://10.9.8.207:3000/ Same
hp02 dashboard :8787 on hp02 (/apu, /dns, /docker, /sites) Fleet view. Metrics over SSH after HOST-APU-01
/apu DASH-32 / REQ-DASH32 Three angles: box (SSH), core (REST ECMP/OSPF), wire (UDP probes)
/dns REQ-D5 Unbound PromQL via SSH → 127.0.0.1:9090
/docker REQ-DF5 APU pulse via SSH → 127.0.0.1:9100 (no Docker on the APUs)

SSH to the APUs is as bodo, pubkey only, password auth off, root login off. Use vlan89 (10.9.8.206 / .207) — that path kept answering when vlan53 had no return route.

What this book is not

It is not a runbook to rewrite FRR, nftables, or Unbound on the live boxes. The applied shape lives in:

  • mikrotik-workspace/tools/handover/2026-09-15-apu-hardening.sh (HOST-APU-01)
  • mikrotik-workspace/scratch/anycast-hc/anycast-healthcheck.sh and tools/handover/anycast-hc/
  • mikrotik-workspace/tools/handover/2026-09-13-apu-anycast-healthcheck.sh (install)
  • hp02 mikrotik-dashboard (apu.py, dns_collect.py, docker_fleet.py)
Back to top