Traefik Dashboard Retirement And Traefik Manager Rollout

Change metadataCopy Anchor Link

• Change id: CR-2026-08-15-traefik-dashboard-retirement
• Date and time: 2026-08-15, VPS / production
• Environment: VPS / production
• Requester and approver: repository owner, explicit approval in the active session
• Executor: Claude
• Status: completed; two self-inflicted outages during execution, both resolved

ScopeCopy Anchor Link

• Disable Traefik's built-in dashboard (api.dashboard: false) and remove its router.
• Keep the Traefik API on, container-network only, as Traefik Manager's data source.
• Deploy Traefik Manager at https://traefik-manager.perspective-v.com, NetBird-only.
• Move dynamic config to a host directory so the UI can write without touching Git.
• Enable file access logging and wire up the Certificates, Logs and Plugins tabs.
• Retire Netdata (covered in the monitoring evaluation bundle).

WhyCopy Anchor Link

The dashboard's only protection was admin-auth — a shared basic-auth file that also holds five CI credentials distributed to build pipelines. Anyone with a pipeline credential could read the full routing topology. Traefik Manager has real per-user authentication: bcrypt cost 12, optional TOTP 2FA, and OIDC/SSO.

Design decisionsCopy Anchor Link

API on, dashboard off. api.insecure: true binds the API on :8080 inside the container. There is no ports: entry for 8080, so it is reachable only from the proxy overlay — not the internet, not NetBird. Accepted trade-off: any container on that overlay can read routing topology. No secrets are exposed (basicAuth is reported as a usersFile path; no TLS private keys are served). Routing api@internal through a Traefik router was rejected as an alternative because Traefik Manager is a container and would be denied by netbird-only.

File-scoped config ownership. Traefik's file provider watches exactly one directory, so repo-managed and UI-managed config must share /var/lib/traefik/dynamic. Ownership is therefore split by file, not by directory:

Only managed.yml is mounted, so a UI edit cannot rewrite a version-controlled file, and install-dynamic-config.sh --check still detects drift on the repo-managed ones. Verified by write test: /data/traefik.yml returns Read-only file system; /data/dynamic.yml writes through to the host.

No Docker socket. The socket is only needed for multi-server management, which does not apply to a single node — and this is an internet-adjacent admin UI.

Data sources wired to the UICopy Anchor Link

Accepted risk — acme.jsonCopy Anchor Link

The ACME store is 568 KB and embeds the private key of every one of 45 certificates, plus the ACME account key. Read-only prevents modification but not disclosure: a compromise of the Traefik Manager container yields every TLS private key on the host.

Accepted 2026-08-15 because the container is NetBird-only, carries its own authentication, and holds no Docker socket. Revisit if any of those three change.

Accepted risk — access log contentsCopy Anchor Link

Access logging previously went to stdout only; it now also writes to /var/log/traefik/access.log. All request and response headers are dropped (fields.headers.defaultMode: drop) — without that, Authorization and Cookie values are logged verbatim.

Traefik cannot redact query strings, and this estate puts JWTs there (/notifications/hub?access_token=eyJ...), so log lines remain credential-bearing. Mitigated by root-only file permissions and /etc/logrotate.d/traefik: 7 compressed days, maxsize 100M. copytruncate is required — Traefik holds the file open and does not reopen it on SIGHUP, so a plain rotate would leave it writing to the renamed inode.

Do not lengthen retention without revisiting this.

Two outages caused during executionCopy Anchor Link

Both took every site down and both presented identically: Traefik healthy and listening, every route 404.

1. traefik.enable=true left on Traefik itself after its loadbalancer port was removed. The swarm provider fails with service "edge-edge-traefik" error: port is missing, and that error is not scoped to the one service — it aborts the entire provider configuration, dropping all routers. Traefik no longer routes to itself, so the label is now simply absent, with a warning comment in the manifest.

2. managed.yml seeded with empty maps (routers: {} / services: {} / middlewares: {}). Traefik rejects the file, and one bad file aborts the whole directory load — every @file middleware vanishes and every router referencing one is disabled. The seed is now comments-only until the manager writes real content.

Diagnostic for both: query http://127.0.0.1:8080/api/http/routers from inside the container. Routers showing disabled with middleware "…@file" does not exist means the file provider failed to load.

Other errors correctedCopy Anchor Link

• Version pinned to 1.10.1, not 1.1.0. GHCR's tag list omits 1.10.1; the digest for latest matches it. Pinning from the visible tag list alone would have downgraded.
• State volume at /app/config, not /app/data. /app/data does not exist in the image and is never written, so a volume mounted there silently persists nothing and every restart re-bootstraps with a fresh auto-generated password. This destroyed one owner password change before it was found. Persistence has since been verified across a redeploy: setup_complete: true, must_change_password: false, zero bootstrap events in the logs.

ValidationCopy Anchor Link

• 31/31 Swarm services healthy; panel_traefik-manager 1/1.
• Public routes unchanged: dbskc.com, nishatcolony.pk, gorsistudio.com, console. = 200, gitea. = 303, auth. = 308, api. swagger = 200.
• NetBird-only routes return 403 from the node itself: cockpit., beszel., traefik-manager.
• traefik.perspective-v.com returns 404 (dashboard gone).
• All four data sources readable inside the container; acme.json and access.log confirmed read-only, dynamic.yml confirmed writable.
• Access log writing and headers dropped as configured.
• operations/traefik/install-dynamic-config.sh --check → no drift.
• operations/monitoring/install-monitoring-config.sh --check → no drift.
• 0 failed systemd units.

Follow-upsCopy Anchor Link

• Retire the DNS A record for traefik.perspective-v.com — it now serves 404.
• TRAEFIK_DASHBOARD_HOST remains in the edge env files for the retained Compose layout only; annotated as legacy in place.
• Re-evaluate the acme.json mount if Traefik Manager ever gains a Docker socket, loses netbird-only, or is exposed publicly.