Already Implemented

• Fresh VPS provisioned on Contabo (Ubuntu 24.04).
• Docker installed and configured.
• Docker Compose installed and configured.
• zsh installed.
• Shared Docker proxy network created for Traefik-routed stacks.
• Edge stack deployed and running (Traefik, Portainer, Kener).
• Edge stack runtime artifacts are split into runtime/stacks/infrastructure/edge/docker-compose.traefik.yml, runtime/stacks/infrastructure/edge/docker-compose.portainer.yml, and runtime/stacks/infrastructure/edge/docker-compose.kener.yml.
• Edge VPS env artifacts are split into per-service folders under runtime/environments/vps/infrastructure/edge/{traefik,portainer,kener}.
• Edge dev env templates are split into per-service folders under runtime/environments/dev/infrastructure/edge/{traefik,portainer,kener}.
• Traefik and Portainer were recreated from split edge artifacts in the approved change window and verified running.
• Portainer split runtime uses portainer/portainer-ee:2.39.1 to preserve compatibility with the existing Business Edition data volume.
• Kener runtime env scaffolding is prepared in runtime/environments/vps/infrastructure/edge/kener/.env and runtime/environments/vps/infrastructure/edge/kener/kener.env (host, origin, Redis URL template, SMTP values, and secret placeholders).
• Kener overlay deployment remains active on VPS from runtime/stacks/infrastructure/edge using the current split edge model.
• Kener container is running and healthy (rajnandan1/kener:v4.0.16) alongside existing edge services.
• Kener public route validation from server source returns HTTP 200 at https://kener.perspective-v.com.
• Kener backend healthcheck returns HTTP 200 with body ok on direct container endpoint (/healthcheck).
• Kener overlay keeps fallback private-only middleware as a commented NetBird line in runtime/stacks/infrastructure/edge/docker-compose.kener.yml.
• Uptime Kuma was retired completely from edge runtime: service removed from edge compose artifacts, container removed, data volume removed, and image removed from VPS.
• Traefik upgraded to v3.6 and pinned with DOCKER_API_VERSION=1.40 for Docker API compatibility.
• Let's Encrypt resolver moved to static Traefik config and verified active.
• NetBird deployed in Existing Traefik mode using combined netbird-server + dashboard.
• NetBird dashboard/API reachable at netbird.perspective-v.com.
• Traefik dashboard is protected by NetBird source allowlist plus Traefik basic auth; Portainer remains NetBird-only with built-in panel authentication; Kener is public and uses built-in panel authentication.
• Server-side NetBird-address validation confirms split admin-route behavior: Traefik dashboard returns HTTP 401 with basic-auth challenge on NetBird path while Portainer returns HTTP 200 app route; default-source checks return HTTP 403 (recorded in docs/operations/validations/2026-04-15-netbird-route-validation.mdx).
• Platform stack deployed from runtime/stacks/infrastructure/platform with Redis and RabbitMQ.
• Redis Insight is deployed in platform stack with active container/runtime env keys; route checks show HTTP 403 from default source and HTTP 200 on NetBird-address path (docs/operations/validations/2026-04-15-netbird-route-validation.mdx).
• Platform Redis host port is published on 6379 for developer access via redis.perspective-v.com, while production containers continue using internal Docker host redis.
• Operations stack deployed from runtime/stacks/infrastructure/operations with Watchtower running, DOCKER_API_VERSION=1.40, and Sunday 03:00 UTC schedule (0 0 3 * * 0).
• Operations stack now runs dual scoped Watchtower services: baseline watchtower (scope=none, weekly schedule) and watchtower-fast (scope=fast, 1-hour interval 3600).
• Fast Watchtower now reads Gitea registry auth from the versioned Swarm secret watchtower_gitea_registry_auth_v1; first authenticated poll is pending.
• Runtime validation confirms active Watchtower commands: baseline --scope none --schedule "0 0 3 * * 0" and fast --scope fast --interval "3600".
• Operations stack now includes WUD (getwud/wud:8.2.2) with persistent /store, Docker socket read-only access, and Discord trigger configuration sourced from VPS Operations .env.
• WUD is exposed through Traefik at wud.perspective-v.com with NetBird-only middleware and security headers.
• WUD container is healthy and serving HTTP on internal port 3000.
• Controlled WUD Discord trigger tests executed successfully through /api/triggers/discord/main.
• Non-NetBird source route check for wud.perspective-v.com returns HTTP 403 as expected.
• WUD custom registry provider custom.pv is configured with authenticated access to https://registry.perspective-v.com and is visible in WUD registries.
• WUD watcher mode is set to discovery-by-default on VPS (WUD_WATCHER_LOCAL_WATCHBYDEFAULT=true) so running containers are inventoried without per-container opt-in labels.
• WUD tag-channel guardrails remain applied on selected services (Redis, RabbitMQ, MySQL, Verdaccio) using wud.tag.include labels.
• Manual WUD full scan was executed through POST /api/containers/watch after watcher-mode change; inventory expanded from 4 to 25 running containers (including dbskc-web).
• Manual pre-upgrade rollback checkpoints were created for Verdaccio, Redis, RabbitMQ, and MySQL volumes under tmp/backups/major-upgrade-20260410-215000.
• Major runtime image upgrades were executed and are live: Verdaccio 6, Redis 8-alpine, RabbitMQ 4.2-management-alpine, and MySQL 9.6-oraclelinux9.
• Post-upgrade checks succeeded: Verdaccio endpoint ping returned HTTP 200, Redis authenticated PING returned PONG, RabbitMQ broker reports running on 4.2.5, and MySQL is healthy on 9.6.0.
• MySQL credential drift was remediated during the major upgrade window by controlled recovery-mode user alignment to live VPS .env values for root and appuser.
• Dedicated wud-monitor credential was added to Registry basic-auth users and Operations WUD registry login was switched from admin to wud-monitor.
• Post-switch validation confirmed Registry /v2/ accepts the dedicated WUD credential (HTTP 200), and WUD still reports custom.pv with active container inventory.
• Registry stack deployed from runtime/stacks/infrastructure/registry with Traefik basic-auth challenge on registry.perspective-v.com and NetBird-only Registry Admin UI on registry-admin.perspective-v.com.
• Registry auth runtime cutover completed to basic-auth model; token-realm auth flow and registry-auth-proxy are removed from active runtime behavior.
• Registry Admin runtime uses internal registry endpoint http://registry:5000 on Docker network.
• Dedicated dbskc-ci registry credential has been created on the deployed registry-admin runtime for dbskc image pipeline usage.
• Shared registry auth file /var/lib/traefik/registry-auth/registry.htpasswd is initialized on VPS with restricted permissions (600) and active user entries.
• Traefik registry router now uses file-provider middleware (registry-basic-auth@file) backed by the shared host htpasswd file.
• Registry Admin is now wired to the same shared htpasswd file path (RA_REGISTRY_HTPASSWD=/app/config/registry-auth/registry.htpasswd) so UI-managed users are shared with registry API auth source.
• Post-rollout validation confirmed registry API still returns HTTP basic challenge and docker login succeeds for admin using password-stdin.
• Registry auth incident follow-up completed for nishatcolony-ci: host/Traefik/Registry-Admin htpasswd checksums and entries were verified aligned, Traefik access logs confirmed user-specific pre-upstream 401s, and restarting only traefik refreshed auth state so docker login now succeeds for the newly managed user.
• Registry auth refresh operating model is finalized as manual Traefik restart (including Portainer restart) after user credential changes when needed; no helper script rollout required for this phase.
• Registry ACL rows were normalized from anchored regex names to exact repository names for scoped CI user access consistency.
• Sync-blocking OCI-only debug tags were quarantined from active registry storage to restore Registry Admin catalog sync.
• Docker schema v2 compatible test image push validated (library/alpine:ra-compat-v2), and Registry Admin catalog database repopulated.
• Owner-approved direct VPS cutover to Docker Distribution v3 was executed by recreating the registry service from runtime/stacks/infrastructure/registry; live container now runs registry:3 (distribution 3.1.0).
• Pre-cutover rollback artifacts for registry v3 migration were captured at tmp/backups/registry-v3-cutover-20260413-225028 including registry_registry_data.tgz, registry_registry_admin_data.tgz, registry_registry_admin_config.tgz, and image reference metadata files.
• Post-cutover verification passed on VPS: registry API challenge remains HTTP 401 with WWW-Authenticate: Basic realm="traefik", and registry logs show successful authenticated repository/tag and manifest reads on existing images.
• Post-cutover route validation from VPS source confirms access policy remains intact: registry-admin.perspective-v.com returns HTTP 403 for non-NetBird source and retired registry-ui.perspective-v.com now returns HTTP 404.
• Feeds stack deployed from runtime/stacks/infrastructure/feeds with BaGet at nuget.perspective-v.com and Verdaccio at npm.perspective-v.com.
• BaGet Traefik basic auth removed; public restore/download works and publish remains API-key controlled.
• Repository-side feeds compose now keeps canonical BaGet and Verdaccio services only in runtime/stacks/infrastructure/feeds/docker-compose.feeds.yml.
• Repository-side feed env contracts now keep canonical feed keys in runtime/environments/dev/infrastructure/feeds/feeds.dev.env and runtime/environments/vps/infrastructure/feeds/feeds.env.
• Repository-side legacy feed bridge bootstrap helper was removed from runtime/stacks/infrastructure/feeds/scripts/.
• Repository-side feed runbook is now aligned to canonical endpoint operations in runtime/stacks/infrastructure/feeds/README.md.
• Repository-side Verdaccio auth policy is now hardened in runtime/stacks/infrastructure/feeds/verdaccio/conf/config.yaml with auth.htpasswd.max_users=-1, anonymous read/install (access: $all), and authenticated publish/unpublish ($authenticated).
• VPS runtime feed validation confirms Verdaccio/BaGet auth behavior: registration blocked (409), anonymous npm read allowed (200), unauthenticated npm write blocked (401), authenticated npm probe package available (200), BaGet no-key push rejected (401), and BaGet API-key push reached duplicate conflict path (409) with evidence in docs/operations/validations/2026-04-16-feeds-verdaccio-baget-auth-validation.mdx.
• Repository-side feed env/runbook/state contracts are aligned to canonical endpoints npm.perspective-v.com (Verdaccio) and nuget.perspective-v.com (BaGet).
• Feed env artifacts now include NPM_FEED_API_KEY in runtime/environments/dev/infrastructure/feeds/feeds.dev.env, runtime/environments/vps/infrastructure/feeds/feeds.env, and live runtime/environments/vps/infrastructure/feeds/.env.
• Azure Console pipeline now supports NPM_FEED_API_KEY as the preferred npm publish secret in runtime/ci/azure-pipelines/build-and-push-console.yml, with backward-compatible fallback to legacy npm feed variables.
• GitHub Actions Console pipeline now supports NPM_FEED_API_KEY as the preferred npm publish secret in runtime/ci/github-actions/build-and-push-console.yml, with backward-compatible fallback to legacy npm feed variables.
• CI consumers using npm feed auth are now synchronized to the same NPM_FEED_API_KEY runtime value, and owner-confirmed CI npm install/publish validation passed (recorded in docs/operations/validations/2026-04-16-ci-npm-feed-api-key-validation.mdx).
• CI consumers using NuGet feed auth are now validated with owner-confirmed CI NuGet restore/publish success against BaGet using NUGET_FEED_URL/NUGET_FEED_TOKEN (recorded in docs/operations/validations/2026-04-16-ci-nuget-feed-token-validation.mdx).
• Approved VPS feed retirement apply was executed from runtime/stacks/infrastructure/feeds/docker-compose.feeds.yml using live runtime/environments/vps/infrastructure/feeds/.env and orphan removal; running legacy proget container was removed.
• Rollback snapshot and checksum for legacy ProGet data were captured at tmp/backups/feeds-proget-retirement-20260416T180458Z (feeds_proget_packages.tar and feeds_proget_packages.tar.sha256).
• Residual legacy ProGet artifacts were removed from VPS runtime: Docker volume feeds_proget_packages and image proget.inedo.com/productimages/inedo/proget:25.0.25.
• Portainer first-install timeout issue resolved via container restart/recreate workflow.
• Uptime Kuma pinned to image version 2.2.1 in repository compose.
• Top-level repository folders renamed from planning to docs and deployed to prod.
• Runtime migration started: docs/docker moved to runtime/stacks and docs/ci moved to runtime/ci.
• dbskc CI templates were added under runtime/ci for GitHub Actions and Azure Pipelines with deploy/dbskc branch trigger and dbskc-web image target.
• Repository-side Identity/Graph/Console CI templates under runtime/ci are now aligned to deploy branches in both CI systems (deploy/identity, deploy/graph, deploy/console).
• Azure pipeline templates under runtime/ci now use docker login --password-stdin with explicit empty-secret guard checks for registry authentication.
• Azure pipeline templates for identity, graph, and console now support feed URL/token contracts with legacy variable fallback compatibility.
• dbskc CI credentials are validated (dbskc-ci login works) and dbskc pipeline push to private registry is successful with image visibility in Registry Admin.
• dbskc VPS runtime env is now organized at runtime/environments/vps/services/perspective-v/dbskc/.env with tracked placeholder runtime/environments/vps/services/perspective-v/dbskc/dbskc.env.
• dbskc website stack path is now organized at runtime/stacks/services/perspective-v/dbskc.
• dbskc stack was re-created with Watchtower fast-scope label (com.centurylinklabs.watchtower.scope=fast) so it is targeted by watchtower-fast.
• Public route validation passed for dbskc.com: HTTPS returns HTTP 200 and HTTP endpoint redirects to HTTPS (308).
• Prepared the separate public dev.dbskc.com Next.js deployment: websites-dev-dbskc Swarm manifest, VPS launcher, private Gitea image target dev-dbskc-web, and fast Watchtower labels. Initial image build and route validation remain pending the application deployment-branch merge.
• nishatcolony VPS runtime env is now organized at runtime/environments/vps/services/perspective-v/nishatcolony/.env with tracked placeholder runtime/environments/vps/services/perspective-v/nishatcolony/nishatcolony.env.
• Repository-side Perspective-V service envs are now mirrored by service under runtime/environments/vps/services/perspective-v/{identity,graph,console,dbskc,nishatcolony} with matching dev templates under runtime/environments/dev/services/perspective-v/{identity,graph,console,dbskc,nishatcolony}.
• Repository-side Perspective-V service runbooks were updated to use service-local env paths inside the new mirrored structure.
• Repository-side Perspective-V compose now allows identity + graph + console startup without api-gateway dependency by removing console hard dependency on api-gateway.
• Repository-side standalone service artifacts were added for identity, graph, and console under runtime/stacks/services/* with matching VPS tracked env templates under runtime/environments/vps/services/*.
• Repository-side syassociates service artifacts were added under runtime/stacks/services/syassociates.pk/{syassociates.pk,service.syassociates.pk} with matching env placeholders already present under runtime/environments/{dev,vps}/services/syassociates.pk.
• Repository-side gateway/service wiring is aligned to the existing external proxy network; no dedicated gateway-only network is required.
• Repository-side routing model now keeps console as direct Traefik public app while identity and graph are marked as Kong upstream backends (no direct Traefik exposure).
• Repository-side Watchtower label compliance sweep is complete for service stacks: all compose files under runtime/stacks/services now set both com.centurylinklabs.watchtower.enable=true and com.centurylinklabs.watchtower.scope=fast.
• Repository-side service hardening sweep is complete for Traefik-exposed stacks: public services under runtime/stacks/services now pin Traefik runtime network with traefik.docker.network=proxy while keeping TLS + Let's Encrypt + security headers labels.
• Repository-side service hardening sweep is complete for container readiness checks: healthchecks are now present on service stack containers including arnexglobal, dbskc, nishatcolony, standalone console, and perspective-v console.
• Healthcheck compatibility remediation was applied for nginx/alpine web stacks (console, arnexglobal, dbskc, nishatcolony, and perspective-v console): bash-dependent checks were replaced with shell-compatible wget probes.
• console container was recreated from runtime/stacks/services/perspective-v/console, and live container health is now healthy.
• Repository-side service compose project-name normalization is complete: all compose files under runtime/stacks/services now declare name: "perspective-v".
• nishatcolony website stack path is now organized at runtime/stacks/services/perspective-v/nishatcolony.
• Public route validation passed for nishatcolony.pk: HTTPS returns HTTP 200 and HTTP endpoint redirects to HTTPS (301).
• Non-production env templates moved to runtime/environments/dev.
• Non-production database env templates reorganized into per-engine folders under runtime/environments/dev/infrastructure/databases.
• Source-of-truth VPS stack artifacts were synced from prod/docker into runtime/stacks (edge, platform, operations, registry, feeds, netbird).
• Active VPS .env files from prod/docker were moved to runtime/environments/vps.
• Git ignore policy now tracks named env files under runtime/environments while keeping plain .env files ignored.
• Residual .env files were removed from runtime/stacks and preserved under runtime/environments/vps.
• Password and secret values in tracked named env files under runtime/environments/vps were replaced with example placeholder strings while preserving host/domain entries.
• Edge Traefik ACME runtime file is host-managed at /var/lib/traefik/letsencrypt/acme.json with file mode 600.
• Traefik service was re-recreated using edge service-local runtime env source-of-truth while keeping host ACME storage mounted at /var/lib/traefik/letsencrypt.
• runtime/stacks/infrastructure/netbird now contains active tracked NetBird artifacts (docker-compose.yml and config.yaml) with active VPS env at runtime/environments/vps/infrastructure/netbird/netbird.env.
• NetBird stack was recreated from runtime/stacks/infrastructure/netbird after latest-image breakage; both netbird-server and netbird-dashboard are running on refreshed images.
• NetBird dashboard compose wiring now loads VPS env via env_file: ../../environments/vps/infrastructure/netbird/.env; post-fix checks confirm required AUTH/endpoint variables are present and https://netbird.perspective-v.com/api/instance returns HTTP 200.
• NetBird VPS env values for embedded IdP dashboard auth were corrected to keep AUTH_CLIENT_SECRET empty in both runtime/environments/vps/infrastructure/netbird/.env and runtime/environments/vps/infrastructure/netbird/netbird.env.
• NetBird dashboard service was force-recreated from runtime/stacks/infrastructure/netbird after env correction, and runtime container env now confirms AUTH_CLIENT_SECRET=.
• Transitional prod folder was removed after runtime sync completion.
• Database env files prepared from runtime/stacks/infrastructure/databases templates and DB engine bind IPs set to 0.0.0.0.
• MSSQL, PostgreSQL, MySQL, and MongoDB containers deployed from runtime/stacks/infrastructure/databases.
• pgAdmin, phpMyAdmin, and mongo-express admin services deployed from runtime/stacks/infrastructure/databases admin profiles.
• Traefik dynamic security config now includes security-headers-allow-sameorigin for pgAdmin-compatible Query Tool framing while keeping strict headers for other routes.
• PostgreSQL stack pgAdmin router now uses netbird-only@docker,security-headers-allow-sameorigin@file to resolve pgAdmin Query Tool iframe refusal.
• pgAdmin container was re-created from VPS PostgreSQL .env, and running container labels now reflect security-headers-allow-sameorigin@file middleware.
• Database stacks were recreated using service-local VPS .env files under runtime/environments/vps/infrastructure/databases/*/.env (not {service}.env placeholders).
• Live DB credentials were aligned to VPS .env values and validated in running containers for MSSQL, PostgreSQL, MySQL, and MongoDB.
• PostgreSQL + pgAdmin were re-recreated from runtime/environments/vps/infrastructure/databases/postgres/.env and verified against live container env/auth checks.
• PostgreSQL persisted credential drift remediation completed: after recreate from .env, live postgres role password was reset to .env value and non-local auth now passes with .env while failing with placeholder postgres.env password.
• Mongo Express built-in auth credentials are now sourced from VPS .env, and mongo-express is running with successful MongoDB authentication.
• NetBird interface and server VPN IP detected on VPS (wt0, 100.83.72.162).
• DB and Redis firewall cutover script executed; UFW now allows DB engine ports and Redis port 6379 only on NetBird interface (wt0).
• Redis developer hostname resolves publicly to VPS (redis.perspective-v.com).
• Friendly DB and DB UI hostnames resolve publicly to VPS.
• Server-side TCP probes to DB ports succeeded on NetBird IP and friendly DB hostnames.
• Object-storage runtime/env layout is now RustFS-only under runtime/stacks/infrastructure/object-storage/rustfs and runtime/environments/{dev,vps}/infrastructure/object-storage/rustfs.
• RustFS host bind-mount directories were created at /var/lib/rustfs/data and /var/lib/rustfs/logs with ownership 10001:10001 and mode 750 on data/logs.
• RustFS server-local runtime env is active at runtime/environments/vps/infrastructure/object-storage/rustfs/.env with explicit non-default API credentials (RUSTFS_ACCESS_KEY/RUSTFS_SECRET_KEY).
• RustFS stack deployed from runtime/stacks/infrastructure/object-storage/rustfs; rustfs container is healthy and is now the sole active object-storage runtime.
• RustFS NetBird-address validation path succeeds using SNI --resolve to server NetBird IP: console UI endpoint /rustfs/console/index.html returns HTTP 200 and API root returns XML AccessDenied (backend reached, auth gate active).
• NetBird client validation succeeded for RustFS key login: console at rustfs.perspective-v.com authenticated successfully when pointing server URL to rustfs-api.perspective-v.com and using access/secret key credentials.
• RustFS routes rustfs.perspective-v.com and rustfs-api.perspective-v.com return expected HTTP 403 responses from non-NetBird sources.
• Traefik ACME state now contains RustFS hostnames (rustfs.perspective-v.com, rustfs-api.perspective-v.com).
• RustFS DNS status is dual-stack complete: rustfs.perspective-v.com and rustfs-api.perspective-v.com both resolve with A (161.97.83.142) and AAAA (2a02:c207:2320:1427::1) records.
• MinIO decommission completed: minio container removed, objectstorage_minio_data volume removed, MinIO images (minio/minio and minio/mc) removed, and MinIO runtime stack/env artifacts removed from repository paths.
• Runtime sweep (including ignored .env files) confirms no remaining MinIO hostnames, MinIO internal endpoints, or MINIO_* app-env references under runtime/environments and runtime/stacks/services.
• Gateway VPS env scaffolding is now present at runtime/environments/vps/infrastructure/gateway/.env (live runtime source) and runtime/environments/vps/infrastructure/gateway/kong.env (tracked placeholder template).
• Gateway runbook now documents VPS env paths in runtime/stacks/infrastructure/gateway/README.md.
• Gateway stack is deployed on VPS from runtime/stacks/infrastructure/gateway using runtime/environments/vps/infrastructure/gateway/.env; kong-migrations completed and Kong is healthy.
• Gateway route validation passed for current baseline: api.perspective-v.com reaches Kong (HTTP 404 before service route provisioning), and non-NetBird access to konga.perspective-v.com is denied (HTTP 403).
• Konga NetBird-address path validation returns HTTP 200 (built-in login route reachable), and PostgreSQL metadata objects for Kong are confirmed present (kong_user role and kong_db database) in docs/operations/validations/2026-04-15-netbird-route-validation.mdx.
• Kong Admin API exposure policy is enforced in runtime: no host-published admin ports, and Konga container can reach http://kong:8001/status over Docker network.
• Ocelot-to-Kong parity artifacts were added under runtime/stacks/infrastructure/gateway: ocelot-to-kong-mapping.md and kong-bootstrap-ocelot.sh.
• Kong bootstrap now manages full Ocelot parity set in DB mode with idempotent upserts for services, routes, request-transformer rewrites, protected-route JWT, route rate-limits, global CORS, global correlation-id, and issuer consumer credential.
• Ocelot migration route inventory is present in Kong with 5 services and 8 routes tagged ocelot-migration.
• Ocelot migration plugin inventory is present in Kong with expected bindings: global cors + correlation-id, per-route request-transformer and rate-limiting, and JWT on protected routes only.
• Ocelot migration JWT credential is present for consumer ocelot-jwt-issuer with key https://identity.perspective-v.com and algorithm HS256.
• Gateway bootstrap reliability hardening is implemented in kong-bootstrap-ocelot.sh with bounded curl connect/max-time and retry controls plus corrected per-route plugin-id resolution by exact plugin name.
• Gateway parity smoke evidence is captured for migrated public/protected routes with JWT allow/deny outcomes and upstream pass-through status codes.
• Dedicated gateway parity validation record is added at docs/operations/validations/2026-04-16-gateway-ocelot-parity-validation.mdx, including route smoke matrix and request-transformer rewrite mapping.
• Gateway compatibility routes are deployed and verified on public endpoint: https://api.perspective-v.com/identity/docs/index.html returns HTTP 200, https://api.perspective-v.com/identity/swagger/v1/swagger.json returns HTTP 200, https://api.perspective-v.com/swagger/v1/swagger.json returns HTTP 200, and https://api.perspective-v.com/resume now resolves to upstream behavior (HTTP 400) instead of Kong route-miss HTTP 404.
• Gateway gopher compatibility routes are deployed and verified with .NET-aligned mapping: Kong now has graph-gopher-credential-public, graph-gopher-serviceprovider-public, graph-gopher-serviceprovideremail-public, graph-gopher-platform-public, and graph-gopher-ui-public; API calls under https://api.perspective-v.com/graph/gopher/{credential,serviceprovider,serviceprovideremail,platform} resolve through explicit public routes, while UI paths /resume and /gopher/{credential,serviceprovider,serviceprovideremail,platform} are passed through as UI endpoints (not rewritten to /graph/*).
• Kong global CORS allow-headers now include Apollo GraphQL client headers (apollographql-client-name, apollographql-client-version) in bootstrap and gateway env templates, removing preflight rejection for console-origin graph API requests.
• Live Kong global CORS origins were updated on VPS to explicitly allow https://console.perspective-v.com and https://hassan.taj.contact; preflight checks on https://api.perspective-v.com/graph/gopher/credential now return HTTP 200 with matching access-control-allow-origin for both origins.
• Live Kong route graph-resume-public now allows POST,OPTIONS, and global CORS origins include https://hassantaj.github.io; preflight checks on https://api.perspective-v.com/graph/resume return HTTP 200 with matching access-control-allow-origin for both https://console.perspective-v.com and https://hassantaj.github.io.
• Repository-side Kong bootstrap now includes additive graph parity routes graph-docs-public (/graph/docs* -> /docs*) and graph-v1-protected (/graph/v1/* -> /api/v1/*) with rate-limiting and JWT on graph-v1-protected only.
• Repository-side gateway artifacts now include graph v1/docs rollout documentation at docs/operations/validations/2026-04-21-graph-v1-kong-access-validation.mdx and docs/operations/change-records/cr-2026-04-21-graph-v1-kong-route-rollout.mdx.
• Repository-side gateway client imports now include Hoppscotch-compatible OpenAPI template runtime/stacks/infrastructure/gateway/client-imports/graph-v1-hoppscotch-openapi.json.
• Owner-approved VPS apply for graph v1/docs gateway extension is complete; kong-bootstrap-ocelot.sh executed successfully and live route inventory now includes graph-docs-public and graph-v1-protected with expected plugin bindings.
• Post-apply graph v1/docs smoke checks are recorded: /graph/docs returns HTTP 500 (public route reached), /graph/v1/health without JWT returns HTTP 401, and /graph/v1/health with valid JWT returns HTTP 500 (JWT allow-path reached upstream).
• Post-apply CORS preflight for OPTIONS /graph/v1/health returns HTTP 200 with expected allow-origin/method/header values for https://console.perspective-v.com.
• Gateway stack now includes decK declarative automation state at runtime/stacks/infrastructure/gateway/kong.yml plus runbook sync guidance in runtime/stacks/infrastructure/gateway/README.md.
• decK is installed natively on VPS host at /usr/local/bin/deck (v1.58.0), local declarative validation now passes for runtime/stacks/infrastructure/gateway/kong.yml, and online connectivity is verified from host binary to Kong Admin API (deck gateway ping succeeded).
• Declarative gateway state drift cleanup is applied in runtime/stacks/infrastructure/gateway/kong.yml: obsolete legacy graph service/routes were removed so decK no longer proposes unintended route/service creation.
• Ocelot-to-Kong graph parity in declarative state is now synchronized for versioned graph routes: graph-resume-version-public was added (/graph/v{version}/resume/{endpoint} -> /api/v{version}/resume/{endpoint}) and graph-protected now matches /graph/v{version}/{endpoint} with methods GET,POST,PUT,DELETE,OPTIONS and rewrite to /api/v{version}/{endpoint}.
• Owner-approved host-native decK sync was executed using rendered state (envsubst < kong.yml) against live Kong Admin API; pre-sync showed route/plugin/JWT drift and post-sync diff returned Created: 0, Updated: 0, Deleted: 0.
• Live JWT credential secret for consumer ocelot-jwt-issuer is now aligned to VPS KONG_JWT_HMAC_SECRET through decK sync.
• Graph resume access-token route is now public: declarative route updated to regex and synced via decK so /graph/v1/resume/GetByAccessToken no longer returns Kong 401.
• Graph service recovery is complete after owner-approved pull and recreate of registry.perspective-v.com/graph:latest; container is now running and healthy (no restart loop).
• Public docs endpoints are now accessible through Kong: https://api.perspective-v.com/graph/docs/ serves Scalar (GraphService API title) and https://api.perspective-v.com/identity/docs/index.html serves Swagger (IdentityService API title).
• Root cause of prior graph docs outage was upstream runtime failure in graph container (GraphService.dll missing at startup), not Kong route mismatch.
• Outbound SSH spike incident triage completed on VPS host 161.97.83.142; active source was a root process ./main 22 bune.txt from /tmp/.ICE-unix/.x/main (non-stack, host-level process chain).
• Malicious process chain and detached screen session were terminated; outbound port 22 sockets from the offending PID dropped to zero.
• Suspicious artifact directory was copied for evidence to /root/incident-2026-04-11/suspect/x-dir-copy and original path was quarantined as /tmp/.ICE-unix/.x.quarantined.<timestamp>.
• Temporary outbound SSH containment was applied in UFW (deny out 22/tcp for IPv4 and IPv6).
• SSH hardening applied: key-only auth enforcement (AuthenticationMethods publickey), password/interactive auth disabled directives, root login constrained to key-only, MaxAuthTries=3, and LoginGraceTime=30.
• Root password was rotated and authorized_keys trust was reset to newly approved SSH keys.
• Contabo VNC/remote console access was disabled after incident containment.
• VS Code SSH and Tabby SSH access were reset and re-approved with fresh trusted key material.
• SSH auth matrix was revalidated after config precedence fix (/etc/ssh/sshd_config.d/50-cloud-init.conf set to PasswordAuthentication no); effective runtime now shows passwordauthentication no, kbdinteractiveauthentication no, and authenticationmethods publickey.
• Unexpected immutable attribute on /var/log was removed to restore normal logging behavior.
• Fail2ban was installed, enabled, and configured with active sshd jail (backend=systemd) and live brute-force banning.
• Post-incident persistence quick checks found no known indicator strings in systemd unit files or cron paths, and no root crontab entries were present.
• Uptime Kuma monitor inventory was audited and confirmed HTTP-only monitors; no SSH-type monitor was configured.
• Incident archive bundle was created in-repo at Incidents/2026-04-12-0015-ssh-outbound-spike/ with markdown reports and raw evidence exports for provider response and audit trail.
• Same-day post-containment revalidation was captured at 2026-04-12T02:09:23+02:00: outbound TCP/22 remained inactive, process attribution showed inbound-only sshd sessions, UFW outbound 22 deny remained active, and SSH/Fail2ban control snapshots were archived in evidence files 23 through 30.
• Permanent outbound SSH policy tooling was implemented at runtime/stacks/infrastructure/operations/ssh-egress-policy.sh with script-managed exception lifecycle, append-only policy logging, GitHub-over-443 SSH client setup, and blocked-attempt scanning.
• Outbound SSH deny was normalized to a logging-enabled dual-stack rule (DENY OUT 22/tcp with (log, out) for IPv4 and IPv6).
• Root SSH client configuration for GitHub was pinned to ssh.github.com:443 and validated with effective config plus TCP reachability checks.
• ssh-egress-policy-maintenance.timer was installed and enabled to run recurring exception-expiry sweeps and blocked-attempt scans.
• Controlled outbound TCP/22 simulations now generate UFW block entries and policy detector events (blocked-ssh-detected) captured in incident evidence.
• SSH egress Discord webhook notifications were upgraded to Markdown templates (quote plus ordered-list structure) for exception open/close, blocked-detection alerts, and alert-test messages.
• SSH egress Discord webhook notification templates were refined to use underlined headings (instead of quote-style titles) plus ordered-list fields in embed descriptions.
• SSH egress alert investigation on 2026-04-17 confirmed recurring Discord alerts were detector false positives: blocked-scan matched inbound UFW BLOCK events and DPT=22xx prefixes (for example 2202 and 2222) as outbound SSH; live socket checks during investigation showed zero outbound connections to destination port 22.
• Repository-side remediation for 2026-04-17 SSH egress false positives is now implemented in runtime/stacks/infrastructure/operations/ssh-egress-policy.sh: blocked-scan keeps only outbound exact DPT=22 events and blocked-alert snippet code fences are escaped to prevent shell command-substitution errors during Discord message formatting.
• Approved VPS validation for the 2026-04-17 SSH egress fix is complete: replaying a known false-positive window produced zero matches under the updated filter, a controlled outbound probe to 198.51.100.22:22 produced outbound exact DPT=22 block matches, and direct blocked-scan execution (webhook-disabled test mode) logged detection without formatter shell errors.
• Tier-1 database backup timer rollout was executed on VPS with repository root override (/home/repo/contabo-server-setup) and tier1-db-backup.timer is active/enabled.
• First successful Tier-1 manual backup run completed for postgres/mysql/mongodb/mssql with artifacts at /home/repo/contabo-server-setup/tmp/backups/tier1-20260415T205728Z.
• Backup manifest with checksum metadata was generated at /home/repo/contabo-server-setup/tmp/backups/tier1-20260415T205728Z/manifest.csv.
• First-run backup evidence was captured at /home/repo/contabo-server-setup/tmp/backups/evidence/tier1-backup-evidence-20260415T205803Z.txt.
• Repository Tier-1 backup policy artifacts were updated to strict rolling retention (keep newest 3) and weekly default schedule (Sunday 03:00 UTC) across db-backup.sh, install-tier1-backup-timer.sh, systemd templates, and related runbooks, including explicit manual rerun guidance.
• Approved VPS Tier-1 policy apply is complete: active systemd units now use ExecStart=...db-backup.sh run --keep-last-backups 3 and OnCalendar=Sun *-*-* 03:00:00.
• Refreshed post-apply evidence was captured at /home/repo/contabo-server-setup/tmp/backups/evidence/tier1-backup-evidence-20260416T222414Z.txt.
• End-to-end PostgreSQL restore validation drill was completed from latest Tier-1 backup with measured RTO 196s and RPO 238s (both within 4h/1h targets), documented at docs/operations/restore-validations/rv-20260415t210554z-postgres-tier1.mdx.
• Restore drill evidence was captured at /home/repo/contabo-server-setup/tmp/backups/evidence/restore-validation-20260415T210554Z.txt.
• Encrypted Tier-1 Google Drive delivery is implemented in the repository: db-backup.sh discovers Swarm tasks, reads mounted secrets, writes PostgreSQL globals plus one collision-safe custom dump for every connectable non-template database, covers all non-system MySQL databases and MongoDB, uploads only behind DB_OFFSITE_ENABLED=true, verifies with rclone cryptcheck, retains 12 remote/3 local sets, and preserves partial staging on failure. MSSQL remains supported but excluded while its service is 0/0.
• Lightweight monitoring baseline is formalized with measured Kener/WUD/Watchtower footprint and no heavy observability stack (docs/operations/baselines/lightweight-monitoring-baseline.mdx).
• Controlled monthly DB maintenance schedule is formalized at first-Saturday 02:00 UTC with pre/post evidence requirements (docs/operations/baselines/monthly-db-maintenance-schedule.mdx).
• One tabletop drill was executed against the incident-response and host-integrity runbooks (docs/operations/tabletop-drills/tt-20260415t220000z-ssh-egress-policy.mdx).
• State/document taxonomy alignment is implemented in tracked docs: decision/state under docs/state, operational artifacts under docs/operations, and setup guidance under docs/runtime with refreshed references.
• Current operator guide set is implemented under docs/runtime, including overview.mdx, per-stack guides, environments.mdx, ci.mdx, wrapper-scripts.mdx, and services/overview.mdx.
• Legacy repository documentation folder docs/ was retired after migration; docs/ is now the sole tracked documentation surface.
• Cross-platform launcher scripts are implemented under runtime/scripts for active infrastructure and service compose targets, backed by shared helper logic for live VPS .env, placeholder fallback, and dev template resolution.
• Zitadel identity provider is deployed from runtime/stacks/infrastructure/zitadel/docker-compose.yml as two containers: core API + console (ghcr.io/zitadel/zitadel:v4.15.2, h2c on :8080) and Login V2 UI (ghcr.io/zitadel/zitadel-login:v4.15.2, :3000), behind the shared edge Traefik on the proxy network.
• Zitadel uses the shared Postgres on postgres-network and auto-creates database zitadel_db and user zitadel_db_usr via the Postgres admin connection on first init.
• Zitadel external domain is auth.perspective-v.com with Traefik TLS termination (ExternalSecure=true, TLS_ENABLED=false, h2c backend); the OIDC issuer is https://auth.perspective-v.com.
• Zitadel Traefik routing splits the single host: /ui/v2/login/* and / route to the login container; /api (prefix-stripped) and all other paths route to the core over h2c.
• Zitadel mints a login-client PAT and an admin service-account PAT (zitadel-admin-sa, IAM_OWNER) at init into the shared zitadel-bootstrap volume; the admin PAT drives post-init API configuration.
• Zitadel instance domain policy sets USERLOGINMUSTBEDOMAIN=false so loginnames are bare usernames (the verified email also works to log in).
• Zitadel org primary domain is set to auth.perspective-v.com (custom, auto-verified) via runtime/stacks/infrastructure/zitadel/set-org-primary-domain.sh, replacing the auto-generated <org>.<instance-domain>; the seeded admin username is normalized from the init-baked domain-qualified form to the bare username via runtime/stacks/infrastructure/zitadel/normalize-admin-username.sh.
• Zitadel runtime env lives at runtime/environments/vps/infrastructure/zitadel/.env; the dev template with placeholder secrets is runtime/environments/dev/infrastructure/zitadel/zitadel.dev.env.
• Zitadel SMTP is configured and active via the Admin API (host fusion.mxrouting.net:465, implicit TLS, sender no-reply@perspective-v.com / "Perspective-V"); the FIRSTINSTANCE_SMTPCONFIGURATION_* env keys only seed at first init, so on an existing instance SMTP is set through the Admin API (/admin/v1/smtp add + _activate) or the console (Instance → Notifications → SMTP).
• Zitadel operational guardrail: the login-client service user must never be deactivated — it backs the Login V2 UI and deactivating it locks out all logins; event-store reactivation only updates one of Zitadel's three projection schemas (projections/auth/adminapi) so token validation stays broken, and recovery requires a re-init. The zitadel-admin-sa IAM_OWNER service account is the one safe to disable when not running API-driven config.
• Zitadel stack setup guide is published at docs/runtime/stacks/zitadel.mdx (registered in docs/docs.json and the Infrastructure Stacks index), covering the dev bring-up from the env template, post-init scripts, SMTP, validation, guardrails, and the re-init procedure.
• Multi-Drive encrypted backup tooling is implemented under operations/: shared env/lock/rclone helpers, Swarm-aware WordPress and database backups, root-only config preparation, safe diagnostics, systemd installers/reference units, and stubbed behavioral tests.
• The four WordPress backup registries use active external Swarm volume names. Gorsi's two sites route to gorsistudio_crypt; dbskc, wcblahore, shared databases, and physical PostgreSQL snapshots route centrally. Both Nishat variants are outside backup scope.
• WordPress archives contain the site database, volume files, and SHA-256 manifest; they are uploaded through rclone crypt, verified before cleanup/pruning, run last-day monthly at 03:00 UTC, and retain four remote archives per site.
• WordPress dry-runs now authenticate to MySQL and require the configured site database; WordPress and Tier-1 MySQL dumps disable GTID statements.
• Monthly PostgreSQL physical backup and guarded restore tooling is implemented under operations/: online pg_basebackup with included WAL/native manifest, four-remote and two-local sliding retention, a first-Sunday 04:00 UTC systemd installer, and isolated scratch restore guards for the PostgreSQL 18 volume layout.
• WordPress, weekly logical database, and monthly physical workflows share a six-hour bounded-wait lock and retain source/upload/verification failures with sanitized evidence.
• The configuration contract is /etc/contabo-backups/backup.env for operations and writable /etc/contabo-backups/rclone.conf for OAuth/crypt state, both root-only; the tracked non-secret template remains runtime/environments/dev/infrastructure/backups/backups.dev.env.
• The encrypted Google Drive setup, multi-account onboarding, validation, diagnostics, restore, and approval-gated rollout guide is published at docs/runtime/stacks/wp-backups.mdx; repository tests pass without real Docker, Google, or production mutations.
• Approved production bootstrap began on 2026-08-14: rclone 1.60.1 is installed and /etc/contabo-backups/{backup.env,rclone.conf} exists with required root ownership and 0700/0600 permissions. The four non-secret rclone remote sections are installed; both OAuth identities/quotas and both crypt configurations are now verified. This bootstrap was followed by the completed activation recorded below.
• Encrypted canaries passed on both contabo_crypt and gorsistudio_crypt: each upload matched with cryptcheck, downloaded through crypt with matching SHA-256, and appeared only as a .bin object through the underlying Drive remote.
• First encrypted WordPress validation passed for store.gorsistudio.com, including manifest verification, 4,411 restored volume entries, 12 restored MySQL tables, and complete scratch cleanup. The existing Tier-1 set also passed central upload, eight-artifact cryptcheck, PostgreSQL download checksum, scratch restore/query, and cleanup. Production services were not restarted and MSSQL remained 0/0.
• Final encrypted backup activation completed on 2026-08-14: the four-site WordPress dry-run/real workflow passed; logical set tier1-20260814T211722Z verified nine PostgreSQL databases plus globals, four MySQL databases, and MongoDB; physical set postgres-physical-20260814T212317Z passed cryptcheck, decrypted checksum validation, native pg_verifybackup, isolated readiness/database-count/query checks, and scratch cleanup.
• DB_OFFSITE_ENABLED and POSTGRES_PHYSICAL_OFFSITE_ENABLED are active with DB_ENGINES=postgres,mysql,mongodb. Weekly database, last-day WordPress, and first-Sunday PostgreSQL physical timers are enabled/active; both Drive destinations connect, final unit results are successful, and MSSQL remains 0/0.
• Large PostgreSQL native tar validation now consumes the complete listing without early-exit SIGPIPE false failures. Failed/unverified physical staging is excluded from automatic local retention; the blocked first attempt remains preserved with sanitized evidence.
• Docker Swarm is live on the VPS (single-node manager, advertise 161.97.83.142): 7 external overlay networks (proxy, platform, postgres-network, mssql-network, mysql-network, mongodb-network, object-storage), 30+ Docker secrets from swarm/secrets/create-all-secrets.sh, and all active database/platform/edge/panel/service/websites-* stacks deployed via swarm/scripts/**.
• Swarm data preservation: every stack maps its data volume to the original compose-named volume with external: true (e.g. netbird_netbird_data, databases_postgres-data, perspective-v_dbskc_data); pre-migration data is intact. Empty infra-*/svc-* volumes created by the first (pre-mapping) deploy are orphans.
• Traefik on Swarm publishes ports 80/443 in mode: host (not the ingress mesh) so it sees real client IPs; without this, ingress SNAT (10.0.0.2) breaks the netbird-only (100.64.0.0/10) allowlist that gates all admin UIs. Admin UIs (portainer, pgadmin, phpmyadmin, rustfs, konga, rabbitmq, redis-insight, mongo-express, registry-admin, wud, kong-admin, traefik dashboard) are reachable only over the NetBird VPN.
• swarm/secrets/create-all-secrets.sh strips surrounding quotes from .env values before creating secrets (quoted values like MYSQL_PWD="…" otherwise store literal quotes and break auth).
• Swarm services without _FILE support are fed secrets via command wrappers: kener injects KENER_SECRET_KEY and SMTP_PASSWORD; vaultwarden injects ADMIN_TOKEN (kept quoted in .env for shell sourcing, so env_file cannot deliver it unquoted).
• Multi-service Swarm launchers load every fragment's owning environment source; website fragments retain per-service database variables.
• Zitadel Login V2 on Swarm needs a dedicated PathPrefix('/ui/v2/login') router to zitadel-login (the core zitadel router excludes that prefix and zitadel-root only matches exact /).
• Traefik dashboard on Swarm requires a dummy traefik.http.services.*.loadbalancer.server.port label on the traefik service or the swarm provider drops the dashboard router ("port is missing"); dashboard basic-auth uses admin-auth@file → /var/lib/traefik/registry-auth/registry.htpasswd (a single-file bind mount — edits need a traefik restart to take effect).
• phpMyAdmin extra themes (darkwolf 5.2, boodark 1.2.0) are extracted to host /var/lib/phpmyadmin/themes/* and bind-mounted in swarm/stacks/panels/phpmyadmin.yml.

Gitea Unified Package Registry Migration (2026-07-01)Copy Anchor Link

• Single Gitea instance deployed as a package-only registry (git features disabled: SSH off, registration disabled, no repos, MAX_CREATION_LIMIT=0, DEFAULT_ALLOW_CREATE_ORGANIZATION=false, org menu trimmed to packages) at gitea.perspective-v.com behind edge Traefik with a valid Let's Encrypt cert; stack service (swarm/stacks/services/gitea/gitea.yml) is 1/1 healthy (~112 MiB).

• The Swarm source now renders pgAdmin, phpMyAdmin, Mongo Express, Konga, Redis Insight, Portainer, WUD, and NetBird Dashboard from eight fragments in the panel stack, preserving the role label, image digests, external volumes, and five-suspended/three-running baseline.

• Vaultwarden, Gitea, and RustFS now share service; Zitadel runs in service-zitadel. Their exact prior images and external data were preserved. Post-cutover checks passed: Vaultwarden /alive 200, Zitadel /debug/ready 200 and root 308, Gitea health 200 and registry challenge 401, and both RustFS routes remain NetBird-restricted.

• Stack launchers parse env values literally, load every service-local env source required by split stacks, and use --resolve-image changed; all 15 canonical launchers render complete DEV service definitions.

• Production naming rollout evidence is recorded in docs/operations/change-records/cr-2026-08-14-swarm-service-naming-rollout.mdx.

• Gitea serves ALL packaging under org owner perspective-v: Docker images (gitea.perspective-v.com/perspective-v/<img>), npm (/api/packages/perspective-v/npm/), and NuGet (/api/packages/perspective-v/nuget/index.json).

• Gitea auth model is Gitea user + access token: write:package scope to push, read:package scope to pull; admin user is pvadmin (Gitea reserves admin).

• Gitea deploy artifacts use the shared swarm/scripts/services/service.{sh,bat} launcher, VPS env runtime/environments/vps/infrastructure/scm/gitea/.env, dev template runtime/environments/dev/infrastructure/scm/gitea/gitea.dev.env, and gitea_db_password in swarm/secrets/create-all-secrets.sh; a deploy-time fix removed DEFAULT_REPO_UNITS/DISABLED_REPO_UNITS (they emptied the fork-unit set and caused a fatal no default fork repository units found).

• All package data migrated into Gitea: NuGet 45 versions (from BaGet), Docker 20 tags across 11 repos (from old registry.perspective-v.com), and npm @pv/core 14 versions (from Azure DevOps pv-ng feed); Verdaccio was empty (nothing to migrate).

• CI/CD pipeline templates repointed to Gitea endpoints + perspective-v namespace (runtime/ci/github-actions/*, runtime/ci/azure-pipelines/*, runtime/ci/README.md).

• WUD reconfigured to watch the Gitea registry: the WUD manifest (now swarm/stacks/panels/wud.yml) and VPS env point to gitea.perspective-v.com, with a dedicated Gitea wud-monitor user + read:package token and recreated wud_registry_password secret; unquoted cron values in the operations env were also fixed.

• All 11 running services use Gitea images. The header-defined stack rollout restored nishatcolony-web from source drift back to its existing Gitea image; both Nishat services are 1/1, nishatcolony.pk returns 200, and the intended WordPress route at new.nishatcolony.pk still exposes its pre-existing 500.

• Legacy 5 package services were RETIRED and removed: infra-registry (docker-registry registry.perspective-v.com + registry-admin registry-admin.perspective-v.com) and infra-feeds (ProGet proget.perspective-v.com + BaGet nuget.perspective-v.com + Verdaccio npm.perspective-v.com); their old hosts now return 404. Related secrets removed: registry_*, baget_api_key, npm_feed_api_key.

• Gitea usage/token guidance is published at docs/runtime/stacks/gitea.mdx; the completed legacy-service retirement record is docs/runtime/gitea-retirement-runbook.mdx.

• Swarm source and launchers are organized by runtime role, with one YAML fragment per active service and multi-file rendering validated for every stack.

• The production role-layout rollout is complete: Redis, RabbitMQ, Watchtower, RustFS, Kong, NetBird, Panels, Traefik, and Kener run under their documented stack/service identities with preserved external state and image digests.

• Kong's migration task now reads its PostgreSQL password from the existing Docker secret and completes successfully instead of remaining failed at 0/1.

• Host-wide backup and systemd tooling is active under operations/; both production timers use the canonical paths and the retired compatibility executables have been removed.

• The header-defined production namespace rollout is complete: all 37 services now use the stack value declared by their manifest, with images, desired replicas, named volumes, secrets, routes, and access policies preserved. Kong's admin router now explicitly binds to its port-8001 Traefik service, and production backup dry-runs resolve the renamed database_* tasks.

• Final rollout evidence is recorded in docs/operations/change-records/cr-2026-08-14-swarm-header-stack-rollout.mdx.

• Pi-hole administration is now available at pihole.home.perspective-v.com through Contabo Traefik with netbird-only + security-headers. The route forwards over NetBird to 100.83.117.37:8053 and uses a dedicated Host: pi.hole backend-header middleware required by Pi-hole v6. Public access returns 403; authorized NetBird access reaches /admin/login with HTTP 200 and a trusted Let's Encrypt certificate. Source/live dynamic-config hashes match and the installer drift check is clean.

Superseded by the Gitea migrationCopy Anchor Link

• SUPERSEDED (retired 2026-07-01): all Registry stack entries above (docker-registry registry:3/distribution 3.1.0, Registry Admin, shared htpasswd basic-auth model, wud-monitor registry credential, registry v3 cutover) — replaced by the Gitea Docker package registry.
• SUPERSEDED (retired 2026-07-01): all Feeds stack entries above (BaGet at nuget.perspective-v.com, Verdaccio at npm.perspective-v.com, ProGet retirement, NPM_FEED_API_KEY/NUGET_FEED_TOKEN CI contracts against BaGet/Verdaccio) — replaced by Gitea npm/NuGet package endpoints.

Homelab LAN certificate export (2026-09-15)Copy Anchor Link

• Added operations/traefik/export-homelab-certs.py to the repository. It reads /var/lib/traefik/letsencrypt/acme.json and emits only the six selected *.home.perspective-v.com PEM pairs needed by the private homelab edge.
• The exporter fails closed on missing or malformed entries and writes no key material to logs. Existing Contabo Traefik routes, ACME renewal, and public listeners were not changed.

Contabo homelab split-DNS documentation (2026-09-15)Copy Anchor Link

• Added docs/runtime/stacks/homelab.mdx, linked it from the runtime stack index and overview, and documented the public Contabo path versus the LAN Pi-hole path.
• Recorded the Namecheap/public-DNS boundary, PTCL F1611A LAN answer, Pi-hole application overrides, NetBird-only dashboard policy, and shared TLS certificate handoff in docs/state/dns-info.mdx.
• Added the exporter invocation and failure-handling guidance to operations/traefik/README.md; no production service, route, secret, or certificate store was changed by this documentation update.

Fumadocs documentation site (2026-10-10)Copy Anchor Link

• Added a Next.js/Fumadocs frontend that serves the shared docs/ corpus while retaining docs.json for the existing Mintlify navigation configuration.
• Added an isolated websites-perspective-v-docs Swarm stack and a registry-authenticated launcher for docs.perspective-v.com.