Header-Defined Swarm Stack Production Rollout

Change metadataCopy Anchor Link

• Change id: CR-2026-08-14-swarm-header-stack-rollout
• Date and time: 2026-08-14 19:29–20:03 UTC
• Environment: VPS / production
• Requester and approver: repository owner, explicit approval in the active session
• Executor: Codex
• Status: completed; known pre-existing application and external follow-ups remain

ScopeCopy Anchor Link

Each fragment's first-line # Swarm stack: value became its authoritative deployed namespace. Fragments with the same value are operated together:

• infra-{postgres,mysql,mongodb,mssql} became database.
• infra-{redis,rabbitmq,watchtower} became platform.
• infra-edge became edge; panels became panel.
• svc-{gitea,rustfs,vaultwarden} became service.
• svc-kong, svc-netbird, and svc-zitadel became service-kong, service-netbird, and service-zitadel.
• The six deployed website groups moved from svc-* to websites-*.
• Syassociates remained undeployed; only its source and launcher identity changed.

Canonical grouped launchers now always deploy the complete fragment set for database, platform, and service. Database and WordPress backup defaults were updated to resolve the new database_* task names.

Preconditions and rollback evidenceCopy Anchor Link

• Captured stacks, services, replicas, exact service image specs, volumes, networks, secrets, and timer state.
• Confirmed the latest scheduled Tier-1 job completed successfully.
• Revalidated all 14 artifacts in tier1-20260814T185509Z by SHA-256.
• Kept MSSQL explicitly at 0/0 in the new manifest.
• Stored sanitized before/after evidence under the ignored local directory tmp/backups/swarm-role-layout-20260814T192920Z-header-rollout/.

Execution and validationCopy Anchor Link

1. Migrated each namespace only after its prior tasks stopped, preventing two identities from mounting the same external state or advertising duplicate routes.
2. database reached PostgreSQL/MySQL/MongoDB 1/1 and MSSQL 0/0; all three running task containers reported healthy.
3. platform reached 4/4; Redis returned PONG and RabbitMQ diagnostics returned Ping succeeded.
4. edge reached 2/2; Kener returned 200.
5. panel contains exactly eight role-labeled services and preserved its five-suspended/three-running baseline. A controlled pgAdmin 0 -> 1 -> 0 cycle passed and the route returned the expected non-NetBird 403.
6. The service stacks reached their expected replicas. Kong's migration task completed, NetBird's instance API returned 200, Zitadel returned ready 200 and root 308, Vaultwarden and Gitea health returned 200, the Gitea registry challenged with 401, and RustFS remained NetBird-restricted.
7. All six active website stacks reached their expected replicas. A drifted Nishat web image and its separate environment source were restored. Both services are 1/1; nishatcolony.pk returns 200, while the intended WordPress route at new.nishatcolony.pk retains its pre-existing 500.
8. Both production backup dry-runs passed against database_*; both systemd timers remain enabled and active.

Preservation checksCopy Anchor Link

• Exact service image specs and desired replica values match the pre-rollout inventory.
• Named volume and Docker secret inventories are unchanged. Anonymous task volumes churned as expected when old containers were replaced.
• The only named network change is infra-watchtower_default -> platform_default.
• All tested public routes and NetBird restrictions returned their expected status codes.

Additional correctionsCopy Anchor Link

• Pinned all four database manifests to their captured production image digests so the namespace change could not trigger an image upgrade.
• Added the explicit kong-admin router-to-service binding required when Kong exposes both proxy and admin ports through Traefik.
• Pinned nishatcolony-web back to its deployed Gitea image after the live env rendered an unintended generic WordPress image during the rename. Its launcher now loads both intentional Nishat environment sources so the two hostnames do not collapse onto one router.

Follow-upCopy Anchor Link

• Restore the VPS Docker client's dedicated svc-puller login and repeat an authenticated Gitea image pull; do not substitute another service token.
• data.arnexglobal.com still resolves away from this VPS, so its external TLS behavior cannot validate this deployment until DNS is corrected.
• Diagnose the existing new.nishatcolony.pk WordPress HTTP 500 separately; its task is healthy and the naming rollout preserved its state and route.

RollbackCopy Anchor Link

Remove the replacement namespace, wait for all of its tasks to stop, then use the pre-change revision to deploy the prior namespace against the same external volumes, networks, and secrets. Roll back one namespace group at a time and validate it before continuing. No rollback was required.