Azure Staging Current Status

This page describes the current repository-owned Azure staging shape. It does not record a past smoke count, workflow run ID, public IP, or container revision; those values become stale. Use the non-mutating smoke and Azure inventory for the deployed state.

Source of truth

Concern	Current source
Azure desired state	infra/tf/azure
Shared service names, routes, and database roles	infra/tf/gcp/service-catalog.json
Staging remote state and variables	Terrakube organization peaksuite, workspace monorepo-azure-staging, directory infra/tf/azure
Runtime image promotion	.github/workflows/container-images-deploy.yml and its Azure actions
Resource/infrastructure smoke	scripts/azure-staging-smoke.sh
Credentialed application smoke	scripts/azure-staging-credentialed-smoke.sh
Terminal certificate/mTLS smoke	scripts/azure-staging-terminal-onboarding-smoke.sh

The shared Terraform root defaults to environment = "staging", resource prefix peakpos-staging, resource group peakpos-staging-rg, and region centralus. Production uses the same folder with a separate Terrakube workspace, backend, variables, state, hostnames, credentials, and resources. Never use the staging workspace as a production plan or apply target.

Current staging architecture

Public and terminal edge

API Management is removed. A single Application Gateway Standard_v2 instance is the Azure public edge:

• one static public IP;
• staging-api.peakpos.co HTTPS listener for the public API;
• optional staging-terminal.peakpos.co mTLS listener on the same IP, separated by SNI;
• path-based routing to private Container Apps and the private Blob /images route; and
• a dedicated terminal backend pool for terminal-api in the internal Container Apps environment.

The Application Gateway has a minimum capacity of one and maximum of two. Staging service health probes run every 300 seconds; production uses 30 seconds. The API certificate and terminal certificate/CA values come from Key Vault/Terrakube variables and gate creation of their listeners.

Cloudflare fronts the API hostname and provides its edge WAF. The Azure Application Gateway is not WAF_v2. The terminal listener authenticates client certificates with an SSL profile and overwrites all forwarded certificate identity headers from Application Gateway server variables before terminal-api sees them. Keep the terminal hostname DNS-only unless a separately reviewed Cloudflare client-certificate design is proven.

Container Apps

Staging intentionally omits kitchen-api through staging_only_pending_microservices; it is not a failed or scaled-to-zero app. The current public Container Apps are:

Route	Service
/merchant	merchant-api
/ai	ai-api
/status	status
/terminal-onboarding	terminal-onboarding
/management	management-api
/customer	customer-api
/integrations	tx-bundler
/auth	auth
/images	private Azure Blob product-images backend

terminal-api runs in the internal terminal Container Apps environment and is reachable through the mTLS listener, not the normal public service map. Every non-pending Container App defaults to one warm replica. Staging uses reduced CPU/memory shapes; pending_microservices, not scale-to-zero, is the intended way to omit an unused service.

Terraform creates each app and a bootstrap image reference, but container-apps.tf ignores later image changes. GitHub deployment automation owns immutable commit-image promotion. An unrelated Terrakube apply must not roll a live revision back to a bootstrap tag.

Data, secrets, and messaging

• Azure Database for PostgreSQL Flexible Server is private-only. The bootstrap Container Apps Job applies repo-owned schema/init SQL, seed data, scoped service roles, grants, and role-password rotation.
• Database-backed apps use the service role from service-catalog.json; they do not use the PostgreSQL administrator login at runtime.
• Key Vault uses RBAC, purge protection, soft delete, default network deny, no public network access, and a private endpoint.
• Azure Storage disables public network access and shared-key authentication. Blob and Queue traffic uses private endpoints and managed identities.
• Product images are anonymously readable at the container level only so Application Gateway can serve /images; the storage endpoint itself remains private, making the gateway the public path.
• management-api, merchant-api, terminal-api, and tx-bundler publish notification requests to Azure Storage queues. Poison queues and Blob pointer envelopes are part of that transport.
• Email/SMS workers and their notification storage may be owned by the shared-services Azure root. The monorepo root owns the producer configuration and, when enabled, private endpoints to the shared storage account; it does not own the worker application lifecycle.

Secret payloads belong in Key Vault. Terrakube stores sensitive secret IDs/version pins, not plaintext payloads. Do not paste Key Vault values, external-account JSON, database passwords, Gateway credentials, certificates, or Terrakube tokens into a plan review or support thread.

Retained non-Azure dependencies

Azure staging is not a zero-GCP environment:

• Firebase projects and tenants remain the authentication/App Check boundary.
• Azure-hosted services use per-service Google external-account credentials through Workload Identity Federation; service-account private keys are not the preferred path.
• Terminal certificate issuance remains on GCP Private CA. The Azure runtime identity used by terminal-onboarding needs the scoped certificate-requester grant.
• Gateway remains an external payment platform reached through environment-specific Gateway URLs and credentials.

The staging smoke rejects SPANNER_* and PGADAPTER_* runtime variables on current Container Apps. That assertion does not authorize removing Firebase, Private CA, Gateway, or other explicitly retained dependencies.

Verify deployed status

Non-mutating resource smoke

Run from an authenticated operator environment:

scripts/azure-staging-smoke.sh --strict

This checks Azure inventory, DNS, Application Gateway/listeners/backend health, Container Apps and Jobs, PostgreSQL, Key Vault, Storage, role/config invariants, public routes, recent logs, and the negative terminal mTLS path. It does not apply Terraform or create application records.

A failure is current evidence; an old successful workflow run is not a waiver. Record the exact failing check and inspect the owning Terraform resource, deployment workflow, or service logs.

Credentialed smoke

Start with the safe preflight:

scripts/azure-staging-credentialed-smoke.sh --preflight-only

The full credentialed smoke can authenticate, create or update test records, provision a demo context when explicitly enabled, mint onboarding material, and send receipts to configured recipients. Run it only with approved staging org/store/terminal IDs or an explicitly authorized demo bootstrap, short-lived tokens, and safe email/phone recipients. --strict makes skipped credential gates fail; it does not make mutations safe. Production additionally requires the script's explicit --allow-prod gate and separate authorization.

Use the focused terminal script only when certificate issuance and mTLS are the intended test. It mints/confirms onboarding state and requests a real client certificate, so it is not a read-only probe.

Apply and recovery boundaries

• Terrakube owns infrastructure plans/applies for monorepo-azure-staging. Run a plan first and review replacements, deletes, role assignments, network changes, and state moves.
• GitHub owns image promotion after Terraform creates the apps. Repair image deployment in the workflow; do not force it through Terraform.
• shared-services owns the email/SMS worker apps and Terrakube control plane. Escalate there when their workers, storage, private DNS, or the IaC platform is the failing boundary.
• Do not open private Key Vault, PostgreSQL, or Storage networking as a generic recovery step. Use the repo scripts' gated temporary access only when the documented operation requires it and confirm restoration.
• Do not trigger Plan and apply, rotate secrets, bootstrap PostgreSQL, recreate a VNet peering, or delete a stuck job without reviewing the exact target and obtaining authorization.
• GCP staging is a separate environment, not an automatic rollback. A rollback requires an explicit data, DNS, image, certificate, and client-routing plan.