What is a safe practice when performing canary deployments for rollback?

What is a safe practice when performing canary deployments for rollback?

• A. Define clear thresholds and automatically roll back if metrics breach
• B. Roll back only after a manual audit
• C. Never roll back once deployed
• D. Ignore metrics during the rollout

Answer: (A)

When releasing changes through canary deployments, the goal is to catch problems quickly and limit impact by having guardrails that respond automatically. The safest practice is to define clear thresholds on key health and performance metrics and have the system roll back automatically if those metrics breach the limits. This creates a fast, consistent safety net: if error rates rise, latency spikes, or availability drops beyond what you’ve tolerated, traffic is redirected back to the known good version without waiting for a human to intervene. It helps protect users and keeps deployments from drifting into unstable states. This approach relies on observable signals tied to reliable service expectations, such as error rate, latency (P95 or P99), and saturation, often aligned with service level objectives. Automatic rollback enforces the policy consistently, reducing recovery time and minimizing blast radius, which is the whole point of canary testing. Rolling back only after a manual audit introduces delays that defeat the purpose of rapid, incremental releases. Never rolling back once deployed leaves you exposed to failures that can worsen over time. Ignoring metrics during rollout eliminates the very guardrail that makes canaries safe.

When releasing changes through canary deployments, the goal is to catch problems quickly and limit impact by having guardrails that respond automatically. The safest practice is to define clear thresholds on key health and performance metrics and have the system roll back automatically if those metrics breach the limits. This creates a fast, consistent safety net: if error rates rise, latency spikes, or availability drops beyond what you’ve tolerated, traffic is redirected back to the known good version without waiting for a human to intervene. It helps protect users and keeps deployments from drifting into unstable states.

This approach relies on observable signals tied to reliable service expectations, such as error rate, latency (P95 or P99), and saturation, often aligned with service level objectives. Automatic rollback enforces the policy consistently, reducing recovery time and minimizing blast radius, which is the whole point of canary testing.

Rolling back only after a manual audit introduces delays that defeat the purpose of rapid, incremental releases. Never rolling back once deployed leaves you exposed to failures that can worsen over time. Ignoring metrics during rollout eliminates the very guardrail that makes canaries safe.