Rollback
The process of returning a system to a previously stable state after a failed deployment.
Detailed Explanation
A fast rollback mechanism is the ultimate safety net for developers. If a new release causes high error rates, a rollback should be as simple as changing a pointer in the load balancer or redeploying the previous 'artifact'. Effective rollbacks rely on immutable artifacts and robust versioning in your deployment pipeline.
Quick Summary
A rollback reverts a system to the previous known-good version after a deploy goes wrong. The faster and more reliable the rollback, the more aggressive the team can be on shipping, fast forward depends on fast back.
Key Takeaways
- Rollback time should be measured in seconds to minutes, not the duration of a re-deploy.
- Immutable artifacts are the precondition: you redeploy the prior artifact, not rebuild from the prior commit.
- Database changes are the hard part, schemas evolve forward, so rollback often means rolling back code while keeping the new schema.
- Feature flags decouple deploy from release; flipping a flag is the fastest "rollback" available.
- Practice rollbacks regularly. The first real rollback during an incident is the wrong time to discover bugs in the procedure.
When to use it
- Reverting a deploy that introduced a regression or outage.
- Backing out a database migration that's causing degraded performance.
- Disabling a problematic feature via a flag without redeploying.
- Restoring a config change applied via GitOps by reverting the commit.
Common Mistakes
- Treating rollback as an emergency-only procedure instead of a normal operational tool, leads to fear of shipping.
- Coupling code and schema changes so tightly that rollback requires data restoration.
- No rollback automation, every reversion is a manual scramble.
- Forgetting to roll back config and feature-flag state alongside the code, leaving a half-rolled-back system.
Rollback, Frequently Asked
Rollback vs. roll-forward?
Rollback restores the previous version. Roll-forward ships a fix for the issue on top of the new version. Roll-forward is preferred when the new version is mostly working and the fix is small; rollback when the new version is broadly broken.
How do I roll back a database migration?
Design migrations as backward-compatible expand/contract steps so you almost never have to roll back the schema. When you must, restore from snapshot or replay write-ahead logs, both expensive and lossy, which is why migration discipline matters.
Should rollbacks be automatic?
For canary failures, yes, automated rollback on metric regression is faster than humans. For broader incidents, a human typically makes the call, but the rollback itself should be one command or one click.