Deployment Strategies

A well-built CI/CD pipeline ensures that your code is tested, linted, and scanned before it reaches production. But how you actually release that code matters just as much. A bad deployment strategy can take your entire application offline, expose every user to a broken release simultaneously, and leave you scrambling to roll back under pressure. A good deployment strategy minimizes risk, limits the blast radius of problems, and gives you a clear path to recovery when things go wrong.

This lesson covers the most important deployment strategies — blue-green, canary, and rolling deployments — along with feature flags for decoupling deployment from release, database migration best practices, smoke tests, and rollback strategies.

Blue-Green Deployments

A blue-green deployment maintains two identical production environments, traditionally called "blue" and "green." At any given time, one environment is live (serving traffic) and the other is idle. When you deploy a new version, you deploy it to the idle environment, run your verification checks against it, and then switch traffic from the live environment to the newly deployed one.

The process works like this:

Blue is live, serving all production traffic.
Deploy the new version to Green (which is currently idle).
Run smoke tests and health checks against Green.
If everything passes, switch the load balancer or DNS to point traffic to Green.
Green is now live. Blue becomes idle (and serves as your rollback target).
If a problem is discovered, switch traffic back to Blue instantly.

The biggest advantage of blue-green deployments is instant rollback. Because the previous version is still running on the idle environment, you can revert by switching traffic back in seconds. There is no need to rebuild, redeploy, or wait for a rollback deployment to complete.

Key insight: Blue-green deployments effectively eliminate deployment downtime. Users never see a "deploying, please wait" message because traffic switches atomically from one healthy environment to another. The new version is fully warmed up and verified before any user sees it.

The tradeoff is cost: you need two complete production environments. For cloud-based applications, this is often acceptable because the idle environment only needs to be running during the deployment window. You can spin it down afterward and spin it up again for the next deployment.

Canary Deployments

A canary deployment gradually rolls out the new version to a small percentage of users first, monitors for problems, and then incrementally increases the percentage until 100% of traffic is on the new version. The name comes from the "canary in a coal mine" — a small group of users acts as the early warning system.

A typical canary rollout might look like this:

Deploy the new version alongside the current version.
Route 5% of traffic to the new version.
Monitor error rates, latency, and key business metrics for 10-15 minutes.
If metrics are healthy, increase to 25% of traffic.
Monitor again. If still healthy, increase to 50%, then 75%, then 100%.
If any stage shows degradation, route all traffic back to the current version.

Canary deployments are especially valuable when you cannot fully replicate production conditions in staging. Some bugs only appear under real production load, with real user behavior, and real data. A canary catches these issues while they affect only a small percentage of users.

Modern platforms like Kubernetes, AWS ECS, and Cloudflare Workers have built-in support for traffic splitting, making canary deployments straightforward to implement. Tools like Flagger and Argo Rollouts automate the entire canary process, including metric analysis and automatic rollback.

Rolling Deployments

A rolling deployment updates instances (servers, containers, pods) one at a time or in small batches. At any point during the rollout, some instances are running the old version and others are running the new version. Once all instances are updated, the deployment is complete.

For example, if you have 10 instances:

Take instance 1 out of the load balancer.
Deploy the new version to instance 1.
Run a health check on instance 1.
Add instance 1 back to the load balancer.
Repeat for instances 2 through 10.

Rolling deployments are the default strategy in Kubernetes (controlled by the maxUnavailable and maxSurge parameters in a Deployment spec) and are straightforward to configure:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  replicas: 10
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1   # At most 1 pod unavailable during update
      maxSurge: 1          # At most 1 extra pod during update

Rolling deployments are resource-efficient (you do not need a second complete environment) and provide zero-downtime updates. However, rollback is slower than blue-green because you need to roll back each instance individually.

When to use which: Use blue-green when you need instant rollback and can afford the infrastructure cost. Use canary when you want to validate with real traffic before full rollout. Use rolling when you need zero-downtime updates with minimal infrastructure overhead. Many teams use a combination: canary for high-risk changes, rolling for routine updates.

Feature Flags

Feature flags (also called feature toggles) are a technique for decoupling deployment (shipping code to production) from release (enabling features for users). With feature flags, you can deploy new code to production at any time while keeping the new functionality hidden behind a flag that you control independently.

// Simple feature flag check
if (featureFlags.isEnabled('new-checkout-flow')) {
  renderNewCheckout();
} else {
  renderLegacyCheckout();
}

// With percentage rollout
if (featureFlags.isEnabled('new-checkout-flow', { userId: user.id })) {
  renderNewCheckout();
} else {
  renderLegacyCheckout();
}

Feature flags can be implemented at various levels of sophistication:

Environment variables: The simplest form. Set a variable like FEATURE_NEW_CHECKOUT=true in your environment configuration. Simple but requires a redeploy to change.
Configuration files: A JSON or YAML file that maps feature names to enabled/disabled. Can be updated without redeploying if loaded at runtime.
Dedicated platforms: Tools like LaunchDarkly, Unleash, Flagsmith, and Split provide real-time flag management with targeting rules (enable for specific users, percentages, geographies), audit trails, and gradual rollout capabilities.

Feature flags are powerful because they let you:

Deploy continuously: Merge code to main and deploy even if the feature is not ready for users. The code is behind a flag.
Run A/B tests: Show the new feature to 50% of users and compare metrics against the control group.
Enable for specific users: Let beta testers or internal users try the feature before public release.
Kill switch: If a feature causes problems in production, disable it instantly without a rollback deployment.

Warning: Feature flags are technical debt if not managed. Every flag adds a conditional branch to your code. Set an expiration date for each flag, and clean up flags (remove the conditional and the old code path) once the feature is fully rolled out and stable. A codebase littered with stale feature flags becomes difficult to understand and test.

Database Migrations During Deployment

Database schema changes are the hardest part of any deployment because they are difficult to roll back. Dropping a column, renaming a table, or changing a data type can break the old version of the application if you need to roll back. The key principle is to make database migrations backwards compatible:

Adding a column: Safe. The old version simply ignores the new column.
Removing a column: Dangerous. Deploy the code change first (stop reading the column), then remove the column in a separate migration.
Renaming a column: Dangerous. Instead, add the new column, backfill data, update code to use the new column, then remove the old column in a later migration.
Changing a column type: Dangerous. Create a new column with the desired type, migrate data, update code, then remove the old column.

The pattern is called expand and contract:

Expand: Add new columns, tables, or indexes without removing anything. Both old and new code can coexist.
Migrate: Deploy the new code that uses the new schema. Backfill data if needed.
Contract: Once the old code is no longer running, remove the old columns, tables, or indexes.

Always run migrations as a separate step in your deployment pipeline, before the application code is updated. This ensures the database is ready before the new code tries to use the new schema.

Smoke Tests After Deployment

A smoke test is a lightweight check that verifies the most critical functionality of your application after deployment. Unlike a full test suite, smoke tests are fast (30 seconds to 2 minutes) and focus on "is the application basically working?" rather than comprehensive coverage.

Common smoke test checks include:

Health endpoint: Does /health or /api/health return a 200 status?
Homepage loads: Does the homepage return a 200 with expected content?
Authentication works: Can a test user log in?
Database connectivity: Can the application query the database?
External dependencies: Can the application reach critical third-party services?
Key user flows: Can a user complete the most important action (e.g., search, add to cart)?

# Simple smoke test script
#!/bin/bash
set -e

BASE_URL="https://myapp.com"

echo "Checking health endpoint..."
curl -sf "$BASE_URL/health" > /dev/null

echo "Checking homepage..."
curl -sf "$BASE_URL/" | grep -q "Welcome"

echo "Checking API..."
STATUS=$(curl -sf -o /dev/null -w "%{http_code}" "$BASE_URL/api/v1/status")
[ "$STATUS" = "200" ] || exit 1

echo "All smoke tests passed."

In an automated deployment pipeline, smoke tests run immediately after deployment. If they fail, the pipeline triggers an automatic rollback. This creates a closed loop: deploy, verify, and revert if anything is wrong — all without human intervention.

Rollback Strategies

No deployment strategy is complete without a rollback plan. The speed and reliability of your rollback directly determines how much damage a bad release can cause. Here are the rollback approaches for each deployment strategy:

Blue-green rollback: Switch traffic back to the previous environment. Takes seconds. The most reliable rollback mechanism available.
Canary rollback: Route 100% of traffic back to the current stable version. Takes seconds since the stable version is still running.
Rolling rollback: Perform another rolling update, this time deploying the previous version. Takes the same amount of time as the original deployment.
Feature flag rollback: Disable the problematic feature flag. Takes seconds and does not require any deployment.
Git revert + redeploy: Revert the problematic commit in Git and trigger a new deployment. The slowest option because it goes through the full pipeline, but it works with any deployment strategy.

The best practice is to maintain the ability to deploy any previous version at any time. Use immutable artifacts (Docker images tagged with Git SHAs, versioned build outputs) so that rolling back means deploying a known-good artifact rather than rebuilding from source.

Best practice: Regularly practice rollbacks. Do not wait for a real emergency to discover that your rollback process is broken. Schedule periodic "rollback drills" where you intentionally deploy and then roll back. This builds team confidence and exposes process issues before they matter.

Choosing the Right Strategy

There is no single "best" deployment strategy. The right choice depends on your application's characteristics, infrastructure, and risk tolerance:

Static sites and JAMstack: Blue-green via CDN (deploy to a new origin, switch the CDN). Instant rollback by switching back.
Containerized applications (Kubernetes): Rolling deployments by default, canary for high-risk changes using tools like Argo Rollouts or Flagger.
Monolithic applications: Blue-green with a load balancer. Invest in feature flags for high-risk features.
Mobile apps: Canary via staged rollouts (Google Play and Apple App Store both support percentage-based rollouts). Feature flags for runtime control.
Databases and data pipelines: Blue-green with expand-and-contract migrations. Always test migrations against a copy of production data first.

Many mature teams combine strategies: rolling deployments for routine changes, canary for high-risk changes, and feature flags for features that need gradual rollout or A/B testing. The goal is always the same — get changes to users safely and quickly, with the ability to recover if anything goes wrong.

Resources

Martin Fowler on Blue-Green Deployment — The original description of the blue-green deployment pattern
LaunchDarkly Documentation — Feature flag management platform with targeting, rollouts, and experimentation