Blue/Green / Canary / Feature Flag / A/B testing: traffic shift, rollback

puml.online
  2026-07-30 18:15 2026-07-30 18:15 Created   2026-08-14 21:34:29 2026-08-14 21:34:29 Updated    

Releasing a new version is not “push to all” — each of the four strategies has trade-offs. This is Blue/Green / Canary / Feature Flag / A/B test architecture, traffic shift mechanism, rollback plans, plus Feature Flag, Argo Rollouts, LaunchDarkly tools in practice.

Four deployment strategies compared

Strategy Traffic shift Rollback speed Risk Best for
Blue/Green 100% one-shot seconds (switch to blue) high major version
Canary 1% → 5% → 25% → 100% minutes (shift traffic) low general
Feature Flag per user/feature seconds (toggle flag) low new features
A/B test per user group comparison changes big low experiments

Strategy 1: Blue/Green Deployment

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
@startuml
title "Blue/Green Deployment"

actor "Users" as Users
participant "Load Balancer" as LB
participant "Blue Env\n(v1.0)" as Blue
participant "Green Env\n(v2.0)" as Green
database "Blue DB" as BlueDB
database "Green DB" as GreenDB

== Deploy v2.0 to Green ==
Users -> LB : ① 100% traffic
LB -> Blue : ② route to Blue
Blue -> BlueDB : ③ read v1.0
BlueDB --> Blue : ④ data

note over Green
  deploy v2.0 to Green
  no traffic
  internal testing
end note

Green -> GreenDB : ⑤ data migration / dual-write test

note over LB : switch traffic
LB -> Green : ⑥ 100% traffic to Green
Green -> GreenDB : ⑦ read v2.0

== Rollback (seconds) ==
LB -> Blue : ⑧ one second back to Blue (emergency)

note right of LB
  DNS / nginx upstream /
  k8s service selector
  one line to switch
end note

@enduml

Blue/Green key points:

  • two complete environments — resources × 2
  • shared database — both versions compatible with same schema
  • switch instant — DNS / LB one command
  • rollback seconds — switch back to blue

Strategy 2: Canary Deployment

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
@startuml
title "Canary Deployment - Gradual Rollout"

actor "Users" as Users
participant "Load Balancer\n(istio / nginx)" as LB
participant "v1.0 (95% traffic)" as V1
participant "v2.0 (5% traffic)" as V2
database "DB" as DB

== Step 1: 1% canary ==
Users -> LB : ① 1000 users access
LB -> V2 : ② 1% → 10 users to v2.0
LB -> V1 : ③ 99% → 990 users to v1.0

note over V2
  monitor v2.0:
  - error rate < 1%?
  - p99 latency < 500ms?
  - CPU normal?
end note

V2 -> DB : ④ read/write
V1 -> DB : ④ read/write
DB --> V2 : ⑤ data
DB --> V1 : ⑤ data

== Step 2: increase traffic ==
LB -> V2 : ⑥ 5% (50 users)
LB -> V1 : ⑦ 95% (950 users)

note over LB
  monitor stable for 10 min
  bump to 25%
end note

== Step 3: 25% canary ==
LB -> V2 : ⑧ 25% (250 users)
LB -> V1 : ⑨ 75% (750 users)

== Step 4: 100% full ==
LB -> V2 : ⑩ 100% switch to v2.0
V1 --> [*] : ⑪ old version offline

note over LB
  on anomaly:
  LB -> V1 : emergency back to 100% v1.0
  seconds rollback
end note

@enduml

Canary key points:

  • progressive traffic bump — 1% → 5% → 25% → 50% → 100%
  • monitor comparison — v1.0 vs v2.0 error rate / latency / CPU
  • fast rollback — shift traffic, don’t touch pods

Strategy 3: Feature Flag

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
@startuml
title "Feature Flag Toggling"

actor "User A" as UA
actor "User B" as UB
actor "User C" as UC

participant "App" as App
participant "Feature Flag Service" as FF
database "Flag Config" as Config

UA -> App : ① request
UB -> App : ② request
UC -> App : ③ request

App -> FF : ④ evaluate "new_checkout"
FF -> Config : ⑤ lookup rule

note right of FF
  rule:
  - User A,B → enabled
  - User C → disabled
  - gradual 50%
end note

Config --> FF : ⑥ rule

FF --> App : ⑦ User A: enabled
FF --> App : ⑧ User B: enabled
FF --> App : ⑨ User C: disabled

App -> App : ⑩ User A goes through new checkout (v2 code path)
App -> App : ⑪ User B goes through new checkout
App -> App : ⑫ User C goes through old checkout (v1)

@enduml

Feature Flag types:

  • Boolean flag — fully on / fully off
  • user whitelist — internal employees / beta users
  • percentage rollout — 50% of users
  • by attribute — country / device / subscription tier
  • A/B experiment — different variants

LaunchDarkly / Unleash in practice:

1
2
3
4
5
6
7
8
9
10
11
12
const LaunchDarkly = require('ldclient-node');
const ldClient = LaunchDarkly.init('sdk-key-xxx');

ldClient.identify({ key: userId, email: user.email });

const enabled = await ldClient.variation('new-checkout', user, false);

if (enabled) {
  return newCheckoutFlow(cart);
} else {
  return oldCheckoutFlow(cart);
}

Strategy 4: A/B Testing

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
@startuml
title "A/B Test Experiment Flow"

actor "User" as U
participant "App" as App
participant "Experiment Service" as Exp
database "Experiment Config" as Conf
participant "Analytics" as Ana

U -> App : ① access
App -> Exp : ② get variant "checkout_color"
Exp -> Conf : ③ lookup experiment config
note right
  experiment:
  - 50% → variant A (blue button)
  - 50% → variant B (green button)
  track metrics:
  - click rate
  - conversion rate
end note
Conf --> Exp : ④ rule
Exp --> App : ⑤ variant = "B"

App -> App : ⑥ render green button (variant B)
U -> App : ⑦ click green button
App -> Ana : ⑧ event {variant:B, event:click}

note over Ana : analyze after 7 days
Ana -> Ana : ⑨ statistics: B conversion +15% (p<0.01)

@enduml

A/B testing key points:

  • control group — variant A (original)
  • experimental group — variant B (new design)
  • sample size calculation — needs enough traffic for p<0.05 significance
  • single variable — only change one thing, otherwise attribution unclear
  • track metrics — primary + secondary + guardrail

Blue/Green vs Canary vs Feature Flag

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
@startuml
title "Strategy Selection Decision"

start

:new version to deploy;

if (is DB schema change compatible)
  if (compatible)
    :continue;
  else
    :do schema migration first (expand-migrate-contract);
  end
end

:new feature vs perf fix?;

if (new feature)
  if (user scope)
    if (internal test)
      :Feature Flag (whitelist);
    else if (5% rollout)
      :Feature Flag (5%);
    else if (full)
      :Feature Flag (100%) + direct deploy;
    end
  end
else
  if (high-risk change)
    :Blue/Green (test full traffic);
  else
    :Canary (1% → 5% → 25% → 100%);
  end
end

:after metrics stable then done;

stop

@enduml

Argo Rollouts canary in practice

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
# rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: user-service
spec:
  replicas: 10
  selector:
    matchLabels:
      app: user-service
  strategy:
    canary:
      steps:
      - setWeight: 5      # 5% traffic to v2
      - pause: {duration: 5m}
      - setWeight: 25     # 25%
      - pause: {duration: 5m}
      - setWeight: 50
      - pause: {duration: 10m}
      - setWeight: 100    # full
      - pause: {duration: 5m}
      canaryService: user-service-canary
      stableService: user-service-stable
      trafficRouting:
        istio:
          virtualService:
            name: user-service-vs
      analysis:
        templates:
        - templateName: success-rate
        - templateName: latency
        startingStep: 2      # analyze from 25%
        args:
        - name: service-name
          value: user-service
  template:
    metadata:
      labels:
        app: user-service
    spec:
      containers:
      - name: user-service
        image: user-service:v2.0
---
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
  name: success-rate
spec:
  metrics:
  - name: success-rate
    interval: 30s
    successCondition: result[0] >= 0.95
    failureLimit: 3
    provider:
      prometheus:
        address: https://prometheus.internal
        query: |
          sum(rate(http_requests_total{status!~"5..",service="user-service"}[5m]))
          /
          sum(rate(http_requests_total{service="user-service"}[5m]))

Argo Rollouts behavior:

  • deploy v2 to canary pods
  • adjust Istio VirtualService traffic
  • pause N minutes each step
  • run Prometheus analysis
  • fail auto-abort, rollback to v1
  • success auto next step

Database schema compatibility

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
@startuml
title "Backward-Compatible Schema Migration"

database "DB v1.0" as V1
database "DB v1.5 (compatible)" as V15
database "DB v2.0" as V2

== Step 1: add new column (keep old) ==
V1 -> V15 : ① ALTER TABLE ADD COLUMN new_field VARCHAR(50)
note right
  v1.0 app: read old_field
  v2.0 app: read new_field (fallback to old_field)
end note

== Step 2: dual write ==
V15 -> V15 : ② app dual-write old_field + new_field
note right
  v2.0 writes new_field
  old app still reads old_field
end note

== Step 3: backfill ==
V15 -> V15 : ③ UPDATE SET new_field = old_field
note right
  historical data new_field also filled
end note

== Step 4: switch read ==
V15 -> V15 : ④ v2.0 reads new_field

== Step 5: cleanup ==
V15 -> V2 : ⑤ DROP COLUMN old_field
note right
  safe to drop, old version no longer uses
end note

@enduml

Core principle: always add before delete — v2.0 deployment still compatible with v1.0 code.

Rollback decision tree

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
@startuml
title "Rollback Decision Tree"

start

:monitor alert;

if (error rate > 5%)
  :immediate rollback (Blue/Green switch back);
  stop
elseif (error rate 1-5%)
  :pause rollout;
  if (recover in 10 min)
    :continue;
  else
    :rollback;
  end
elseif (error rate < 1% but latency high)
  :check upstream dependency issue;
  if (upstream issue)
    :continue after upstream recovers;
  else
    :pause or rollback;
  end
elseif (business metric anomaly)
  :check if experimental group issue;
  if (new version introduced)
    :Feature Flag off;
  else
    :no rollback;
  end
end

stop

@enduml

Field foot-guns

  • DB schema incompatible — v2.0 deployment, v1.0 code can’t connect. Enforce expand-migrate-contract.
  • Canary pod doesn’t get traffic — Istio VirtualService not configured. canaryService / stableService both needed.
  • Feature flag not cleaned — 100% on, flag still in code. Periodic audit + tool enforce cleanup.
  • A/B test insufficient samples — small traffic, not significant. Use power analysis for minimum sample size.
  • Rollback too slow — Canary manual each step. Use Argo Rollouts auto-rollback.
  • Rollback causes data inconsistency — v2.0 wrote data, v1.0 can’t read new field. Dual-write compatibility period.
  • Monitoring blind spot — only HTTP status codes, not business errors. Business events + error aggregation.
  • Feature Flag service down — entire app flag-loading timeout. Fallback to default + local cache.

Decision tree

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
What to deploy?
├─ New version full release → Blue/Green (high-risk) / Canary (standard)
├─ New feature gradual → Feature Flag
├─ Experiment validation → A/B test
└─ Emergency fix → Blue/Green + immediate switch

Risk level?
├─ Low (frontend only / perf optimization) → Feature Flag direct enable
├─ Medium (backend logic) → Canary
└─ High (DB migration / architecture change) → Blue/Green + full test

Rollback requirement?
├─ Seconds → Blue/Green
├─ Minutes → Canary + automation
└─ Rollback can be deferred → Feature Flag (cleanup in next release)

Minimum viable: Feature Flag (Unleash self-hosted) + Canary manual. Production: Argo Rollouts + Prometheus analysis + LaunchDarkly (commercial).

Remember: no zero-risk deploy, only low-risk deploy + fast rollback. Monitoring + automation + data compatibility are three cores. Watch monitoring 2 hours after deployment day, don’t just push and leave.

  • Title: Blue/Green / Canary / Feature Flag / A/B testing: traffic shift, rollback
  • Author: puml.online
  • Created at : 2026-07-30 18:15:00
  • Updated at : 2026-08-14 21:34:29
  • Link: https://puml.online/blog/plantuml-blue-green-canary-en/
  • License: This work is licensed under CC BY-NC-SA 4.0.
On this page
Blue/Green / Canary / Feature Flag / A/B testing: traffic shift, rollback
  1. Four deployment strategies compared
  2. Strategy 1: Blue/Green Deployment
  3. Strategy 2: Canary Deployment
  4. Strategy 3: Feature Flag
  5. Strategy 4: A/B Testing
  6. Blue/Green vs Canary vs Feature Flag
  7. Argo Rollouts canary in practice
  8. Database schema compatibility
  9. Rollback decision tree
  10. Field foot-guns
  11. Decision tree