Home

Product notes

Feature Flag Rollout Strategy: Shipping to Some Users Before All of Them

4 min read

Shipping a feature to everyone at once is a big bet. If something is wrong, everyone sees it. If you can only fix it with a new app release, everyone keeps seeing it for days.

Feature flags change that. The code ships, but the feature is only switched on for some users. If it works, you widen the rollout. If it doesn't, you switch it off.

Moving more behaviour behind server-side configuration was one of the most useful changes we made at Travelxp, especially for the Smart TV apps where releases were slow and hard to undo. It turned a lot of risky launches into gradual ones.

What a flag actually does

At its simplest, a feature flag is a switch in configuration that decides whether a user sees a feature. More advanced setups let you target by platform, country, user group or a percentage of traffic.

Martin Fowler's article on feature toggles is a good overview of the different kinds and how teams use them. 1

Choose the first audience carefully

First audienceGood forWatch out for
Internal teamCatching obvious bugsNot representative of real users
Small percentage of all usersRealistic usage, limited riskNeed enough traffic to see problems
One platform or countryIsolating a specific environmentResults may not apply elsewhere
Opted-in beta usersGetting feedback from willing usersBeta users are often more forgiving

I'd usually start with the internal team, then a small percentage of real users, then widen in stages. For multi-platform products, I'd often roll out on the platform where rollback is easiest first.

Plan the stages and the watch list

Before starting, write down:

  • The stages, like 1%, 10%, 50%, 100%.
  • What you'll watch at each stage: errors, key metrics, support contacts.
  • How long you'll wait at each stage before moving on.
  • What would make you stop or roll back.

The rollback criteria are the important part. Decide them before you're looking at a worrying chart, when it's easier to think clearly. This links closely to the go/no-go checklist, just spread out over time.

Keep a kill switch

For any feature that touches payments, playback, sign-in or anything critical, I'd want a kill switch that can turn it off for everyone in minutes, without a deploy. You hope never to use it. When you need it, it saves you.

Flags aren't experiments

A staged rollout and an A/B test both show a feature to some users and not others, but they answer different questions. A rollout asks "is this safe?" An experiment asks "is this better?"

If you want to measure impact, design the experiment properly, with a clear hypothesis, the right sample size and a control group. A rollout where you glance at the metrics isn't the same thing.

Clean up old flags

Flags pile up. Every one adds a branch to the code and a question about which version a user is seeing. After a feature is fully rolled out and stable, remove the flag.

I'd keep a simple list of active flags with an owner and an expected removal date. Old flags that nobody remembers become a source of confusing bugs, which is a form of technical debt all on its own.

Tell support

When a feature is only on for some users, support will get questions like "my friend has this and I don't." Let them know what's rolling out, to whom, and what to say.

Staged rollouts take a little more planning than a single launch. In return, the worst day of a launch is a lot less bad.