Watch time is the number everyone in streaming wants to talk about. It's also the number that tells you the least when something goes wrong.
If watch time drops this week, was it the content slate? A bad app release on one TV brand? A CDN problem in one country? Or just a holiday? On its own, the headline number can't tell you.
At Travelxp I worked on an OTT platform that ran on Web, iOS, Android and Smart TVs across more than 40 countries, and part of my job was setting up the telemetry that told us how streaming quality and retention were actually doing. This post is the order I'd set those metrics up in if I were starting again.
Start with whether the video plays at all
Before engagement, check whether people can actually watch. A few basics:
- Play attempts that fail. Someone pressed play and got an error or nothing.
- Time to first frame. How long between pressing play and seeing video.
- Rebuffering. How often playback stalls, and for how long, once it's started.
- Exits before video start. People who give up while the spinner is still going.
These are quality-of-experience metrics. They sound like engineering numbers, but they decide whether everything else on the dashboard means anything. A great catalogue doesn't help if a big chunk of play attempts on one platform never start.
Segment, or the average will lie to you
A streaming service is many products sharing a backend. An old Smart TV model on a slow connection behaves nothing like an iPhone on office Wi-Fi.
| Cut | Why it matters |
|---|---|
| Platform and device model | TV apps and older devices fail in ways mobile doesn't |
| App version | Tells you if a release caused the change |
| Country and network | CDN and bandwidth issues are often regional |
| Content type | Live, long-form and short clips behave differently |
| New vs returning viewers | New users are far less forgiving of a bad first play |
When a number moved, the first thing I'd do was break it down by platform and app version. More often than you'd expect, the "engagement problem" turned out to be one release on one device family.
Then look at discovery
Once playback is healthy, ask how people find something to watch. Useful signals:
- How long from opening the app to starting a play.
- How many sessions end without any play at all.
- Which rows or entry points actually lead to plays, and which just get scrolled past.
A session with no play is a strong signal. The person opened the app wanting something and left without it. That's usually a discovery or catalogue problem, and it shows up long before churn does.
Engagement and subscription health last
Now watch time, completion rate and return visits start to mean something. For a subscription service I'd pair them with trial conversion, renewal and voluntary cancellation, read by cohort so you can see whether newer subscribers behave differently from older ones.
Google's HEART framework is a handy way to connect goals to signals and metrics across happiness, engagement, adoption, retention and task success. 1 It helped me stop treating every number as equally important.
Define each metric once, in writing
"Watch time" can mean total minutes, minutes per viewer, or minutes per session. "Active user" can mean opened the app or played something. Pick one definition, write it down, and use it everywhere. Otherwise two teams will present the same week with different numbers and spend the meeting reconciling them.
For the adoption side of this, I wrote separately about choosing denominators. The same discipline applies here.
If I could only look at one thing on a Monday morning, it wouldn't be total watch time. It would be failed plays by platform and app version. When that's healthy, the rest of the dashboard is worth reading.