Measurement
Part of Why video marketing agencies need a measurement plan from day one
Video marketing agencies measurement mistakes: six reporting errors to fix, with a decision table
A buyer's listicle on the video marketing agencies measurement mistakes that distort reporting, with a decision table on situations, choices and what to avoid.
A measurement mistake is any reporting habit that makes a number mean something other than what the reader assumes. Video agencies make six common ones. Each is fixed by writing a definition down before the campaign starts, not by buying a new tool.
What to take away
- Report views and impressions as delivery checks, not as outcomes.
- Ask for a retention curve or distribution before accepting an average watch time.
- Agree a view-through window in writing and state what it cannot prove.
- Keep one definitions page per report so unlike events are never stacked.
- Do not claim cause without a stated comparison, and do not claim uplift without a baseline.
Mistake 1: delivery reported as an outcome
Views, impressions and reach are not interchangeable. Treat them as one figure and month-to-month comparison breaks. A video can be served a million times and change nothing, so a delivery count cannot stand in for an outcome.
Pair every reach metric with a downstream signal, such as a form start or a booked call. If the agency cannot name that signal in the plan, reporting defaults to views.
Mistake 2: averages with no spread
A mean watch time can sit on top of many viewers who leave after a few seconds and a few who watch to the end. The average hides that shape. Ask for the distribution or a retention curve.
As an illustration only, a mean of fifteen seconds could describe a very different audience from a mean of fifteen seconds with a long tail. Agree which of the key metrics the agency reports, and in which form.
Mistake 3: last-click reporting that erases the video
Many viewers see a video and convert later by search or direct. Last-click attribution credits the search visit and erases the video, which does not mean the video did nothing.
Agree a view-through window before the campaign starts, write down its length and state what it cannot prove. The attribution methods article compares the main options and their limits.
Mistake 4: mixed definitions in one chart
A view means something different on each platform, and completion rates are calculated on different bases. Stack them in one chart and the total means nothing.
A useful data quality test has two parts. Consistency is the degree to which values in a data set do not contradict other values for the same entity, and validity is whether data is in the range and format expected. Apply both to every metric, and read the Government Data Quality Framework for the other four dimensions.
Mistake 5: correlation sold as cause
A rising spend line and a rising sales line can move together because of season, price or distribution. Ask what else changed in the period. A basic evaluation principle carries over: decide the comparison before you launch, not after the results arrive.
Where you cannot run a hold-back region, say so in the report and label the result as correlation. The return on investment guide shows how to state that limitation.
Mistake 6: uplift claimed with no baseline
Without a pre-campaign baseline, an uplift claim cannot be checked. Hold back a comparable region where you can, and state the limitation where you cannot. Record the baseline date and method in the same definitions page as the metric.
Measurement and consent
Advertising measurement does not require a separate consent, as the ICO guidance on online advertising explains, because collecting information to measure the effectiveness of campaigns is linked to the purpose of online advertising.
Online advertising uses of storage and access technologies still require consent. Ask the agency which technologies it uses and on what basis. See the measurement and reporting guide for how to document this in the report.
One definitions page for every report
Every report should open with a definitions page naming each metric, its platform, the counted event and the window. Everything after that page can quote a metric by name without explaining it again. Check the figures against benchmark research before you quote them to a board.
Illustrative example only, not client data. A report leads with a headline view count. The corrected version keeps that number, labels it a delivery check, and adds a definitions page, a seven-day view-through window, a retention curve and a hold-back region.
Decision table: situation, choose, avoid
| Situation | Ask for | Avoid |
|---|---|---|
| Small budget, one platform | One primary metric plus one quality metric | Dashboards with twenty tiles |
| Long consideration cycle | An agreed view-through window with stated limits | Last-click only |
| Several platforms | One shared definitions page | Stacked charts of unlike events |
| Rising sales during the campaign | A stated comparison or hold-back | Claiming cause from timing |
| Brand uplift claim | A baseline and its date | Uplift without a start point |
How should you challenge a weak report?
Ask three questions, in order:
Challenge a weak report
- Which source produced this number, and on what date?
- What population does the denominator cover?
- What decision would change if the number were different?
If the answers are vague, the report is decoration.
Common questions
Are impressions ever useful?
Yes, for delivery checks and cost per thousand comparisons within one platform. They are weak evidence of intent, so keep them out of outcome summaries.
How often should the metric set change?
Only when the objective changes. Constant changes destroy comparability, which is the main value of a monthly report.
Does attribution software remove these mistakes?
No. Software applies the rules you give it, and unclear rules produce confident-looking numbers nobody can defend.
