A major advertiser nearly cut £70M in branded search spend. An incrementality measurement experiment proved it was driving over £1B in revenue through a conversion path attribution had missed entirely. By MASS Analytics.
- →How a Fortune 100 company nearly cut £70M in branded search — and what stopped them.
- →Why platform attribution missed over £1B in revenue, and the structural reason it always will for certain channels.
- →How incrementality experiments can validate MMM recommendations independently — not just evaluate channels for cuts.
- →Which channels benefit most from incrementality testing — and why they are the ones attribution is least equipped to handle.
There’s a principle borrowed from carpentry that applies with precision to marketing: measure twice, cut once. Before you remove something — a beam or a budget line — you want to be certain of what it’s doing. Some things look removable until you take them out and the ceiling falls in.
Branded search is one of marketing’s most reliably misunderstood channels on the question of incrementality. In attribution models it tends to look impressive: high last-click conversion rates, strong ROAS reported by the platform. But attribution measures the last step of a journey, not whether the journey would have happened without the ad. The real incrementality measurement question is: are these customers buying because of our search ad, or would they have found us anyway?
The Decision That Almost Happened
A Fortune 100 technology company was running over £70 million in annual branded search spend. The platform numbers looked solid. But internal pressure to cut costs had put the channel under review. The question: is this spend genuinely incremental, or are we paying to capture customers who were already coming?
It’s exactly the right incrementality measurement question to ask. And it has exactly one reliable method for answering it: a controlled experiment.
When the question is whether a channel is genuinely incremental, there is only one reliable answer: a controlled experiment. Platform reporting cannot settle it. Attribution cannot settle it. Only a properly designed geo or audience test can.
Why Attribution Missed It
Platform attribution measures the digital journey from ad impression to online conversion. It cannot see offline conversions: phone calls, in-store visits, or purchases made on a different device. Branded search was triggering behavior that completed outside the digital funnel entirely. From attribution’s perspective, the channel looked marginal. The incrementality experiment revealed it as critical.
This is the structural limitation that incrementality measurement is designed to address. The question isn’t whether attribution is doing its job — because it is. The question is whether the conversions it tracks represent the full causal story. For branded search at this company, they represented a small fraction of it.
“The real risk is not measuring incrementality. It’s making budget decisions on channels you don’t understand — and cutting the ones that were genuinely driving growth.”
Attribution is not broken. It is doing exactly what it was designed to do: track digital journeys to digital conversions. The gap is between what it can see and what the channel is actually driving. Incrementality measurement closes that gap.
A Second Example: Proving MMM Recommendations Work
Incrementality experiments aren’t only used to evaluate individual channels for cuts. They’re also a powerful tool for validating whether MMM-driven optimization recommendations actually deliver in the real world — independently of the team that built the model.
One of our clients, a major retailer, ran exactly this kind of validation. They implemented a structured A/B test across their store network: 10% of locations adopted an MMM-optimized marketing strategy; 90% continued with their usual approach. Before the experiment, cost performance between the two groups was negligible. After the optimized strategy was implemented, the cost difference shifted significantly in the treatment group’s favor — lower costs, maintained performance.
Of the network ran the MMM-optimized strategy in the treatment group
Cost differential emerged in favor of the treatment group after optimization
Validation: the experiment confirmed MMM’s recommendations worked in practice
The lesson from both examples is the same: incrementality measurement removes the guesswork from high-stakes decisions. Whether you’re evaluating a channel for cuts or validating an optimization strategy, the experiment provides the causal evidence that model outputs and platform reporting simply cannot.
- Branded search — capturing offline conversions and cross-device completion paths
- TV and connected TV — brand equity effects and delayed purchase decisions
- Radio and audio — in-store and phone behavior with no digital trace
- Out-of-home — awareness effects invisible to any digital funnel
- Retail media — halo effects on in-store purchase beyond the platform’s reporting
Frequently Asked Questions
What is incrementality measurement and why does it matter?
Incrementality measurement determines whether a marketing channel or tactic is genuinely causing additional sales — or whether those customers would have converted anyway. It matters because platform attribution can only track digital journeys to digital conversions. For channels like branded search, TV, or out-of-home, the causal story plays out largely offline, where attribution is blind.
Why can’t platform attribution answer the incrementality question?
Platform attribution is designed to map the digital path from ad impression to online conversion. It cannot see offline conversions — phone calls, in-store visits, or purchases completed on a different device. A channel can appear marginal in attribution while driving substantial revenue through conversion paths it cannot observe. Branded search at the Fortune 100 company described above is a precise example of this gap.
Can incrementality experiments be used to validate MMM recommendations?
Yes — and this is one of the most valuable use cases. A structured A/B test across store locations or geographic markets can confirm whether an MMM-optimized strategy delivers the improvements the model predicted. This provides independent, causal validation that no model output or platform report can replicate.
Which channels benefit most from incrementality testing?
Any channel where a significant portion of the customer journey completes offline: branded search, TV and connected TV, radio and audio, out-of-home, and retail media. These are also the channels where attribution is structurally weakest — making them the highest-priority candidates for a controlled experiment.
Ready to measure what attribution misses?
Talk to MASS Analytics about building an incrementality measurement program that gives you the causal evidence to make confident budget decisions.
Incrementality Measurement: Key Takeaways
- ✓Measure twice before you cut. A channel that looks removable in attribution may be holding up conversions you can’t see. A controlled experiment is the only way to know.
- ✓Attribution is not broken — it’s limited by design. It tracks digital journeys to digital conversions. The gap is structural, not a flaw. Incrementality measurement closes it.
- ✓Incrementality experiments validate MMM, not just evaluate channels. A structured A/B test can confirm whether an optimized strategy delivers in practice — providing causal evidence that model outputs alone cannot.
- ✓The highest-risk channels are the ones attribution handles worst. Branded search, TV, radio, out-of-home, and retail media are where offline conversion paths dominate — and where incrementality testing delivers the most insight.