What this article covers
- Why incrementality experiments and MMM form a two-way improvement loop — and why neither simply consumes the other’s output
- Three levels of calibration rigor and how to choose the right one for your measurement maturity
- How to align KPIs, time granularity, and carryover effects so experiment data and model data speak the same language
- The five specific things that change in the model when incrementality calibration works correctly
- A real D2C case where one well-integrated experiment produced a 26% increase in measured incremental sales contribution
Calibrating MMM with incrementality evidence is about getting closer to the truth instead of trusting what the model produces alone. MMM infers causal effects from patterns in data. Incrementality experiments anchor those inferences to something firmer: directly observed evidence of what advertising caused.
When you combine both properly, you get a measurement system that is substantially more reliable than either approach alone — one where each method makes the other more useful over time.
The Two-way Relationship
It is worth being explicit about something that often goes underappreciated. Incrementality experiments calibrate MMM. But MMM also guides better experiments. The model can tell you how many weeks a test needs to run to detect a meaningful signal, which regions provide the best test-control match, and what level of spend variation gives the experiment its best chance of yielding a usable incrementality estimate. Neither is simply a consumer of the other’s outputs — they improve each other.
The Key Relationship
MMM and incrementality experiments form a reinforcing loop. The model guides better experiment design; the experiment produces evidence that sharpens the model. Each method makes the other more reliable over time.
Three Levels of Calibration
Not all calibration is equal. There are three approaches, each with a different level of rigor:
The Rule
Choose the calibration level that matches your experimental infrastructure and data maturity. A level-one calibration done rigorously outperforms a level-three calibration done carelessly.
Getting the Comparison Right: KPIs and Carryover
Before you can calibrate, the experiment data and the MMM data must speak the same language. This is where calibration efforts most often break down. If the experiment measures online conversions but your MMM targets total revenue, you need to transform one into the other. If the experiment is daily and the MMM is weekly, you need to aggregate. Mismatched units produce calibration that is worse than no calibration.
Carryover effects require particular care. Experiments typically measure short-term impact within the test window. MMM accounts for the full effect including carryover — the weeks or months of continued influence after the campaign ends. The experiment’s iROAS will often look like a lower bound compared to the model’s estimate. The right approach is to compare short-term effects like-for-like, not to treat the experiment’s number as the complete picture.
The Fix
Align KPIs, time granularity, and carryover handling before calibrating. The experiment’s iROAS is a short-term lower bound — compare like-for-like and never treat it as the complete picture of advertising impact.
What Changes in the Model When Calibration Works
What changes after incrementality calibration
- Channel contribution corrected to reflect the measured causal incrementality
- ROAS rising to align with the experimentally validated lift
- Optimization recommendations shifting to account for the calibrated channel
- Model fit improving: better statistical performance, more stable coefficients
- Budget recommendations becoming more defensible to finance and boards
In one D2C client engagement, properly calibrated MMM produced a 26% increase in measured incremental sales contribution, a 24% improvement in measured ROI, and a 33% increase in budget allocation to the tested channel — all from a single well-designed incrementality experiment feeding into a properly integrated model.
“When your MMM stops guessing and starts learning from observed incrementality evidence, the whole organization’s relationship with measurement changes.”
Previous article: The $1 Billion Incrementality Question
Next article: Which Channels Need Incrementality Testing First?
Frequently Asked Questions
What does it mean to calibrate an MMM with incrementality evidence?
Calibration means using the results of a controlled incrementality experiment — such as a geo test or matched-market test — to constrain or correct the coefficients your MMM assigns to a given channel. Instead of relying purely on statistical inference from historical patterns, you anchor the model’s estimate to directly observed evidence of what advertising caused.
Do incrementality experiments and MMM always produce the same estimate?
Not always — and the difference is usually expected. Experiments measure short-term lift within a test window; MMM accounts for full carryover effects that continue after a campaign ends. The experiment’s iROAS is typically a lower bound of the model’s full-period estimate. Mismatched KPIs or time granularity can also create gaps — aligning these before calibrating resolves most conflicts.
How does MMM help design better incrementality experiments?
The model can estimate how many test weeks are needed to detect a meaningful signal, identify regions with the best test-control match, and indicate what spend variation gives the experiment its best chance of producing a usable iROAS estimate. This is the two-way nature of the relationship — MMM makes experiments more efficient before a single conversion is measured.
What results can a well-integrated incrementality experiment produce?
In the D2C engagement described in this article, one properly integrated experiment produced a 26% increase in measured incremental sales contribution, a 24% improvement in measured ROI, and a 33% increase in budget allocation to the tested channel. The key is integration — the experiment’s evidence must feed correctly into the model, not just sit alongside it.
Incrementality Measurement and MMM: Key Takeaways
✓
The two-way relationship. Incrementality experiments calibrate MMM. MMM guides better experiments. Neither method simply consumes the other’s output — they improve each other.
✓
Not all calibration is equal. Three levels of rigor exist. Choosing the right one depends on your experimental infrastructure and data maturity.
✓
Mismatched data breaks calibration. KPIs, time granularity, and carryover handling must all be aligned. Mismatched units produce calibration that is worse than no calibration.
✓
Calibration changes five things in the model. Channel contributions, ROAS, optimization recommendations, model fit, and the defensibility of budget recommendations all improve when calibration is done correctly.
✓
The business impact is material. One well-integrated experiment: 26% more incremental sales contribution, 24% better ROI, 33% more budget directed to the right channel.
Related Articles
Why Most MMM Programs Optimize Reports, Not Outcomes
The insight-to-decision gap is organizational, not statistical. Here’s the fix.
Incrementality
Measurement Guide
The Complete Guide to Incrementality Measurement
The full series — from fundamentals to calibration in practice.
Incrementality
Measurement
Incrementality Measurement at MASS Analytics
How we help you design, run, and integrate incrementality tests into your MMM program.
Ready to sharpen your MMM with incrementality evidence?
Talk to MASS Analytics about integrating incrementality measurement into your marketing mix model.