Running Marketing Mix Modeling is an intricate process. To achieve its desired results, the project needs to be handled with a lot of care and attention to detail, at every stage. If a mistake happens at any given part of the project, its impact will eventually affect the outcome of the model.
In this article, we share some guidelines and hints that will help you avoid the most common mistakes in MMM, evaluate your model properly, and check its consistency with reality.
- →Why skipping variable graphing leaves data quality issues invisible until it’s too late.
- →How abusing data transformations forces a model to fit reality rather than reflect it.
- →Why nonsignificant variables create unstable coefficients that break on the next model update.
- →How leaving response curves to the last minute makes budget optimization impossible to sign off.
- →Why a statistically sound model is not enough — and what a compelling business story requires.
Mistake 1: Not Graphing All Your Variables
As part of Data Exploration — one of the early phases of an MMM project — it is very important that you graph all the variables at hand. This will allow you to spot any changes and variations in your variables, which might indicate a change in behavior or strategy.
Extensive graphing will also allow you to ask the right questions upfront and apply the right transformation where needed. To a minimum, and before delivering the project, you should graph and investigate the shape of every variable that went into the final model.
Three recommendations are important at this stage:
- Compare the shape of the independent variable to the shape of the dependent variable to spot any endogeneity issue.
- Analyze the shape of the independent variable and be aware of any “unusual” shape that could indicate a change in strategy, as this might imply further testing involving variable split.
- Compare the shape of the raw variable to the final processed variable to spot any processing problems, and make sure that the shape of the processed variable makes sense when compared to its raw version.
Never deliver a model without graphing every variable that went into it. A shape you haven’t looked at is a problem you haven’t found yet.
Mistake 2: Abusing Data Processors
Part of what makes MMM an art is data transformation. But be careful — this can easily backfire if misused.
We usually use standard transformations such as AdStock, Diminishing Returns, and Lags to make the variables reflect the realistic relationship with the dependent variable. But sometimes an analyst might use a combination of transformations to force the variable to fit the model. That is wrong.
“As an analyst, you should be able to find an explanation and interpretation behind each variable included in the model. If you cannot explain why a transformation was applied, it should not be there.”
Always ask yourself the following questions before applying any transformation:
- Why am I applying the transformation?
- What hypothesis am I trying to test?
- What should I expect?
- How can I interpret the result?
If you cannot answer all four questions above for a given transformation, do not apply it. Forcing a fit is not modeling — it is curve-fitting with extra steps.
Mistake 3: Using Nonsignificant Variables
It is very important to make sure models are robust by including as many significant variables as you possibly can (T-stat > 2 or p-value < 5%). This makes the coefficients stable across multiple model updates.
In certain cases, you can tolerate non-significant variables. When you do, you need to let your business stakeholders know and share relevant explanations — for example, that the variable has a low spend below the cut-through point, and there are signs of a potential positive impact based on other reports.
T-stat > 2 or p-value < 5% is the baseline for significance. Any exception must be documented and communicated to stakeholders — not silently absorbed into the model.
Mistake 4: Leaving Response Curves to The Last Minute
Imagine you’ve developed your model, calculated your return on investment (ROI) and contribution, and communicated your findings to the business. Now, the company requests you to optimize the media budget based on this model.
As you try to optimize, you find out that certain channels are heavily saturated, while others remain entirely unsaturated. This leads any optimization algorithm to allocate no funds to the saturated channels and invest the entire budget into the non-saturated ones. No business is going to sign off on that.
“Optimization and modeling are not sequential steps. They are the same step. A model without a validated response curve is not ready for budget decisions.”
To avoid this, build the diminishing returns function into your variables during processing and modeling. Then continuously monitor the optimization results and iterate as needed. Only when modeling, optimization, contribution, and ROI align can you consider the project done.
Before signing off on a model, confirm that modeling, optimization, contribution, and ROI all tell a consistent story. If they don’t, the response curve needs more work — not the stakeholder presentation.
Mistake 5: Ignoring the Business Story
Having a statistically viable model is a necessary, but not sufficient, condition to succeed in your MMM project. Your stakeholders are coming to you for advice, recommendations, insight, and benchmarks. They are not only after numbers — they are after stories they can easily disseminate to their colleagues and bosses.
To build a compelling story, add context to your measurements by quoting external research, industry benchmarks, past experience, or ground truth data. A number without context is just a number. A number with context is a recommendation.
Every finding in your output should answer two questions: what does this number mean, and what should we do about it? If it only answers the first, the business story is not finished.
Frequently Asked Questions
What is AdStock and why does misusing it break an MMM model?
AdStock is a transformation that models the carry-over effect of advertising — the idea that a campaign’s impact on sales persists beyond the week it ran. Misusing it means applying AdStock rates that are not grounded in the channel’s actual decay behavior, often to make a coefficient look more significant. This produces a model that fits historical data well but generates unreliable forecasts and misleading ROI estimates.
How do I know if my MMM model is robust enough to present to stakeholders?
A robust model passes three tests: all included variables are statistically significant (or their exceptions are documented), response curves are validated before optimization runs, and the model’s output tells a consistent story across contribution, ROI, and scenario planning. If any of these three are missing, the model is not ready for a stakeholder presentation — it is ready for more iterations.
Can a model be statistically correct but still produce bad budget recommendations?
Yes, and this is one of the most common failures in MMM. A model can have excellent fit statistics while still recommending budget shifts the business cannot act on — typically because response curves were not built in during modeling, or because non-significant variables are inflating or deflating certain channel coefficients. Statistical validity is necessary, but it is not the same as commercial validity.
Key Takeaways
- ✓Graph every variable before you deliver. Shape anomalies you catch during data exploration are problems you won’t have to explain after the model is built.
- ✓Every transformation needs a hypothesis. If you cannot explain why a processor was applied, it should not be in the model.
- ✓Significant variables make stable models. Non-significant variables are a risk to document and communicate — not silently include.
- ✓Response curves are part of modeling, not post-modeling. Build diminishing returns in from the start, or the optimization output will be unusable.
- ✓A number without context is not a recommendation. The business story is not a slide deck — it is the difference between findings that change decisions and findings that get filed away.
Building an MMM program that actually moves budgets?
Talk to MASS Analytics about getting your model, optimization, and business story aligned — from the first cycle.