5 Common Mistakes To Avoid in Marketing Mix Modeling
Running Marketing Mix Modeling is an intricate process. To achieve its desired results, the project needs to be handled with a lot of care and attention to detail, at every stage. If a mistake happens at any given part of the project, its impact will eventually affect the outcome of the model.
In this article, we share some guidelines and hints that will help you avoid some of the most common mistakes in MMM, evaluate your MMM model properly, and check its consistency with reality.
Mistake 1: Not Graphing
All Your Variables
As part of Data Exploration—one of the early phases of an MMM project—it is very important that you graph all the variables at hand. This will allow you to spot any changes and variations in your variables, which might indicate a change in behavior/strategy.
Extensive graphing will also allow you to ask the right questions upfront and apply the right transformation where needed.
To a minimum, and before delivering the project, you should graph and investigate the shape of every variable that went into the final model.
Three recommendations are important at this stage:
- Compare the shape of the independent variable to the shape of the dependent variable to spot any endogeneity issue.
- Analyze the shape of the independent variable and be aware of any “unusual” shape that could indicate a change in strategy, as this might imply further testing involving variable split.
- Linked to the above, compare the shape of the raw variable to the final processed variable to spot any processing problems, and make sure that the shape of the processed variable makes sense when compared to its raw version.
Mistake 2: Abusing
Data Processors
Part of what makes MMM an art is data transformation. But be careful, this can easily backfire if misused!
We usually use standard transformations such as AdStock, Diminishing Returns, and Lags to make the variables reflect the realistic relationship with the dependent variable. But sometimes an analyst might use a combination of transformations (processors) to force the variable to fit the model, and this is wrong!
It is highly recommended to avoid using this as a reason to apply a specific set of transformations.
As an analyst, you should be able to find an explanation & interpretation behind each variable included in the model. Always ask yourself the following questions before applying any transformation:
- Why am I applying the transformation?
- What hypothesis I am trying to test?
- What should I expect?
- How can I interpret the result?
Mistake 3: Using
Nonsignificant Variables
It is very important to make sure models are robust by including as many significant variables as you possibly can (T-stat > 2 or p-value < 5%). This makes the coefficients stable across multiple model updates.
In certain cases, you can tolerate non-significant variables, for this, you need to let your business stakeholders know about it while sharing relevant explanations (e.g., the variable has a low spend below the cut through point and there are good signs of a potential positive impact based on other reports.)
For more on this, we wrote an article explaining the statistical tests you can use to evaluate your model’s robustness.
Mistake 4: Leaving Response
Curves to The Last Minute
Imagine you’ve developed your model, calculated your return on investment (ROI) and contribution, and communicated your findings to the business. Now, the company requests you to optimize the media budget based on this model.
As you try to optimize, you find out that certain channels are heavily saturated, while others remain entirely unsaturated.
This situation would lead any optimization algorithm to allocate no funds to the saturated channels and invest the entire budget into the non-saturated ones. No business is going to sign off on that!
To avoid such a scenario, it’s recommended to build the diminishing returns function into your variables during processing and modeling. Then, you can continuously monitor the optimization results and iterate as needed. Only when the modeling, optimization, contribution, and ROI align, can you consider your project done.
Mistake 5: Ignoring the
Business Story
Having a statistically viable model is a necessary, but not sufficient, condition to succeed in your MMM project. Your stakeholders are coming to you for advice, recommendations, insight, and benchmarks. Remember that they are not only after numbers, they are also after stories that they can easily disseminate to their colleagues and bosses.
To build a compelling story, make sure you add context to your measurements by either quoting external research, benchmarks, past experience, ground truth, etc.
Conclusion
Remember, your MMM project demands meticulous data exploration, clear variable interpretation, robust statistical modeling, careful optimization, and the creation of a meaningful narrative. By avoiding the mistakes we covered here, you will save your company from wasted marketing spend that could have happened from a bad model.
