In MMM, it is inevitable that some variables are not relevant to the model. Often they contain such a large measurement error that they become irrelevant for the analysis.
The question that should be asked now is: what is the procedure that should be applied to ensure only relevant variables are kept?
That’s where model selection comes in as part of the Modeling stage of the standard marketing mix modeling workflow.
What is Model Selection?
Model selection, or feature selection, is the process of selecting the most relevant variables to use as inputs for your marketing mix model. This is crucial to do because it leads to:
- Better performance of the final model
- A simpler and easier to understand model
- Faster results (because it’s faster to compute)
To reap the benefits of this step in the MMM workflow, you could apply multiple techniques. The next part of this blog is a dive into a few of these methods.
Model Selection Using Stepwise Methods
Stepwise procedures are techniques whereby the choice of the selected variable is done through the deployment of an automatic procedure. Let’s look at the three ways you can do this:
The Forward Selection:
The idea here is that the analyst begins with no variables in the model, and then they start adding them gradually. They only add the plausible ones that can improve the quality of the model based on the chosen criteria, for example, R Squared, and Durbin Watson.
The Backwards Elimination
This method starts with all the variables in the model, and then it eliminates the ones whose removal from the model will improve its quality. Similar to the Forward Selection, this is done based on a set of criteria that the analyst defines upfront.
The Bi-Directional
This is based on a combination of the Forward Selection and the Backwards Elimination. In this case, both techniques are applied simultaneously.
Modeling in MassTer
Building models in the Marketing Mix Modeling tool, MassTer can be done in 3 ways:
- Automatically; by using the auto-modeler built into the tool, all you need to do is specify to MassTer the set of independent variables you want to consider in the search for the best model.
- Manually; simply select the variables you want to include in the model
- Hybrid; build models automatically and later refine them manually using the built-in helper.
How We Do it at MASS Analytics: Genetic Algorithm
The method used to select the best combination of variables in the software MassTer is based on the concept of genetic algorithms. As the name suggests, this algorithm follows the same logic of genetic mutation found in nature to optimize the results we need. We first start by constructing a population of random chromosomes.
Random Forest techniques are also used in this context to create a range of initial populations. Each chromosome represents a combination of included and excluded variables.
Then, we iteratively and randomly mutate those chromosomes, while monitoring the fitness score until the latter can’t be improved any further.
The fitness score could be a combination of R squared, VIF statistic, Durbin Watson etc.
Other Methods You Can Use
There are many methods you can use that we did not go through in this article. For further study, we recommend exploring the literature on supervised feature selection algorithms as those are the ones that are applicable to marketing mix modeling.
They are split into 3 categories:
- Filter methods
- Embedded methods
- Wrapper methods
Tools to Implement Model Selection
A resourceful team of data scientists can code their own model selection script using existing libraries.
Open-source has been around, and you can use some helpful machine-learning libraries to facilitate this process. Do note that this will require in-depth mathematical and programming experience to get it right.
But if you’re looking to save time and get the best and most accurate results, you can use specialist MMM tools that can handle this entire process for you, all while providing you with the support, knowledge, and extensive experience of the specialists behind the technology.
Our very own MassTer has the auto-modeler feature, generating thousands of models and selecting the best one without you having to worry about the technical side of the process.
Conclusion
In this article, we looked at what model selection is, and why it’s important to the MMM project. We also looked at the different methods and tools you can use to implement this process. The next step is model evaluation, where we evaluate the robustness of the model and reliability of the results, see you there!