Why Data Processing in Marketing Mix Modeling projects is Instrumental ?

Now that the analyst is familiar with their charted data, in-depth work should be conducted to successfully transform the variables and test various hypotheses to get a better idea about the reality behind the model inputs. This article explores a set of transformations that could be applied during the Data Processing phase.

The rationale behind Data Processing in Marketing Mix Modeling

Data processing in Marketing Mix Modeling involves transforming raw data into useful formats that can be analyzed effectively. This phase comes third in the MMM workflow:

Figure 1: MMM Workflow

The idea behind data processing is to create additional variables (they stem from Data Exploration). Those variables are then tested through the modeling stage, to either confirm or refute the hypothesis that had been established.

The new variables that the analyst could create at this stage are divided into two main groups:

Calendar variables: They are created from scratch. They are not based on raw variables that exist in the project. For example, dummies, or trends.

Calendar Variables in MassTer

Creating calendar variables can be done with a couple of clicks on your MMM Software. Select the type of variable you would like to create, define your period and you’re good to go. Once created, you can further process it in the Data processing tab by simply selecting your processor and creating your transformed calendar variable

Processed variables: They are transformations applied to raw variables. For example, if decay is applied to a TV GRP, that results in a processed variable. ⇒ Diminishing returns, lags, seasonality, adding up variables, and splitting variables are also examples of processed variables that we will explore in this article.

MassTer allows you to select a data series and choose one or more processors to be applied among predefined processors of the MMM Software. This is a unique and standout feature in MassTer that will add so much ease in your MMM journey. These processors include:

• Seasonality
• Multiply / Multiply by Scalar / Power
• Moving Average / Median Filter
• Diminishing Returns
• Window (Real Values)
• Diminishing Returns
• Splitter / Split Regions
• Edit
• Decay
• Divide / Divide by Scalar
• Min / Max Ratio
• Added / Weighted Sum

a. Dummy variables: They are generally created to model events, whether they are known or unknown. Analysts should be very careful when using dummy variables. The exaggerated and non explained use of dummy variables may result in an overfitted model.

Example: An out of stock, which happens 3 times a year, is a situation where 3 dummies could be added to the model to represent this event.

Figure 2

b. Trends and Part Trends: They are created to model the gradual increase, or decrease, in the modeled KPI. The delisting of a product is a case in point, and so is the launch of a new product, as illustrated in figure 3.

⇒ In the retail sector, it could be modeling the opening or the closure of a store.

Figure 3

c. Periodic: This is generally used to model a frequent event, for example: Payday (figure 4). People get paid around the 25th of the month; so, an increase in consumption is expected. In this case, the analyst could create a variable that takes the value “1” on the week where the 25th of every month falls. This could be done using the Periodic transformation.

Figure 4

Another example is seasonality during a specific month of the year such as June. Here, a periodic seasonality in June could be created. And if the data modeled is over the course of three years, we will end up with three sequences of those using the periodic transformation.

Figure 5

Processed variables are created either out of the raw variables that are already collected as part of the kickoff meeting and the data request, or out of the Calendar Variables. What is advisable before creating any processed variables, is to graph the raw variable against the dependent variable.

Based on the comparison, the analyst should choose the transformations that would be adapted to the raw variable.

This leads them to creating the shape that they are aiming to model at a later stage. These transformations include:

a. Decay is one of the most popular transformations in Marketing Mix Modeling. It is commonly called Adstock. The theory behind it is as simple as saying that there is a prolonged (or carry-over) effect of the advertising activity. So, when the business advertises at a time “t”, that activity will not have an impact on consumption only at time “t”. It will also carry over to the following weeks. Mathematically, it is perceived as a stock of advertising that is influenced by two factors:

1: How much activity, or how much advertising, the consumer is exposed to in that particular week.

2: A carryover effect that is coming from the stock of advertising from previous weeks.

⇒ Decay will determine how much memory the consumer takes with them week after week.

The following is an illustration of the Ad Stock concept:

• The dependent variable is graphed in red.
• The raw TV GRP is graphed in blue.
• Two types of Adstock were created here:

In the first one, which is orange, we are assuming a memory decay of 10%. Here, the carryover stretches over a long period because the consumer only forgets 10% of the message they have been exposed to.

⇒ That means 90% of the message is taken over week on week.

In the second transformation, which is the 40%, the stock of advertising that is taken over week on week is less than in the first case of 10%.

⇒ It is noticeable that the green line falls probably two or three weeks following the end the TV activity.

Figure 6

The idea behind Adstock is to create different variations of Decay. Then when the analyst moves to the modeling stage, they choose the best variable that best matches the variations seen in sales.

How do we determine where to stop the Decay?

Another variation of the decay function is to specify a Cutoff point. It means that the user decides to stop the tail starting from a certain point. So, all what they need to do is to enter the cutoff point, and the software they use should be able to automatically create 2 variables: 

• One before the cutoff point to measure the short-term effect. 
• Another after cutoff point to measure medium/long term effect.

Half-life Value

If you want the carry-over effect to stop post the specified number of periods, you can tick the Cutoff Barrier option (the tail would be cut).
MassTer is the only Marketing Mix Modeling tool on the market that allows you to do that. It will also ensure that you account for the short and/ or long-term contributions.
Half-life Value For a cut-off point equal to 0 (i.e. 0 periods), the decayed variable will be the exact match of the raw variable as the decay takes into account 0 periods of the prior observation.
A cut-off barrier of 4, means that any delay beyond that will not be considered in the shortterm variable but counted in the long-term one. Thus, summing up short-term and long-term
variables results in the original decay variable.

b. Lag: is generally applied to shift a variable to model its delayed effect. A good example of this is promotions (see figure 5): what one would expect is that when promotions are on, sales would be increasing as a result. However, depending on the strength of the promotion, that could easily result in a drop in sales in subsequent weeks. Let’s take the case of “buy one get one free”, what generally tends to happen is that customers will stock the products they buy in times of promotion. Consequently, they won’t do their shopping as usual in the subsequent weeks.

What the analyst needs to do here is create a lag of the promotion variable. While the raw variable of promotion will be used to model the immediate effect of promotions, the lagged version will be used to model the brought forward sales, as well as a potential decrease in the sales line.

Figure 7

Figure 8

For the client to get a sound optimization, the analyst needs to build a Diminishing Return in every single channel that they are modeling. They can use several mathematical transformations to this end, for example, the Exponential function, the Adbudg function, or the Atan2 function.

The answer is yes, and it is called Chain Processing. 

Chain Processing in MassTer

Applying chain processing is straightforward in MassTer. Start by selecting your variable and clicking on the desired transformation.
To add the 2nd processor, click on [+] and
MassTer would create a series of transformed variables based on the parameters you defined

The analyst can apply different processors, in sequence, to the same raw or calendar variable. Here we can cite two examples:

a. Applying the decay function to TV spend (GRP), to create Adstock. Then, applying Diminishing Returns to optimize variables like TV, outdoor, search, etc.

b. A typical transformation in MMM is to create a seasonal factor, then smooth it either using a median filter or a moving average. This will result in a shape that the analyst can then use in the model as an explanatory variable.

The time spent on data processing in Marketing Mix Modeling will tremendously help the analyst to better represent and visualize his variables and accurately display how they are behaving.

It is through Data Processing that the analyst shapes the true impact of the variables/events that are behind the real KPI variations.

Marketers and data analysts need to follow a well-structured methodology in order to perform data driven actions and make decisions. This process will make the path clearer for everyone involved in a Marketing Mix Modeling Project to build a statistically and commercially robust model that fits the actual market reality. This couldn’t be done without properly explored and processed data