On 21 May 2025, we hosted a webinar on how to measure creative effectiveness within MMM. We had a lively session led by Dr. Ramla Jarrar, co-founder of MASS Analytics, alongside Ian Forrester, CEO of DAIVID, who’s spent years figuring out how to decode the science behind great advertising.
Here’s a quick recap of what we covered, and why it matters.
Why Creative Matters More Than Ever
We kicked things off by stating the obvious: not all impressions are equal. A brand can spend the same budget on two campaigns and get wildly different outcomes, often because of the creative. Thinkbox research was cited showing that, apart from brand size (which you can’t really control in the short term), creative quality is the single biggest driver of marketing ROI.
But if creative is such a big deal, why has it been so hard to measure it properly in MMM?
The Problem with “Classic” MMM
Ramla explained how MMM has historically dealt with creative: by adding separate variables for each ad or campaign and hoping the model can make sense of it. That worked back in the day when we had fewer creatives and channels. But now? With hundreds of assets running across dozens of platforms, that kind of granularity can break your model literally. You end up with overfitting, noisy results, and not a lot of trust in the insights.
A Smarter Way: Creative Scores at Scale
That’s where Ian and the team at DAIVID come in. They’ve built a tool that uses AI to analyze creative content — not just at the surface level, but by looking at how real people respond to ads. Their model considers attention (did it grab people?), emotion (did it make them feel something?), and memory (did they remember the brand?). These become the building blocks of a creative effectiveness score.
The best part? This can be done at scale. Thousands of assets. Multiple platforms. No problem.
How It Fits into MMM
So how do you actually use this creative data in an MMM project?
Ramla laid it out in three levels:
- No integration: treat MMM and creative analysis as totally separate.
- Parallel analysis: run MMM and creative tests side by side and compare the patterns.
- Full integration: bring creative scores into the MMM model, weighting media inputs based on creative quality.
Naturally, we’re all about that third option. When you factor in creative scores directly, you get models that are more accurate, more insightful, and way more actionable.
The Magic of Weighted Variables
One of the coolest ideas discussed was the use of weighted variables. Instead of treating every impression the same, you adjust them based on the quality of the creative. So an ad with a high creative score has more “weight” in the model. This makes your media variables smarter and more reflective of reality.
Yes, this means more complexity. But with the right tools and modeling approach, it’s manageable. You just have to be careful not to drown in granularity. As Ramla put it, MMM is part art, part science. You’ve got to know when to roll up and when to dig in.
What You Can Learn from This
The benefits of this approach are as follows:
- You can see which creatives actually move the needle: not just on sales, but across the whole funnel (awareness, consideration, conversion).
- You can identify winning creative elements: like what kind of emotions or storytelling techniques work best for your brand.
- You can guide future creative and media decisions: with actual data, not gut feel.
Because MMM shouldn’t just be about explaining the past. It should help you make better choices for what’s next.
Audience Q&A: What You Asked Us
The Q&A section of the webinar was packed with sharp, thoughtful questions. Here are a few highlights that sparked great discussion:
Q: How do you manage having multiple creative scores (i.e., one campaign with 50 different creatives), knowing that MMM is not isolating results at a creative level?
Ramla: “We use creative scores that have been trained to measure how good a particular asset is. Then we combine those creatives within campaigns and campaigns within channels, allowing us to measure their impact. And just to clarify, it’s not about creating a variable for every single creative because that would quickly lead to multicollinearity issues. The key is to use the creative scores, which are based on solid data, and aggregate them efficiently.”
How do you generate the index of creative effectiveness?
Ian: “Our standard creative effectiveness metric is a composite score made up of three things: attention in the first second, the percentage of people who felt one or more intense positive emotions, and brand recall. So essentially, it’s attention, emotion, and memory — those are the three core drivers of success for any brand outcome. Now, the cool thing about MASS Analytics is that we produce a lot more data than just that broad score. We can isolate different metrics depending on the goal of the creative. For example, if the goal of a creative is awareness, you’d focus on attention and emotional connection. But for a lower-funnel piece of creative, you’d focus more on things like search intent or purchase intent. So the beauty here is that we have a depth of data that allows us to adjust the model based on what’s needed for the specific goal of the creative.”
Q: How do you distinguish creative effectiveness in-market versus in-lab?
Ian: “When we’re training our model, we look at the creative in isolation – we control for everything else. That’s our business: creative data. We need to look at the creative by itself and ask, versus a norm, how is it performing in terms of capturing attention, generating positive emotion, and driving memory? So we create that data for the creative in a clean environment.
Now obviously, when that creative goes out into the wild – whether it’s on TV, online, or wherever – its actual performance is going to be affected by things like distribution strategy. That might boost or dampen its impact. The cool thing about the MASS Analytics approach is that it doesn’t just use creative data. It also includes media data. So we know when and where creatives are used, and how they’re deployed. And that lets us understand not only the creative strategy, but also the media strategy – and how they work together to drive outcomes like sales or brand lift.”
Q: How do you deal with overfitting when working with creative data?
Ramla: “It’s something we’re very aware of. And as an analyst, you have to know where to draw the line. Just because you have the data at the creative level doesn’t mean you should use it that way. Sometimes it’s better to aggregate – group creatives by family or category. There has to be a common logic. Then you test at that level. This is something we face not just with creative data, but also with audience segments, spot lengths, and so on.
That’s why I always say MMM is part art and part science. You have to use judgment and experience to know when the data is too granular and noisy to be helpful. Even if the data exists and technically you can model it, it doesn’t mean it makes sense. At that point, you have to step in and say, ‘Let’s go one level up and manage the client’s expectations accordingly.’ Because often, brands want to go deeper and deeper – but too much detail can lead to misleading results. That’s when it’s your job to bring experience and offer a clear explanation of why you’re keeping it at a higher level.”
Q: What kind of processing or transformations do you apply to create the model variables from creative data?
Ramla: “We use what we call weighted sums. These are more sophisticated processors, where we can vary the weights of different creatives or different campaigns depending on the data we have. Then we run this through the auto modeler to understand which combination works best for the model.”
Q: How can you operationalize the results to inform your creative and media strategy?
Ian: “So, first benefit: the model improves. Second: marketers can now understand the actual impact their creative has on outcomes – which is obviously a really important thing to know. Third: you can identify which creatives are strong performers and which ones are weaker. Then, we dive into those top-performing creatives – just like we did with Mars and the other examples – to understand: ‘What was it about those creatives that drove the outcome?’ ‘Which metrics were most strongly correlated with success?’ ‘What did those creatives do to move those metrics?’ And once we know that, we can generate really useful creative strategic insight. You can then use that to brief your creative agencies, brief your influencers, or write better prompts – if that’s how you’re generating content. So, it becomes a very actionable tool not just for measurement, but also for informing creative and media strategy.”
Q: Does the model account for decay, carryover effects, and creative effectiveness at the same time?
Ramla: “Yes, definitely. This is actually something I’m really passionate about in MMM — it keeps you on your toes because there’s always something new to think about. So, when we treat media, the idea is: You take your media variable — say, impressions or GRPs — and you apply adstock to model the carryover effect. Then you model the saturation effect using diminishing returns. And on top of that, you apply a weight to those impressions or GRPs based on the creative effectiveness score we talked about earlier. So it’s not one or the other — you’re layering all of these things together to build a more accurate picture. Now, to do that, you need sophisticated tools and software.
One of the questions was whether Meridian handles this. I don’t think so, not in its current version. It’s too simplistic for this level of complexity. So you’d need something more advanced that can handle media decay, saturation, and creative effectiveness all at once.
Once you’ve done that, and you get your measurement, you can then decompose the effect into different components. Then, for each of those components, you dig deeper to understand what features and attributes of a specific creative made it work. That’s the general process you have to follow.”
