Everything old is new again.  Never could I, your humble author, have foreseen that the fashions I donned as a middle-schooler in the late 90s would rear their ugly heads again in my lifetime.  And yet, here we are in 2024, where scarcely a day passes without spotting a teenager in baggy, JNCO-style jeans.  Yes, it appears that we have come full circle.

Questionable fashion trends aside, media mix modeling (MMM) is also in the midst of a renaissance after the “holy grail” promise of multi-touch attribution was beset by myriad issues such as cross-device browsing behavior, privacy restrictions, walled-gardens, and time-consuming and difficult to maintain implementations.  The inability to reliably stitch together user-level interactions has led many organizations to embrace MMM once again.

The good news is that while the fundamentals of media mix modeling remain the same, there have been a number of advancements in terms of both statistical approach as well as data collection that have increased the accuracy and repeatability of model results.

More rigor around time series & seasonality

Example of Bayesian Structural Time Series Decomposition

Media mix modeling is inherently a time series problem –  i.e. media delivery and sales data is collected at regular intervals, typically daily or weekly. By analyzing historical sales data alongside advertising expenditure across various channels, MMM builds a model that captures how these factors influence sales over time.  This time series element introduces complexities that are challenging for models to capture – a company’s sales today are not only a function of today’s advertising dollars, but also historical trends, seasonality, and the effect of recent media, which is captured to varying degrees of effectiveness by adstock transformations.

Most MMMs built twenty years ago utilized multiple regression models that, while functional, did not account for some of the nuances of time series data.  Today’s approaches, like Robyn’s generalized additive models, or Slingwave’s own Bayesian structural time series, bring substantially higher degrees of statistical rigor and do a better job to help marketers understand baseline sales, i.e. the combination of organic growth and seasonality.

S-curve transformations have become nearly ubiquitous, allowing models to capture saturation effects with higher resolution than the simple logarithmic or exponential transformations of the past.  Even adstock methods have evolved to be more complex, now able to incorporate Weibull transformations to model delayed impact of spend.

Bayesian approaches

Bayesian modeling has grown by leaps and bounds in recent years thanks to advancements in computing power.  One of the primary advantages to taking a Bayesian approach as it pertains to MMM is the ability to incorporate prior information into models.

At a bare minimum, it’s probably safe to assume that your marketing efforts have a positive effect on sales, implying that their estimated coefficients in the model must also be positive.  While there were ways to enforce that requirement using the old least-squares regression approach, they were somewhat cumbersome and added more complexity to an approach that was already plagued with multicollinearity.

Bayesian regression, on the other hand, not only embraces prior knowledge, but is reliant upon it.  Priors can be set up to be relatively informative or non-informative depending on a practitioner’s upfront knowledge of a channel’s effectiveness.  For example, if an incrementality test has recently been conducted on generic paid search and found that the ROAS likely lies between 5 to 10, that information can be incorporated into the model.  This allows us to eliminate models that are not plausible and reduce churn trying to otherwise coerce the model to produce a believable result.

Data management

MMM engagements of yore were onerous processes, requiring meticulous project management and coordination that resulted in the collection and transfer of data in flat files, which were then cleaned and assembled by the MMM provider.  Today, most organizations and/or their agencies maintain data marts with all of their media delivery data in a relatively consistent format.

The days of passing spreadsheets back and forth have ceded to API feeds and automated file drops, allowing companies to iterate on models much more quickly.  Many companies now run MMM refreshes on a quarterly basis, rather than annually.  This allows companies to pivot marketing investment allocation more quickly, and can also aid in forecasting.

Still a complex problem with many hyper-parameters that is partially art and partially science

Source: Robyn

While the tools at our disposal are more advanced than ever before, practitioners still face a number of issues inherent in media mix modeling.  For instance, we can model complex saturation curves via hill or s-curve functions, but how exactly do we determine the shape of these curves?  Similarly, how do we decide the magnitude of carry-over effect, or the shape of its impact?

In order to answer these questions, one of two approaches are typically used: build a more complex model that directly incorporates these parameters, or treat them as hyperparameters and build a hundreds, or even thousands of relatively simpler models to sort out what the true values should be based on the models’ ability to fit the data.

Google’s Lightweight MMM (now Meridian) takes the former approach, while Meta’s Robyn uses the latter.  Neither is inherently superior, but there are pros and cons to each depending on the use-case that are too nuanced for this article.  Suffice it to say that it can be easy to fall into “analysis paralysis” when needing to make final decisions on media saturation and carry-over.

Not useful in a vacuum – needs support from incrementality testing and platform measurement

MMM, while powerful, isn’t a standalone solution. Relying solely on MMM data, in a vacuum, can lead to misleading results and ultimately, suboptimal marketing decisions. Here’s why:

Firstly, MMM focuses on historical data, revealing trends and correlations between marketing efforts and sales. However, it can’t differentiate between correlation and causation. For example, a rise in sales might coincide with a TV ad campaign, but external factors like a competitor’s stumble or a seasonal surge in demand could be the true driver. Without additional context, MMM might wrongly credit the TV campaign.

Secondly, MMM doesn’t account for the intangible aspects of marketing. Creative quality, brand reputation, and customer sentiment all play a role in sales, but these aren’t captured in the model. Solely relying on MMM could neglect the importance of strong creative or positive brand perception in driving sales.

Can only speak to media effectiveness in broad strokes

The more complex the advertising landscape, the more data a model needs to accurately estimate all of the parameters and hyperparameters involved.  As a result, sometimes marketing channels need to be “rolled up” or combined into a higher level of aggregation than would be ideal for deriving actionable insights from the model.

Accurately accounting for seasonal effects is another complexity that introduces trade-offs.  While day-of-week seasonality, e.g. the difference in sales between a typical Sunday vs. a Wednesday can be estimated with less data, cycles that occur regularly every year, like the summer season, the holiday season, back to school, and which profoundly affect most retailers sales, require multiple years’ worth of data to quantify.

The trade-off with modeling over a long time horizon is that macro-level variables begin to play a larger role, and these can be difficult to identify and collect.  Additionally, a channel’s modeled effectiveness will be an aggregated estimate over the time period of the model, and not necessarily indicative of more recent and relevant performance.  While there are some emerging models that allow for effectiveness to evolve over time like Uber’s Orbit KTR, these models again introduce added complexity.

Struggles reconciling channels that are at different funnel stages

While it might seem counterintuitive, MMM models have a tendency to be biased towards lower funnel tactics that are ‘closest to the cash register,’ like retargeting ads and branded paid search.  These typically have a clearer line of sight to conversions compared to upper funnel brand awareness campaigns, but are not necessarily the influencing factors and may be merely taking credit for an outcome where they had little to no impact. By isolating the influence of these tactics on sales uplift, MMM can highlight their effectiveness in driving immediate results. This allows marketers to optimize their budget allocation, potentially favoring channels with strong lower funnel performance that directly translate to sales. However, it’s important to remember that MMM doesn’t paint the whole picture. Brand awareness built by upper funnel tactics can indirectly influence the success of lower funnel efforts, which MMM might not fully capture.

MMM is ultimately part art, part science

While modeling techniques continue to evolve to handle more complex marketing problems, media mix modeling will continue to require skilled practitioners with a deft understanding of the trade-offs involved with these techniques, at least for the foreseeable future.

They will need to work hand-in-hand with marketers to determine the most important questions the model should answer, and then make modeling decisions accordingly.  For instance, are we strictly trying to understand historical performance, or does the model need to be more generalized in order to forecast financial results for the upcoming quarter?  Will a model be used for tactical optimization, or do we want to more broadly understand media’s contribution vs. baseline?  The answers to these questions could lead to strongly divergent modeling approaches.

Nevertheless, there is reason to be excited about the resurgence of media mix modeling: there are more open source tools available than ever before, as well as customized and purpose-built applications for marketers like Slingwave MMM, that will continue to advance and play an increasingly critical role in decision making as tracking becomes more and more difficult.  We just hope the results age a little better than that old pair of JNCO jeans.

About Us

Slingwave empowers marketers to optimize omnichannel media and drive superior business results.  Slingwave’s marketing analytics platform enables clients to make better marketing investment decisions, leveraging predictive modeling that combines data science and machine learning with unified marketing data to predict optimal cross-channel marketing mix.