Incrementality: How Do We Measure It?

We’ve all heard of the so-called “butterfly effect,” a theoretical concept wherein a small change in the initial conditions of a complex system can lead to large, unpredictable differences in a later state. It posits that even a seemingly insignificant event, like a butterfly’s wing flap, can trigger a chain reaction with huge, unforeseen consequences unfolding later.

The butterfly effect is all about the unexpected and unforeseen, which are things marketers don’t waste much time on. That’s because skilled marketers know that with careful study of data and proper planning, advertising campaigns can largely be predictably effective and successful, and that they can be improved upon and optimized even once underway.

Both new marketing campaigns, as well as changes made within an existent marketing campaign, can lead to great success, but unlike with the butterfly effect, these successes are anything but random or unexpected. Instead, they are planned for, tracked, and measured.

What is Incrementality?

Incrementality in marketing measures the additional lift a marketing activity provides, showing how much a specific action directly contributes to a desired outcome that wouldn’t have happened otherwise — this can be more sales, more sign-ups, more clicks or likes, or other actions taken by targeted users. It can be used to study a campaign as a whole (as compared to sales that would or would not have occurred without the ads) or to changes within a given campaign.

Unlike standard attribution, which assigns credit to touchpoints, incrementality uses test-and-control examination to determine the true, causal impact of marketing efforts, helping advertisers eliminate waste, find growth opportunities, and optimize return on ad spend (ROAS).

What incrementality is decidedly not is simply spending more money or running the same advertising another time; it refers to conscious, specific changes that can be measured and learned from.

Why Do Advertisers Need Incrementality?

Advertisers need to consider incrementality to understand a campaign’s true, additional impact by differentiating actual sales driven by the ad, from those that would have occurred anyway. This allows them to optimize budget allocation, eliminate wasted spend, identify truly effective channels, and improve overall return on investment (ROI) by making data-driven decisions rather than relying on potentially inflated attribution metrics.

Helps Prove The Value of Marketing

At the most basic level, paying attention to incrementality helps prove that marketing and advertising are worth it. Tracking the lift in sales (or other desired actions by users) after a marketing campaign kicks off can help show its value, and can be compared to previous or concurrent efforts, as well.

Maximizes Return on Ad Spend

By seeing what creative is demonstrably effective when you launch or tweak an ad campaign, you can see exactly how well your ad dollars are — or aren’t — being spent. You can use this information to make sure you’re getting the best possible ROAS and are making changes as needed.

Eliminates Bias and Guesswork

Traditional attribution models can’t account for conversions that would have happened organically. Incrementality corrects for this bias by effectively comparing a test group, namely people exposed to ads, with a control group of those who were not exposed to them, such as would-be consumers prior to the launch of a campaign.

Key Incrementality Terms Explained

Incrementality

The term refers to the impact of a marketing activity on a specific advertising goal, such as an app install or a sale, beyond what would have occurred naturally. It answers the question: “What happened because we did this?”

Attribution

In marketing, attribution is the practice of matching marketing “touches” (clicks or views, for example) to specific conversions like sign-ups or sales. Incrementality, meanwhile, focuses on the causal impact rather than just the association.

Counterfactual Analysis

This refers to the hypothetical scenario (or the actual outcome) that would have occurred if a specific marketing action had not been taken.

Holdout Testing

This is a form of A/B testing wherein a portion of a potential target audience is excluded from a campaign to serve as a control group. I’ll break this down below.

How Can We Test Incrementality?

Define the Goals

Before you try to see how successful an ad campaign or a change to a campaign was, know what you’re hoping to accomplish. Identify the specific marketing element (channel, message, ad, and so on) and the key performance indicators (KPIs) you want to measure, such as sales or leads.

Implement Ads or Changes to Marketing Materials In Measurable Ways

You might release ads with geo-fenced parameters to select demographics, only at certain times, and so on; however you plan to run your advertising, know all the parameters that you’re putting in place so you can have a sense of where they appeared and who potentially saw them — and who almost surely did not.

Collect and Analyze Data

Once you’ve run new or updated ads for a predetermined amount of time, gather performance data for both the test and control groups of people not exposed to the ads. Compare the outcomes of the test and control groups by subtracting the conversion rate of the control group from the test group’s conversion rate to find the incremental lift.

Requirements to Measure a Campaign Incrementally

Clearly Defined Goals and Metrics

Before launching a campaign or an update to advertising materials, you must define the precise objective of your testing and note the key performance indicators you will track.

Test and Control Groups

These facets of the campaign are arguably the most critical requirement for determining true cause and effect of marketing. The test group is exposed to the marketing campaign, while the control group is not. The groups must be as identical as possible in size, demographics, and behavior, to avoid bias.

Fixed Testing Duration

You must run a marketing test long enough to collect sufficient data, but short enough to avoid outside influences like seasonality, holidays, market fluctuations, other promotions, or other factors. A typical test may run anywhere from two to six weeks, depending on your sales cycle and traffic volume.

How Do We Measure Incrementally?

Incrementality in marketing is usually measured using controlled experiments that compare a test group exposed to a given campaign with a control group that is not exposed to the materials. The difference in performance (incremental lift, e.g.) between the groups shows the actual contribution of the campaign to results like sales or conversions.

Geo Holdout Testing

In this common approach to measuring incrementality, advertisers divide the market into geographic regions and withhold the campaign from a specific region (the control) while running it in another (the test). The difference in revenue (or other KPIs) seen between these groups shows the incremental impact of the campaign in that area.

Audience Split Testing

Here, a randomly selected segment of the target audience is withheld from the campaign (the control group), while the rest are exposed (the test group). This is a direct way to see how many users convert only because of the marketing effort.

User-Level Testing

This granular approach involves withholding ads from users at the individual level. The difference in conversions between the exposed and unexposed users demonstrates the campaign’s incremental contribution.

How to Calculate the Campaign Uplift

To calculate campaign uplift, you need to compare a test group exposed to a campaign with a control group that was not exposed, then calculate the difference in outcomes (like sales or conversions), and divide it by the control group’s performance to find the percentage lift.

Baseline Performance

Establish baseline performance by the performance of the control group, which represents what would have happened without the campaign, to see how effective your efforts actually were.

Incremental Lift

Take note of the additional sales or conversions generated by the campaign, beyond what would have occurred naturally. This is incremental lift, and it’s your main goal in terms of data and marketing success.

How to Relate Your Campaign Incrementality to Attribution Measurement

Marketers can relate campaign incrementality to attribution by using them together. Attribution identifies which marketing touchpoints led to a conversion, while incrementality validates that those touchpoints genuinely drove new business rather than conversions that would have happened anyway. This pairing creates a feedback loop, allowing for more effective budget allocation, future media planning, and a truer understanding of the marketing effort’s impact.

Attribution

This shows the customer journey and credits specific channels, campaigns, or interactions for a conversion.

Incrementality

This measures the true, additional impact of a campaign by comparing conversion rates in a test group exposed to the campaign, versus a control group that wasn’t.

Key Takeaways

In marketing, “incrementality” refers to the measurement of the true impact of a marketing activity — the additional value or conversions that can be attributed directly to a specific campaign, channel, or tactic, beyond what would have happened anyway, meaning organically or through other unrelated marketing efforts.

Marketing campaigns can appear successful due to factors like seasonality, brand loyalty, or external influences; incrementality helps isolate what actually works, ensuring you spend money on what drives true value instead of mistaking mere correlation.

For advertisers, attribution identifies which marketing touchpoints a customer interacts with before a conversion, while incrementality measures the true impact of those marketing efforts by determining if a customer would have converted anyway. Attribution uses data and correlations, such as clicks and impressions, to give credit to various channels, whereas incrementality uses controlled experiments with test and control groups to establish a causal link between a marketing campaign and a conversion.

Frequently Asked Questions (FAQs)

Is incrementality measurement difficult?

Yes, often incrementality measurement in marketing is difficult. This is due to the complexity of isolating marketing effects from the overall customer journey; the need for specialized expertise in experimental design and data science; the technical challenges of setting up tests; the potential opportunity costs of excluding customers from tests; and the difficulty of measuring across increasingly complex, interconnected marketing channels.

What questions can be answered with incrementality measurement?

Incrementality measurement answers fundamental questions about true marketing effectiveness, such as:

• What is the actual revenue or sales impact of our advertising campaigns?
• Which media channels or tactics are truly driving new business?
• How should I allocate my marketing budget to maximize growth?

It helps determine if sales would have occurred without the marketing effort, allowing businesses to optimize spend, avoid cannibalizing existing sales, and prove the genuine value of marketing investments to stakeholders.

What is campaign incrementality vs. revenue?

Campaign incrementality measures the additional impact or uplift a campaign generates beyond what would have occurred naturally, whereas revenue is the total money earned from sales or services over a given period. (All incrementality can be counted as revenue, whereas not all revenue comes from incrementality.) Incrementality answers whether a campaign caused something new, while revenue simply states how much money was brought in.

If, for example, a company has sold $1,000 worth of widgets every month for the past year without any new advertising, then it runs ads and sees monthly widget sales jump to $1,500, that $500 increase is the incrementality. That $1,500 is revenue, just like the $1,000 was before it, but $500 of that larger figure can be classified differently.