What does Incrementality testing mean?

What is incrementality testing?

Incrementality testing is a method used to measure the real impact of a marketing effort.
The aim is to find out what actually creates extra results and what would have happened anyway.

When companies advertise across channels such as Google, Meta, email or affiliate, a key question often arises:
How much of the sales are directly attributable to marketing and how much would have happened without the campaign?

This is where incrementality testing becomes important.
The method helps isolate the additional value an activity creates to make better decisions about budget, channel selection and strategy.

In Danish, the term can be translated to something like Measuring incremental effect or testing the powerlift.
In practice, however, the term is often used in marketing, analytics and performance work.

Why is incrementality testing important?

Many marketing metrics show correlation, but not necessarily cause and effect.
For example, an advert can get credit for a sale even if the customer had already decided to buy.

This means that classic reporting metrics such as clicks, impressions, conversions and ROAS can give a skewed picture.
They often show what has been recorded, but not always what has actually moved the business.

Incrementality testing attempts to answer a more precise question:
What is the additional impact of this activity compared to a scenario where the activity did not take place?

• Reduce waste in your marketing budget
• Identify channels that actually generate additional revenue
• Reveal overrated campaigns and tactics
• Support more accurate budget allocation
• Improve collaboration between marketing, management and finance

In an era of limited tracking, stricter privacy regulations and less reliable attribution, incrementality testing has become even more relevant.
It provides a more robust basis for decision-making than many traditional measurement methods.

How the method works in practice

At the heart of incrementality testing is comparison.
It compares a group that is exposed to a marketing effort with a group that is not.

If the two groups are similar enough, the difference in results can be used as an estimate of the incremental effect.
That is, the value that the campaign has actually created on top of what would have happened otherwise.

In many ways, it resembles a classic experiment.
The better the test is designed, the more credible the result will be.

Test group and control group

A test group is exposed to the activity you want to measure.
It could be an ad campaign, a discount code, an email series or a branding effort.

However, the control group is not exposed to the same activity.
If the control group is otherwise similar to the test group, the difference between the groups may point to the actual effect of the campaign.

For example, a company may show adverts in one region but not in another comparable region.
If sales increase significantly more in the test region, the increase may be a sign of incremental effect.

The holdout method

One of the most commonly used forms of incrementality testing is a holdout test.
Here, the campaign is deliberately withheld from part of the target audience to create a basis for comparison.

It may feel counterintuitive not to market to everyone, but this retention makes it possible to measure the added promise.
Without a control group, it's much harder to determine what the campaign actually contributed.

Time period and statistical certainty

A good test typically requires an appropriate duration and enough data.
If the test is too short or the target group too small, the results can be uncertain or misleading.

That's why you often work with statistical significance, confidence levels and clear definition of success criteria.
The more disciplined the setup, the greater the value of the test in practice.

What do you typically measure with incrementality testing?

Incrementality testing can be used for much more than just measuring sales.
The choice of metric depends on your organisation's goals, channel and customer journey.

• Turnover
• Number of purchases
• Leads
• Subscription sign-ups
• App installations
• Visit on website
• Repurchase and customer loyalty
• Fire lift and awareness

Performance marketing often focuses on conversions and revenue.
In longer-term efforts, you can also test the effect on demand, search behaviour or branded search.

The most important thing is that the goal is clearly defined from the start.
If you don't know exactly what you want to prove or disprove, the test can easily become too imprecise.

Difference between incrementality testing and attribution

Incrementality testing is often mentioned alongside attribution, but the two are not the same thing.
Attribution is about distributing credit for a conversion between touchpoints in the customer journey.

For example, an attribution tool can show that a Facebook advert received credit for a purchase because the customer clicked on the advert shortly before the conversion.
But that doesn't necessarily tell you if the advert was instrumental in making the purchase.

Incrementality testing goes one step further.
This examines whether the activity created an additional result that would not otherwise have occurred.

• Attribution: Who gets credit for the conversion?
• Incrementality testing: Did the effort create an extra conversion at all?

Both methods can be useful.
But if the goal is to understand real business impact, incrementality testing is often more valuable for decision making.

When does incrementality testing make sense?

The method is especially relevant when a company wants to find out if a channel or campaign is actually generating growth.
This applies to both digital marketing and broader media efforts.

• When multiple channels affect the same conversion
• When the last click gives a misleading image
• When branded search seems to take too much credit
• When remarketing is powerful, but may only capture ready-to-buy users
• When you want to test the effect of increased or reduced budget
• When privacy restrictions make tracking less accurate

Incrementality testing is also relevant in situations where management wants evidence of marketing's business value.
Here the test can create a more reliable link between activity and result.

Examples of incrementality testing in marketing

The concept may seem technical, but the application is very concrete.
Below are some typical situations where incrementality testing is used in practice.

Paid search on brand name

Many companies advertise their own brand name in search engines.
The question is whether the ads generate additional sales or whether customers would have clicked on the organic result anyway.

By switching off branded search for selected periods or areas, you can test the real added effect.
The result is often surprising because some of the traffic comes naturally without ad impact.

Remarketing

Remarketing is often measured as a strong channel because it reaches users close to purchase.
But that's why it can look better in attribution models than its actual lift warrants.

A holdout test can show if remarketing actually moves sales or if it mostly converts people who were already on their way to buy.
This can have a big impact on budget allocation.

Email marketing

Email campaigns can be tested by not sending to a small, representative portion of recipients.
If the recipient group performs significantly better than the holdout group, it indicates a real incremental effect.

This is especially useful when you want to find out how much extra revenue an automated flow series, newsletter or promotional offer actually generates.
Not just how many sales were registered after a click.

Social media and display

On channels like Meta, YouTube or display networks, incrementality testing can be used to measure whether campaigns create demand beyond existing levels.
This is especially relevant for the upper and middle parts of the funnel, where the effect is not always visible in the last click.

Geotests, exposure control or the platform's own lift studies can be relevant methods here.
The choice depends on budget, data and the complexity of the media mix.

Benefits of working with incrementality testing

There are several good reasons why more marketing teams are prioritising incrementality testing more than ever before.
The method provides a more realistic picture of what drives growth.

• Better understanding of cause and effect
• More efficient use of marketing budgets
• Stronger arguments to management and stakeholders
• Less reliance on uncertain tracking data
• Possibility to trim or stop inefficient activities
• Better foundation for scaling what works

When companies know the real contribution of channels, it becomes easier to invest wisely.
This creates both higher efficiency and greater confidence in marketing efforts.

Challenges and limitations

While incrementality testing is powerful, it is not without its challenges.
A poor test setup can lead to conclusions that seem accurate but in practice are uncertain.

• It can be difficult to create fully comparable groups
• Small amounts of data lead to uncertain results
• External factors can affect the outcome
• Testing can cost short-term revenue in the holdout group
• The organisation may lack the skills to interpret the results correctly

Seasonality, price changes, competitor activity and inventory issues can all disrupt a test.
Therefore, results should always be assessed in context and not stand alone.

It's also important to remember that one test doesn't necessarily provide an eternal answer.
Markets change, audiences evolve, and channel impact can vary over time.

How to get started with incrementality testing

If a company wants to work more systematically with incrementality testing, it's beneficial to start simple.
The key is not to test everything at once, but to choose the areas where the uncertainty is greatest and the potential is high.

• Define a clear business question
• Choose one channel or campaign to test
• Designate test group and control group
• Set KPIs and test period in advance
• Ensure sufficient data volume
• Analyse the result soberly and in context
• Use the learning for the next test and budget decision

A good first step might be to test branded search, remarketing or email campaigns.
These are areas where the measured performance often looks strong, but the incremental impact may be lower than expected.

Over time, the organisation can build a testing culture where decisions are based on experimentation rather than assumptions.
It enhances learning, efficiency and strategic maturity.

Incrementality testing in a modern marketing strategy

In modern marketing, the customer journey is rarely linear.
A customer can be influenced by search, social media, email, organic traffic, reviews and word-of-mouth before a purchase is made.

This makes it difficult to understand which efforts are actually making a difference.
Incrementality testing is an important complement to dashboards, ad platforms and classic analytics tools.

For Danish companies that want better control over their marketing impact, incrementality testing is more than a technical buzzword.
It's a way to get closer to the truth about what creates real growth.

When you get serious about incremental impact, marketing is measured not just by activity, but by actual incremental contribution.
It's often the difference between optimising reports and optimising business.

Conclusion

Incrementality testing is basically about measuring whether marketing creates something extra.
Not just what can be tracked, but what actually moves results.

It is particularly valuable in a digital reality where attribution is often incomplete and many channels compete for credit for the same conversion.
By comparing test and control groups, organisations can get a more credible picture of impact.

For businesses, marketers and analysts, incrementality testing is therefore an essential tool.
This leads to better decisions, sharper prioritisation and greater certainty that the budget is spent where it makes a real difference.