Incrementality Testing
Incrementality Testing explained
A is credited with sales. That alone does not establish that it caused additional sales. helps you investigate this question for a specific budget decision.
**The question defines the experiment.** Are you comparing a channel with no advertising, an increase in spend, or new creative with the existing version? These comparisons answer different questions. Define the outcome, comparison condition, analysis method and decision rule before starting. Randomly assign people or regions where possible. A simple before-and-after comparison can mistake seasonal demand for advertising impact.
**Plan for an informative result.** Data volume, background variation, expected effect and purchase cycle determine whether a test can detect the change that matters. There is no universal minimum budget or duration. Overlap between groups, changing prices and concurrent promotions can complicate interpretation.
**Turn the finding into a decision.** Report the estimated effect alongside its uncertainty interval. An inconclusive result does not establish that there is no effect. Consider margin, capacity and costs before reallocating spend. Findings apply to the tested activity, audience and period; they do not automatically extend to larger budgets.
In Creative Engineering, the measurement question connects strategy with creative development: which idea should change which behaviour? AI can help prepare analysis and identify data issues. Study design, interpretation and conclusions still require review. We take responsibility for the concept and quality.
Examples
Hypothetical: Additional orders
A well-designed test estimates 1,100 orders in the treatment group against 1,000 expected without the activity. Estimated absolute lift is 100 orders; relative lift is 10 percent of the comparison baseline. Uncertainty and study assumptions determine how much confidence this calculation deserves.
Hypothetical: New creative
A retailer compares an explanatory product idea with its existing ad under comparable delivery conditions. The test estimates the effect of changing the creative. It does not automatically establish the impact of the entire channel compared with no advertising.
Key Points
- Define the question and comparison condition in advance.
- Separate additional impact from attributed conversions.
- Check randomisation, measurement quality and statistical power.
- Report effect size together with uncertainty.
- Consider margin and implementation costs in budget decisions.
Practical application
Start with a pending budget decision. Establish the smallest commercially relevant change, whether the available data can detect it and which control condition is feasible.
Useful measures
Absolute and relative lift
Estimated additional impact and its ratio to an explicitly defined comparison baseline.
iROAS
Incremental conversion value divided by the media spend basis appropriate to the test; not a profit measure.
Uncertainty interval
Expresses the precision of the estimate under the chosen method’s assumptions.
Common mistakes
- Selecting only successful outcomes after seeing the data.
- Interpreting an uncertain finding as proof of no effect.
- Using different outcome definitions or time windows across groups.
- Extending one result indefinitely to other budgets and audiences.
Sources and context
- Google Ads: About Conversion Lift
Official explanation of user- and geography-based lift studies; product availability depends on the account.
- Vaver & Koehler: Measuring Ad Effectiveness Using Geo Experiments (2011)
Methodological foundation for randomised geographic advertising experiments.
- Google Ads: Geography-based Conversion Lift metrics
Definitions of incremental conversion value, incremental cost and uncertainty intervals in the Google product.
Frequently Asked Questions about Incrementality Testing
Attribution allocates measured outcomes to touchpoints using a rule or model. Incrementality asks about additional impact against a comparison condition. Both can be useful, but they answer different questions.
There is no reliable universal threshold. Data volume, variation, the minimum detectable effect and the importance of the decision matter. Assess statistical power before committing the budget.
Only if that is the comparison you want to study. A control group can retain the existing spend or creative. The difference between groups defines the effect being estimated.
The test cannot sufficiently distinguish the relevant possibilities. Examine the interval, data quality and study design. An inconclusive result does not automatically mean the activity has no effect.
An experiment examines a bounded change. A marketing mix model considers longer trends and multiple influences. Appropriately scoped experiments can inform the model; differences in period, outcome and spend basis must be accounted for.
Related links
Loading related terms…
All TermsArticles about Incrementality Testing
CTV & Retail Media: The Performance Upgrade for Your TV Budget
Two megatrends are converging. Learn how to leverage retail data to generate measurable sales from your CTV ads and finally steer your TV budget with performance in mind.
YouTube Ads with Demand Gen: Turning good videos into measurable business
Good videos alone do not tell you whether a campaign brings the right business. How to connect data, creative buying arguments and landing pages for Demand Gen.
Meta Ads: Why targeting starts with the idea
Automated delivery makes the idea a strategic task. How brands translate reasons to buy into distinct creative concepts and learn from testing.