A recent eMarketer report says only 42% of us consistently use incrementality testing to see what’s actually working. We know it’s better than last-click for finding real ROI, but there’s a huge disconnect between knowing this and actually doing it in a growth marketing plan. It just means a lot of organizations are leaving easy, measurable growth on the table.
Key Takeaways
- To measure the true lift in brick-and-mortar sales from your digital ads, you have to run A/B tests using control and test groups in different, isolated geographic areas.
- You should dedicate at least 10% of your marketing budget to running these kinds of incrementality experiments if you want to find out where your money is actually making a difference.
- To make sure you’re measuring your campaign’s impact and not just a random market fluctuation, use more advanced statistical methods like synthetic control groups to get a clean read.
- Start by running incrementality tests on your highest-spend channels. That’s almost always where the most significant wasted ad spend is hiding.
- Take what you learn from your incrementality tests and use it to constantly adjust your attribution models, otherwise you’re just going to keep misallocating your budget.
20% Lift from Geo-Targeted Ad Campaigns
I saw this work firsthand with a retail client in late 2024. They were running these huge, broad digital campaigns across multiple states, just assuming that more impressions automatically meant more people walking into their stores. We convinced them to try a series of geo-targeted experiments instead. We carved out five different market areas, set two aside as control groups that got no extra digital ads, and then cranked up the ad spend in the other three for six weeks. The results were immediate. The test markets, after we corrected for their baseline sales and seasonality, saw a 20% incremental lift in foot traffic and conversions compared to the controls. Our careful experimental design showed this was actual causation, not just some random correlation. The control groups which had similar demographics and sales histories, saw no such lift, proving that the targeted digital ads were directly responsible for driving new business. If they hadn’t run this test, they would’ve just credited their big, general campaign for all the sales and kept overstating its effectiveness while missing the real power of targeted spending.
$1.2 Million in Annualized Savings from Paused Campaigns
Here’s another one. A subscription service was pouring money into a specific social media platform because their last-click attribution model said it was a top performer for new sign-ups. It looked great on paper. We came in and ran a ghost ad test, where we simply excluded a small slice of their target audience in a few regions from seeing any ads from that channel for a whole month. The rest of the audience saw the ads as usual. If the channel was really driving new sign-ups, we expected to see a big drop in conversions from the group that saw no ads. The reality was surprising. The conversion rate was statistically identical between the two groups. It turned out that a huge portion of the conversions being credited to that channel were from people who were going to sign up anyway, or were being influenced by other marketing efforts. Armed with this data, the client slashed their spend on that platform and moved the money to channels that actually proved their incremental value. That single move resulted in an estimated $1.2 million in annualized savings and had no negative impact on their overall subscriber growth. It’s a classic example: high last-click conversions don’t always mean you’re creating new customers.
15% Underestimation of Brand Search Impact
Attribution models are notoriously bad at valuing branded search. Because it often happens late in the funnel, it gets less credit than it deserves. We worked with a consumer electronics brand that had tons of branded search traffic but treated it as a low-priority channel for new investment. We set up a holdout test where we turned off their branded search ads in a few select geographic markets for three weeks, leaving everything else running normally. The common thinking was that people searching for the brand by name would find the website and convert anyway, with or without an ad. The data told a completely different story. The test regions saw a 15% drop in total conversions, not just branded search conversions, compared to the control regions. This proved the branded search ads were doing more than just catching people who were already on their way. They were reinforcing buying intent, prompting immediate sales, and probably defending against competitor ads bidding on their terms. It was actively converting existing demand much more efficiently. After seeing this, the brand properly recognized the incremental value of the channel and increased its investment in branded search.
The Conventional Wisdom is Often Flawed
So many marketers (and I’ve been guilty of this myself) just default to the metrics staring them in the face, like the ROAS or CPA that Google Ads or Meta Business Suite spits out. I get it. The dashboards are clean and the numbers seem authoritative. But relying only on what the platforms report, without running your own incrementality tests, is a huge strategic error. Those platforms are built to demonstrate their own value, not the total net new value they create for your business, and they are inherently bad at understanding cross-channel influence or conversions that would’ve happened organically. Conventional wisdom says if a platform has a high ROAS, you should pour more money into it. I disagree. A high ROAS can easily hide cannibalization of other channels or just show that you’re good at capturing demand that was already there. The question I always ask is, what would’ve happened if we *didn’t* spend that money? That’s the core of incrementality, a perspective you won’t find in a standard platform dashboard. We have to be willing to challenge the platform’s view of the world by designing our own experiments that prove causation.
Simply seeing a positive ROAS and concluding that your marketing is working is a dangerous oversimplification. I’ve had so many clients who were absolutely convinced a channel was critical to their business, but when we ran an incrementality test, we discovered its actual contribution was tiny, or sometimes even negative once you factored everything in. This isn’t a knock on the platforms themselves. They’re powerful tools. It’s a knock on how we’re interpreting their data without applying the scientific rigor of incrementality. My experience shows, again and again, that the marketers who put in the effort to run real tests (even with all the upfront complexity) are the ones who build more sustainable, profitable growth. They stop investing based on what a dashboard tells them and start investing in what actually moves the needle.
Getting stakeholders on board to turn off a campaign that looks like it’s performing well is always the hardest part of the job. It just feels wrong to purposely cut your reach or spend when the numbers look good on the surface. But every single time we’ve managed to push past that initial hesitation, the clarity we’ve gained has been worth it, leading directly to smarter budget decisions and a much clearer picture of what drives the business. The only other option is to keep operating based on platform-reported metrics, which means you’re basically just guessing your real impact.
Conclusion
Incrementality testing is a fundamental shift in thinking about how we should be allocating budgets and judging our campaigns. It’s a mindset. By using experiments to measure the actual causal impact of our marketing, we can find hidden efficiencies and drive real growth that goes way beyond what a simple attribution model can show you. You have to apply the scientific method to your marketing. It’s the only path to knowing what really works.
What is incrementality testing in growth marketing?
It’s a method for measuring the true, causal impact of your marketing. You do this by comparing a test group (people who see your ads) to a control group (people who don’t). This tells you how many sales or sign-ups happened *only* because of your campaign.
How does incrementality testing differ from traditional attribution models?
Traditional attribution (like last-click) just divides up credit for a sale after the fact. Incrementality uses a real scientific experiment to determine if your marketing *caused* the sale in the first place, or if it would have happened anyway.
What are common methods for conducting incrementality tests?
The most common methods are geo-lift tests (comparing different regions), ghost ad tests (showing a blank ad to a control group), and simple holdout groups (randomly excluding some users from seeing a campaign). The right one depends on the channel you’re testing.
Why is it challenging for some marketers to adopt incrementality testing?
It can be complex to set up, you need enough data for the results to be meaningful, and it’s often a tough sell internally. Convincing stakeholders to intentionally *not* market to a group of potential customers feels risky, even though it’s the only way to learn.
What kind of results can a business expect from successful incrementality testing?
You get a much clearer picture of which channels and campaigns are actually driving new business and which are just wasting money. This almost always leads to smarter budget allocation, a higher *real* ROAS, and more sustainable growth.