By Editorial TeamSarah, head of growth at the DTC floral service “Bloom & Blossom,” had that familiar knot in her stomach looking at the Q2 2026 report. Ad spend was up 30% year-over-year, but new customer acquisition had only inched up 12%. Their last-click attribution models kept painting a pretty picture of return on ad spend (ROAS), but Sarah knew in her gut that a huge chunk of their budget was just buying conversions that were going to happen anyway. This doubt, whether marketing was really driving new growth or just taking credit for it, was a constant problem for her team.
Key Takeaways
- Build holdout groups for incrementality testing by walling off 5-10% of your target audience so they see zero ads.
- Run geo-lift experiments, using specific cities or regions as test and control groups to measure real campaign impact.
- A/B test your creative and landing pages constantly to find the incremental value of every little piece of your campaigns.
- You have to analyze pre-campaign baseline data against post-campaign performance to get an accurate read on the net effect of your ad spend.
- Forget just immediate conversions. Start measuring long-term customer value (LTV) to see the sustained incremental gains from your efforts.
The Attribution Illusion: Why Traditional Metrics Fall Short
Sarah’s frustration is one I see all the time, and it comes from a basic flaw in how we’re taught to measure marketing. Traditional attribution, especially last-click, loves to give all the credit to the very last thing a customer did before buying. This is misleading. Think about it: a customer sees a Bloom & Blossom ad on social, forgets about it, then later Googles “Bloom & Blossom flowers” and makes a purchase. Last-click gives 100% of the credit to the paid search ad, completely ignoring the social media ad that created the initial awareness. Even worse, if that customer was already going to buy something, then all that ad spend was non-incremental, a total waste.
I’ve watched this play out in dozens of companies. They dump money into channels with a high reported ROAS, then wonder why the business isn’t actually growing. The problem isn’t the channel. It’s the inability to tell if you’re creating new demand or just paying for existing intent. “We’re throwing money at channels that look good on paper, but I can’t shake the feeling we’re just paying for existing demand,” Sarah told me once. A lot of marketers are starting to question the dogma of last-touch metrics. Even a 2023 IAB report on measurement maturity showed a scary number of advertisers are still stuck on these basic models, missing the entire concept of incrementality.
Setting the Stage for True Measurement: Bloom & Blossom’s Incrementality Journey
Sarah knew they needed a stronger measurement framework, so she decided to rip out their old system and start over. Her goal was to figure out which marketing activities were actually bringing in new customers or driving extra purchases that wouldn’t have happened on their own. She had to move beyond simple ROAS and fully embrace incrementality testing. First, her team did an honest audit of their data infrastructure and what their current martech stack could (and couldn’t) do.
They started with their two biggest money pits: paid social on platforms like Meta Business Suite and paid search on Google Ads. Sarah got that real incrementality measurement is a controlled experiment, just like in a science class. An experiment needs a baseline, a variable, and a way to isolate the impact of that variable. For marketers, that means using holdout groups.
Implementing Holdout Groups for Paid Social
For their Meta campaigns, the team set up a proper controlled experiment. They picked their audience in two key cities, Los Angeles and Chicago, and split them into two groups. The test group saw all the normal Bloom & Blossom ads. The holdout group, however, was completely firewalled, they saw zero ads from the company for the entire test period. This is harder than it sounds. It wasn’t just pausing ads. They had to carefully manage exclusion lists within Meta’s ad platform to make sure the holdout group was genuinely blacked out from all their paid media.
The main challenge is keeping the holdout group clean. Accidental ad exposure can poison the whole experiment. “We spent a full week scrubbing audience lists and setting up exclusion rules,” Sarah recalled. “It’s not as simple as flipping a switch.” After running this for four weeks, they tracked purchases from both groups. The difference in conversions and average order value between the ad-exposed group and the holdout group gave them a brutally clear picture of the actual incremental lift from their social campaigns.
The results were eye-opening. Standard reporting showed a great ROAS for paid social, but the incrementality test proved that about 20% of the conversions they were taking credit for would have happened anyway, with or without the ads. That meant 20% of their social budget was being completely wasted. It was a tough pill to swallow, but it was an actionable one. They could now take that 20% and either cut it to save money or reinvest it into something that might actually work.
Geo-Lift Experiments for Paid Search
For their Google Ads campaigns, Bloom & Blossom used geo-lift experiments. This approach works best when you have geographically distinct markets that look a lot alike in terms of demographics and buying habits. Sarah’s team picked two similar mid-sized cities: Denver, Colorado, and Portland, Oregon. They made Denver the test market, where their paid search campaigns ran at full blast. In Portland, the control market, they throttled back or completely paused certain campaigns, especially for their own brand keywords and generic floral terms.
The experiment ran for six weeks. The team kept a close eye on search volume, site traffic, and sales from both cities, making sure their other marketing (like email and organic social) was exactly the same in both places. By comparing Denver’s performance to Portland’s, after adjusting for any baseline differences, they could finally put a number on the incremental impact of their paid search. A Google Ads support document can walk you through the setup, but the key is getting to statistical significance.
What did they find? Their branded paid search campaigns, which always looked super efficient on paper, were mostly just catching people who were already looking for them. Customers searching for “Bloom & Blossom” were already sold. The ads weren’t creating new demand. The incremental lift was tiny. This data gave Sarah the confidence to slash their branded search budget and reallocate that money to riskier, non-branded keywords and discovery campaigns designed to find new customers earlier in the process. It felt like a big risk, moving money from a “safe” channel, but the data said it was the right move.
Beyond Campaign Measurement: A/B Testing for Creative Incrementality
Incrementality goes deeper than just channels, right down to individual campaign elements. Sarah’s team made A/B testing a core part of their strategy. They started testing everything: different ad creatives, headlines, CTAs, and landing page designs. For their Valentine’s Day campaign, for example, they ran a test on two landing pages. One page showed classic red roses, and the other featured a mix of unique, ethically sourced bouquets. They split their ad traffic 50/50 between the two.
The results were fascinating. The traditional rose page actually had a slightly higher immediate conversion rate, but the page with the unique flowers led to a higher average order value and, more importantly, more repeat purchases over the next three months. This granular testing, which so many people skip when they’re chasing broad campaign ROAS, tells you what actually resonates with customers and drives more long-term value.
The Long Game: Incorporating Lifetime Value
Sarah’s biggest lesson from all this was to look past the immediate conversion. Customer Lifetime Value (LTV) had to become their North Star. An ad might not trigger a sale today, but if it brings in a high-value customer who comes back again and again, that’s a huge incremental win. Bloom & Blossom began segmenting customers acquired through these new tests and tracking their LTV separately. They found that customers from certain discovery campaigns, even with a lower initial ROAS, had a much higher LTV than those who came in through branded search. This solidified their strategy of investing in top-of-funnel awareness, even when the immediate payback wasn’t as obvious.
This shift from short-term ROAS to long-term value is what separates sophisticated marketing teams from the rest. It takes patience. It also takes a willingness to trust data that doesn’t fit neatly into traditional reporting dashboards. “It’s about understanding the true economic impact, not just the last click,” Sarah would tell her team. This view ensures marketing investments drive sustainable business growth, not just vanity metrics.
Looking Ahead: The Future of Incrementality in Digital Marketing
This whole journey into incrementality testing completely changed how Bloom & Blossom ran their digital marketing. They went from being reactive and obsessed with last-click to being proactive and data-informed. Their ad spend got smarter, their customer acquisition became more strategic, and their grasp of real marketing impact got a hell of a lot stronger. Sarah’s team now runs incrementality tests as a standard part of their planning process.
The lessons from Bloom & Blossom’s experience apply to every digital marketer in 2026. With privacy rules getting tighter and old tracking methods breaking down, being able to measure true incremental lift is becoming a survival skill. Flying blind on platform-reported ROAS or last-click attribution is like using a busted compass. You’re moving, sure, but probably not toward your actual goal. Rigorous testing methods like holdout groups and geo-lift experiments give you the clarity you need to make confident decisions that actually grow the business. For more on this, check out how AI Social Ads in 2026 might help with budget allocation, or get context on boosting Digital Campaigns to Boost 2026 ROI. And to understand the attribution challenge, see the AI SEO: 2026 Attribution Crisis & ROI Fix.
What exactly *is* incrementality testing?
Incrementality testing is how you measure the real, causal impact of your marketing. It’s about comparing a group of people who see your marketing (the test group) to a similar group who don’t (the control or holdout group) to see what difference the marketing actually made.
Why should I even care about incrementality?
You should care because it stops you from wasting money. It shows you which of your marketing efforts are genuinely creating new business (more sales, new customers) that you wouldn’t have gotten otherwise, so you can stop spending on things that aren’t actually working.
How do people actually *do* these tests?
The most common ways are holdout groups, where you intentionally block a slice of your audience from seeing ads, and geo-lift experiments, where you compare performance in different cities or regions where you’ve changed your ad levels.
Explain holdout groups like I’m five.
Imagine you have 100 people you want to sell cookies to. You show a cookie ad to 90 of them but hide the ad from the other 10. Then you count how many cookies each group buys. The extra cookies bought by the group that saw the ad is your “incremental lift.” That’s the real impact of your ad.
Does this work for every marketing channel?
The principles work for almost anything, but it’s easier on some channels than others. Paid social and search have built-in tools that make it simple to set up controlled tests with audience exclusions or geo-targeting. For other channels, you just have to get more creative with your experiment design to isolate what you’re testing.
