Urban Roots: Incrementality Testing in 2026

When AI agents strip UTMs and referrers, accurate incrementality testing becomes a critical challenge for marketers. This isn’t just about losing some data points; it’s about fundamentally misattributing performance and making poor investment decisions.

I’ve seen firsthand how the proliferation of AI agents and privacy-focused browsers has made traditional last-click attribution a fool’s errand. It’s not just cookies disappearing; it’s entire segments of user journeys becoming invisible. We recently ran a campaign for “Urban Roots,” a fictional DTC plant subscription service, and it perfectly illustrated the complexities of incrementality testing when AI agents strip UTMs and referrers.

This wasn’t a small client; Urban Roots had secured a significant Series B round and needed aggressive, measurable growth. Their previous marketing efforts, while seemingly successful based on last-click data, lacked true insight into what was actually driving new customer acquisition versus simply capturing demand that would have converted anyway. My team at GrowthMetrics (our fictional agency) was brought in specifically to unravel this.

Campaign Teardown: Urban Roots’ “Green Living” Spring Acquisition Drive

Objective: Drive new subscriber acquisition for Urban Roots’ premium plant subscription box, focusing on audience segments interested in sustainable living and home decor.
Budget: $750,000
Duration: 8 weeks (March 1st, 2026 – April 26th, 2026)
Target Audience: US-based adults, 25-54, with interests in gardening, home decor, sustainability, and eco-friendly products. We segmented further into “Beginner Botanists” (new to plants) and “Lush Life Enthusiasts” (experienced plant parents).
Key Performance Indicators (KPIs): Incremental Subscribers, Return on Ad Spend (ROAS), Cost Per Incremental Lead (CPIL), Brand Search Lift.

Strategy: Embracing a Multi-Method Incrementality Approach

Knowing that AI agents stripping UTMs and referrers would be a significant hurdle, we developed a multi-pronged incrementality strategy. We couldn’t rely solely on pixel-based attribution. This is where many marketers get it wrong – they try to patch up old methods rather than adopting new ones. My opinion? If you’re not planning for this reality, you’re already behind.

1. Geo-Lift Testing (Primary Method): We divided 20 major US DMAs into test and control groups.

• Test Group (15 DMAs): Full ad spend allocation across Meta Ads, Google Search, and connected TV (CTV) platforms.
• Control Group (5 DMAs): Minimal “brand awareness” spend (primarily organic social and PR, no paid acquisition ads). These were carefully selected DMAs with similar demographic profiles and historical Urban Roots purchase patterns. We used Nielsen’s geo-demographic data to ensure comparability, a step I insist on for any geo-test.

1. Holdout Group (In-Platform): Within our paid channels (Meta Ads, Google Ads), we allocated 10% of the eligible audience to a ghost ad or public service announcement campaign. This meant these users saw an ad, but it wasn’t for Urban Roots – it was a neutral message, allowing us to measure the baseline conversion rate of an exposed but uninfluenced group. This is different from a pure control group, as they still saw an ad, but it helps isolate the effect of our specific advertising.
2. Brand Search Lift Analysis: We monitored branded search queries (“Urban Roots,” “Urban Roots subscription,” etc.) in both test and control DMAs, as well as for the holdout groups. This is a robust indicator of brand awareness and intent, less susceptible to direct attribution issues.
3. Survey-Based Brand Lift: Partnering with a third-party research firm, we conducted weekly brand perception surveys within exposed and unexposed segments to gauge changes in awareness, consideration, and purchase intent. According to a recent IAB report on privacy-centric measurement, brand lift studies are gaining significant traction as a reliable indicator of campaign effectiveness in a cookieless world.

Creative Approach: Storytelling & Education

Our creative focused on the emotional benefits of plants and sustainable living, rather than just product features.

• Video Ads (Meta & CTV): Short, engaging narratives showing people transforming their homes with plants and the joy of nurturing them. We used A/B testing on headlines and calls to action (CTAs).
• Image Ads (Meta & Google Display): High-quality, aspirational imagery of lush interiors and close-ups of beautiful plants, with clear value propositions like “Curated for Your Home” and “Sustainable Sourcing.”
• Search Ads (Google): Targeted long-tail keywords related to “easy houseplants,” “sustainable plant delivery,” and “indoor gardening kits.”

Targeting: Precision with a Safety Net

We used a combination of interest-based targeting, custom audiences (lookalikes of existing subscribers), and contextual targeting. Crucially, we implemented Privacy Sandbox compliant targeting features available on platforms like Google Ads, which allowed for some level of audience segmentation without relying on individual user data. This isn’t perfect, but it’s the direction things are moving.

What Worked (and What Didn’t): A Data-Driven Post-Mortem

| Metric | Test DMAs (Paid Exposure) | Control DMAs (Minimal Exposure) | Holdout Group (Paid, Neutral Ad) |
| :————————– | :———————— | :—————————— | :——————————- |
| New Subscribers | 12,500 | 4,200 | 1,100 |
| Total Conversions | 15,000 | 5,000 | 1,200 |
| CPL (Attributed) | $50 | N/A | N/A |
| ROAS (Attributed) | 1.8x | N/A | N/A |
| CTR (Paid Channels) | 1.8% | N/A | 1.1% |
| Impressions | 65M | 8M | 15M |
| Branded Search Lift | +35% | +8% | +12% |
| Survey Brand Awareness | +15% | +3% | +5% |

Initial Last-Click Attribution (Pre-Incrementality): Based on platform reporting, we were seeing an average CPL of $50 and a ROAS of 1.8x. This looked good on paper, but I always tell my clients: don’t trust platform numbers blindly. They’re designed to make their platform look good.

Incrementality Results:

• Geo-Lift:
• Incremental Subscribers: (12,500 in Test DMAs / 15 DMAs) – (4,200 in Control DMAs / 5 DMAs) = 833 – 840. Wait, what? This initial calculation suggested negative incrementality. This was our first “oh shoot” moment.
• Correction: We realized that while the DMAs were demographically similar, the baseline organic growth in the control group was slightly higher due to existing community groups and local events we hadn’t fully accounted for. After normalizing for baseline growth rates (using pre-campaign data), the adjusted incremental subscribers from geo-lift analysis was approximately 2,800. This yielded an Incremental CPL of $267 and an Incremental ROAS of 0.38x. Ouch.
• Holdout Group:
• The 10% holdout group (who saw neutral ads) generated 1,100 new subscribers from the same audience pool as the paid group.
• The paid group (90% of the audience) generated 12,500 subscribers.
• Baseline Conversion Rate (Holdout): 1,100 subscribers / (Total Audience * 10%)
• Paid Conversion Rate: 12,500 subscribers / (Total Audience * 90%)
• This analysis showed that approximately 30% of paid conversions would have occurred organically or through other channels. This meant our incremental subscribers from this method were roughly 3,750.
• Brand Search Lift: The 35% lift in branded searches in test DMAs versus 8% in control DMAs strongly indicated that our paid media was driving genuine new interest, not just capturing existing demand. This is a critical qualitative indicator when quantitative attribution falters.

The Hard Truth: Our initial ROAS of 1.8x was highly inflated. The true incremental ROAS, combining insights from geo-lift and holdout groups, was closer to 0.6x. This meant we were spending $1 to get $0.60 back, which is clearly unsustainable. This is the editorial aside I’m talking about: anyone who tells you that their last-click ROAS is the only thing that matters is either naive or trying to sell you something.

Optimization Steps Taken:

1. Reallocated Budget: We immediately shifted 40% of the Meta Ads budget from broad interest targeting to lookalike audiences of our most profitable existing subscribers (those with high lifetime value and repeat purchases). This was a direct response to the low incremental ROAS.
2. Increased CTV Frequency Capping: Our CTV ads had high reach but low frequency. We increased the frequency cap from 2x per week to 4x per week to improve message retention, particularly in the geo-test areas showing positive brand lift.
3. Server-Side Tagging Implementation: This was a major technical undertaking. We worked with Urban Roots’ development team to implement a server-side Google Tag Manager (sGTM) setup, processing conversion data directly from their server to advertising platforms via APIs. This significantly reduced data loss from AI agents stripping UTMs and referrers, as the data wasn’t relying on client-side browser events. This is non-negotiable for serious marketers in 2026. If you’re still relying purely on client-side pixels, you’re bleeding data.
4. Enhanced First-Party Data Collection: We refined Urban Roots’ onboarding process to collect more explicit consent for email and SMS marketing, allowing us to build more robust first-party audiences for retargeting and exclusion, further reducing reliance on third-party identifiers.
5. Creative Refresh: We introduced new video creative that was more direct in its value proposition and included clearer calls to action, addressing feedback from our survey data that some initial ads were too abstract. We also tested new landing pages with improved conversion funnels.

Refined Metrics (Post-Optimization – Last 4 Weeks of Campaign):

| Metric | Test DMAs (Paid Exposure) | Control DMAs (Minimal Exposure) | Holdout Group (Paid, Neutral Ad) |
| :————————– | :———————— | :—————————— | :——————————- |
| New Subscribers | 10,200 | 3,800 | 950 |
| Total Conversions | 12,000 | 4,500 | 1,050 |
| CPL (Attributed) | $45 | N/A | N/A |
| ROAS (Attributed) | 2.1x | N/A | N/A |
| CTR (Paid Channels) | 2.5% | N/A | 1.3% |
| Impressions | 40M | 6M | 10M |
| Branded Search Lift | +48% | +10% | +15% |
| Survey Brand Awareness | +22% | +4% | +7% |

After these adjustments, our incremental subscriber count for the latter half of the campaign improved dramatically. The geo-lift analysis, adjusted for baseline growth, showed approximately 3,500 incremental subscribers, leading to an Incremental CPL of $171 and an Incremental ROAS of 0.88x. Still not profitable on first purchase, but a significant improvement. The holdout group analysis also showed a reduced percentage of “would have converted anyway” users, indicating our targeting was becoming more effective at reaching truly new customers.

This experience solidified my belief that incrementality testing is no longer a “nice-to-have” but a fundamental requirement, especially with the persistent challenge of AI agents stripping UTMs and referrers. You simply cannot trust your raw attribution data anymore. You need to build a robust framework that accounts for the invisible.

To truly understand marketing impact, marketers must pivot from solely relying on platform-reported attribution to a rigorous incrementality testing framework, integrating server-side solutions and multi-touchpoint analysis.