Are your Google Ads driving real growth - or just taking credit for sales that would’ve happened anyway? Incrementality testing helps you find out by measuring the true impact of your SEM campaigns. Instead of relying on clicks, impressions, or last-click attribution, this method shows how many conversions your ads actually create versus those that would’ve occurred naturally.
Key Takeaways:
- Incrementality Testing: Compares groups exposed to ads vs. those not exposed to measure the extra value ads bring.
- Core Metrics:
- CTR (Click-Through Rate): Tracks ad engagement.
- CPC (Cost-Per-Click): Measures ad cost efficiency.
- Conversion Rate (CVR): Shows how many clicks turn into actions (e.g., sales).
- Incremental Conversions & Revenue: Quantifies conversions and revenue driven solely by ads.
- Incremental ROAS: Revenue generated per dollar spent, beyond organic growth.
- Testing Setup:
- Create test and control groups (user-based or geographic).
- Measure differences in conversions to calculate incremental lift.
- Actionable Insights: Use results to adjust budgets, refine strategies, and focus on campaigns with higher incremental ROAS.
This guide helps you move beyond surface metrics to uncover which campaigns truly drive growth. Let’s dive deeper into how to design these tests and use the results to optimize your SEM strategy.
How do I test for Incrementality?
Key SEM Metrics for Incrementality Testing
To truly understand the impact of your SEM campaigns, it’s essential to track both baseline metrics and those specific to incrementality testing. Together, these metrics reveal whether your ads are driving measurable growth or simply capturing conversions that might have happened organically.
Primary SEM Metrics to Track
Start by monitoring the core SEM metrics that provide insights into your campaigns’ baseline performance.
Click-Through Rate (CTR) reflects how often users click on your ads after seeing them. For Google search PPC campaigns, the average CTR is 3.17%. Comparing CTR across ad variations can highlight what resonates with your audience. For instance, in the financial services sector, including a headline like "No Hidden Fees" boosted CTR from 2.4% to 4.1%.
Cost-Per-Click (CPC) indicates how much you’re paying for each click. Quality Score plays a big role here - a score of 6 or higher can cut CPC by 16–50%, while scores below 4 can increase costs by 25–400%. One education client reduced CPC by 23% after optimizing landing pages for their top 10 keywords, improving their Quality Score from 5/10 to 8/10.
Conversion Rate (CVR) measures the percentage of clicks that lead to desired actions, such as purchases or sign-ups. The average CVR is 2.35%, but top-performing campaigns can achieve rates three to five times higher. For example, a Singapore-based e-commerce client reduced their Cost Per Acquisition (CPA) from $45 to $28 by refining their conversion tracking and optimization efforts - all while maintaining the same conversion volume.
Return on Ad Spend (ROAS) shows how much revenue your ads generate for every dollar spent. The average ROAS is 2.87:1, meaning $2.87 in revenue for every $1 spent. However, ROAS alone doesn’t account for whether the revenue would have occurred without your ads, which is why incrementality metrics are key.
Quality Score impacts both ad placement and costs. Google evaluates this score based on ad relevance, expected CTR, and landing page experience. A higher Quality Score not only improves ad positions but also reduces costs.
Once you’ve established these baseline metrics, it’s time to dive into incrementality-specific measures to uncover the additional value your campaigns bring.
Incrementality-Specific Metrics
While standard SEM metrics provide a performance overview, incrementality metrics are designed to measure the extra impact of your ads.
Incremental Conversions quantify the conversions driven solely by your ads. To calculate this, compare a test group exposed to ads with a control group that isn’t. For example, if the test group achieves a 2% conversion rate and the control group achieves 1.5%, the incremental lift is calculated as (2% – 1.5%) / 1.5%, resulting in a 33% increase.
Incremental Revenue focuses on the extra revenue generated by reallocating spend to high-performing keywords.
Incremental ROAS measures revenue derived exclusively from additional conversions. A 2022 study revealed that 78.4% of senior-level marketers in the US still rely on last-click attribution, often overlooking the incremental value their ads deliver.
Causal Sales Lift captures the direct sales increase attributable to your ads. This is especially critical for branded campaigns, where studies suggest that 80% of customers would convert even without branded search ads.
By combining these metrics, you can distinguish between conversions that your ads created versus those that would have happened organically. Standard metrics count total conversions, while incrementality metrics isolate the additional ones.
At Experiment Driven, we prioritize metrics like nCAC, LTV, and incrementality multipliers to identify campaigns that drive genuine growth. For example, an analysis of $14 million annually spent on Google paid search campaigns revealed that 80% of revenue came from branded terms, while 80% of the spend on non-branded competitor keywords accounted for only 20% of the results. These insights lay the groundwork for designing effective incrementality tests.
How to Set Up Incrementality Tests for SEM Campaigns
Running successful incrementality tests requires careful planning and execution. The objective is to create a controlled setup where you can accurately measure the true impact of your SEM campaigns beyond what would naturally occur.
Test and Control Group Setup
The backbone of any incrementality test is establishing two distinct groups: a test group that sees your ads and a control group that does not. These groups should be as similar as possible in terms of demographics, behavior, or location - and they must not overlap.
To ensure the groups are comparable, segment your audience based on shared characteristics. For example, you could use factors like location, age, or online behavior. It’s also crucial to minimize external influences during the testing period to maintain clean, reliable data.
Once your groups are set, measure the incremental lift by comparing conversions. For instance, if your test group achieves 120 conversions while your control group records 100, the incremental lift would be 16.7%.
A practical example comes from a retail business that conducted an incrementality test using well-segmented groups. Their efforts led to a 34% increase in sales attributed to the campaign.
After defining your groups, the next step is to carefully plan the details of your experiment.
Planning Your Experiment
For incrementality testing to yield actionable insights, you need clear goals, adequate test duration, and control over external factors.
Start by defining your measurement goals. Are you looking to evaluate ROAS (Return on Ad Spend), cost per acquisition, or revenue lift? These objectives will shape the structure of your test.
Determine the appropriate length of your test. A minimum of one week is usually recommended, but if your campaign has lower traffic, extending the duration can help achieve statistical significance.
Timing is also critical. Avoid running tests during periods when customer behavior deviates from the norm, such as major shopping seasons or holidays. These events can skew results, making it harder to isolate the true impact of your campaign.
Lastly, calculate the minimum sample size needed to detect meaningful differences between your test and control groups. This ensures your results are statistically reliable.
When planning, you’ll also need to decide whether to use a user-based or geographic testing approach.
User-Based vs. Geographic Tests
The choice between user-based and geographic testing depends on your goals, audience size, and the type of insights you’re seeking.
User-based tests work like traditional A/B tests. They randomly divide your audience at the individual level, making them ideal for smaller campaigns or when quick insights are needed. These tests usually deliver results in about a week. However, they have limitations: they can’t measure cross-channel lift, rely on ad networks for execution, and face tracking challenges due to privacy restrictions.
Geographic tests, on the other hand, group audiences by location - such as cities, regions, or zip codes. These tests take longer (2–4 weeks) but are better suited for measuring the combined impact of campaigns across multiple channels, including harder-to-track media like TV or radio. Geographic testing has gained popularity as privacy changes have made user-level tracking more difficult.
Here’s a quick comparison:
| Test Type | Best For | Time to Results | Key Limitations |
|---|---|---|---|
| User-Based | Small audiences, quick insights | ~1 week | Cannot measure cross-channel lift |
| Geographic | Cross-channel measurement, larger campaigns | 2–4 weeks | Requires additional expertise |
When deciding, consider your priorities. If you need to understand how SEM campaigns interact with other marketing channels, geographic testing offers a broader perspective. For faster optimizations of individual campaigns with enough traffic, user-based tests are more practical.
At Experiment Driven, we often recommend geographic testing for a more thorough evaluation of campaign incrementality. While these tests require more setup time and expertise, they provide transparent and reliable results without depending on tracking methods that may be restricted. The effort typically pays off with insights that are both actionable and trustworthy.
sbb-itb-d4df0e6
How to Use Incrementality Test Results
Take the insights from your test and control group comparisons and turn them into actionable strategies to fine-tune SEM performance and spending.
Budget Allocation Based on Results
One of the quickest ways to apply incrementality test results is by refining your budget allocation. Instead of depending on last-click attribution or gut instincts, you can now base your decisions on which campaigns genuinely drive incremental growth.
Start by analyzing incremental ROAS (Return on Ad Spend) across campaigns, keywords, and ad formats. This metric shows how much additional revenue each dollar of ad spend generates beyond what would have occurred naturally. Campaigns with a higher incremental ROAS should receive more budget, while those with low or negative incrementality should be scaled back or even paused.
Not all campaigns will deliver positive returns. If you see little to no lift, it’s a sign to rethink your strategy or creative direction before committing more resources. For example, try increasing budgets for promising campaigns by 15–20% and allow 10–14 days to assess the impact before making further changes.
These findings also provide a foundation for evaluating the overall effectiveness of your SEM strategy.
Testing SEM Strategy Effectiveness
Incrementality testing helps you determine whether your SEM strategies are genuinely effective or if they just seem successful due to attribution quirks. This is especially helpful for branded search campaigns, where it’s often unclear if new conversions are being driven or if existing demand is simply being captured.
For example, you might find that some keyword categories yield strong incremental lift, while others mainly capture conversions that would have happened anyway. Testing can also uncover which ad formats, bidding strategies, or audience targeting methods contribute to real growth rather than vanity metrics.
Consider the case of a mattress retailer working with Acceleration Partners. They shifted their focus to SEM partners that influenced customers during the research phase rather than just at the point of conversion. This change led to a 160% incremental ROAS, a 15% boost in conversion rates, and $1.44 million in incremental revenue.
Incrementality testing also helps fine-tune the timing of re-engagement campaigns. By examining post-click or post-install windows, you can pinpoint the best moments for follow-up ads, ensuring maximum incremental lift from retargeting efforts.
Making Data-Driven SEM Decisions
Use incrementality insights to enhance campaign performance by digging deeper than surface-level metrics. The goal is to identify the real causal relationships between your ads and business outcomes.
For example, compare incremental ROAS with customer lifetime value (LTV) to gauge true profitability. A campaign might show strong short-term returns but attract customers with lower long-term value. On the flip side, campaigns with modest immediate results could bring in high-value customers who generate substantial revenue over time.
When evaluating top-performing campaigns, look at metrics like conversion rate and cost-per-lead. However, always view these through the lens of incrementality. A campaign with a high conversion rate but low incremental lift might simply be capturing demand that would have converted elsewhere.
These insights should also influence your creative and targeting strategies. If certain audience segments show strong incremental lift, consider expanding your reach to similar groups. Similarly, if a specific ad creative drives genuine incremental conversions, use it as a blueprint for future campaigns.
KURU Footwear provides a great example of this approach. By leveraging incrementality insights, they identified the true value of their marketing efforts, optimized their paid social channels, and scaled their social ad spend by 350% - all while maintaining profitable growth.
Avoid making decisions on autopilot. Instead, combine conversion data with incrementality results to validate and refine your SEM efforts over time. This creates a feedback loop where each test informs better decisions, driving continuous improvement.
At Experiment Driven, this data-driven approach has consistently transformed SEM performance. Marketers who rely on their own customer data to guide decisions see a 30% boost in performance compared to those sticking with traditional attribution methods. The key is to treat incrementality testing as an ongoing process, not a one-off task, ensuring it informs every major SEM decision you make.
Using SEM Metrics for Long-Term Growth
Building on earlier discussions about driving action through incrementality testing, let’s dive into how to achieve sustained growth in SEM. A long-term strategy requires a deliberate, data-driven approach that incorporates insights from incrementality testing into every decision. Here’s how you can make that happen.
Key Steps to Keep in Mind
Start by defining unified metrics that give you a complete view of your SEM performance. While traditional metrics like click-through rates (CTR) and conversion rates are helpful, they only scratch the surface. Focus on metrics that highlight true incremental gains, such as total SERP share, the lift in organic CTR when paid ads are running, and blended ROI/ROAS across both paid and organic efforts.
"Combining paid and organic data provides a fuller picture of high-performing keywords, allowing for more effective targeting across both channels."
Another critical metric to track is customer lifetime value (CLV). This helps you understand how customers acquired through different SEM campaigns contribute to your business over time. It’s a great way to distinguish between campaigns that attract high-value, loyal customers and those that deliver quick, but short-lived, conversions.
Additionally, keep an eye on total SERP visibility - your brand’s overall presence across both paid and organic search results. This metric gives you a broader view of your market presence and highlights areas where you can refine your strategy to maximize impact.
Finally, centralize your experiment results. By doing so, you can avoid repeating past mistakes and build on what’s already proven to work.
Embracing a Testing-First Strategy
Once you’ve established unified metrics and centralized data, the next step is to adopt a testing-first mindset. Make incrementality testing the cornerstone of your SEM growth strategy. Run frequent, data-driven experiments to quickly identify what works and scale those approaches across your campaigns.
"Experiment Driven is like a SWAT team for marketing. They come in to tackle our biggest challenges, such as wasted ad spend and misaligned teams. They run fast, data-driven experiments to find out what works best for our audience and help us optimize our strategies."
- Experiment Driven Customer
Align your paid and organic teams around a single, shared metric. When both teams are working toward the same incrementality goals, you can optimize your entire search presence instead of managing disjointed efforts.
Use real-time dashboards to monitor performance across all channels. This allows you to make quick, informed adjustments based on incrementality data, helping you spot trends early and shift budgets before underperforming campaigns drain resources.
Update your media mix regularly using the latest incrementality data rather than relying on outdated attribution models. As search behavior evolves - especially with advancements in AI - your testing methods should adapt accordingly.
Companies that commit to continuous experimentation and data-driven decisions often see long-lasting results. By rethinking their marketing stack and refining their measurement processes, they create a foundation for growth that builds over time.
At Experiment Driven, this approach to incrementality testing has consistently shown results. By focusing on metrics like Customer Acquisition Cost, Lifetime Value, and incrementality multipliers across marketing channels, we help businesses achieve sustainable growth that compounds year after year.
FAQs
How can I tell if my SEM campaigns are driving real growth or just capturing existing demand?
To figure out if your SEM campaigns are genuinely driving new growth or just pulling in existing demand, incrementality testing can be a game-changer. Here's how it works: you split your audience into two groups - a control group that doesn’t see your ads and a test group that does. Then, compare key metrics like conversions and sales between the two groups. If the test group shows a noticeable boost in conversions, it’s a clear sign that your campaigns are bringing in new customers, not just redirecting existing ones.
Another useful method is A/B testing. By tweaking ad exposure between groups and tracking how customer behavior changes, you can pinpoint the real impact of your SEM campaigns. This kind of data-driven analysis gives you the insights needed to fine-tune your approach and make smarter marketing decisions that drive long-term growth.
What’s the difference between user-based and geographic testing for incrementality, and how do I decide which is best for my campaign?
User-based testing and geographic testing each offer distinct methods and insights when evaluating incrementality.
User-based testing works by splitting users into randomized groups to assess how specific marketing efforts influence individual behaviors. This method shines when you need detailed insights into metrics like customer acquisition costs (CAC) or lifetime value (LTV). It’s particularly useful for campaigns aimed at specific audience segments, allowing for a more focused analysis.
In contrast, geographic testing evaluates campaign performance by comparing results across different locations. This method provides a broader perspective, revealing how factors like regional preferences or local economic conditions impact outcomes. It’s often easier to execute and is especially helpful when user-level data isn’t available.
When deciding between the two, think about your campaign’s objectives. If your goal is to dig into individual user behavior, user-based testing is the way to go. But if you’re more interested in understanding regional patterns or market dynamics, geographic testing will likely serve you better.
How can I use incrementality testing to optimize my SEM budget and improve campaign performance?
Incrementality testing lets you gauge the actual impact of your paid search campaigns by pinpointing the conversions that are directly driven by your ads. By comparing a group exposed to ads with a control group that isn’t, you can figure out which campaigns are pulling in new customers versus those simply converting users who would have purchased anyway.
Armed with this information, you can fine-tune your SEM budget by focusing on campaigns and channels that generate the most incremental growth. Adjusting bids, targeting niche keywords, and reallocating spending can help you get the most out of your budget while boosting ROI. This data-driven strategy ensures your marketing efforts are working smarter to fuel long-term growth.
