Landing Page A/B Tests for Paid Traffic in 2026: Testing Without Breaking Smart Bidding
The familiar version of this story: a team swaps a headline, sees 3.8% conversion against 3.1% after four days, declares a winner. A month later the difference is gone, but the campaign has been more expensive ever since. The hypothesis was not the problem — the method was. Landing page A/B tests on paid traffic measure the page and disturb bid strategy learning at the same time, unless they are run carefully. Here is how to test so the result holds up and the campaign does not pay for the experiment twice.
What you get: four split methods and their real cost, a sample size formula with reference numbers, a day-by-day run order, the biases that make conclusions worthless, and the cases where you should not test at all.
Why a landing page split is not an ad split
When you test two ads, the system distributes impressions and scores them inside the same auction. Testing two landing pages adds three effects that creative tests never have.
- Landing page experience feeds Ad Rank. Different pages mean different speed, structure and relevance. If version B is slower, you are measuring load time, not the headline. What counts as page experience is in our breakdown of Quality Score and Ad Rank.
- Smart Bidding learns from conversions. While half your traffic hits the weaker page, the strategy receives a blended signal and re-prices towards the weighted average — that is the hidden cost of the test.
- Different URLs mean different reporting. Without disciplined tagging you cannot separate the two cohorts a week later; at minimum you need a stable parameter layer.
A landing page test on paid traffic always costs more than it looks. The price is not only the spend on the losing version — it is a week of bidding on a noisy signal. So you test what changes the economics, not everything that could be changed.
Four ways to split, and what each one costs
| Method | How it works | Upside | Cost |
|---|---|---|---|
| Server-side or client-side split on one URL | One final URL; a testing tool serves a version per visitor and remembers the assignment | Campaign structure untouched; Smart Bidding sees a single URL and does not re-solve | Needs a sound tool; client-side splits can add latency and content flicker |
| Campaign experiment | A campaign copy with a different final URL and a defined traffic share | Clean auction-level separation and campaign-level reporting | Fragments learning data; no conclusions on thin volume |
| Different URLs in parallel ad groups | Twin ad groups with identical keywords, different pages | Fast to assemble | The dirtiest option: impressions distribute unevenly and the groups differ on rank |
| Geo split | Version A in some regions, B in others | Measures the business outcome; suits large changes | Needs volume and comparable regions |
The working rule: if you are changing one element inside the page — headline, form, offer, block order — prefer the single-URL split. If you are replacing the page or the user path (a form instead of a phone call), use a campaign experiment or a geo split. How experiments differ from drafts is covered in campaign experiments in Google Ads, and the geo methodology sits in Conversion Lift for Search and Performance Max.
How much traffic you need: arithmetic instead of instinct
This is the main source of false winners. The required volume depends on your baseline conversion rate and the uplift you expect to detect, and the relationship is non-linear: the smaller the uplift, the steeply larger the sample.
A serviceable planning estimate for comparing two proportions at 95% significance and 80% power: visitors per version ≈ 16 × p × (1 − p) / d², where p is the baseline rate as a decimal and d is the absolute uplift as a decimal. Approximate, but good enough to plan with.
| Baseline conversion | Uplift you want to detect | Visitors per version (reference) |
|---|---|---|
| 2% | +0.5 pp (to 2.5%) | about 12,500 |
| 2% | +1 pp (to 3%) | about 3,100 |
| 5% | +1 pp (to 6%) | about 7,600 |
| 5% | +2 pp (to 7%) | about 1,900 |
| 10% | +2 pp (to 12%) | about 3,600 |
Treat those as planning references, not promises. Two practical conclusions follow. First: at a 2% baseline and 300 visits a day, an honest test for a half-point uplift needs roughly 25,000 visits across both versions — more than two months. That test is not worth starting. Second: on thin traffic you only test large changes where you expect a multiple, not a few percent. Everything else is handled by removing obvious defects rather than by splitting.
How to run landing page A/B tests: eight steps
- Write the hypothesis down before launch. “Replace the headline with a price-led version; expect form conversion to move from 3% to 4%, because price is the first question in support chat.” A hypothesis without an expected magnitude becomes a licence to fit the result.
- Calculate the sample and the duration with the formula above. If the honest duration exceeds six weeks, make the hypothesis bigger.
- Verify technical parity. Speed, mobile layout, a working form, identical analytics events. Speed differences are the most common cause of a false result; what to measure is in landing page speed and Core Web Vitals.
- Freeze the campaign. For the duration, do not touch bids, budgets, strategy, keywords or ads. Any change inside the window makes the conclusion unprovable.
- Run in whole weeks. Weekday and weekend behaviour differ; a partial week skews the result.
- Look, but do not decide early. Watching the data is fine; stopping at the first crossover is not — repeated peeking almost guarantees a false winner.
- Score in money, not conversion rate. A version with higher conversion and a lower average order value is the loser. How to collapse this into one number is in budget pacing and spend economics.
- Allow a recalibration window after the finish. Seven to fourteen days after routing all traffic to the winner is recovery, not failure.
What spoils results most often
- Traffic mix skew. If version B happened to receive more brand traffic, it wins regardless of content. Compare the composition of each cohort, not just the totals.
- Different attribution windows. If one version drives an instant form fill and the other a next-day call, a short window names the wrong winner. Model mechanics are in attribution models: DDA versus last click.
- Double-counted conversions. The classic: the new page fires the event twice. Before celebrating, check for discrepancies using the method in Google Ads and GA4 conversion discrepancies.
- Junk leads. An uplift won by shortening the form is frequently an uplift in junk. Filters are in fake leads and form checks.
- Automation changed the page for you. In expansion modes the system can substitute a different final URL, sending half the traffic somewhere you did not plan. Check the settings against final URL expansion in PMax and AI Max.
- A promotion inside the test window. A discount, an email blast or seasonal demand all shift the baseline. A test that lands on a sale gets rescheduled.
When not to test
- An obvious defect. The form fails on mobile, the page takes six seconds, the offer is unreadable on a phone — fix it, do not test it.
- Not enough traffic, provably. See the sample maths: if the honest duration is over two months, the test cannot answer the question but will still cost full price.
- The change cannot move the decision. Button colour on a single-screen page with one form almost never shifts the economics enough to be detectable at your volume.
- The decision is already made. If the result will not change what you do, the test is a ritual, not a measurement.
A useful alternative at low volume: sequential comparison against a control over a long window, plus qualitative input — session recordings, heatmaps, five customer interviews. That is not statistical proof, but at fifty leads a month it is more useful than a split that can never reach significance. A reliable source of hypotheses is the landing pages report in Google Ads, which shows where paid traffic dies after the click and usually nominates the page to work on.
Sequencing page tests with ad tests
Changing creative and the landing page at once is a reliable way never to learn the cause. The working order: fix the technical layer first, then test the offer on the page, then align ads with the winning offer. Ad testing methodology is covered separately in RSA ad A/B testing. If what you are changing is the traffic structure rather than the copy — reallocating budget across channels, for instance — finish that change first; its mechanics are in today’s companion piece on channel prioritization and budget caps. And if the real problem is which queries arrive at all, start with keyword match types and close variants.
For how we run this on live accounts, see the Google PPC agency overview; access and structure questions for manager accounts are under Google Ads agency accounts.
What to test: ranking hypotheses by money
Teams usually test what is easiest to change rather than what moves the result. Hence months of work with no measurable effect. The working hierarchy, in descending order of expected impact:
| Tier | What changes | Expected impact | Volume required |
|---|---|---|---|
| 1 | The offer itself: what the visitor gets, at what price, with what guarantee | Multiples | Minimal — the gap shows almost immediately |
| 2 | The action type: form, call, messenger, instant quote | Tens of percent | Moderate |
| 3 | Page structure: what sits above the fold, block order, form length | Single to low double digits | Moderate |
| 4 | Wording: headline, subhead, button label | A few percent | High |
| 5 | Styling: colours, fonts, imagery | Fractions of a percent | Out of reach for most accounts |
Start at the top, because only there is the effect large enough to detect on your traffic. A button-colour test at 300 visits a day is not a test; it is waiting for a random number.
Where hypotheses come from, other than imagination: the questions people ask in chat and on the phone before buying; the objections your sales team repeats; the queries actually arriving (if someone searches for pricing and the page has no price, that is a tier-one hypothesis); mobile session recordings. The query link matters — the page has to answer the intent that brought the visitor, and how that intent is formed is covered in keyword match types and close variants.
Scoring the result: from conversion rate to profit
Conversion rate is an intermediate metric. The decision is made in money, and four numbers per version are enough: visits, actions, the qualified share of those actions, and average revenue per qualified action.
A pattern that recurs constantly. Version A: 5,000 visits, 150 leads (3.0%), 60% reach a conversation, average deal $2,000. Version B: 5,000 visits, 190 leads (3.8%), but the qualified share drops to 40%, same deal size. On conversion rate B wins comfortably: 3.8% against 3.0%. In money: A delivers 90 qualified leads, B delivers 76. Version A won — and a team deciding on conversion rate would have made the result 15% worse while sincerely believing they improved it by 27%.
Hence the rule: agree the deciding metric before launch and write it down. If quality data does not exist in your systems, build it before the test rather than after. How to translate revenue into an allowable cost per click is in unit economics and LTV for media buying.
Second wrinkle: the window. If deals close in two weeks, the revenue verdict arrives two weeks after the lead verdict. Standard practice is to decide on qualified leads — not on all leads — and then re-verify against revenue over the longer window.
A test log: without it the team walks in circles
Six months on, nobody remembers what was already checked. The same hypothesis gets tested twice, and the conclusions of past tests get recalled from memory with a convenient sign. A minimal log is one table with eight columns:
- Start and end date.
- What changed, in one line.
- The hypothesis with its expected magnitude.
- Split method and traffic share.
- Volume per version: visits and actions.
- The deciding metric, agreed in advance.
- Result: winner, loser or “no difference”.
- What was done next, and when.
The column teams usually forget: external events inside the window — promotions, seasonal spikes, campaign edits. Six months later it is the one that explains why a strange result was strange. If you keep a general change log for the account, test entries belong there alongside campaign edits, which makes it far easier to tie a metric dip to a specific action. How that log fits into regular account review is in our Google Ads account audit checklist.
Last in order, not in importance: record the negative results too. “No difference” is a finding that saves the next person three weeks and a budget.
FAQ
How long should a landing page A/B test run?
At least one full week, in whole-week increments, and until the calculated sample is reached. If the calculation points past six weeks, enlarge the hypothesis or skip the test.
How much traffic does a test need?
It depends on the baseline rate and the uplift you want to detect. As a reference, a 5% baseline and a one-percentage-point uplift needs roughly 7,000–8,000 visitors per version.
Does split testing break Smart Bidding?
A single-URL split barely touches it, since the system sees one address. Different final URLs change the learning data, so the strategy re-prices; allow 7–14 days to stabilise after the test ends. See the Smart Bidding learning period for what normal recalibration looks like.
Campaign experiment or single-URL split?
For changes inside the page, the single-URL split. For replacing the page or the user path entirely, a campaign experiment or a geo split.
Can I just build two ad groups with different pages?
Technically yes; methodologically it is the weakest option, because impressions distribute unevenly and the groups differ on Ad Rank, which makes the cohorts incomparable.
Why can’t I stop the test once a difference appears?
Because repeated checks sharply raise the odds of seeing a “significant” difference by chance. Fix the duration and sample in advance and see them through.
What if the higher-converting version earns less?
Score in money. The winner is the version with better profit or better cost per qualified action, not the better conversion percentage.
Does page speed affect the test result?
Substantially. A speed gap between versions is the most common source of a false result, which is why technical parity is verified before launch.
Should I pause a test during a promotion?
Yes, reschedule it. Promotions change both traffic mix and baseline conversion, so a result measured inside a sale does not transfer to normal periods.
How do I test with fifty leads a month?
A statistically sound split will not assemble at that volume. Test only large changes, lean on session recordings, heatmaps and customer interviews, and decide on a long-window comparison against a control.
Can I change ads during a landing page test?
No. Any change inside the window blends the effects and the cause becomes unrecoverable.
What happens after the test ends?
Route all traffic to the winner, record the result and give the bid strategy 7–14 days to re-solve. Launch the next hypothesis only after it stabilises.
