Testing a New Lead Source: Sample Size, Pass Marks and When to Scale

Test a new lead source on a pilot batch of 300 to 500 leads before you move a campaign to it. Verify the whole batch and record how much fails. Then send only the valid rows with your usual copy, and measure bounces, replies and complaints against pass marks you wrote down beforehand. Smaller tests mislead: at 100 sends, one bounce moves the rate by a full percentage point. Once a source passes, keep tagging its leads, because quality drifts and you will only see it in per-source numbers.

This is about choosing what to scale. If you have already bought a file and need to check the delivery, our guide to checking a lead list's quality before you pay covers acceptance testing.

Why small tests mislead

A bounce rate from a small batch has a wide margin of error. The table uses the standard approximation for a 95% interval around a measured 2% bounce rate:

Emails sent in the test	Bounces at 2%	Rough 95% range
100	2	about 0% to 4.7%
300	6	about 0.4% to 3.6%
500	10	about 0.8% to 3.2%
1,000	20	about 1.1% to 2.9%

At 100 sends, a source with a real 4% bounce rate and one with a real 1% rate can both show "2 bounces". You cannot tell them apart.

Zero bounces is not proof either. A quick rule from statistics, the "rule of three", says that if you see no bounces in n sends, the true rate could still be as high as about 3 divided by n. Zero bounces in 100 sends only tells you the rate is probably under 3%. Zero in 300 tells you it is probably under 1%.

For reply rates the problem is worse, because replies are rarer than bounces. If you care about reply rate as a pass mark, plan on a larger test, or judge replies over the first few weeks after scaling.

Stage 1: verification tells you about the data

Run every lead in the pilot through verification before any of it is sent. Record:

Measure	What it tells you about the source
Share invalid	How much is stale, guessed wrong or made up
Share catch-all	How many leads sit at domains that accept everything, so cannot be confirmed
Share unknown or timed out	Usually temporary; retry before you count it
Share role addresses (info@, sales@)	Whether the source gives you people or inboxes
Share freemail on a B2B file	Whether contacts are personal addresses rather than work ones
Duplicates	Whether the export was cleaned at all

A source that returns a large invalid share has a data problem you will pay for twice: once to buy it and again to verify it. A large catch-all share is not a flaw in the source, but it means fewer addresses can be confirmed. Our guide to reading a verification report explains each status.

Free tools for this stage: the email list sampler draws a random pilot from a bigger file, remove duplicate emails cleans repeats, and the role-based email filter and free vs business email filter count the other two.

Stage 2: the send tells you about the real world

Send only the valid rows. That protects your inboxes and measures what verification cannot see.

1. Hold everything else constant. Same ICP filter, same copy, same inboxes, same days of the week. If you change the copy at the same time, you are testing the copy.
2. Send from inboxes that are already warm and healthy, at their normal daily cap.
3. Run the full sequence, or at least two steps, because late bounces and decay show up on later steps. See why follow-up emails bounce more than step 1.
4. Record per lead: delivered, hard bounce, soft bounce, reply (positive, negative, out of office), unsubscribe, spam complaint if your tool reports it.
5. Read the bounce text. Run bounces through Bounce Doctor. Bad-address bounces on verified-valid rows point to a stale source. Gateway or policy rejections point to the targeting or the sending, not the data.

Set pass marks before you start

Write down what "good enough to scale" means before the data arrives. Otherwise a source gets kept because it was cheap, or because someone championed it.

An example scorecard, with illustrative pass marks you should replace with your own:

Measure	Pass mark	Source A	Source B
Invalid at verification	Under 10%
Catch-all at verification	Informational
Hard bounce on valid sends	Under 1%
Positive replies per 100 delivered	At or above current source
Unsubscribes and complaints	No worse than current source
Cost per verified valid lead	Within budget

Cost per verified valid lead is the number that compares fairly. Divide what you paid for the pilot by the number of rows that came back valid. A cheap source with a high invalid share can cost more per usable lead than an expensive one. The cold email bounce cost calculator and cost per lead calculator help with the arithmetic.

Comparing two sources fairly

If you are choosing between two vendors, run them side by side:

• Same filters. Ask both for the same titles, industries, company sizes and regions.
• Overlap check. Use compare email lists to see how many leads appear in both. Overlapping leads let you compare the two sources' data for the same person directly.
• Same batch size, sent in the same weeks, from inboxes of similar age.
• Split the inboxes evenly. Do not send Source A from your oldest inboxes and Source B from new ones.

Where both sources return a different email for the same person, that is worth a look on its own. Our post on what to do when a lead has more than one email address covers picking one.

After you scale: keep the source tag

A source that passes in month one can drift by month four. Vendors refresh data on their own schedule, scrapers break, and enrichment tools change providers.

• Keep a source column on every lead, through the sequencer and into the CRM.
• Report bounce and reply rates per source monthly. A campaign-level bounce rate hides a source that went bad.
• Re-verify before each campaign, not only at purchase. Our guide to how often to re-verify gives a cadence by list type.
• Re-run the pilot if a source's numbers slip past your pass marks two months in a row.

Where SimpleVerifier fits

Stage 1 of the test is a verification run, and the report is your data-quality score for the source. SimpleVerifier returns valid, risky or invalid with a reason for each row. Catch-alls are graded rather than guessed, and rows we could not check come back for retry instead of being counted as valid. Upload the pilot CSV on the verify page; the first 100 checks are free with a Google sign-in. Plans start at $9 a month for 2,500 checks, and Pro is $39 a month with the first 100,000 checks a month at full speed, then slower, never cut off. See pricing.