All posts

How to Remove Duplicate Phone Numbers (Normalise First, Then Dedupe)

··4 min read

To remove duplicate phone numbers, convert every number to one format first, then remove duplicates on that column. The format to use is E.164: +14155552671. Skip the first step and duplicates slip through, because (415) 555-2671, 415.555.2671 and +1 415 555 2671 are the same phone line written three ways, and a plain duplicate check sees three different values.

Why duplicate phone numbers hide

Phone numbers arrive in a list from forms, imports, enrichment tools and people typing them in. Each source writes them its own way:

How it was written E.164
(415) 555-2671 +14155552671
415.555.2671 +14155552671
+1 415 555 2671 +14155552671
1-415-555-2671 +14155552671
07911 123456 (UK) +447911123456
+44 (0)7911 123456 +447911123456

Six rows, two numbers. Normalising collapses each group into a single value, and only then can any duplicate tool see them as the same.

Duplicates cost you more than space. A sales team calls the same person twice. An SMS campaign texts one phone twice, doubling the cost and the annoyance. And CRM contacts that should be one record split into several, each with half the history.

Step 1: Normalise to E.164

E.164 gives every phone number exactly one written form: +, country code, national number, no spaces. To get there:

  1. Strip everything that isn't a digit (keep track of whether there was a leading +).
  2. Work out the country. Numbers with + or 00 already carry it. For numbers without one, you need to know which country the list is from.
  3. Drop the national trunk prefix: the leading 0 in the UK, Germany, Australia and most other countries, or the leading 1 in North America.
  4. Add + and the country code.

The tricky part is the (0) that people write in international numbers, like +44 (0)7911 123456. That zero is never dialled from abroad, so +44 07911… is wrong. A numbering-plan library such as Google's libphonenumber handles these cases for every country. We go through the format in detail in E.164 phone number format.

Step 2: Remove the duplicates

In Excel

With E.164 numbers in column C, mark duplicates with:

=COUNTIF($C$2:$C$10000,C2)>1

To delete extra copies, select the data, choose Data → Remove Duplicates, and tick only the E.164 column. Excel keeps the first row of each group. Our Excel phone validation guide has the formulas for step 1, and formatting phone numbers in Excel covers E.164 conversion.

In Google Sheets

Flag duplicates with the same COUNTIF, or get a de-duplicated list in one go:

=UNIQUE(C2:C)

See validating phone numbers in Google Sheets for the normalising formulas.

In code

In Python, the phonenumbers package (a port of libphonenumber) does step 1, and a set does step 2:

import phonenumbers

def to_e164(raw, region="US"):
    try:
        n = phonenumbers.parse(raw, region)
    except phonenumbers.NumberParseException:
        return None
    if not phonenumbers.is_valid_number(n):
        return None
    return phonenumbers.format_number(n, phonenumbers.PhoneNumberFormat.E164)

seen, unique_rows = set(), []
for row in rows:
    key = to_e164(row["phone"])
    if key and key not in seen:
        seen.add(key)
        unique_rows.append(row)

There's more in validating phone numbers in Python and in JavaScript.

Step 3: Decide what a duplicate means

Not every shared number is a mistake. Before deleting, ask what the duplicate is:

Situation What to do
Same person, same number, two rows Merge into one record, keep the most complete data
Different people at one company, same switchboard number Keep both; the number is shared, the people aren't
Family members sharing a landline Keep both, or keep one for calling and both for email
Same number on a lead list and your do-not-contact list Remove from the lead list

A sensible default for outreach lists: keep one row per number for calling and texting, but don't throw away the other contacts' emails.

Check validity while you dedupe

Normalising exposes broken numbers too. A number that can't be converted to E.164, because it's too short, too long or in a range that doesn't exist, can't be dialled either. Removing those at the same time saves a second pass. For more on what "valid" does and doesn't mean, see how to check if a phone number is valid.

Do it in one upload

SimpleVerifier normalises and checks every number in a single pass. Upload a CSV on the verifier, and each row gets:

  • phone_e164: the number in E.164, ready to de-duplicate on;
  • phone_status: valid, invalid or empty;
  • phone_country and phone_type: country and numbering-plan line type.

Numbers without a country code are read as US numbers, so add country codes to international lists first. Then open the file and remove duplicates on phone_e164. Any email column in the same file gets a live mailbox check in the same upload.

Format checks are unlimited on the $29.99 Pro plan. The Email + Phone plan ($49.99/month) adds live carrier lookups, with the carrier name and real line type (mobile, landline or VoIP) for 5,000 numbers a month, and failed lookups don't count. For one number, try the free phone number validator.

Common questions

Why doesn't Remove Duplicates find all my duplicate phone numbers?

Because the same number is written in different ways. (415) 555-2671, 415.555.2671 and +1 415 555 2671 are one phone line but three different strings, so a plain duplicate check sees three numbers. Convert every number to E.164 first, then remove duplicates on that column.

What's the best format for de-duplicating phone numbers?

E.164: a plus sign, the country code and the national number with no spaces or punctuation, such as +14155552671. Every phone number has exactly one E.164 form, so two rows match only if they're the same number.

Should I remove rows with the same phone number?

Not always. Several contacts at one company can share a switchboard number, and family members can share a landline. Flag duplicates first and decide per case: merge true duplicates of one person, keep different people who share a number.

Can I de-duplicate phone numbers from different countries?

Yes, as long as every number carries its country code. The same national digits can be a London number in one row and a different number in another country in the next, and only E.164 keeps them apart. Numbers without a country code have to be assigned one before you compare them.

Verify unlimited addresses for $29.99/month

Real SMTP mailbox checks. No credits, no per-email fees.

Get Started