All posts

How to Clean and Validate a List of Domains (URLs, Emails, Duplicates, Punycode)

··3 min read

To clean a list of domains, strip each entry down to its domain, lowercase it, convert accents to Punycode and remove duplicates. The domain list cleaner does that for a mixed paste of domains, URLs and email addresses, and marks each entry Already clean, Cleaned, Duplicate or Invalid.

The by-hand way

  1. Delete https://, the path, the query string and the port.
  2. For an email address, keep only what follows the @.
  3. Lowercase everything and remove a trailing dot.
  4. Sort and remove duplicates.
  5. Check each name for illegal characters.

In a spreadsheet this takes several formulas per column, and www.Acme.com and acme.com are still two rows.

Step 1: paste the list

Paste domains, URLs or email addresses, separated by new lines, commas or spaces. To pull domains from a column of website URLs in a file, the extract domain from URL tool is built for that; this one is for a mixed paste.

Step 2: choose two options

Choose whether to remove www., and whether to cut subdomains. Cutting turns blog.acme.com into acme.com so a list of hosts becomes a list of companies. It uses a short list of two-part suffixes such as co.uk and com.au, so a name under a less common one can be cut in the wrong place. Leave it off when the subdomain matters, as with mail or tracking hosts.

Step 3: read the result

Mark Meaning
Already clean Nothing changed
Cleaned Something was removed or converted; the note says what
Duplicate Same as an earlier entry after cleaning
Invalid Dropped, with the reason

Accented and other-script names are written in Punycode, the xn-- form DNS and registrars use. Duplicates are found after cleaning, so https://www.Acme.com/pricing and acme.com are one entry.

What makes a name invalid

The tool applies the syntax rules of RFC 1035 and RFC 1123: letters, digits and hyphens; each part 1 to 63 characters; 253 in all; no part starting or ending with a hyphen; an extension that is not all digits. It also flags underscores.

It checks the shape of the name, not the name. A tidy name can still be unregistered; use the bulk domain availability checker to see which are free.

Step 4: copy or download

Copy the clean names, or download the CSV with the input, result and notes. See also how to check many domain names at once.

What this tool cannot do

  • It checks syntax only and does not look names up.
  • Subdomains are cut with a short suffix list, not the full Public Suffix List.
  • A registry can still refuse a name that converts to Punycode.

Setting up a sending domain? Verify the list first

Setting up a sending domain? Check that the names are free with the bulk domain availability checker, then verify the list you will send to first. A new domain has no sending reputation, and early bounces from dead addresses count against it. The first 100 email checks on the verify page are free with a Google sign-in, no card. Each address comes back valid, risky or invalid with a reason, and nothing is sent to anyone on the list. For larger lists, see bulk email verification and pricing.

Clean your list now

Open the domain list cleaner.

Common questions

How do I clean a list of domains?

Strip the https://, path, query string and port, keep the part after an @ in email addresses, lowercase, and remove duplicates. A domain list cleaner does all of it and explains each change.

What makes a domain name invalid?

Characters other than letters, digits and hyphens, a part over 63 characters, a name over 253, a part starting or ending with a hyphen, an all-digit extension, or an underscore.

Does it check that the domain exists?

No. It checks syntax only. A tidy name can still be unregistered. Use an availability checker to see which are free.

What does cutting subdomains do?

It reduces blog.acme.com to acme.com, using a short list of two-part suffixes such as co.uk, not the full Public Suffix List.

Does my list leave my browser?

No. It runs in your browser.

Check your list before you send

Real mailbox checks over SMTP, and nothing is sent to the address. Sign in with Google and your first 100 email checks are free, no card. Plans for bigger lists are on the pricing page.