Remove Duplicates in SQL

The Excel version

Data → Remove Duplicates, with the Email column ticked

The SQL equivalent

This example assumes: customers(id bigint, email text, name text, created_at timestamptz). It is written in PostgreSQL syntax, and your table and column names will differ from these.

SELECT id, email, name, created_at
FROM (
  SELECT id,
         email,
         name,
         created_at,
         ROW_NUMBER() OVER (
           PARTITION BY LOWER(TRIM(email))
           ORDER BY created_at, id) AS rn
  FROM customers
  WHERE email IS NOT NULL
) t
WHERE rn = 1
ORDER BY id;

How it works

• PARTITION BY LOWER(TRIM(email)) defines what counts as a duplicate, like the columns you tick in Excel's dialog.
• ORDER BY created_at, id decides which row is first, so the survivor is the earliest record.
• WHERE rn = 1 keeps only that row; this query selects the cleaned list and does not delete anything.

Where the results differ from Excel

• Excel's Remove Duplicates permanently deletes rows from the sheet; this SQL only selects the rows to keep and leaves the table untouched.
• Excel compares text case-insensitively; the LOWER and TRIM make SQL behave the same way.
• Excel always keeps the first row in sheet order; in SQL you must state the order, or the survivor is arbitrary.

Skip the query: ask in plain English

If you would rather not write SQL, type the question. SchemaChat turns it into SQL against your connected database or uploaded file, runs it, and shows the SQL and the figures it used so you can check the answer before you share it. Check that the result matches how your spreadsheet defines the number.

• Show me one row per email address, keeping the earliest customer record.
• How many duplicate customer records do we have?