Remove Duplicate Lines
Paste a list and strip out every repeated line. Keep the first or the last occurrence, optionally ignore case or spacing, and download a clean file. Your data never leaves the browser.
Paste your list here, one item per line…
Comparison uses a normalised key, so your original lines are never rewritten. Nothing is uploaded — everything runs in your browser.
How to use
- Paste your list into the left box, one entry per line.
- Choose whether to keep the first or the last occurrence of each duplicate.
- Turn on “Ignore case” or “Ignore leading/trailing spaces” if near-identical lines should count as one.
- Copy the result or download it as a .txt file.
Details
Deduplicating looks simple until the details matter. Two lines that look identical on screen can differ by a trailing space, a full-width versus half-width character, or a capital letter. This tool separates the key used for comparison from the text that is output: you decide what should be considered “the same”, and the lines that survive are always your original, unmodified text.
Order is preserved. The output keeps the relative sequence of the surviving lines rather than re-sorting them, so a deduplicated contact list, keyword list or log extract still reads the way it did before. Blank lines can be treated as a single duplicate or removed outright, and line endings can be written as LF for Unix or CRLF for Windows.
FAQ
Does “ignore case” also change the lines I keep?
No. Comparison uses a normalised key, but the line that is kept is always your original text, with its original capitalisation and spacing. Turning the option on only affects which lines are judged to be duplicates — it never rewrites the output.
What does “keep last occurrence” do differently?
With “first occurrence”, the earliest version of a repeated line survives. With “last occurrence”, the latest one survives. Both preserve the relative order of the surviving lines, so the result stays readable as a list.
Can I also delete blank lines or collapse repeated spaces?
Yes. “Delete blank lines” removes empty lines entirely, and “Collapse inner spaces” turns runs of spaces or tabs inside a line into a single space. Both run as part of the same pass, before duplicates are resolved.
Is there a limit?
The tool accepts up to 20 MB of text or one million lines. Beyond roughly twenty thousand lines it hands the work to a background thread so the page stays responsive. Larger inputs should be split first.