Remove duplicate lines

How duplicate removal works

This tool processes text line by line, removing any line that has already appeared earlier in the input. The algorithm splits the input on newline characters, iterates through each line, and uses a Set (or equivalent deduplication structure) to track lines seen so far — only outputting a line the first time it appears.

Options available: Case-sensitive comparison (default) treats "Apple" and "apple" as different lines. Disabling this normalises all lines to lowercase before comparing, so all case variants of the same word are treated as duplicates. Sort output alphabetically orders the deduplicated lines, useful for creating clean sorted lists. Trim whitespace strips leading and trailing spaces from each line before comparison, preventing near-duplicates caused by accidental spaces. The original order of first appearances is preserved when sorting is disabled. This is useful for deduplicating email lists, log entries, word lists, CSV rows, and any line-based data.