Back to Free Tools
FREE TOOLS
Word Deduplicator
Keep each admitted word’s first spelling and order, with optional lowercase comparison.
Pasted text, One UTF-8 text file
unique-words.txt, word-deduplication.json
Browser only
Runs locally in your browser. Your files, pasted content and generated output are not uploaded. Reset clears this tool’s working data.
This tool needs JavaScript. If it does not load, refresh the page and try again.
How to use this tool
- Paste the admitted text or choose one UTF-8 source file. A selected file takes precedence.
- Choose the explicit operation and options, then inspect the stated conventions and limits.
- Select Deduplicate words and download unique-words.txt, word-deduplication.json.
Examples
alpha beta alpha produces the word list alpha followed by beta.
Limitations
- At most 200,000 UTF-16 units / 800,000 UTF-8 bytes; 50,000 logical lines and 65,536 units per line; at most 5 MiB complete combined output and 30 seconds. A selected valid UTF-8 file takes precedence over paste. File BOM and original CR/LF spans are preserved for reporting; native textarea input can normalize newlines.
- At most 50,000 admitted word tokens and distinct keys. The result is a word list; original punctuation and layout are not retained. Lowercase comparison is distinct from full Unicode case folding.
Related tools
No items found.