Say it once, where it matters.

What it does

This tool finds words that appear more than once and removes the extra copies according to the scope you choose. It works on whole whitespace-separated tokens, so “cat” and “cats” are different words, while “Word,” and “word” can be treated as the same word when punctuation and case are ignored.

It is not a prose editor. It cannot tell that “the report” and “the document” refer to the same thing, and it cannot tell that a repeated word is intentional. Its job is narrower: remove exact repeated tokens under a clear matching rule, and show you what was removed.

Line scope vs text scope

Line scope is the default and the safer choice for ordinary writing. It only collapses repetitions within the same line. A recurring word at the start of three different paragraphs stays in all three, because those are separate lines. This is useful for fixing accidental repeats such as “the the” or “word word” without flattening a document into a unique-word list.

Text scope treats the entire input as one document. Only the first occurrence of a term is kept, and every later occurrence is removed. On a keyword or tag list, that is exactly the desired deduplication. On a paragraph, it will remove articles, names, and important repeated terms, often producing text that no longer reads as English. Use it deliberately, not as a default.

Case and punctuation

By default, matching ignores case and punctuation at the edges of a token. “Word,” “word”, and “WORD!” count as the same term. The copy that survives keeps its original punctuation and capitalization; the tool does not normalize the word it keeps. If you need “API” and “api” to remain distinct, turn off “Ignore case”. If a comma is semantically important, turn off punctuation ignoring.

“Keep the last copy” changes which occurrence survives. The earlier copies are removed and the final one stays, along with its surrounding whitespace. That is useful when a list has been appended to over time and the newest entry is the authoritative one. It is not a merge: the tool does not combine the two entries or prefer the longer one.

Examples

The quick brown fox jumps over the lazy dog.
The quick brown fox runs into the forest.
A final line with with a repeated word.

The quick brown fox jumps over the lazy dog.
The quick brown fox runs into the forest.
A final line with a repeated word.

Only the repeated “with with” disappears. “The quick brown fox” appears on two lines and survives on both, because line scope does not compare across line boundaries. The repeated word list in the tool shows the term that triggered a removal.

Release notes are ready. The new release page is live. Launch checks passed.

Release notes are ready. The new page is live. Launch checks passed.

This is what text scope does to prose: “release” survives once, and the later mentions vanish. The result is shorter but not necessarily better. Text scope is included because keyword lists and controlled vocabularies need it, not because a paragraph usually does.

Limits worth knowing

• Exact tokens only. “Report” and “reports” are different words. The tool does not stem, lemmatize, or compare meanings. “Car” and “automobile” are unrelated to it.
• Whitespace defines a word. A hyphenated compound such as “state-of-the-art” is one token, so its internal “the” is not removed. That is usually the correct behavior.
• Punctuation handling is edge-only. A token such as “can't” keeps its apostrophe. A quote mark attached to a word is ignored for matching but not removed from the surviving copy.
• Deliberate repetition is not protected. “Very very”, “no no”, and a repeated name can all be reduced. Use line scope and review the output, or edit the input before running the tool.
• Spacing can look unusual after removal. The tool removes one adjoining whitespace run with each removed word, but a line with several consecutive spaces can still show the original gaps. Clean Text can normalize them afterward if needed.

Page updated 27 October 2026