Duplicate Word Remover

In short Removing duplicate words comes in two modes that do quite different things. Consecutive-only removes a word that immediately repeats itself, which is the classic typing error — "the the" and "and and" survive proofreading remarkably well because the eye reads over them. That mode is safe on prose, because a word genuinely repeated next to itself is almost always a mistake. Removing every repeat across the whole document is a different operation entirely and destroys ordinary text, since normal English repeats its common words constantly. Its use is on lists: a line of comma-separated tags or keywords where each should appear once. The one place the consecutive rule is wrong is deliberate repetition for emphasis — "very very good", or a name like "New York, New York" — so read the changes rather than accepting them blindly. Case handling is a choice too: "The the" is a duplicate to a case-insensitive comparison and two different words to a strict one, and for the typing-error case you want the insensitive version.