Find and remove consecutive duplicated words while ignoring case

Question

I am trying to find and remove duplicated words in a LaTeX file.

Right now I am using query-replace-regexp with the regular expression \b\(\w+\)[ #]+\1 where # stands for line-break (C-q C-j). This does not ignore case, however, and misses the following repetition of the word 'the':

I am the new organ player. The the new keymaster.

Are regular expressions sufficient for this or it's better to program a lisp function to go word by word, checking whether consecutive words are the same?

Answer 1

You can query-replace-regexp as follows:

\(\b\w+\b\)\W+\1\b → \1

This means, match a whole word (\b\w+\b), followed by non word characters (\W+), followed by the first word (\1) and a word ending (\b). Replace with the first word only.

Case sensitivity is handled by options case-fold-search and case-replace.