I'm editing a text in org-mode, several lines are repeated since they where part of different files from the same topic.

I want to use delete-duplicate-lines to delete the repeated lines, however, the command removes the blank lines too, which is something I don't want (otherwise I wouldn't have paragraphs!). I want to ignore certain words that exists alone in lines, for instance, "Resume", it is repeated a lot of times in the text, but I want to keep it since is required.


Ignoring blank lines

You can tell delete-duplicate-lines to ignore blank lines by calling it via

C-u C-u C-u M-x delete-duplicate-lines RET

If you don't want to have to hit C-u that many times when you call delete-duplicate-lines, you can wrap it in a custom command and bind that command to a key sequence of your choice:

(defun delete-duplicate-lines-keep-blanks ()
  (delete-duplicate-lines (region-beginning) (region-end) nil nil t))

(global-set-key (kbd "C-c d") 'delete-duplicate-lines-keep-blanks)

Ignoring lines matching regexp

As for the second part of your question, I don't think you can achieve what you want using the built-in version of delete-duplicate-lines. You can, however, use a modified version of the command (which also keeps blank lines by default):

(defun delete-duplicate-lines
    (beg end keep &optional reverse adjacent keep-blanks interactive)
     (list (region-beginning) (region-end)
           (read-string "Keep lines matching regexp: ") ; Prompt for regexp to keep
           (equal current-prefix-arg '(4))
           (equal current-prefix-arg '(16))
           t                                            ; Keep blanks by default
  (let ((lines (unless adjacent (make-hash-table :test 'equal)))
        line prev-line
        (count 0)
        (beg (copy-marker beg))
        (end (copy-marker end)))
      (goto-char (if reverse end beg))
      (if (and reverse (bolp)) (forward-char -1))
      (while (if reverse
             (and (> (point) beg) (not (bobp)))
               (and (< (point) end) (not (eobp))))
        (setq line (buffer-substring-no-properties
                (line-beginning-position) (line-end-position)))
        (if (or (and keep-blanks (string= "" line))
                (string-match keep line))               ; Ignore line if it
                                                        ; matches regexp to keep
            (forward-line 1)
          (if (if adjacent (equal line prev-line) (gethash line lines))
                (delete-region (progn (forward-line 0) (point))
                               (progn (forward-line 1) (point)))
                (if reverse (forward-line -1))
                (setq count (1+ count)))
            (if adjacent (setq prev-line line) (puthash line t lines))
            (forward-line (if reverse -1 1))))))
    (set-marker beg nil)
    (set-marker end nil)
    (when interactive
      (message "Deleted %d %sduplicate line%s%s"
               (if adjacent "adjacent " "")
               (if (= count 1) "" "s")
               (if reverse " backward" "")))

This version of delete-duplicate-lines will prompt you for a regexp and keep all lines that match the regexp. For instance, to keep all lines consisting of the word Resume you would do:

M-x delete-duplicate-lines RET ^Resume$ RET

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.