Fast/efficient way to keep a running word count

Question

I am trying to implement a minor mode that displays in the modeline the current word count and the number of new words since the file was last opened.

There are some existing implementations with this type of functionality (e.g., nanowrimo.el). But as far as I can tell, all of the implementations I've found so far seem to re-count the entire buffer after each change (i.e. after each character insertion or deletion, kill, yank, undo, etc.) This makes emacs a bit slow and unresponsive on on large files. This would seem to be a very common problem, but I haven't found a solution.

I am trying to keep track of the current word count in a variable and use before-change-functions and after-change-functions to update that count after every insertion, deletion, kill, yank, undo, etc. This involves counting only the changed character(s) and the immediately surrounding characters. (For example, if a space is inserted between two non-word characters, the word count increases by 1.) This approach is much faster/more efficient - but I haven't quite figured out all the details to make it work.

Is there existing code that already does this, either using change hooks or any other method? Or, any thoughts about a good approach to do this?

Answer 1

I went with an approach that uses before-change-functions and after-change-functions to update the word count after each buffer modification. A simplified version of the code is:

;; Function that counts words in buffer
(defun wc-buffer ()
...
)

;; Function that counts words in a region
(defun wc-region (rbeg rend)
...
)

(defvar-local a1 nil)
(defvar-local a2 nil)
(defvar-local curr-wc nil)

(defun init-function ()
  (interactive)
  (save-excursion
    (setq curr-wc (wc-buffer))
    )
  )

(defun wc-update-before (change-beg change-end)
  (setq pos1 (max 1 (1- change-beg)))
  (setq pos2 (min (point-max) (1+ change-end)))

  (setq a1 (wc-region pos1 pos2))
)

(defun wc-update-after (change-beg change-end prev-len)
  (if (bound-and-true-p a1) 
      (progn
    (setq pos1 (max 1 (1- change-beg)))
    (setq pos2 (min (point-max) (1+ change-end)))

    (setq a2 (wc-region pos1 pos2))

    (setq curr-wc (+ curr-wc (- a2 a1)))
    )
    nil)
  )

(init-function)
(add-hook 'before-change-functions 'wc-update-before nil t) 
(add-hook 'after-change-functions 'wc-update-after nil t)
(setq inhibit-modification-hooks nil)

(Code for the wc-buffer and wc-region functions needs to be added to use this.) The variable curr-wc holds the current word count. This runs efficiently and causes no noticeable slowdown even on large files (tested with 100,000 words). It seems to work correctly for all modification types I've tried (inserts, deletes, yanks, kills, and undos). The main concern with the approach is that, according to the documentation, the before-change-functions and after-change-functions are not necessarily called in pairs. But in my testing so far this doesn't seem to have affected the accuracy of the count.