How to get emacs count text words only without counting words in comments, code blocks, and markups in markdown?

Question

Publishers set a word count limit you have to meet in writing a manuscirpt. I am trying to write up the manuscript using markdown in emacs. Consider the example below.

MWE

# Results {-}

## Topic 1 {-}

This is just a random text with a citation in markdown \@ref(fig:pca-scree)).
Below is a code block.

```
{r pca-scree, echo = FALSE, fig.align = "left", out.width = "80%", fig.cap = "Scree plot with parallel analysis using simulated data of 100 iterations (red line) suggests retaining only the first 2 components. Observed dimensions with their eigenvalues are shown in green."}
    
   knitr::include_graphics("./plots/PCA_scree_parallel_analysis.png")
```

## Topic 2 {-}

<!-- todo: a comment that needs to be avoided by word count hopefully-->

So when I hit M-x count-words it shows 88 words but when I select the text region only and hit M-= I get 20 words including the undesired \@ref(fig:pca-scree)) which by itself was considered 4 words!

How to do word counting right and text only the emacs way?

Edebug Marko

pandoc-count-buffer
 [2 times]
Result: 1 (#o1, #x1, ?\C-a)
 [2 times]
Result: 2037 (#o3765, #x7f5)

/bin/bash: line 1: marko: command not found
Result: 127 (#o177, #x7f, ?\C-?)
 [3 times]
Result: "*Shell Command Output*"
 [3 times]
Result: 1 (#o1, #x1, ?\C-a)
 [2 times]
Result: 45 (#o55, #x2d, ?-)

Result: (html nil (body nil (p nil "/bin/bash: line 1: marko: command not found\n")))

Result: (html nil (body nil (p nil "/bin/bash: line 1: marko: command not found\n")))
 [4 times]
Result: +
 [3 times]
Result: (html nil (body nil (p nil "/bin/bash: line 1: marko: command not found\n")))

Result: ((p nil "/bin/bash: line 1: marko: command not found\n"))
 [4 times]
Result: (p nil "/bin/bash: line 1: marko: command not found\n")

Result: "/bin/bash: line 1: marko: command not found\n"

Result: ("/bin/bash:" "line" "1:" "marko:" "command" "not" "found")

Result: 7 (#o7, #x7, ?\C-g)

Result: (7)

Update

This post has a follow up question here.

Answer 1

EDIT another alternative

A third easy alternative is to use some external markdown parser like e.g. marko (install using pip install marko).

Then (for Marko) use the following command from your .md buffer to count only text words (or only words parsed into html 'p' elements, to be more specific)

(defun marko-count-words ()
  (interactive)
  (shell-command-on-region (point-min) (point-max)
                           "marko")
  (let ((dom (with-current-buffer shell-command-buffer-name
               (libxml-parse-html-region (point-min) (point-max)))))
    (print (apply #'+
                  (apply #'append
                         (mapcar (lambda (n)
                                   (mapcar (lambda (s) (if (stringp s)
                                                           (length (split-string s))
                                                         0))
                                           (cddr n)))
                                 (dom-by-tag dom 'p)))))))

At least it works on your MWE where it counts 17 words.

END EDIT

There are several/many solutions possible, but I will post the two most straightforward solutions (for me) here.

Your first option is to use pandoc, with the the wordcount filter (see installation instructions). Then you could use the following command to count words in your file

(defun pandoc-count-words ()
  (interactive)
  (shell-command-on-region (point-min) (point-max)
                           "pandoc --lua-filter wordcount.lua"))

However, on a quick test, the filter does not seem to count only 'text' (but maybe you can read more about the filter its usage).

My other suggestion, would be to write your file in org-mode and use the org-wc package for counting. Then when finished simply export the file to markdown. However, also the counting total behavior of org-wc seems a little strange to me, but that can easily be fixed with the following bit of lisp (also you could check the customization options using M-x customize-group org-wc, but I can not find anything that looks relevant):

(defun org-wc-total ()
  (interactive)
  (print  (apply #'+ (mapcar (lambda (o)
                               (let ((str (overlay-get o 'display)))
                                 (if str
                                     (plist-get (text-properties-at 0 str) :org-wc)
                                   0)))
                             (overlays-in (point-min) (point-max))))))

Execute this command after doing M-x org-wc-display in some org buffer.