I would like to traverse the dom tree, extract the text and build a list after visiting each node.
Have you tried using the functions dom-strings (returning a list) or dom-texts (returning a string) from the built-in library dom.el?
I'm confused on implementing the recursive function text-tag to perform the said operation and I need your assistance.
It would be helpful if you described in more detail which text precisely you are interested in extracting.
The current attempt leads to an error of:
Unknown push pattern: (push (dom-text node) text-list)
Maxim Kim's answer already correctly identifies this as arising from invalid use of pcase.
A call to pcase follows the pattern (pcase EXP (PATTERN CODE...)...).
The call to (push (dom-text node) text-list) is in the place where a PATTERN was expected, but there is no built-in pcase pattern that starts with push, hence the error.
Some inline comments:
(setq text-list '())
When defining your own variables and functions, it is best to name them with a unique prefix such as my-, to ensure that they don't clash with other Elisp code. Also, when a variable is expected to be used by others (such as a function you define), it is best to define it with defvar instead of setq, for example:
(defvar my-text-list ()
"List of HTML strings.")
(defun text-tag (x)
(let ((node (dom-node x)))
When text-tag is first called in apply-extraction, it is passed a DOM node (specifically body).
So passing the body to dom-node is both unnecessary and potentially invalid.
(pcase (dom-tag node)
((and (pred stringp) str) str)
This pcase call is inspecting the dom-tag (which is meant to be a symbol) of node, but the pattern is looking for a string.
So the pattern should be matched against node, not (dom-tag node).
('comment '())
What about script?
((push (dom-text node) text-list)
(mapconcat #'text-tag (dom-children node)))
This simultaneously builds a list of strings, and also returns their concatenation.
Are both operations really needed, or can you get away with doing only one or the other?
Putting this all together, we would get something like:
(pcase x
((pred stringp)
(push x text-list)
x)
((app dom-tag (or 'comment 'script))
())
(_
(mapconcat #'text-tag (dom-children x))))
(defun apply-extraction (tag-entries)
(with-temp-buffer
(insert tag-entries)
(let* ((mytags (libxml-parse-html-region (point-min) (point-max))))
(text-tag (car (dom-by-tag mytags 'body))))))
dom-by-tag returns a list of all children nodes whose tag is body, but most of the time all you need is the HTML's first body child.
For that, you can use dom-child-by-tag instead.
(let ((mytree "<html>
<head>
<title>Demo: Gradient Slide</title>
</head>
<link href=\"https://fonts.googleapis.com/css?family=Nunito+Sans\" rel=\"stylesheet\">
<link rel=\"stylesheet\" href=\"dist/build.css\">
<body>
<header data-table=\"testtable\">
<h1 data-field=\"testfield\">Skeleton Screen</h1>
<h1 data-field=\"newfield\">Skeleton Screen</h1>
</header>
<div class=\"wrap\"> Sample Text <p class=\"format\" data-attribute=\"somethingsomething\">Welcome to this page</p>
<div class=\"flex-grid generic-card\">
<!-- ADD .loading CLASS TO ANY ELEMENT TO SHIMMER/ANIMATE -->
<div class=\"card loading\">Text </div>
<!-- ADD .loading CLASS TO ANY ELEMENT TO SHIMMER/ANIMATE
this multiline comment
-->
<div class=\"card loading\"></div>
</div>
</div>
</body>
</html>"))
(apply-extraction mytree))
Note that libxml-parse-html-region preserves in the returned DOM all blanks present in the original HTML. Is this desirable in your use-case?
To wrap up, here is how I would refactor this:
;; -*- lexical-binding: t -*-
(require 'dom)
(eval-when-compile
;; For `string-join' prior to Emacs 29.
(require 'subr-x))
(defun my-dom-strings (node)
"Return a flat list comprising all string nodes under HTML NODE.
Like `dom-strings', but also ignores HTML `comment' nodes."
(cond ((stringp node)
(list node))
((not (memq (dom-tag node) '(comment script)))
(mapcan #'my-dom-strings (dom-children node)))))
(defun my-html-strings (html)
"Collect string nodes under HTML's `body' in a list."
(with-temp-buffer
(insert html)
(let ((dom (libxml-parse-html-region (point-min) (point-max))))
(my-dom-strings (dom-child-by-tag dom 'body)))))
(setq html "\
<html>
<head>
<title>Demo: Gradient Slide</title>
</head>
<link href=\"https://fonts.googleapis.com/css?family=Nunito+Sans\" rel=\"stylesheet\">
<link rel=\"stylesheet\" href=\"dist/build.css\">
<body>
<header data-table=\"testtable\">
<h1 data-field=\"testfield\">Skeleton Screen</h1>
<h1 data-field=\"newfield\">Skeleton Screen</h1>
</header>
<div class=\"wrap\"> Sample Text <p class=\"format\" data-attribute=\"somethingsomething\">Welcome to this page</p>
<div class=\"flex-grid generic-card\">
<!-- ADD .loading CLASS TO ANY ELEMENT TO SHIMMER/ANIMATE -->
<div class=\"card loading\">Text </div>
<!-- ADD .loading CLASS TO ANY ELEMENT TO SHIMMER/ANIMATE
this multiline comment
-->
<div class=\"card loading\"></div>
</div>
</div>
</body>
</html>")
(setq strings (my-html-strings html))
;; => ("..." "Skeleton Screen" ...)
(string-join strings)
;; => "... Skeleton Screen ..."
