How to add complex syntax highlighting in a minor mode?

Question

I'd like to highlight code with various faces in a minor mode.

Here's a screenshot that's close to what I want:

One thing that I'm missing is having the comment chars # in font-lock-comment-face. The idea is to have comments that "belong" to an outline highlighted as plain text, so it's easier to read. While having regular comments with their usual less prominent face.

Here's the code that I used:

(setq-local font-lock-defaults
            '(python-font-lock-keywords
              nil nil nil nil
              (font-lock-syntactic-face-function
               . lpy-font-lock-syntactic-face-function)))

(defun lpy-font-lock-syntactic-face-function (state)
  "Return syntactic face given STATE.
Returns 'defalt face for comments that belong to an outline."
  (cond ((nth 3 state)
         (if (python-info-docstring-p state)
             font-lock-doc-face
           font-lock-string-face))
        ((save-excursion
           (while (and (> (point) (point-min))
                       (progn (move-beginning-of-line 0)
                              (eq (char-after) ?\#))))
           (forward-line 1)
           (looking-at "#\\*+ "))
         'default)
        (t
         font-lock-comment-face)))

The thing is, I have no clue about the interface on which font-lock-syntactic-face-function operates, other than it receives a complex data structure state, has different point state, and returns a face.

Could someone explain this interface? Is there a better one perhaps?

Answer 1

font-lock-syntactic-face-function is a regular variable from Font Lock, more specifically from the Syntactic Font Lock phase (emphasis mine):

If this variable is non-nil, it should be a function to determine which face to use for a given syntactic element (a string or a comment). The value is normally set through an other-vars element in font-lock-defaults.

The function is called with one argument, the parse state at point returned by parse-partial-sexp, and should return a face. The default value returns font-lock-comment-face for comments and font-lock-string-face for strings (see Faces for Font Lock).

parse-partial-sexp in turn returns a list which describes Emacs' current syntactic state, which is essentially the result of the application of the syntax table to the current buffer. The list is rather complex, hence I'll spare it here; you can see the complete reference in the docstring of parse-partial-sexp. The purpose of this function is to change the face applied to a syntactic element under certain rules. The beginning of your function demonstrates this: If the current string is a docstring, use a different face for it.

However, the face always applies to the whole syntactic element, i.e. the whole string or comment. You can't highlight individual parts with this function, and you should only look at the given state for this purpose—like (python-info-docstring-p state) does in your code. Do not use point at this place; I'm not even sure whether the value of point is properly defined at this stage of font-locking.

---

Putting the pieces together, you're using the wrong function for your purpose, which is why you can't get it to work.

I haven't tried to implement your desired highlighting, but I think you dug way, way too deep for your purpose. If I understand things aright, you'd just like highlight outlines in a comment.

If I'm right, then you just need font-lock-keywords in a special way, namely:

(my/find-outline-in-comment-p 0 'outline-face t)

where outline-face is the face you'd like to apply to the headline, t means to override any prior font-locking at this place, and my/find-outline-in-comment is a matcher function (see the docstring of font-lock-defaults) which takes a position and searches for the first outline in a comment between (point) and that position, returning the extents of the outline to be highlighted in the match data.

To find the outline, you'd scan forward for comments (using font-lock-comment-face or the syntactic state) and then uses looking-at to check whether the comment has an outline.

Answer 2

Consider to define font-lock-syntactic-face-function like that:

(setq font-lock-syntactic-face-function
      (lambda (state)
    (cond ((nth 3 state)
           font-lock-string-face)
          ((and (nth 4 state)(nth 8 state))
            MY-COMMENT-FACE
          (t  font-lock-comment-face))))

This tested with python-mode.el, leaving a section starting with "#*" without comment-face:

(setq py--font-lock-syntactic-face-function
      (lambda (state)
    (cond ((nth 3 state)
           font-lock-string-face)
          ((and (nth 4 state)(nth 8 state)
            (progn (save-excursion
                 (goto-char (nth 8 state))
                 (looking-at (concat comment-start (regexp-quote "*"))))))
           nil)
          (t font-lock-comment-face))))

While it's administered with the mode:

(font-lock-syntactic-face-function
                    . py--font-lock-syntactic-face-function)

Instead of nil, any valid face should work.