1

See https://lists.gnu.org/archive/html/emacs-devel/2015-12/msg01046.html for more background.

YAML defines a comment as a # either preceded by the beginning of the line or whitespace (more specifically, TAB or SPC). This means that foo#bar is a normal token, not foo with a #bar comment. You can get most of the way there with the traditional syntax rules:

(modify-syntax-entry ?# "<")
(modify-syntax-entry ?\n ">")

Unfortunately this will highlight foo#bar incorrectly. I've been pointed to syntax-propertize-function which allows for more powerful syntax rules, but behaves weirdly. Depending on the specific code I write, either insertion of new characters or deletion invalidates the existing fontification.

Example 1 (marking # preceded by non-whitespace as _):

(modify-syntax-entry ?# "<")
(modify-syntax-entry ?\n ">")

;; invalidated by deletion
(syntax-propertize-rules
  ("[^ \t]#" (0 "_")))

Example 2 (marking # preceded by whitespace as <):

(modify-syntax-entry ?\n ">")

;; invalidated by insertion
(syntax-propertize-rules
  ("^#" (0 "<")))
  ("[ \t]#" (0 "<")))

If I rewrite either of these in terms of a function scanning for appliable parts and edebug it, edebug does not take me through the function for the invalidating case. Any idea why?

Bonus - Syntax propertize functions:

Example 1:

(defun yaml-mode-syntax-propertize-function (beg end)
  (save-excursion
    (goto-char beg)
    (while (search-forward "#" end t)
      (save-excursion
        (forward-char -2) ;; move to char preceding #
        (when (looking-at "[^ \t]")
          (forward-char 1) ;; move to # again
          (put-text-property (point) (1+ (point))
                             'syntax-table (string-to-syntax "_"))))))

Example 2:

(defun yaml-mode-syntax-propertize-function (beg end)
  (save-excursion
    (goto-char beg)
    (while (search-forward "#" end t)
      (save-excursion
        (forward-char -1)
        (if (bolp)
            (put-text-property (point) (1+ (point))
                               'syntax-table (string-to-syntax "<"))
          (forward-char -1)
          (when (looking-at "[ \t]")
            (forward-char 1)
            (put-text-property (point) (1+ (point))
                               'syntax-table (string-to-syntax "<"))))))))

Test file:

keyword: token#token
#comment comment
token #comment
1

yaml-mode adds a function to the obsolete variable font-lock-syntactic-keywords. It appears as though this interferes with syntactic propertize functions.

My suggestion is to combine the two into one and only use the modern syntax-propertize-function method.

Btw. your first example works (when font-lock-syntactic-keywords isn't set), but you need to check if the # is at the beginning of a line:

(defun yaml-mode-syntax-propertize-function (beg end)
  (save-excursion
    (goto-char beg)
    (while (search-forward "#" end t)
      (save-excursion
        (forward-char -1)
        (unless (bolp)
          (forward-char -1)           ;; move to char preceding #
          (when (looking-at "[^ \t]")
            (forward-char 1) ;; move to # again
            (put-text-property (point) (1+ (point))
                               'syntax-table (string-to-syntax "_"))))))))

Also, it can be made more efficient if you use something like (memq (preceding-char) '(?\s ?\t)), as in:

(defun yaml-mode-syntax-propertize-function (beg end)
  (save-excursion
    (goto-char beg)
    (while (search-forward "#" end t)
      (save-excursion
        (forward-char -1)
        (unless (bolp)
          (unless (memq (preceding-char) '(?\s ?\t))
            (put-text-property (point) (1+ (point))
                               'syntax-table (string-to-syntax "_"))))))))
  • Thank you! I wouldn't have expected that to interfere, but once I removed the change to that variable, no more invalidation happens. – wasamasa Feb 17 '16 at 11:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.