How to use tree-sitter parser for syntax highlighting?

Question

I'm trying to setup Emacs for editing Roc (https://www.roc-lang.org/) source code. I have an LSP server for Roc and tree-sitter parser for Roc installed.

Unfortunately I failed to find tree-sitter documentation for absolute beginners. Initially my configuration was copy-pasted from this article and other examples all over the Internet:

;;; Roc

(define-derived-mode roc-mode fundamental-mode "Roc")
(add-to-list 'auto-mode-alist '("\\.roc\\'" . roc-mode))
;; LSP
(require 'lsp)
(with-eval-after-load 'lsp-mode
  (add-to-list 'lsp-language-id-configuration
    '(roc-mode . "roc"))

  (lsp-register-client
    (make-lsp-client :new-connection (lsp-stdio-connection "roc_ls")
                     :activation-fn (lsp-activate-on "roc")
                     :server-id 'theme-check)))
;; tree-sitter
(require 'treesit)
(require 'tree-sitter)
(require 'tree-sitter-langs)

(defun roc-mode-setup ()
  (interactive)
  (treesit-parser-create 'roc)
  (global-tree-sitter-mode)
  (tree-sitter-hl-mode)
  (treesit-major-mode-setup))

(tree-sitter-load 'roc "/home/yshirokov/.emacs.d/tree-sitter/libtree-sitter-roc")
(add-to-list 'tree-sitter-major-mode-language-alist '(roc-mode . roc))
(add-hook 'roc-mode-hook (lambda ()
                           (roc-mode-setup)
                           (setq treesit-font-lock-level 4)))

Well, with this configuration I have an LSP connection, I can see the types under the cursor in the modeline and can visualize the syntax tree and navigate through it with treesit-explore-mode. So, both LSP and tree-sitter parser are probably working. But I have no syntax highlighting.

So I added explicit highlighting rules:

;; tree-sitter
(require 'treesit)
(require 'tree-sitter)
(require 'tree-sitter-langs)

(defun roc-mode-setup ()
  (interactive)
  (treesit-parser-create 'roc)
  (setq-local treesit-font-lock-feature-list
              '((comment)
                (keyword)))
  (defvar roc-font-lock-rules
    '(:language roc
      :override t
      :feature comment
      '((comment) @font-lock-comment-face)

      :language roc
      :override t
      :feature keyword
      '((keyword) @font-lock-keyword-face)))

  (global-tree-sitter-mode)
  (tree-sitter-hl-mode)
  (setq-local treesit-font-lock-settings
              (apply #'treesit-font-lock-rules
                     roc-font-lock-rules))
  (treesit-major-mode-setup))

But it changed nothing: I still haven't syntax highlighting.

Is it possible to have syntax highlighting for a new language with existing tree-sitter parser? Maybe, there is some step-by-step guide for absolute beginners in tree-sitter mode I missed?

Answer 1

Answering my own question. TL;DR: yes, explicit highlighting rules are needed; patterns for highlighting rules should match real fragments of syntax tree (in my case Roc syntax tree hasn't comment and keyword nodes); tree-sitter library is not needed in modern Emacs (thank you, @jirassimok!).

Working config for Roc:

;;; Roc

(define-derived-mode roc-mode fundamental-mode "Roc")
(add-to-list 'auto-mode-alist '("\\.roc\\'" . roc-mode))
;; LSP
(require 'lsp)
(with-eval-after-load 'lsp-mode
  (add-to-list 'lsp-language-id-configuration
    '(roc-mode . "roc"))

  (lsp-register-client
    (make-lsp-client :new-connection (lsp-stdio-connection "roc_ls")
                     :activation-fn (lsp-activate-on "roc")
                     :server-id 'theme-check)))
;; tree-sitter
(require 'treesit)

(defun roc-mode-setup ()
  (interactive)
  (defvar roc-font-lock-rules
    (treesit-font-lock-rules
     :language 'roc
     :feature 'string
     '((string) @font-lock-string-face)

     :language 'roc
     :feature 'comment
     '((line_comment) @font-lock-comment-face)

     :language 'roc
     :feature 'definition 
     '((value_declaration_left
        (identifier_pattern
         (long_identifier (identifier) @font-lock-constant-face))))

     :language 'roc
     :feature 'function
     '((application_expression
        caller: 
        (long_identifier_or_op
         (long_identifier (identifier) @font-lock-function-name-face))))))

  (setq-local treesit-font-lock-settings
              roc-font-lock-rules)
  (setq-local treesit-font-lock-feature-list
              '((string comment definition function variable)))
  (treesit-major-mode-setup))

(add-hook 'roc-mode-hook (lambda ()
                           (roc-mode-setup)
                           (lsp)
                           (setq treesit-font-lock-level 4)))

Problem was solved after reading this manual, advised by @jirassimok too. It contains all the missing parts of the puzzle:

• how syntax highlighting with tree-sitter works;
• what is a query;
• what is a pattern;
• what is a capture name;
• what is a feature.

I still don't understand how to set the level of the syntax highlighting features, but to be honest I just don't care, it's not an emacs-mode-as-a-software-pet-project.

Source code is already readable enough for me, and anyway I always can add or modify syntax rules, now I know how. Config above contains the rules to highlight string literals, comments, value definitions and function calls: