15

I want to perform a variant of anchored font lock matching. I have function definitions that start with a list of names, and I want those names to be highlighted inside the function body.

I have created a function that does this and registered it as a jit-lock function with jit-lock-register, however, performance is pretty poor and scrolling lags in larger files.

  • How can I measure performance? If I just time calling my function on a large file (with float-time before and after or with elp) I get wildly varying performance, it takes anything from 0.65 to 12 seconds. Is there a recommended way to benchmark font lock performance?
  • Is there any difference in performance between an anchored matcher defined in font-lock-keywords and adding a function via jit-lock-register?

Edit: It seems that the variability in performance is related to garbage collection, invocations of my jit-lock function get successively slower with each invocation until garbage collection is run, at which point they get fast again.

5
  • For the first item, try the profiler.
    – Malabarba
    Dec 14, 2014 at 2:14
  • I can (and have) used the profiler too see what parts of my code that takes time, but since performance is so inconsistent it is hard to tell whether any changes I make are an improvement or not. Dec 15, 2014 at 5:20
  • Do you have some code we can test? That might help us a lot.
    – PythonNut
    Feb 25, 2015 at 5:33
  • 2
    Although not about profiling or micro-optimizations, per se: I've found the font-lock-studio package to be another helpful tool for understanding font-lock performance. It can help the same way as any other interactive stepping debugger can help -- you might discover the execution paths are not what you expect, and that's the main performance issue. Oct 16, 2015 at 15:41
  • Thanks for the tip about font-lock-studio, it's awesome! Doesn't help with jit-lock-functions though, but sure does with everything else. Oct 16, 2015 at 19:28

1 Answer 1

9

It turns out that the wildly varying performance was related to garbage collection. Each call to the function would get slower until a garbage collection was run. With stock emacs, gc was run every couple of seconds, but I had a line in my init.el to improve startup time that set gc-cons-threshold to 20 MB, and that meant gc was run much more infrequently, causing benchmarks to report slower and slower timing until a gc was run after a couple of minutes, then times would plummet and be fast again.

After reverting to the default gc-cons-threshhold, benchmarking became easier.

I then profiled for memory with the built in profiler (M-x profiler-start), and discovered that calls to syntax-ppss caused the most allocations, so after some optimization to call syntax-ppss less often I achieved acceptable performance.

Using jit-lock-mode (adding a function via jit-lock-register) seems to be the easiest way to get multi line font locking to work reliably, so that was the method I chose.

Edit: After discovering that performance was still not good enough in very large buffers I spent a lot of time optimizing cpu use and allocation, measuring the performance improvements with the built in Emacs profiler (M-x profiler-start). However, Emacs would still stutter and hang when scrolling quickly through very large buffers. Removing the jit-lock function I registered with jit-lock-register would remove the stuttering and hangs, but profiling showed the jit-lock function to complete in around 8 ms which should be fast enough for smooth scrolling. Removing the call to jit-lock-registerand instead using a regular font-lock-keywords matcher solved the issue.

TLDR: Doing this was slow and would stutter:

(defun my-font-lock-function (start end)
"Set faces for font-lock between START and END.")

(jit-lock-register 'my-font-lock-function)

Doing this was fast and would not stutter:

(defun my-font-lock-function (start end)
"Set faces for font-lock between START and END.")

(defun my-font-lock-matcher (limit)
    (my-font-lock-function (point) limit)
   nil)

(setq font-lock-defaults
  (list 
     ...
    ;; Note that the face specified here doesn't matter since
    ;; my-font-lock-matcher always returns nil and sets the face on
    ;; its own.
    `(my-font-lock-matcher (1 font-lock-keyword-face nil))))
7
  • Could you share the code you used? Your solution might help others looking to achieve the same thing. Oct 16, 2015 at 7:24
  • I didn't really use any specific code, I just called syntax-ppss less. You can check out the code in question here: bitbucket.org/harsman/dyalog-mode/src/… Look for dyalog-fontify-locals. Oct 16, 2015 at 19:18
  • I guess dyalog-fontify-locals-matcher should be my-font-lock-matcher and one of the end should be limit. Anyway, really interesting discovery! Dec 31, 2016 at 11:49
  • 1
    Re: gc-cons-threshold, if you're messing with internal values purely to improve start-up time, I suggest you use emacs-startup-hook to restore them afterwards.
    – phils
    Jan 15, 2017 at 2:02
  • 1
    I also noticed that large gc thresholds cause stuttering, but I have no idea why that is happening. My system has plenty of free RAM laying around ...
    – HappyFace
    May 12, 2021 at 18:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.