13

I want to perform a variant of anchored font lock matching. I have function definitions that start with a list of names, and I want those names to be highlighted inside the function body.

I have created a function that does this and registered it as a jit-lock function with jit-lock-register, however, performance is pretty poor and scrolling lags in larger files.

  • How can I measure performance? If I just time calling my function on a large file (with float-time before and after or with elp) I get wildly varying performance, it takes anything from 0.65 to 12 seconds. Is there a recommended way to benchmark font lock performance?
  • Is there any difference in performance between an anchored matcher defined in font-lock-keywords and adding a function via jit-lock-register?

Edit: It seems that the variability in performance is related to garbage collection, invocations of my jit-lock function get successively slower with each invocation until garbage collection is run, at which point they get fast again.

  • For the first item, try the profiler. – Malabarba Dec 14 '14 at 2:14
  • I can (and have) used the profiler too see what parts of my code that takes time, but since performance is so inconsistent it is hard to tell whether any changes I make are an improvement or not. – Joakim Hårsman Dec 15 '14 at 5:20
  • Do you have some code we can test? That might help us a lot. – PythonNut Feb 25 '15 at 5:33
  • 1
    Although not about profiling or micro-optimizations, per se: I've found the font-lock-studio package to be another helpful tool for understanding font-lock performance. It can help the same way as any other interactive stepping debugger can help -- you might discover the execution paths are not what you expect, and that's the main performance issue. – Greg Hendershott Oct 16 '15 at 15:41
  • Thanks for the tip about font-lock-studio, it's awesome! Doesn't help with jit-lock-functions though, but sure does with everything else. – Joakim Hårsman Oct 16 '15 at 19:28
8

It turns out that the wildly varying performance was related to garbage collection. Each call to the function would get slower until a garbage collection was run. With stock emacs, gc was run every couple of seconds, but I had a line in my init.el to improve startup time that set gc-cons-threshold to 20 MB, and that meant gc was run much more infrequently, causing benchmarks to report slower and slower timing until a gc was run after a couple of minutes, then times would plummet and be fast again.

After reverting to the default gc-cons-threshhold, benchmarking became easier.

I then profiled for memory with the built in profiler (M-x profiler-start), and discovered that calls to syntax-ppss caused the most allocations, so after some optimization to call syntax-ppss less often I achieved acceptable performance.

Using jit-lock-mode (adding a function via jit-lock-register) seems to be the easiest way to get multi line font locking to work reliably, so that was the method I chose.

Edit: After discovering that performance was still not good enough in very large buffers I spent a lot of time optimizing cpu use and allocation, measuring the performance improvements with the built in Emacs profiler (M-x profiler-start). However, Emacs would still stutter and hang when scrolling quickly through very large buffers. Removing the jit-lock function I registered with jit-lock-register would remove the stuttering and hangs, but profiling showed the jit-lock function to complete in around 8 ms which should be fast enough for smooth scrolling. Removing the call to jit-lock-registerand instead using a regular font-lock-keywords matcher solved the issue.

TLDR: Doing this was slow and would stutter:

(defun my-font-lock-function (start end)
"Set faces for font-lock between START and END.")

(jit-lock-register 'my-font-lock-function)

Doing this was fast and would not stutter:

(defun my-font-lock-function (start end)
"Set faces for font-lock between START and END.")

(defun my-font-lock-matcher (limit)
    (my-font-lock-function (point) limit)
   nil)

(setq font-lock-defaults
  (list 
     ...
    ;; Note that the face specified here doesn't matter since
    ;; my-font-lock-matcher always returns nil and sets the face on
    ;; its own.
    `(my-font-lock-matcher (1 font-lock-keyword-face nil))))
  • Could you share the code you used? Your solution might help others looking to achieve the same thing. – Manuel Uberti Oct 16 '15 at 7:24
  • I didn't really use any specific code, I just called syntax-ppss less. You can check out the code in question here: bitbucket.org/harsman/dyalog-mode/src/… Look for dyalog-fontify-locals. – Joakim Hårsman Oct 16 '15 at 19:18
  • I guess dyalog-fontify-locals-matcher should be my-font-lock-matcher and one of the end should be limit. Anyway, really interesting discovery! – Lindydancer Dec 31 '16 at 11:49
  • @Lindydancer: Yes thank you. Fixed. – Joakim Hårsman Dec 31 '16 at 22:26
  • 1
    Re: gc-cons-threshold, if you're messing with internal values purely to improve start-up time, I suggest you use emacs-startup-hook to restore them afterwards. – phils Jan 15 '17 at 2:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.