Line height with unicode characters

Question

Some Unicode characters cause the line they are displayed on to be quite large. For example, a grave accent "̀ " adds about 2.5 lines of space above and below it. Other characters that cause this behavior include the Greek letter "ϕ" (phi) or subset-equal "⫅". Other characters such as maps-to "⤇" add only about 0.5 lines on either side.

I ran into this problem when reading the source of julia-mode.el which contains a large number of such characters for LaTeX substitution.

Why does this occur, and can it be fixed?

Edit: I'm using Ubuntu 14.04 LTS with Emacs 24.3.1. By default I use the "Ubuntu Mono 13" font but other fonts are sometimes used to display characters. I have installed no packages to explicitly manage unicode, and this problem is reproducible using the command emacs -Q.

Answer 1

I use agda-mode with a lot of mathematical symbols and have had the same problem. Used to be the only real solution was as @shosti suggests: customizing the font mappings. In my case, I had to disable a number of fonts because even though I had fonts installed with particular glyphs defined, emacs would often choose the wrong one (displaying boxes). Seemed to be worse on OSX but I've seen it on Linux too. So just having the right fonts installed wasn't enough.

What I do now is use Roland Walker's excellent unicode-fonts package. (I would recommend installing it through MELPA.) This has pretty much eliminated the issue entirely.

I put this in my init file:

(unicode-fonts-setup)
(set-frame-font "PragmataPro 12")

I use PragmataPro by default but any other font with good coverage will work too and you can customize things further with the package if needed.

Answer 2

This occurs because Emacs uses different fonts for different parts of the Unicode character set. You can verify what font is being used by positioning the cursor over a character and pressing C-u C-x =. For instance, with my configuration over an ASCII character I get nil:-apple-Consolas-medium-normal-normal-*-14-*-*-*-m-0-iso10646-1 (#x88), but over a ⧺ I get nil:-apple-Symbola-medium-normal-normal-*-14-*-*-*-p-0-iso10646-1 (#xCE1) (in other words, it's using Consolas for ASCII and Symbola for some special characters). Different fonts have different heights, and Emacs will always give enough space in the line height to display the tallest font. By default, Emacs will try to use system-dependent fonts that work, but this often leads to ugly results.

The solution I've found to the line height problem is unfortunately rather annoying: I go through the problematic characters I find and use set-fontset-font to set them to a font/size that doesn't affect the line height. The code to do this is available in my configuration, but it's ugly and still not perfect. If anyone has a better solution I'd be interested in hearing it.

Answer 3

I already had this problem in a different unanswered question here. I would expect different fonts at the same size to have the same heights, but this doesn't seem to be the case, so I will show my ad-hoc approach to fixing this.

Leaving aside accents, and focusing only on plain characters, different fonts are necessary to display all the different characters simply because some fonts omit entire blocks of characters that are present in other fonts. To make sure that uncommon characters (such as "𝚫") have at least one font for them, install a font such as Symbola, and see the list of fonts in the readme of the unicode-fonts package.

If you install the package unicode-fonts and a number of good fonts, all characters should be supported, but some will have wrong displayed heights in emacs.

Suppose that Monaco has incorrect heights, but Symbola appears to have correct heights for mathematical symbols (such as SUBSET OF OR EQUAL TO ⊆; use C-x 8 RET or insert-char to test different characters.). In my case, not using Monaco, Noto Sans Symbols and Apple Symbols was sufficient; one good font for me was DejaVu Sans Mono.

The first thing is that you can forbid unicode-fonts from using Monaco by adding it to unicode-fonts-skip-fonts; whatever font it selects next might have the right height. Alternatively, you can tell unicode-fonts to use a specific font for a Unicode block (such as Mathematical Operators; here is a list of all blocks) by modifying an entry of unicode-fonts-block-font-mapping.

The second is that you can easily do this manually for a very precise set of characters by using set-fontset-font. If Symbola is a good font for mathematical symbols (in this case the range 0x2100..0x23ff), the following should work:

(set-fontset-font t '(#x2100 . #x23ff)
  ;; this should throw an error if there is no such font
  (font-xlfd-name (find-font (font-spec :family "Symbola"))))

Other ranges I needed to modify myself were 2000..206f, 27c0..27ff, 2900..2bff, 1d400..1d7ff.

Finally, it is not necessary to search for misconfigured characters by hand. Assuming unicode-fonts is installed, the following function will generate a list of all characters with incorrect heights:

(defun debug-unicode-heights (&optional block-name)
  "Find all characters in a given block that have incorrect heights.

BLOCK-NAME can be anything that
`unicode-fonts-debug-insert-block' accepts, such as `'all-math',
or a string naming a Unicode block."
  (interactive "sBlock name:")
  (unless block-name (setq block-name 'all-math))
  (let ((buffer (generate-new-buffer (format "debug-unicode-heights:%s" block-name)))
        expected-height
        bad-characters)
    (pop-to-buffer buffer)
    (with-current-buffer buffer
      (unicode-fonts-debug-insert-block block-name)
      (goto-char (point-min))
      (setq expected-height (line-pixel-height))
      ;; (message "Expected height %d" expected-height)
      (while (< (point) (point-max))
        (if (or (= (line-pixel-height) expected-height)
                ;; Some characters are invalid, they have no name
                ;; (their name is just their hex code), and their
                ;; heights do not matter to us.
                (looking-at-p "^.[^\"]*\"#"))
            (delete-region (line-beginning-position)
                           (1+ (line-end-position)))
          (push (char-after (line-beginning-position)) bad-characters)
          (forward-line))))
    ;; (display-message-or-buffer buffer)
    (apply #'string (reverse bad-characters))))

For example:

M-: (debug-unicode-heights 'all-math)

and then

M-: (debug-unicode-heights 'all-greek)

would show all bad mathematical symbols. You can then examine which font they are displayed with, and change it.

I have OS X 10.9.5, so my font settings would likely be different from yours. It is not necessary to install unicode-fonts; it is possible to manually specify your preferred font choices entirely through set-fontset-font whenever you find that emacs's default choices don't work well.

P.S. There is an alternative: when a font is consistently too large/small, you can add an entry to face-font-rescale-alist to tell emacs to always multiply that font's size by a factor of, say, 0.95, like so:

(add-to-list 'face-font-rescale-alist (cons (font-spec :family "STIXGeneral") 0.95) t)

When I tried this, it didn't quite work right (my bug report is here), but this is also a possible approach.

Answer 4

This is a bug; it is fixed in trunk. The issue is caused by bad line height information on TeX fonts, which happen to be the ones that Emacs falls back to if it can't find a character.

The bug thread on emacs-bug has a lot more info.