Visual corruption in terminal-emacs

Question

I run Emacs in my terminal, which is primarily kitty on macOS, but I'm experiencing the same (what I'm about to describe) with other terminal emulators:

In my shell LANG is en_US.UTF-8 and I have fonts installed so that pretty much any unicode point can be displayed, e.g. Symbols for Legacy Computing.

If a text file (C source code or whatever) contains such symbols and I open it in Emacs and scroll up and down, the buffer gets visibly "corrupted". It's especially noticeable with vertical splits, where the corruption involves visual spill over between the buffers. It's a purely visual thing, the file on disk is not affected.

Other CLI and TUI programs don't seem to have that problem: nano, vim, cat'ing the file to less. Is there something I can do to fix the problem in Emacs, or is it a known limitation/bug?

Example file: abi/constants.nim, scroll to the end.

If I run Emacs bare, it's not as severe but the problem is still evident

I'm running macOS 13.3, kitty 0.27.1, Emacs 28.2. TERM is xterm-kitty.

With that setup I've also tried iTerm2 3.5.0-beta10, Terminal.app 2.13, WezTerm 20230326-111934-3666303c (TERM is xterm-256color for those) — I experience the same problem, so it doesn't seem to be terminal-specific.

From searching commits: kitty supports Unicode 15, iTerm2 supports at least Unicode 13 and likely newer, WezTerm supports Unicode 15. Terminal.app (ships with macOS) is not open source, but symbols added in Unicode 13 are displayed correctly.

I've also tried with the Emacs GUI (28.2, downloaded from emacsformacosx.com) and experience a similar problem.

Running the GUI bare doesn't make a difference.

Answer 1

Your terminals just aren’t rendering those characters at the same width that Emacs expects them to be rendered. Emacs expects some of those characters to take up two character cells, but your terminal is rendering them in one.

Emacs stores this information in a variable called char-width-table. You can query the width that Emacs expects using the function `char-table-range.

For example, you can check the expected width of one of the angle block characters like this:

(char-table-range char-width-table ?\U0001fb42)

For me this returns 2, but your terminal rendered that character inside a single cell. (You can find out what character is at point using C-x =.)

You could improve the display by carefully setting values in char-width-table using (setf (char-table-range char-width-table ?\U0001FB42) 1) and similar instructions (use C-h f to see the help for char-table-range for more information about specifying larger ranges in one go).

Or you could use the Emacs GUI, which will use the exact pixel widths of the characters as reported by the font used to display them, and avoid all this nonsense. This is my recommendation; there is a lot that Emacs cannot do when stuck inside of a terminal.