Emacs converting spaces to tabs when using X11 primary selection in a terminal window

Question

When I use emacs in a terminal window on linux, I am often getting spaces unexpectedly converted to tabs when I cut and paste using the X11 primary selection. What reproduces the problem for me is the following:

Open a terminal window. (I tested using both lxterminal and aterm.) Start emacs with the -nw option. Type the following text character by character on the keyboard:

1234567 1234567

(If you copy and paste it from the stackexchange question, it won't reproduce the problem.) Triple-click on this line of text to copy it to the X11 primary selection. Open a new terminal window, and in that window do od -a. Click the middle mouse button to paste the text into the terminal. This is echoed by the od command with an ht rather than an sp.

I'm pretty sure this is emacs doing this, because if I use the lightweight editor mg instead, it doesn't happen. I've also tested that this still happens if I hide my .emacs file, so that it's not resulting from any customizations that I've done there.

Can anyone explain how to get rid of this behavior?

Update: Gilles' comment shows that this happens at line 265 of this file https://github.com/emacs-mirror/emacs/blob/master/src/cm.c , which has the following code: if (tabcost < (deltax * tty->Wcm->cc_right)) { . I would like to find a solution for this problem that doesn't involve my having to make my own private fork of emacs. I would be happy to see an answer that would suggest an approach to doing this by submitting a patch to emacs. Perhaps this behavior could be turned off by setting a certain flag in lisp. If that seems like a reasonable approach, then it would be great to see an answer saying so, and maybe pointing to an example of how well-behaved emacs C code would access such a flag. If I was an emacs maintainer, I would just rip all the code out, because I just don't think this is reasonable behavior for a terminal-based application in the year 2020, but possibly some people depend on this behavior and it would cause problems if it were simply removed.

Answer 1

Analysis

There's a space in the buffer, but in certain circumstances, when Emacs needs to print a sequence of spaces, it chooses to print a tab instead because that requires writing fewer characters to the terminal. This is what's happening when you insert the characters one by one. It doesn't happen when Emacs is printing the line all in one go, for example when you paste the whole line, or if you scroll away then back (you need to scroll enough to make the line disappear to be sure), or if you switch to another buffer and then back.

The decision to print a tab happens deep inside the C code, in calccost() in cm.c. I don't fully understand how this works: in this particular case, writing one space or writing one tab would have the same effect, but apparently Emacs prefers to write one tab (I can confirm this with Emacs 26.3 on my machine).

I can't find a way to disable this behavior from Lisp code. It's parametrized by tty->Wcm->cc_tab, tty->Wcm->cm_usetabs and tty->Wcm->cm_tabwidth (which in some places are written TabCost (tty), UseTabs (tty) and TabWidth (tty)). As far as I can tell these are derived only from terminal characteristics and can't be influenced by Lisp code. TabWidth is read from the tw termcap value if present (it isn't on modern Linux, I think it's obsolete), defaulting to 8. UseTabs is true if TabWidth is 8 on modern Unix-like platforms (tabs_safe_p). TabCost is calculated from TabWidth and UseTab so it's no help. So the only way to disable this behavior (without recompiling Emacs) is to convince Emacs that the width of a tab is not 8.

Workaround: use a terminal that doesn't copy tabs

In xterm, unlike the other terminals I tried (lxterminal, gnome-terminal, rxvt), when you copy the line, the clipboard contains a space, not a tab.

Workaround: redisplay

Anything that redraws the screen makes the tab go away, for example M-x redraw-display or C-l (recenter-top-bottom).

This works even for a line like 1<TAB>2 where a tab is present in the buffer and a tab would be actually advantageous for display.

Workaround: tell Emacs that your terminal doesn't support tabs

It's a bit complicated for what it does, and it can have a secondary effect in other applications that have behavior based on the terminal name (for example, loss of colors). But you can change your terminal's description to pretend that its tab width is not 8. Any value other than 8 will make Emacs avoid tabs, but I recommend choosing a large value to avoid confusing other applications.

Create a file with the following content. I'll call it my-terminal-descriptions.terminfo; the name doesn't matter since you only need it once and for all.

xterm-256color-notab|xterm with 256 colors avoiding tabs,
 tw#9999,
 use=xterm-256color,

Run the following command to compile this new terminal description:

tic -x my-terminal-descriptions.terminfo

Run Emacs with TERM set to xterm-256color-notab. For example:

alias emacs='env TERM=xterm-256color-notab emacs'