Problems with keybindings when using terminal

Question

I have the following line in init.el:

(global-set-key [(control \;)] 'comment-region)

It works very well in GUI window, but if emacs is invoked with the -nw option then C-; doesn't comment region, only inserts ; character. M-x comment-region RET works fine however.

I tried follow this answer on similar problem, but C-q C-; returns just ; and I obviously don't want to bind bare ;.

How to achieve desired effect?

Answer 1

First, the more general issue: terminal emulators are often limited in the control and escape sequences they can send. So: it may be the case that the terminal swallows your special characters before they reach Emacs. As a general diagnostic, you can hit C-h l (or M-x view-lossage) to see if your key combinations make it into Emacs.

For more discussion of this issue, have a look at this Stack Overflow thread and the links therein.

To summarize the back-and-forth in the comments, your specific problem suggests it's the terminal rather than Emacs that is the problem. When you tried C-; and then M-x view-lossage, you got blah blah blah c ; ESC v i e w - l o s s a g e RET. That looks like a terminal issue: if the key combination were reaching Emacs, the c ; part would look like C-;.

Answer 2

In Shift+Up isn't recognized by Emacs in a terminal I explain how terminals translate most function keys into escape sequences, because the interface between applications and terminals transmits characters (or rather bytes), not keys. Only a few modifier+character combinations have their own character:

• Ctrl plus a letter or one of @[\]^_ turns into bytes 0–31 (ASCII control characters).
• Often Ctrl+? turns into byte 127 and Ctrl+Space is equivalent to Ctrl+@ (byte 0).
• Some function keys are equivalent to control characters: Tab = Ctrl+I, Return = Ctrl+M, Esc = Ctrl+[.
• And Backspace = Ctrl+H or Ctrl+? depending on the configuration. Ctrl+? is more convenient for Emacs, since Ctrl+H is help.
• Meta+character is sent as Esc followed by character.

So what about other combinations such as Ctrl+; or Ctrl+Shift+letter? Since there's no corresponding character, the terminal has to either reuse a character or send an escape sequence. Many terminals ignore modifiers when there's no corresponding character, so you end up with Ctrl+; sending ;, Ctrl+Shift+letter equivalent to Ctrl+letter, etc.

Terminal vendors kept doing the simple thing for a long time. There was no standard for escape sequences, which was self-perpetuating — terminal vendors don't implement it, applications don't support it, users don't expect it. Some terminal emulators can be configured to send arbitrary escape sequences, so if yours can, you can configure it and declare the escape sequences to Emacs (more on this later).

Lately, the situation is changing, because there have been two proposals to standardize on escape sequences. One is LeoNerd's libtermkey with the syntax ESC [ codepoint ; modifier u. Another is Thomas Dickey's xterm with the syntax ESC [ 2 7 ; modifier ; codepoint ~. Current versions of xterm (≥216) can be configured for either syntax by setting the formatOtherKeys resource; the feature must be activated by setting the modifyOtherKeys to a nonzero value.

If your terminal emulator doesn't support these syntaxes but can be configured, pick either.

Since Emacs 24.4, Emacs automatically turns on the modifyOtherKeys feature when it detects that the terminal is xterm version ≥216. Emacs's detection of escape sequences to encode keys works through the variable local-function-key-map. As of Emacs 24.4, not all escape sequences are supported. You can use the following code in your init file to complete the job.

;; xterm with the resource ?.VT100.modifyOtherKeys: 1
;; GNU Emacs >=24.4 sets xterm in this mode and define
;; some of the escape sequences but not all of them.
(defun character-apply-modifiers (c &rest modifiers)
  "Apply modifiers to the character C.
MODIFIERS must be a list of symbols amongst (meta control shift).
Return an event vector."
  (if (memq 'control modifiers) (setq c (if (or (and (<= ?@ c) (<= c ?_))
                                                (and (<= ?a c) (<= c ?z)))
                                            (logand c ?\x1f)
                                          (logior (lsh 1 26) c))))
  (if (memq 'meta modifiers) (setq c (logior (lsh 1 27) c)))
  (if (memq 'shift modifiers) (setq c (logior (lsh 1 25) c)))
  (vector c))
(defun my-eval-after-load-xterm ()
  (when (and (boundp 'xterm-extra-capabilities) (boundp 'xterm-function-map))
    (let ((c 32))
      (while (<= c 126)
        (mapc (lambda (x)
                (define-key xterm-function-map (format (car x) c)
                  (apply 'character-apply-modifiers c (cdr x))))
              '(;; with ?.VT100.formatOtherKeys: 0
                ("\e\[27;3;%d~" meta)
                ("\e\[27;5;%d~" control)
                ("\e\[27;6;%d~" control shift)
                ("\e\[27;7;%d~" control meta)
                ("\e\[27;8;%d~" control meta shift)
                ;; with ?.VT100.formatOtherKeys: 1
                ("\e\[%d;3u" meta)
                ("\e\[%d;5u" control)
                ("\e\[%d;6u" control shift)
                ("\e\[%d;7u" control meta)
                ("\e\[%d;8u" control meta shift)))
        (setq c (1+ c))))))
(eval-after-load "xterm" '(my-eval-after-load-xterm))

If the TERM environment variable isn't set to xterm or a variant such as xterm-256color, Emacs won't activate those sequences. If Emacs already has support for your value of TERM, you can add support by defining a function similar to the one above, to be executed after loading the Lisp file whose name is the value of TERM. If Emacs has no such support, you can add it by creating a subdirectory called term somewhere in your load-path, and creating a Lisp file called term/$TERM.el where $TERM is the value of TERM, defining a function called terminal-init-$TERM.

As I write, it seems that few terminal emulators other than xterm have adopted these escape sequences. On OSX, you can configure iTerm2 by selecting an escape sequence for each key combination, one by one.

Answer 3

Gilles' answer stated: On OSX, you can configure iTerm2 by selecting an escape sequence for each key combination, one by one. I had a little trouble getting there, so here a detailed explanation in case you are like me, confused.

If, when running Emacs in a terminal (e.g., Mac's default Terminal.app), C-x C-; does not run comment-line, and you would like that functionality, then you must switch to iTerm2 (Terminal.app does not have the capability) and create a Key Mapping under Profiles...Keys as follow:

^;     ^[[59;5u

This is accomplished by clicking on the + button, which brings up a little window called "Keyboard Shortcut"; this window has initially two fields; the top has a value of "Click to Set" and the bottom has a value of "Ignore." Click the top button and type C-;. This sets the key combination that will run the action and code. Click the bottom field, "Action", and find and click on "Send Escape Sequence", which is down a little more than half way. After you click that, a third field will appear, called "Esc+". Inside this field, type:

[59;5u

The 59 is comma's decimal ASCII code, and the 5 is the code for Ctrl. Then press "OK" to finish. The correct key combo to key sequence will now be included in iTerm2's preferences. When you start up Emacs in iTerm, you will now have access to C-x C-; functionality.

Answer 4

One further update for macOS/iTerm2 users: iTerm now ships with a "Report modifiers using CSI u" option within preferences that configures LeoNerd's libtermkey for you with a single checkbox. As of this writing (iterm2 3.3.9) it can be found in Preferences -> Profiles -> Keys. There is no longer any need to remap keychords one by one in iTerm, although I found that a small handful of keystrokes still needed to be registered with emacs.

Answer 5

I created a package which allows configuring Emacs and many terminal emulators to recognize any combination of key/modifiers (including C-;):

https://github.com/CyberShadow/term-keys

Answer 6

What I've got here works with iTerm2 3.3.11 + tmux 3.0a + Emacs 26.3 and is a combination of some previous answers here with a couple of tweaks.

The biggest problem I had was that despite them matching the CSI u spec, the codes in tarsius' answer did not support case-sensitive chords. I confirmed by going into sed -n l and trying out various key combinations that instead of any modifiers relating to shift chords, iTerm2 sends non-shifted chords with the different base key code. e.g., for ctrl+shift+a it sends ^[[65;5u (C-A) instead of ^[[97;6u (C-S-a). This is not necessarily "wrong" (as far as I know) but we need to work around it because Emacs does not differentiate keybindings for C-A vs C-a: you need to specify the S in C-S-a. The solution is to add xterm CSI code mappings from those particular upper-case codes to their lower-case+shift counterparts. Without that fix Emacs does not recognize that the shift key has been pressed.

1. In iTerm Preferences -> Profiles -> Keys for the tmux profile to tick the "Report modifiers using CSI u" box. It will prompt you to erase existing mappings - you want to do this. (credit to ambirdsall's answer)
2. On your tmux host's ~/.tmux.conf, set the following:

set-option -g xterm-keys on
set -g default-terminal "xterm-256color" # or some other "xterm" variant

You must restart tmux for changes to take effect. Use tmux kill-server. A worse solution is to directly export+set the TERM environment variable from your shell init (.zshrc, .zprofile, etc.). This is not desirable in case you actually use different terminal emulators, but this will work fine if you just want to test it out.

3. In an Emacs init file (e.g., ~/.emacs.d/init.el) add the following (heavily based on tarsius' answer):

;; xterm with the resource ?.VT100.modifyOtherKeys: 1
;; GNU Emacs >=24.4 sets xterm in this mode and define
;; some of the escape sequences but not all of them.
(defun character-apply-modifiers (c &rest modifiers)
  "Apply modifiers to the character C.
MODIFIERS must be a list of symbols amongst (meta control shift).
Return an event vector."
  (if (memq 'control modifiers)
      (setq c (if (or (<= ?@ c ?_)) (<= ?a c ?z)))
                  (logand c ?\x1f)
                (logior (lsh 1 26) c))))
  (if (memq 'meta modifiers) (setq c (logior (lsh 1 27) c)))
  (if (memq 'shift modifiers) (setq c (logior (lsh 1 25) c)))
  (vector c))
(defun my-eval-after-load-xterm ()
  (when (and (boundp 'xterm-extra-capabilities) (boundp 'xterm-function-map))
    (let ((c 32))
      (while (<= c 126)
        (mapc (lambda (x)
                (define-key xterm-function-map (format (car x) c)
                  (apply 'character-apply-modifiers c (cdr x))))
              '(;; with ?.VT100.formatOtherKeys: 0
                ("\e\[27;3;%d~" meta)
                ("\e\[27;5;%d~" control)
                ("\e\[27;6;%d~" control shift)
                ("\e\[27;7;%d~" control meta)
                ("\e\[27;8;%d~" control meta shift)
                ;; with ?.VT100.formatOtherKeys: 1
                ("\e\[%d;3u" meta)
                ("\e\[%d;5u" control)
                ("\e\[%d;6u" control shift)
                ("\e\[%d;7u" control meta)
                ("\e\[%d;8u" control meta shift)))
        (setq c (1+ c))))
    ;; Override the CSI u codes for 'C-A' and 'C-M-A' as 'C-S-a' and 'C-M-S-a' for A-Z
    (let ((c 65))
      (while (<= c 90)
        (mapc (lambda (x)
                (define-key xterm-function-map (format (car x) c)
                  (apply 'character-apply-modifiers (+ 32 c) (cdr x))))
              '(;; with ?.VT100.formatOtherKeys: 0
                ("\e\[27;5;%d~" control shift)
                ("\e\[27;7;%d~" control meta shift)
                ;; with ?.VT100.formatOtherKeys: 1
                ("\e\[%d;5u" control shift)
                ("\e\[%d;7u" control meta shift)))
        (setq c (1+ c))))))

(eval-after-load "xterm" '(my-eval-after-load-xterm))

N.B.: I'm sure there's a better way to write this but my elisp skills are rather poor. =)