how to map <C-M-left>, <C-M-right> etc to the correct hex codes in iterm2

Question

I'm looking to configure iterm2 to output the correct hexcodes that correspond to in emacs.

There's a whole bunch of mappings predefined but it's very cryptic.

I'm a little bit lost as to how these mappings are translated and if there is a table of how these mappings work.

Answer 1

You can add:

For C-M-left: Send escape sequence: [1;7D
For C-M-right: Send escape sequence: [1;7C
For C-M-up: Send escape sequence: [1;7A
For C-M-down: Send escape sequence: [1;7B

And this will work with Emacs if you set the term in iTerm2 to be reported as xterm.

To know how to figure all this out, well its complicated, and I just went down this rabbit whole myself, so here I'll explain it all.

How terminals work

First off, you have to understand how Terminals work. Basically a terminal will get the input from the keyboard or mouse directly. So when you type or use the mouse, the terminal receives the exact keys that were pressed.

The terminal's job is to translate those pressed keys or mouse events into actual text. So when you press on the key A, the terminal receives that signal from the keyboard, and its the terminal that decides if this key should map to the character a. You could imagine a terminal in a different keyboard layout might actually choose that the A key on your US keyboard when in AZERTY should actually map to the character q.

So the terminal's job is to translate from actual keyboard and mouse events into characters to send to the connected shell or command line application.

That means when an application is running in your shell, it does not receive the actual keyboard and mouse events, but instead it receives characters as translated by the terminal.

Now because terminals are super old, this translation of keyboard and mouse events to characters only supported the ASCII character range, which is composed of 128 total characters.

By default, most terminal will do the right thing when translating the simple keyboard events that have clear direct mappings to ASCII.

For example, all pressed single letter keys a-z (as per the keyboard layout) becomes their corresponding ASCII (97 to 122 range). Now all of those keys pressed with the Shift modifier held or CapsLock turned on will map to their corresponding upper-case character A-Z (65 to 90 range).

So when it comes to those normal mappings that have a simple logical correspondence to the ASCII characters, generally it all works, and no custom mapping in your Terminal need to be configured.

Where things get weird is that 128 characters is not enough to represent all possible keyboard and mouse events.

What happened since then is that first, Terminals were extended to support Unicode with UTF-8 encoding. UTF-8 was chosen, because it is backwards compatible with ASCII. That means that if you were to press the keys on some keyboard that logically map to some Unicode character, say the character Բ, well the Terminal would send over that character using UTF-8 encoding to the shell or application its connected too.

But even though they now support Unicode, there are some keyboard and mouse event that have no representation in Unicode, because Unicode is not meant for keyboard and mouse events, its meant for representing text. So the question (ASCII or Unicode) is what character should map to some keyboard event like: Ctrl+Alt+Home ?

Logically, none of these are characters, the Home button doesn't represent a character, neither does Ctrl or Alt. Now in Unicode you have some characters for these, for example the MacOS Ctrl character is ⌃ which is Unicode U+2303, but even then, you wouldn't want the Ctrl key to map to U+2303, because then what if you wanted to actually type U+2303 so that inside VIM it typed ⌃? So you can't use the Unicode for indicating a Ctrl key-press using the macOS Unicode control character or it would now be ambiguous with if you are actually trying to type the control character or are just pressing the Ctrl key.

Now in ASCII, it turned out people had something called Function Keys, basically the range 0 to 31 and 127. Those are called as such because they were meant to perform a function instead of printing a character. So back in the days, people thought that apart from typing a few characters to have them printed, there would be a common set of typing functions someone could run and those would map to the character codes in ASCII 0 to 31 and 127.

For example, ASCII 127 is DEL (Delete), which was meant to call the delete character function to delete a typed character. ASCII 13 was meant to run the carriage return function, to introduce a new line, and so on.

Naturally, for those functions that still make sense, Terminals also generally do the correct mapping for those. So pressing the Delete key will send the ASCII 127 character to the shell or application.

So this will work great for TAB, Backspace, Escape, Return, and Delete. The other functions don't make as much sense nowadays. For example, you have start of header which is too concrete and not really useful, it goes as follows:

Character  Dec  Hex   Octal  HTML  Function
^A         1    0x01  0001   ^A    SOH start of header

But as you see, these characters were also called Control Keys I think because you did have to press Control+A to execute the start of header function. So what Terminals do here is they continue to use all these Control+ ASCII characters as a way to send the keyboard event Ctrl+. But again, there is only a few of these (range 0 to 31).

So if you press Ctrl+A the terminal maps it to ASCII 1.

What's annoying is there isn't even enough for all Ctrl+english letter combination, because some of them are already used by TAB and Return:

Character  Dec  Hex   Octal  HTML  Function
^I         9    0x09  0011   ^I    HT horizontal tab [\t]
^M         13   0x0d  0015   ^M    CR carriage return [\r]

You also get a few extras that allow for Ctrl+ one of @,\,],^,_ and you could have Ctrl+[ but that's taken for Escape so it will collide.

Alright, so now if you have been following, you can go to https://www.barcodefaq.com/ascii-chart-char-set/ to see the list of ASCII characters and you should have a good idea of what keypress will be sent to your shell or application by your terminal.

Examples:

Ctrl+@ => ASCII DEC: 0, CHAR: ^@, INTERPRETED AS: Ctrl+@
Ctrl+B => ASCII DEC: 2, CHAR: ^B, INTERPRETED AS: Ctrl+B
Ctrl+[ => ASCII DEC: 27, CHAR: ^[, INTERPRETED AS: ESC
Shift+a => ASCII DEC: 65, CHAR: A, INTERPRETED AS: A
a => ASCII DEC: 97, CHAR: a, INTERPRETED AS: a
DEL => ASCII DEC: 127, CHAR: <invisible>, INTERPRETED AS: Delete

How terminals send keyboard and mouse events that don't fit the ASCII range?

As we saw, there's not enough characters to send all the keyboard and mouse events, especially those that don't have a logical character associated to them like Ctrl+Alt+Home.

In order to support these, different terminals came up with their own scheme, no standard was established, and everything just got messy, but overtime some terminals have become the "most used" and their way of doing things is often the ones best supported by applications and shells.

Generally, what terminals will do is that for keyboard or mouse events with no direct mapping to ASCII, they will make up a combination of ASCII characters that they use to represent more keyboard and mouse events. So for example, they might say that Ctrl+Alt+Home will be represented as the string of characters: ^[[1;7H

This is just ASCII once more, so a terminal could decide that:

Ctrl+Alt+Home => ASCII (in decimal): [27,91,49,59,55,72]

Back to iTerm2 Key Mappings

What you see in iTerm2's Key Mapping are all those additional mappings for stuff that fall outside the ASCII range. Though I think you could also use it to remap how iTerm2 maps the normal keys even within the ASCII range if you wanted.

You have two useful settings:

1. Send hex codes

This lets you tell iTerm2 that a specific keyboard shortcut should send the following sequence of ASCII characters (where it takes the ASCII chars in their hexadecimal representation).

So refer too: https://www.barcodefaq.com/ascii-chart-char-set/ to see the hex code for each ASCII character.

With that you can tell iTerm2 to send any arbitrary sequence of ASCII characters for any arbitrary keyboard key press event.

2. Send escape sequence

But generally, terminals like xterm have used a sequence of ASCII characters that always starts with the ASCII Escape character ^[ which is ASCII decimal 27 or ASCII hex 0x1b.

Because that's so common, iTerm2 has a "Send escape sequence" where it lets you type the ASCII characters using your keyboard (so you can input them as text instead of hex or decimal) which it will automatically prepend with the ASCII Escape char or ^[ aka ASCII decimal 27 or hex 0x1b.

How do I know what I should map those shortcuts outside the ASCII range so they work in most shells and terminal applications?

This is a little tricky, because each shell or application running in the terminal might not understand the same ASCII sequence to mean the same key-presses. And like I said, there is no real standard for it either. But some older terminals had established some convention for some additional key-press combinations, that's where for example in iTerm2 you can load the xterm defaults preset. This will load the set of added key-presses for which the xterm terminal (an old popular terminal) had itself added by default. Because xterm is popular, most shells and terminal applications will understand those.

Beyond that, it really depends on the terminal application in question, and you need to check their documentation and hopefully they document that stuff.

There's really no standards?

Like I said, each old popular terminal like VT-102, xterm, etc. will have had their own additional key-presses, and those are mostly "standard". But none of them offered a scheme that covered all possible mouse and keyboard events.

But, since then, there are 2 conventions that emerged which attempt to systematically cover all keypresses:

1. Fixterm (aka libtickit or libvterm)

The convention for this is:

^[[<char-code>;<modifer-code>u

I'm not sure where the list of valid <char-code> and <modifier-code> comes from though.

2. xterm's modifyOtherKeys

The convention for this is:

^[27;<modifer-code>;<char-code>;~

I'm also not sure where the list of valid <char-code> and <modifier-code> comes from though, but I believe they are the same as with fixterm.

Please note the xterm convention predates fixterm convention by many years. Neither are yet very popular, and I would say they are trying to be a "standard", but don't expect this to work with all apps.

If you really just care about Emacs?

For me, most exotic keyboard shortcuts I care about are for use with Emacs. Other apps tend to have shortcuts within the ASCII range, which works as-is.

What I found for Emacs is that if you inspect the emacs-lisp variable: input-decode-map it'll show you all the ASCII sequences that Emacs support, it uses ASCII decimals to do so. For each combinations of sequence it shows what EMACS interprets the sequence as.

So you can use that to bind the shortcut in iTerm2 to the sequence Emacs is configured to interpret it as.

Also you can use the function: (read-key-sequence "?") and press some keyboard combination and Emacs will show you what it interprets it too.

Useful trick

A useful trick is to run cat -v in your terminal and now it will show you the ASCII characters of all the key combos and everything you press to see what your terminal currently maps them too. It won't show invisible stuff like Delete though.

What about Alt?

I forgot to mention that a lot of terminals will also make it so that Alt or Meta are mapped to ASCII 27 (the Escape character). So when you press Alt+key it will send the sequence: ^[<key> so it'll send ASCII 27 for Escape followed by the character or character sequence that maps to the key you typed along with it.

That's why in terminal Emacs, ESC is used for almost all M- style shortcuts as well, so M-x can also be performed with ESC x. Emacs will thus interpret ESC key as M-key.

What about shift?

Shift is normally not sent by the terminal, instead it normally simply change the character to be sent, like Shift+a sends the upper-case A character. Or pressing Shift+5 will send % (on a US layout`.

You can map a custom sequence for Shift+5 instead which you could have Emacs recognize as S-5 instead of %.

Revisiting fixterm and xterm modifyOtherKeys

I also didn't mention that newer Emacs by default have in input-decode-map most of the mappings necessary to interpret both fixterm and xterm modifyOtherKeys generic schemes. That said, the way Emacs does it is hard-coded, it misses some of them, so not all of them will work as-is. Also both conventions are not super well defined when there exist another way to send the same, as when will they use the convention versus try to be backwards compatible with some prior sequence or strategy that was used.

Anyways, that means that for Emacs and iTerm2, you can tick the Report modifiers using CSI u in iTerm2 Keys config, and a lot of things will now "just work" in Emacs (though not all).

Answer 2

I could post a thousand-word essay on how all of the various layers of software interact to provide both backwards-compatibility with hundreds of historical systems as well as customizability for dealing with new ones, but the truth is that it's really a waste of time. Just run Emacs as a normal gui application, and it'll all work much better.

If you really insist on running Emacs inside a terminal emulator, know that the OS tells the terminal emulator directly what keys you pressed. then the terminal emulator then translates this into a sequence of bytes to send to the application running inside the terminal, and finally that application translates it back into some idea of what keys you pressed.

The idea behind this configuration window in iTerm is to allow you to tell iTerm what the application you're running expects. Unfortunately, Emacs can be configured to work in a wide variety of environments, and you haven't given us enough information to figure all of that out.

I would suggest using the Emacs help functions to tell you what Emacs thinks is going on. Specifically, if you type C-h k, followed by any keyboard shortcut, Emacs will tell you what keys Emacs thinks you pressed and what command they normally trigger, if any. This will tell you what Emacs thinks of the bytes or escape codes that iTerm is sending.