shell-command and C-h

Question

If I create a string from the output of a shell command, using shell-command-to-string, I sometimes see ^H characters in the string, when I guess those characters were supposed to result in backspacing.

For example, I can produce such a result on my Debian-stable machine by doing

(shell-command-to-string "fortune -m MacKay")

for that particular fortune, the result contains the following substring The subspace _^HW inherits (where I've replaced a "non-printing character" by the string "^H", because that is what I "see" in Emacs).

If I just do fortune -m MacKay in a terminal (e.g. in eshell) the result contains instead the substring The subspace W inherits.

If I do something like

(insert (shell-command-to-string "fortune -m MacKay"))

in the *scratch* buffer, and then do "M-x describe-string" with point on the non-printing character (which - as I said earlier - appears in emacs as ^H) it is (unsuprisingly) described as

             position: 1442 of 2745 (52%), column: 15
            character: C-h (displayed as C-h) (codepoint 8, #o10, #x8)
    preferred charset: ascii (ASCII (ISO646 IRV))
code point in charset: 0x08
               script: latin
               syntax: _    which means: symbol
             to input: type "C-x 8 RET 8" or "C-x 8 RET BACKSPACE"
          buffer code: #x08
            file code: #x08 (encoded by coding system prefer-utf-8-unix)
              display: no font available
       hardcoded face: escape-glyph

What should I do to make shell-command-to-string "treat these deletion characters correctly?"

I doubt this has much to do with the Emacs version, but anyhow I'm using Emacs 25.1.50.18

Answer 1

On old teletypewriters, the BS control character, written ^H, \008 or \b, would backspace the print head by one character position. Underlining could be achieved by overstriking a character with an underline (_^HH_^Hi), and bolding could be achieved by overstriking a character with itself (B^HBy^Hye^He).

Teletypewriters no longer exist, but a number of Unix commands (such as nroff and man) still produce this kind of sequences, and they are embedded in old text files (such as the fortune data files). Unix pagers (more, less) interpret them by converting to the right terminal control sequences for underline and bold.

In your case, the simplest solution would be to discard the overstriking by removing the character preceding the BS and the BS itself:

(replace-regexp-in-string ".\008" "" string)

A better solution would be to interpret the overstriking and generate text properties for bold and underline. There are examples of that in the Emacs sources, e.g. Man-fontify-manpage in man.el or woman-man-buffer in woman.el.