How to `ediff-buffer` with ansi-colors (script output)? `ediff-file` works!

Question

Context: files with ANSI and other terminal color codes.

I often record terminal transcripts using the traditional script program or expect. Colored ls output, colored grep matches, mc and other interactive programs, everything works great and record is faithful!

What works: replaying, opening in emacs

• One can replay such files in a terminal, either fast e.g. cat or on own rhythm using less -r (or less -R to see backspaces ^H instead of obey them).
• One can open them in emacs and even have emacs automatically apply relevant mode, for example my .emacs.d applies this to any file matching script_*.txt:

(add-hook 'text-mode-hook
          (lambda ()
        (when (string-match "script_.*\.txt$" (buffer-name))
              (progn
        (format-decode-buffer 'backspace-overstrike)
        (format-decode-buffer 'ansi-colors)
        (hide-dos-eol)
        (face-remap-add-relative 'default '((:foreground "white" :background "black")))
        ))))

What fails: emacs accepts ediff-files, refuses ediff-buffers. Why? How to solve?

Detail of what works: comparing with ediff-files works

This opens files straight into ediff, perfectly:

emacs --eval '(ediff-files "script_file1.txt" "script_file2.txt" )'

Differences are shown, colors from ANSI codes are blended with colors from ediff which was not expected as obvious.

Calling ediff-files interactively also works.

The fact is, I just figured out ediff-files works, while searching to write this question. I never ediff-files because it's much more convenient to open files with any of the convenient ways (dired, etc) then ediff-buffer after opening. ediff-buffer offers straightforward completion among the opened buffers, which is more selective/relevant.

Detail of what fails: opening files interactively then invoking ediff-buffers fails

ediff doesn't start, minibuffer shows:

"Sorry, `ansi-colors' format is read-only"

Notice that the very same buffers that have just been compared using ediff-files, when closing ediff then calling ediff-buffers yield same failure.

Detail of what fails: saving file

Trying to save a file decoded with tty-format/ansi-colors fails, too.

How to solve?

A good netizen searches before they ask. Error message appears in package tty-format. Browsable copies of source code appear in various places, e.g. on https://github.com/grantdhunter/dotfiles/blob/8c11b417a373d4a8eb7a4b3772ebf956e825edfc/emacs.d/tty-format.el#L226

It looks like there is an avoidable obstacle.

It looks like buffers opened with ansi-color encoding have "lost something", they have been reencoded internally by emacs and cannot be saved again. I can understand that ANSI codes that move cursor around to produce a composite buffer make not obvious what to do when saving again.

Possible workaround: recovering convenience of ediff-buffer

I have an idea of a possible workaround but am not savvy enough in emacs-lisp.

Q1: Can someone make a function which would offer completion by buffer names (like ediff-buffers) but would actually figure out underlying files and calls ediff-files on them?

Any better idea is welcome.

Possible workaround for impossibility to save: instruct emacs to save them into another format, possibly warning about lost information

This makes sense when saving snippets of text, e.g. as plain text, as color HTML, whatever. (Notice: htmlize-buffer does not fit, practically does no better than plain text.)

Q2: Is there a way to tell emacs "when saving this file, instead of refusing at all, offer me to save to a different location and format"?

I expect this would have a side-effect of allowing ediff-buffers to work immediately and lose color information, but maybe not.

Something else?

Perhaps there is another way to handle files with ANSI color codes that doesn't have these problems at all?

Thanks anyway.

Answer 1

Replacing tty-format with xterm-color appears to provide some of the expected benefits stated in the question text, while adding a risk of destroying information by overwrite with plain text.

Cf. above "recovering convenience of ediff-buffer"

ediff-buffer works again.

Cf. above "instruct emacs to save them into another format"

As soon as a buffer is opened, it is marked as "modified".

Saving it saves a plain-text version of the text, keeping normal characters, plus the obvious linefeed, plus BS (a.k.a. 0x08, ^H) and CR (0x0D, ^M).

How to get this behavior

Here's how I adjust in .emacs my startup script to associate this to all files matching script_*.txt.

(require 'xterm-color)

(add-hook 'text-mode-hook
          (lambda ()
            (when (string-match "script_.*\.txt$" (buffer-name))
              (progn
                (xterm-color-colorize-buffer)
                (hide-dos-eol)
                (face-remap-add-relative 'default '((:foreground "white" :background "black")))
                ))))

What is not fully satisfying

One has to remember not to save file over the same name because it immediately destroys all existing color information.