Characters replaced with angled bracket sequence, possibly due to encoding

Question

I have files in Markdown, saved in a Dropbox folder. One of them replaces characters with angled bracket sequences. For example a git diff no-index between the saved file and a copy-paste of the text to a new file gives:

-# `Package <81><B4>julia-mode-0.3<81><B4> is unavailable.`
+# `Package ´julia-mode-0.3´ is unavailable.`

Other sequences are <91><A1><AD> for ellipsis (...), <81><A0> for a space, <81><E2> for â, <86><A4> for the euro sign €, <91><A1><B0> for opening double quote, <91><A1><B1> for closing double quote, <86><A1> for opening single quote, <86><A2> for closing single quote.

The variable buffer-file-coding-system is emacs-mule-unix in the original buffer and utf-8-unix in the buffer with the copy-pasted content.

Many other files in the same folder have recently stopped showing with proper formatting. They list Fundamental in the mode line instead of Markdown. The characters display properly. Many of these buffers have a variable buffer-file-coding-system of undecided-unix.

As I started having these issues around the same time, I believe they are related. Since the characters display fine in Emacs, it's not a question of converting the buffer from one encoding to another (as many of the threads I have found).

How can I save all of these buffers back to utf8-unix, and what triggered this behavior?

Answer 1

Interesting problem. To start with, I doubt that Emacs has replaced a horizontal ellipsis with the string "<91><A1><AD>". Instead, I suspect that what you're seeing is what your shell prints out when it sees a byte sequence that aren't valid in utf-8.

Indeed, <91><A1><AD> is not the byte sequence that utf-8 specifies for a HORIZONTAL ELLIPSIS character; I would expect to see <E2><80><A6>. (But of course, if your shell saw that byte sequence it would have drawn the glyph for the ellipsis as expected.)

It looks like the emacs-mule-unix coding system isn't compatible with utf-8 (the documentation on it isn't very specific). I would update the Emacs configuration so that it always saves files in utf-8-unix instead.