How do I convert a character code in one coding system (windows-1252) to another?

Question

I'm working on a script that needs to decode some markup in .rtf format.

Within rtf, as documented here, the escape sequence \' followed by hex digits specifies a character by numeric code, with the catch that this is not a Unicode code point but a character in the Windows-1252 character set (at least in the files I'm dealing with).

So the question:

What is the cleanest way to convert a numeric code in a different character set to a Emacs multibyte character or string. Please note that I would like to avoid converting the whole file; I only want to convert a sequence of numeric codes in one encoding (Windows-1252) to an equivalent sequence in another one.

The following code more or less works but seems needlessly verbose and wasteful:

    (defun convert-rtf-byte (byte)
      (let ((file (make-temp-file "rtf")))
        (with-temp-file file
          (insert (unibyte-string byte))
          (setq coding-system-for-write 'no-conversion))
        (with-temp-buffer 
          (let ((coding-system-for-read 'cp1252))
            (insert-file-contents file)
            (buffer-substring-no-properties (point-min) (point-max))))))

Answer 1

I don't know how I missed this in the manual, but the function I was looking for is decode-char, e.g. (decode-char 'cp1252 #x93)).

Answer 2

Everything is in the manual; the manual should be the first place you look. Questions should ideally indicate what you searched for in the manual. Use C-h i to open the Info viewer, which lets you read all of the manuals installed on your system.

The functions you need are documented in chapter 34.10.7 Explicit Encoding and Decoding of the Emacs Lisp Manual. I found it by choosing the Elisp manual from the list, then using C-s to search for “encoding”. The first result was for Base64, so I skipped that one. The second is for I/O, which you want to avoid. The third is the one you want.

You should read the whole thing, but probably you want this function:

 -- Function: decode-coding-string string coding-system &optional nocopy
          buffer
     This function decodes the text in STRING according to
     CODING-SYSTEM.  It returns a new string containing the decoded
     text, except when NOCOPY is non-‘nil’, in which case the function
     may return STRING itself if the decoding operation is trivial.  To
     make explicit decoding useful, the contents of STRING ought to be a
     unibyte string with a sequence of byte values, but a multibyte
     string is also acceptable (assuming it contains 8-bit bytes in
     their multibyte form).

Honestly the names of all of these functions are pretty bad. WTF is a “coding string”?? Anyway, if we ignore the function names it is easy:

(defun convert-rtf-byte (byte)
  (decode-coding-string (unibyte-string byte) 'cp1252))

M-x man cp1252 tells me that decimal 128 is Euro sign; that seems like an easy test. Executing (convert-rtf-byte 128) in my scratch buffer gives me #("€" 0 1 (charset windows-1252)), which looks just right. You can use C-x = to verify that the buffer contains U+20AC, and that it is encoded as #xE2 #x82 #xAC (assuming your scratch buffer, like mine, is UTF-8).