I have a string with characters that are unencodable in UTF-8, and I need to remove these characters before writing the string to a file.
For instance, the following snippet creates a file that contains an invalid UTF-8 byte sequence. I would like this sequence to just not be present in the output file.
(let ((coding-system-for-write 'utf-8)) (write-region "This is invalid: \x19008e" nil "~/outputfile"))
By checking this character with
describe-char, I can see Emacs knows it is not encodable through UTF-8. So there must be a way to detect these characters and remove them.
I have an emacs-lisp script which converts docstrings to html files, which must be entirely in UTF-8. The following snippet shows essentially what I've been doing so far.
(with-temp-file output-file (insert symbol-name "\n" symbol-documentation) (set-buffer-file-coding-system 'no-conversion))
symbol-documentation is the string returned by
describe-variable (the content of the Help buffer).
This seemed to work fine at first, but now I've realised this was
causing some files to contain byte-sequences that are not valid UTF-8.
Some perpetrators are
composition-function-table. By manually
checking their documentation, I can see my Emacs indeed fails to
comprehend part of the content.
Q: How can I ensure only valid UTF-8 byte sequences are written to the output file?
I'm perfectly fine with skipping these invalid sequences entirely (instead of trying to understand them). I just want to make sure the output file is valid.
I tried changing
utf-8, but then my script gets
interrupted by a
Select coding system: prompt, and the problem