I have a python file which defines a function and calls it.
def remove_whitespace(input_file, output_file): with open(input_file) as f: f.readlines() # do something
If I execute this file in Emacs (version 24.5.1), I recieve the error:
UnicodeEncodeError: 'ascii' codec can't encode character '\u2013' in position 1798: ordinal not in range(128)
I followed the error message and found many known issues relating to encoding. I altered my function to try filtering out anything outside of the ascii range 0-128 (leaving standard-ish only characters). I got the same error messages. So python is unable to read the file in the first place it seems. I should've recognised that sooner.
I then used the
codecs module to open the file:
import codecs with codecs.open(my_file, encoding="utf-8") as f: # do something
... but still received the same error.
Just out of curiousity, I exectued the file from terminal with the following:
which worked! No errors, the output file was as expected.
I looked into elpy's congig with
M-x elpy-config and saw that the
shell-interpreter-arguments contains an additional
-i flag. This simply puts the shell into interactive mode after executing the script, so I assume cannot affect anything.
I then preformed one last test - executing the file from an Emacs terminal:
eshell. This gave me the same error as before, which again points the blame to Emacs itself (and less so at elpy!)
Can anybody shed light onto this Emacs/Elpy issue? Where is Emacs assuming/enforcing some encoding? I am not printing those non-ascii characters within my function, so didn't imagine Emacs even tries to interpret them itself.