Shell coding system

Question

If, before M-x shell, I run C-x RET c (universal-coding-system-argument) I can change the coding system used by the shell. Therefore, given

echo Straße > foo 

foo will be encoded in the related CS.

How can I change the coding system after issuing M-x shell?

More generally, how can I set comint input/output encoding?

Answer 1

As regards Linux, the behaviour can be controlled by with:

(setenv "LANG" ENCODING)

before calling M-x shell and after from the shell buffer with:

(set-buffer-process-coding-system INPUT-ENCODING OUTPUT-ENCODING)   

Both should be set to a value compatible with the string to be represented. UTF-8 encoding family should work in most of the cases.

To learn about current encoding, we can use:

(getenv "LANG")
(process-coding-system (get-buffer-process (current-buffer)))

Note that (setenv "LANG" ENCODING) affects all subsequently created processes, which will inherit the LANG value; (set-buffer-process ... and (process-coding-system... are buffer specific, so they require that the related (shell) buffer is current when called.

For the most cases this should work

M-S-: (setenv "LANG" "en_US.UTF-8") 
M-x shell
M-S-: (set-buffer-process-coding-system 'utf-8-unix 'utf-8-unix)
echo Straße > foo

On the contrary, using (setenv "LANG" "en_US") (without UTF-8), would print in the console or the file weird characters.

Chances there are that the proper values (UTF-8 based) are already the default ones.

As regards Windows, the function set-buffer-process-coding-system still applies, but there is no LANG variable in Windows. Windows represents the encodings in terms of code page numbers. Code pages are often associated to several languages, for example most of the Western languages (though not US English) are encoded through code page 850. To learn about the system code page consult the registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls\CodePage\OEMCP

When in the language setting you change Windows default input language, you might end up changing this value, e.g. switching from US to UK English changes OEMCP value from 437 to 859. You can manually change this value and there is a code page equivalent to UTF-8, that is 65001. While 65001 (UTF-8) could be the standard in the future (like already is for Linux), given the relevant changes, you might break some non-unicode apps with CP 65001. Luckly, it is possible to easily set the code page in the Windows command prompt, rather than system wide, with the chcp command (change code page). Under the Emacs perspective this boils down to:

M-x shell
chcp 65001
dir 
type foo.txt

In practice, assuming you have an English Windows, the command dir and type should be able to correctly show respectively filenames and file content in Russian or Chinese.