4

I have two simple text files on my D: drive, as shown below.

D:\我.txt

I

D:\你.txt

You

After I execute M-x diff and type the correct filenames (Yes, I can input Chinese and filename autocomplete works fine for Chinese), the output give the following.

diff -u "d:/我.txt" "d:/你.txt"
diff: "d:/我.txt": No such file or directory
diff: "d:/ä½ .txt": No such file or directory

Diff finished (diff error).  Wed Jan  4 02:13:20 2017

I have MSYS2 installed with diff residing on D:\msys32\usr\bin\diff.exe.

I run Emacs 25.1.1 on Windows 7 (en-US), so the w32-unicode-filenames is set to t, which means the native Windows Unicode APIs is used when passing file names to the OS. This is good, Unicode file names display correctly in dired-mode, and autocomplete for Unicode file names also works seamlessly. So I suspect that Unicode file names only cause problem when creating subprocesses with Unicode arguments.

To verify that, I execute M-x shell and do the following.

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

$ cd
cd
d:\

$ type 我.txt
type 我.txt
The system cannot find the file specified.

$ type 你.txt
type 你.txt
The system cannot find the file specified.
Error occurred while processing: \344\275.
The system cannot find the file specified.
Error occurred while processing: .txt.

$ where cat
where cat
D:\msys32\usr\bin\cat.exe

$ cat 我.txt
cat 我.txt
cat: ''$'\303\246\313\206\342\200\230''.txt': No such file or directory

$ cat 你.txt
cat 你.txt
cat: ''$'\303\244\302\275\302\240''.txt': No such file or directory

Autocomplete for Unicode file name also ceases to work in shell-mode.

And I also try similar commands in eshell-mode invoked by M-x eshell.

Welcome to the Emacs shell

d:/ $ which which
eshell/which is a compiled Lisp function in ‘esh-cmd.el’
d:/ $ which cat
eshell/cat is a compiled Lisp function in ‘em-unix.el’
d:/ $ cat 我.txt
I
d:/ $ cat 你.txt
You
d:/ $ which ls
eshell/ls is a compiled Lisp function in ‘em-ls.el’
d:/ $ ls 我.txt 
我.txt
d:/ $ ls 你.txt 
你.txt
d:/ $ which ls.exe
d:/msys32/usr/bin/ls.exe
d:/ $ ls.exe 我.txt 
/usr/bin/ls: cannot access ''$'\303\246\313\206\342\200\230''.txt': No such file or directory
d:/ $ ls.exe 你.txt 
/usr/bin/ls: cannot access ''$'\303\244\302\275\302\240''.txt': No such file or directory

It almost confirms to me the issue is about the subprocess encoding. Calling internal Elisp functions with Unicode arguments works fine but calling external executables with Unicode arguments doesn't work. I also experiment with various encoding for subprocess. I set them in my init file like this.

(setq default-process-coding-system '(raw-text-dos . raw-text-unix))

But to no avail, none of my attempts work.

  • 3
    I recommend you M-x report-emacs-bug to send this question to someone who actually knows what's going on. Handling of non-ASCII in Windows is fairly convoluted; starting subprocesses in Windows is fairly convoluted; and the combination of the two, well you can guess. – Stefan Jan 4 '17 at 8:06
  • This comment in w32.c may be relevant. – npostavs Jun 30 '17 at 22:04
0

This is not an Emacs issue per se. It has to do with the command interpreter default code page. You need to change the "shell" code page to utf8 with chcp 65001. By default it sets the code page to Latin-1 or CP850 (which I believe are the same).

Updated answer after doing some more research

It turns out Emacs has a history of problems with Chinese fonts. There's a mode called chinese-fonts-setup here I installed it and that fixed the problem in my machine (I'm running GNU Emacs 24.5.1 (i686-pc-mingw32) on Windows 8.1). BTW install instructions are in Chinese, had to Google Translate + guess

  • No, this has nothing to do with code page. The shell thing is just to demonstrate the idea that subprocess causes problem. In the very first problem, I want to use the M-x diff in Emacs, which has nothing to with command prompt and calls diff.exe directly via Windows API. – Lei Zhao Jan 3 '17 at 20:38
  • Also, I don't believe Emacs' own eshell has anything to do with code page. And the real command prompt can deal Chinese file name without any problem. Only the command prompt buffered through Emacs' shell-mode has difficulties with Unicode file name. – Lei Zhao Jan 3 '17 at 20:54
  • I see your point, I have default-process-coding-system to '(utf-8-dos . utf-8-dos) and still can't get the chinese character. Will try some true type fonts and come back to you. – yorodm Jan 3 '17 at 21:01
  • First, thank you for doing research and informing me a good package that aligns Chinese and Western character in a monospace fashion. But this is not related to my question. My question is primarily concerned with encoding/decoding filename but not with font selection. Font and encoding are related but different concepts. – Lei Zhao Jan 4 '17 at 7:37
  • As pointed in my original question, I have no problem displaying or editing Chinese in Emacs. My problem is when Emacs calls a subprocess with Chinese (Unicode in a general case) file name arguments, the subprocess doesn't work correctly. This has nothing to do with fonts per se but with the underlying encoding/decoding procedure. – Lei Zhao Jan 4 '17 at 7:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.