6

I have two simple text files on my D: drive, as shown below.

D:\我.txt

I

D:\你.txt

You

After I execute M-x diff and type the correct filenames (Yes, I can input Chinese and filename autocomplete works fine for Chinese), the output give the following.

diff -u "d:/我.txt" "d:/你.txt"
diff: "d:/我.txt": No such file or directory
diff: "d:/ä½ .txt": No such file or directory

Diff finished (diff error).  Wed Jan  4 02:13:20 2017

I have MSYS2 installed with diff residing on D:\msys32\usr\bin\diff.exe.

I run Emacs 25.1.1 on Windows 7 (en-US), so the w32-unicode-filenames is set to t, which means the native Windows Unicode APIs is used when passing file names to the OS. This is good, Unicode file names display correctly in dired-mode, and autocomplete for Unicode file names also works seamlessly. So I suspect that Unicode file names only cause problem when creating subprocesses with Unicode arguments.

To verify that, I execute M-x shell and do the following.

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

$ cd
cd
d:\

$ type 我.txt
type 我.txt
The system cannot find the file specified.

$ type 你.txt
type 你.txt
The system cannot find the file specified.
Error occurred while processing: \344\275.
The system cannot find the file specified.
Error occurred while processing: .txt.

$ where cat
where cat
D:\msys32\usr\bin\cat.exe

$ cat 我.txt
cat 我.txt
cat: ''$'\303\246\313\206\342\200\230''.txt': No such file or directory

$ cat 你.txt
cat 你.txt
cat: ''$'\303\244\302\275\302\240''.txt': No such file or directory

Autocomplete for Unicode file name also ceases to work in shell-mode.

And I also try similar commands in eshell-mode invoked by M-x eshell.

Welcome to the Emacs shell

d:/ $ which which
eshell/which is a compiled Lisp function in ‘esh-cmd.el’
d:/ $ which cat
eshell/cat is a compiled Lisp function in ‘em-unix.el’
d:/ $ cat 我.txt
I
d:/ $ cat 你.txt
You
d:/ $ which ls
eshell/ls is a compiled Lisp function in ‘em-ls.el’
d:/ $ ls 我.txt 
我.txt
d:/ $ ls 你.txt 
你.txt
d:/ $ which ls.exe
d:/msys32/usr/bin/ls.exe
d:/ $ ls.exe 我.txt 
/usr/bin/ls: cannot access ''$'\303\246\313\206\342\200\230''.txt': No such file or directory
d:/ $ ls.exe 你.txt 
/usr/bin/ls: cannot access ''$'\303\244\302\275\302\240''.txt': No such file or directory

It almost confirms to me the issue is about the subprocess encoding. Calling internal Elisp functions with Unicode arguments works fine but calling external executables with Unicode arguments doesn't work. I also experiment with various encoding for subprocess. I set them in my init file like this.

(setq default-process-coding-system '(raw-text-dos . raw-text-unix))

But to no avail, none of my attempts work.

3
  • 3
    I recommend you M-x report-emacs-bug to send this question to someone who actually knows what's going on. Handling of non-ASCII in Windows is fairly convoluted; starting subprocesses in Windows is fairly convoluted; and the combination of the two, well you can guess.
    – Stefan
    Jan 4, 2017 at 8:06
  • 1
    This comment in w32.c may be relevant.
    – npostavs
    Jun 30, 2017 at 22:04
  • @npostavs Are you involved in emacs Development? Because this made sense as long as the only thing windows had was conhost. OpenConsole has proper unicode support and the interaction with Emacs on Windows with the shell, assuming that it doesn't handle Unicode properly, is just arcane.
    – JohnDoe
    Jul 21, 2023 at 7:30

1 Answer 1

0

This is not an Emacs issue per se. It has to do with the command interpreter default code page. You need to change the "shell" code page to utf8 with chcp 65001. By default it sets the code page to Latin-1 or CP850 (which I believe are the same).

Updated answer after doing some more research

It turns out Emacs has a history of problems with Chinese fonts. There's a mode called chinese-fonts-setup here I installed it and that fixed the problem in my machine (I'm running GNU Emacs 24.5.1 (i686-pc-mingw32) on Windows 8.1). BTW install instructions are in Chinese, had to Google Translate + guess

8
  • No, this has nothing to do with code page. The shell thing is just to demonstrate the idea that subprocess causes problem. In the very first problem, I want to use the M-x diff in Emacs, which has nothing to with command prompt and calls diff.exe directly via Windows API.
    – Lei Zhao
    Jan 3, 2017 at 20:38
  • Also, I don't believe Emacs' own eshell has anything to do with code page. And the real command prompt can deal Chinese file name without any problem. Only the command prompt buffered through Emacs' shell-mode has difficulties with Unicode file name.
    – Lei Zhao
    Jan 3, 2017 at 20:54
  • I see your point, I have default-process-coding-system to '(utf-8-dos . utf-8-dos) and still can't get the chinese character. Will try some true type fonts and come back to you.
    – yorodm
    Jan 3, 2017 at 21:01
  • First, thank you for doing research and informing me a good package that aligns Chinese and Western character in a monospace fashion. But this is not related to my question. My question is primarily concerned with encoding/decoding filename but not with font selection. Font and encoding are related but different concepts.
    – Lei Zhao
    Jan 4, 2017 at 7:37
  • As pointed in my original question, I have no problem displaying or editing Chinese in Emacs. My problem is when Emacs calls a subprocess with Chinese (Unicode in a general case) file name arguments, the subprocess doesn't work correctly. This has nothing to do with fonts per se but with the underlying encoding/decoding procedure.
    – Lei Zhao
    Jan 4, 2017 at 7:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.