0

Update:

The problem is that systemd fails to start emacs.service, and kills it after a timeout period of 90s. Why systemd fails to start emacs.service, I don't know.

I have a problem which was raised in a previous question and not answered - emacsclient exits unexpectedly without any error

My question is how to debug this? I am using Emacs 29.4 (stable release) with several additional compilation flags, attached here in case any of them are relevant.

export CC="/usr/bin/gcc-13"; ./configure --with-native-compilation=aot --with-tree-sitter --with-gif --with-png --with-jpeg --with-rsvg --with-tiff --with-pgtk --with-imagemagick --with-x-toolkit=gtk3 --with-xwidgets

I ran emacsclient with GDB as was recommended in other posts about debugging a "crash." The exact command was

gdb -ex=r --args emacsclient -c

After a few minutes, emacsclient exits seemingly randomly, and GDB just prints

[Inferior 1 (process 11115) exited normally]

so it is just a sudden unexpected shutdown of emacsclient rather than a crash. The journalctl log (posted below) seems to indicate that the emacs server died and was restarted by systemd. Can I find information about what raised the signal to terminate the program?

I am using systemd via the command posted here - systemctl --user enable emacs

This appears to have generated the following files on my machine: /usr/local/lib/systemd/user/emacs.service, which contains

[Unit]
Description=Emacs text editor
Documentation=info:emacs man:emacs(1) https://gnu.org/software/emacs/

[Service]
Type=notify
ExecStart=/usr/local/bin/emacs --fg-daemon

# Emacs will exit with status 15 after having received SIGTERM, which
# is the default "KillSignal" value systemd uses to stop services.
SuccessExitStatus=15

# The location of the SSH auth socket varies by distribution, and some
# set it from PAM, so don't override by default.
# Environment=SSH_AUTH_SOCK=%t/keyring/ssh
Restart=on-failure

[Install]
WantedBy=default.target

and ~/.config/systemd/user/default.target.wants/emacs.service which is a symlink to the other one.

I have not been able to reproduce the problem without systemd (the problem has not yet occurred if I just put emacs --daemon in my /etc/profile).

While it is running, htop shows about 8 copies of gdb -ex=r --args emacsclient -c running under different process names, and one copy of /usr/local/bin/emacsclient -c.

The ps for the latter shows

ps 7364
    PID TTY      STAT   TIME COMMAND
   7364 pts/0    S+     0:00 /usr/local/bin/emacsclient -c

while it is running, before it crashes.

The crash happens within 1-2 minutes of emacsclient startup.

Here is the output of journalctl over a time interval of about 2 minutes, from the client start to the surprise shutdown.

Sep 07 12:21:09 patrick-ThinkPad-X1-Carbon-Gen-10 rtkit-daemon[2124]: Supervising 2 threads of 2 processes of 1 users.
Sep 07 12:21:09 patrick-ThinkPad-X1-Carbon-Gen-10 rtkit-daemon[2124]: Supervising 2 threads of 2 processes of 1 users.
Sep 07 12:21:21 patrick-ThinkPad-X1-Carbon-Gen-10 gnome-shell[2988]: meta_window_set_stack_position_no_sync: assertion 'window->stack_position >= 0' failed
Sep 07 12:22:17 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: start operation timed out. Terminating.
Sep 07 12:22:17 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: Failed with result 'timeout'.
Sep 07 12:22:17 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: Failed to start emacs.service - Emacs text editor.
Sep 07 12:22:17 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: Consumed 4.740s CPU time.
Sep 07 12:22:18 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: Scheduled restart job, restart counter is at 844.
Sep 07 12:22:18 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: Starting emacs.service - Emacs text editor...
Sep 07 12:22:18 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Due to a limitation in GTK 3, Emacs built with PGTK will simply exit when a
Sep 07 12:22:18 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: display connection is closed.  The problem is especially difficult to fix,
Sep 07 12:22:18 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: such that Emacs on Wayland with multiple displays is unlikely ever to be able
Sep 07 12:22:18 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: to survive disconnects.
Sep 07 12:22:18 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Loading /home/patrick/.emacs.d/config/appearance.el (source)...
Sep 07 12:22:18 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Loading /home/patrick/.emacs.d/config/appearance.el (source)...done
Sep 07 12:22:18 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Loading /home/patrick/.emacs.d/config/bedrock-misc.el (source)...
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Loading /home/patrick/.emacs.d/config/bedrock-misc.el (source)...done
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Loading /home/patrick/.emacs.d/extras/base.el (source)...
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Loading /home/patrick/.emacs.d/extras/base.el (source)...done
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Loading /home/patrick/.emacs.d/extras/dev.el (source)...
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Loading /home/patrick/.emacs.d/extras/dev.el (source)...done
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Loading /home/patrick/.emacs.d/config/org.el (source)...
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Clearing removed files...
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Clearing removed files...done
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Processing modified files...
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Processing modified files...done
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Loading /home/patrick/.emacs.d/config/org.el (source)...done
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Loading /home/patrick/.emacs.d/extras/researcher.el (source)...
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Clearing removed files...
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Clearing removed files...done
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Processing modified files...
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Processing modified files...done
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Loading /home/patrick/.emacs.d/extras/researcher.el (source)...done
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Loading /home/patrick/.emacs.d/config/langs.el (source)...
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]:  ■  Error (use-package): Cannot load agda2-mode
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Loading /home/patrick/.emacs.d/koka-mode.el (source)...
Sep 07 12:22:19 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Loading /home/patrick/.emacs.d/koka-mode.el (source)...done
Sep 07 12:22:21 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Loading /home/patrick/.emacs.d/config/langs.el (source)...done
Sep 07 12:22:21 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[179169]: Starting Emacs daemon.

To me what this journal log shows is that the emacs daemon /server shuts down (without any apparent error message or warning), and then is restarted with systemd; it fails to restart and times out; then it tries again and successfully restarts.

The crash is not caused by any kind of user event. I can just sit passively and watch it for a minute and it'll eventually crash of its own accord.

Update (September 7): I no longer think emacsclient has anything to do with it. I think that the emacs server is crashing, exiting or being killed by systemd once every minute or two, and emacsclient exits when its host server disappears.

Update: The problem is reproducible with -Q, so I don't think it's a problem with an add-on. Although, as I mentioned before, I did compile it with several flags.

Update: I realize now what's happening, there's a distinctive pattern to the crashes.

Sep 07 12:52:45 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: start operation timed out. Terminating.
Sep 07 12:52:45 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: Failed with result 'timeout'.
Sep 07 12:52:45 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: Failed to start emacs.service - Emacs text editor.
Sep 07 12:52:46 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: Scheduled restart job, restart counter is at 1.
Sep 07 12:52:46 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: Starting emacs.service - Emacs text editor...
Sep 07 12:52:46 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[183514]: Starting Emacs daemon.
Sep 07 12:54:16 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: start operation timed out. Terminating.
Sep 07 12:54:16 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: Failed with result 'timeout'.
Sep 07 12:54:16 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: Failed to start emacs.service - Emacs text editor.
Sep 07 12:54:16 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: Scheduled restart job, restart counter is at 2.
Sep 07 12:54:16 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: Starting emacs.service - Emacs text editor...
Sep 07 12:54:16 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[183623]: Starting Emacs daemon.
Sep 07 12:55:46 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: start operation timed out. Terminating.
Sep 07 12:55:46 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: Failed with result 'timeout'.
Sep 07 12:55:46 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: Failed to start emacs.service - Emacs text editor.
Sep 07 12:55:47 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: Scheduled restart job, restart counter is at 3.
Sep 07 12:55:47 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: Starting emacs.service - Emacs text editor...
Sep 07 12:55:47 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[183684]: Starting Emacs daemon.
Sep 07 12:57:17 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: start operation timed out. Terminating.
Sep 07 12:57:17 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: Failed with result 'timeout'.
Sep 07 12:57:17 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: Failed to start emacs.service - Emacs text editor.
Sep 07 12:57:17 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: Scheduled restart job, restart counter is at 4.
Sep 07 12:57:17 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: Starting emacs.service - Emacs text editor...
Sep 07 12:57:17 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[183831]: Starting Emacs daemon.
Sep 07 12:58:47 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: start operation timed out. Terminating.
Sep 07 12:58:47 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: Failed with result 'timeout'.
Sep 07 12:58:47 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: Failed to start emacs.service - Emacs text editor.
Sep 07 12:58:48 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: Scheduled restart job, restart counter is at 5.
Sep 07 12:58:48 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: Starting emacs.service - Emacs text editor...
Sep 07 12:58:48 patrick-ThinkPad-X1-Carbon-Gen-10 emacs[183875]: Starting Emacs daemon.
Sep 07 13:00:18 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: start operation timed out. Terminating.
Sep 07 13:00:18 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: Failed with result 'timeout'.
Sep 07 13:00:18 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: Failed to start emacs.service - Emacs text editor.

The crashes are regularly spaced at intervals of 90 seconds. So systemd thinks the program failed to start, and after the 90 second timeout period, systemd kills emacs. So the bug is that somehow Emacs is not successfully communicating to systemd that it is up and running, and the start operation times out.

Improve this question
7
  • The linked question was answered: the OP tried it on a stable release and it worked. I do understand that you are asking a different question, but I wanted to correct the record. Be that as it may: Are you on an unstable release? If so, have you tried with a stable release? What are you doing to trigger this exit? You are asking us to imagine a hypothetical situation (hypothetical, since we are not experiencing the problem) and come up with ways to debug it: there could be millions of reasons and thousands of possible debugging techniques. You need to narrow down the problem.
    – NickD
    Commented Aug 30 at 22:33
  • You asked two specific questions: 1. Am I on an unstable release, and 2. How is the exit triggers. The answers are 1. No and 2. not clear, seemingly random
    – Patrick Nicodemus
    Commented Aug 31 at 0:02
  • If you want more information to narrow down the problem please asks for specific additional information, I will supply whatever specific information is asked for. It is Emacs 29.
    – Patrick Nicodemus
    Commented Aug 31 at 0:06
  • 1
    How do you run the emacs server? Are you using systemd? How exactly does the unit file look? How do you run emacsclient? Are you running emacs or emacsclient under GDB? Did you do a ps of process 11115? If not, can you repeat the experiment and do so for the emacs process and the emacsclient process, so that when one or the other or both die, we have an idea of who died? Basically, if I wanted to reproduce the problem, how do I set up as similar an environment to yours as possible? And please add the information to your question as you go - not in comments.
    – NickD
    Commented Aug 31 at 1:26
  • @NickD I added some information to the post. Do you know how I can insert gdb into the emacs process, rather than emacsclient, given that it is being run by systemd
    – Patrick Nicodemus
    Commented Aug 31 at 3:15

1 Answer 1

Reset to default
2 
Sep 07 12:22:17 patrick-ThinkPad-X1-Carbon-Gen-10 systemd[2704]: emacs.service: start operation timed out. Terminating.

This is the key line in the log file. This is happening because your emacs.service file has Type=notify in it. You can read the systemd documentation (man systemd.exec) for the details, but basically this type of service is supposed to send a DBus notification to systemd once it has successfully started. Either use Type=simple instead or figure out why Emacs isn’t sending the notification.

Improve this answer
3
  • Thank you. I know nothing about systemd so it will take me some time to read and understand the documentation but this seems like a promising lead.
    – Patrick Nicodemus
    Commented Sep 7 at 17:15
  • 1
    +1 - but a couple of nits: 1) notify does not send a DBus notification, it uses sd_notify whose man page says: These functions send a single datagram with the state string as payload to the socket referenced in the $NOTIFY_SOCKET environment variable. 2) The man page of systemd.service mentions that Type=exec is generally preferred: Typically, Type=exec is the better choice, see below. But fixing it so that it uses notify should be fairly easy: the OP opened a bug and one of the Emacs maintainers noticed that the OP built Emacs without libsystemd support.
    – NickD
    Commented Sep 9 at 4:05
  • 1
    The problem was that emacs was compiled without being linked to libsystemd. At this moment AFAIK there is no flag you can pass to the "configure" script that will make sure emacs is compiled with libsystemd linked in, it just links it in if it finds it in your system, otherwise it silently compiles without it. (You can see if it found it by reading the output of the configure script.) On Ubuntu I solved the problem by installing the libsystemd-dev apt package and recompiling.
    – Patrick Nicodemus
    Commented Oct 14 at 20:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.