How does Emacs manage to start instantaneously with many el-files?

Question

As every Emacs'er will know, I'm currently suffering from my extensive dot-Emacs configuration. All my packages are within the containers of use-package, and I bytecompiled all of my .el-files. Even with that, Emacs starts in 6.4 seconds, and then loads the rest of the packages (around 40 of them) afterwards.

I was thinking about another ways to solve the long startup time, then I noticed something. The default Emacs (without user configuration) uses many .el libraries, which are included with every Emacs. They're located in \shares\emacs\version number\lisp\.

Even with many lisp files, it manages to get started in one second. When I was inspecting the files of many packages that are included with the default Emacs, I found nothing extraordinairy that could explain why Emacs manage to start within one second. Could anyone tell me how Emacs manages that, even with thousands .el-files?

Answer 1

Could anyone tell me how Emacs manages that, even with thousands .el-files?

Emacs "manages" that by not loading at startup thereby not holding up the loading of the core application. This in turn as the effect of returning keyboard control to the user faster.

But when is it loaded? On first use of that function, mode, or feature.

Doesn't it slow down? Yes, on first use. That's the trade-off. Do you want to slow-down at emacs startup time or on first use.

Is it noticeable? Loading at startup time appears to take longer as other core libraries are also loaded. But on first use it feels faster as only that subset feature is loaded.

Then why would anyone pick load on startup? Because some do not mind waiting to load all frequently used libraries on startup so once loaded all operations perform snappily thereafter.

How can I choose? As Drew and others have pointed out in their responses to this very question, you can use autoload and similar tricks to control. But the most important consideration should be your usage pattern. If you happen to use emacs like vi, constantly opening and closing, yeah startup time becomes painfully obvious. But on the other hand, if you use emacs running all the time, startup time of 1 second or 1 minute won't be as noticeable or important enough to care.

Note you can use batch mode or Zile for instant startup while testing, running, or otherwise using emacs like vi.

My preference is to load on startup so any errors are caught upfront. I prefer not to have to deal with any load errors while in the midst of a workday when I have countless buffers, modes, and compile states active along with several remote locations managed by TRAMP. Debugging autoload errors in such conditions is not very pleasant.

Answer 2

Many of the included libraries are not loaded when you start up.

Some commands etc. are autoloaded, which means that Emacs recognizes them and knows how to load them. When you try to use a command that is autoloaded, Emacs then loads the library that defines it, if it has not yet been loaded.

You can create your own autoloads, whether for your own commands or commands in libraries you did not write. See the Elisp manual, node Autoload.

Answer 3

In addition to the other answers (which explain how the majority of libraries are actually only loaded on demand), there is also the matter of the pre-loading of many core elisp libraries within the emacs executable itself, which provides an illusion of it loading a lot of things incredibly quickly.

This is achieved by running a so-called "bare" version of Emacs (which is what was actually compiled, and which is fully-functional, but contains only the elisp interpreter and other core functionality written in C), and telling that to load all the elisp libraries which ought to be pre-loaded, before finally "dumping" out the actual emacs binary with those libraries built in.

This mechanism is detailed in the elisp manual:
C-hig (elisp) Building Emacs RET

If you have compiled Emacs yourself then you can experiment with this process, and can even dump alternative versions of the final executable if you wish (not generally recommended, but the facility is there).

The compiled temacs binary can be found in the src directory, and you can compare the difference in start times by running each version like so:

$ time ./temacs -l loadup --batch
$ time ./emacs --batch

On my system the former takes ~4 seconds (during which 111 elisp libraries are loaded), while the latter takes ~0.02 seconds.