11

I would like to build a very minimal variant of Emacs trunk for unit-testing my Emacs Lisp package. The build needs no GUI, no image support, etc. It should should essentially just be a minimal Emacs Lisp interpreter with the core Emacs Lisp libraries, and it should build fast, ideally in less than five minutes.

Currently, I'm passing --with-x-toolkit=no --without-x --without-all to ./configure. After it has finished, it tells me that all Emacs features are disabled, but unfortunately the build still takes almost ten minutes.

I understand that it might be impossible to get Emacs build faster, but what makes me wonder is that with the very same flags Emacs 24.5 builds in just about two minutes.

What is the reason for this gross difference, and can I get Emacs trunk to build as fast as Emacs 24.5?

And, in a related question, how to I get Emacs to build quietly? Currently almost 80% of my unit test output is Emacs building. Ideally, I'd like to have make install printing no output at all.

Improve this question
5
  • Are you planning doing this on some sort of CI platform? If not what kind of computer are you using? Obviously the build speed is going to be very dependent on your processor, but for me ./configure --with... && make -j (number of cores * 1.5) finishes in 30 seconds. If you are running on a local machine, make sure to use the -j argument to make. Is there a good reason for you to do make install? This will add a little bit of time you could avoid if you just run emacs from the src directory.
    – Jordon Biondo
    Commented Jul 17, 2015 at 15:16
  • It's Travis CI, but I don't see why that matters? It's the gross difference between two different Emacs versions in the same machine that I'd like to have explained. IOW why does trunk that five times longer to build on the same system?
    – user227
    Commented Jul 17, 2015 at 15:21
  • Building from a repository needs to create certain files, which are already present in the distributed tarballs.
    – politza
    Commented Jul 17, 2015 at 21:38
  • @politza What files? I know that I need to run ./autogen.sh to generate configure, but that's a matter of seconds, not minutes.
    – user227
    Commented Jul 18, 2015 at 8:34
  • 2
    @lunaryom You have three separate questions here. 1: how to build emacs fast, 2: why does emacs 25 build slower than emacs 24.5 and 3: how to make make install run silently. So please split these into 3 questions so they can be tracked separately and edit this accordingly to stick with one question.
    – rocky
    Commented Oct 17, 2015 at 11:51

2 Answers 2

Reset to default
9

The reason 24.5 builds so quickly for you is that the .elc files are actually distributed in the tarball, see make-dist. When building from git, the majority of the time is spent compiling the .el files into .elc. By optimizing the C code, the Lisp compilation can go faster, but it still takes a long time. Compare build times using your original settings (~14 vs ~1 minutes) with a build using CFLAGS='-O2 -march=native' (~9 vs ~1.5 minutes).

Also, cloning from git takes about a minute, while downloading and unpacking the tarball takes about 5 seconds. Compare build times between versions when downloading the git archive from github (~5, ~6, ~8 minutes for v24.5, master, and emacs-25, respectively. As you can see, when not using a distribution tarball, all the build times are at least the same order of magnitude. (not sure why emacs-25 was slower than master, could be random variation, or some obsolete code was removed in master?).

And, in a related question, how to I get Emacs to build quietly? Currently almost 80% of my unit test output is Emacs building. Ideally, I'd like to have make install printing no output at all.

You can always redirect the output to /dev/null. For my experiments I piped the make install output to grep -E '^(make|[A-Z])' to cut down on the output (the Travis CI javascript that formats the log on the web was having trouble with the full output).

can I get Emacs trunk to build as fast as Emacs 24.5?

No (or yes in the sense that you can get Emacs 24.5 to build (almost) as slow as Emacs trunk :p). But what you can do is save the the Emacs build, and just download that cached result for unit-testing. I have implemented this in the upload branch of my emacs-travis fork, here is an example use by yasnippet: Emacs install time is ~2.5 seconds.

Improve this answer
3

Here are various suggestions.

  1. No elc files.

As stated below, compiling all of the lisp files accounts for at least 10% of the time. One way to disable that is to edit the loaddefs target in file lisp/Makefile and change that to: 

    $(lisp)/loaddefs.el: $(LOADDEFS)
          true
  1. No compiler optimization or debugger symbol tables

Currently, I'm passing --with-x-toolkit=no --without-x --without-all to ./configure.

I was able to reduce C compilation time to 1/4 the time (from a little less than minute to 16 seconds) in the src just changing the default compile flags. The default CFLAGS I got had been: -g -O3.

So use instead CFLAGS=''.

  1. Don't run make install, but just run the built emacs from inside the src directory.

As stated below another 10% of the time is in building docs. Although I didn't time it, there is no doubt a lot of time copying files and compressing elisp files in make install. So don't. If you want to redo the timing charts, run remake --profile.

The above observations are based on below....

First step is to understand where the time is spent to figure out how to reduce it. Fortunately for something like Emacs, I recently wrote (or rather extended) a tool to help you find out. I added a --profile option to remake (a fork of GNU make) which will tell you how much time is spent in specific targets.

I tried that building a recent snapshot and yes it takes about 10 minutes. If you don't have remake installed, I have a gist of the profiling information you can use of my run. I use kcachegrind to display the information but there may be other tools out there for visualization tools. There is a png in the gist which is a screenshot of the run.

Now to the details...

In my run, about 20% of the time is spent in building lisp and info files which you don't really need to do. Actually a little more is spent in the lisp files than in the info files. You could probably change the Makefile to skip that.

It might be interesting to compare with emacs 24. My guess is that the sizes of both of these has grown proportionally.

If there is interest (which you can show by upvotes), I'll suggest specific hacks to the Makefile. However this by itself should be enough for someone motivated to work from.

Improve this answer
6
  • "20% of the time is spent in building lisp" - I cloned @lunaryorn's emacs-travis repo it looks like make lisp is taking around %60 percent for Emacs 25: travis-ci.org/npostavs/emacs-travis/builds/91107858. And a large chunk of make src is also compiling lisp, so I wonder how to reconcile these measurements. In Emacs 24 it looks like it only compiles cc-*.el files during make lisp, is it a bug?
    – npostavs
    Commented Nov 14, 2015 at 13:57
  • Also, this may be on optical illusion, but it looks to me that your profiling picture only adds up to about 50% total.
    – npostavs
    Commented Nov 14, 2015 at 14:56
  • @npostavs remake only profiles the time of the target, it doesn't include itself. It is possible and likely that with all of the files and directories under lisp, there is considerable time spent in "remake"/"make" in calculating what to remake. Also, even though 'remake" is a fork of make, so what they do is similar, for closer comparisons you should compare times of remake's profiling output with "remake" without profiling, not "make". And also with the same remake/make version. Lastly, although one can quibble with the %'s and so on, the overall suggestions seem to apply using your data.
    – rocky
    Commented Nov 14, 2015 at 17:35
  • Also, my measurement was using CFLAGS='' which makes C compilation faster and lisp compilation slower. As it turns out, using CFLAGS='-O2 -march=native' is faster overall for Emacs 25, though slower for Emacs 24.5: travis-ci.org/npostavs/emacs-travis/builds/91142923
    – npostavs
    Commented Nov 14, 2015 at 19:53
  • @npostavs You observe: setting CFLAGS "makes C compilation faster and lisp compilation slower." But if you are going to run a single test, I am not sure whether the overall time: some-sort-of-optimized C compile/build + LISP compile + LISP test run will be less than unoptimized C compile/build + no LISP compile + LISP test run.
    – rocky
    Commented Nov 14, 2015 at 21:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.