What does a byte compiled Emacs file contains and how is it executed?

Question

I didn't know byte compiled emacs files (.elc) were actually version-specific until Malabarba pointed that to me.

So this made me ask what do those files actually contain and how does their execution compare to the execution of normal .el files? What is the benefit of compiling an emacs file and how does emacs takes advantage of this?

Answer 1

The Emacs manual node on byte compilation explains that elisp

... has a compiler that translates functions written in Lisp into a special representation called byte-code that can be executed more efficiently. The compiler replaces Lisp function definitions with byte-code.

These byte-codes are basically numeric codes that are not human readable (byte-car, for example, is 64, and byte-forward-char is 123). There is, however, a disassembler provided to "satisfy a cat-like curiosity" if you want to see how the sausage gets made.

Here's what the all-knowing Wikipedia page on Bytecode has to say:

Bytecode... is a form of instruction set designed for efficient execution by a software interpreter. Unlike human-readable source code, bytecodes are compact numeric codes, constants, and references... which encode the result of parsing and semantic analysis of things like type, scope, and nesting depths of program objects. They therefore allow much better performance than direct interpretation of source code.

The manual gives a simple example of the speedup from using a byte-compiled versus interpreted function -- 2 to 3 times in their looping example.

New bytecodes get added to Emacs as new versions are released -- hence, new versions know about the old versions' bytecodes, but not the other way around. To see some of this in action, you can browse through the bytecomp library (or M-x find-library RET bytecomp) to take a look at what's in it. Note that, at various points in the code, it defines new bytecodes for newer versions of Emacs (look for comments around byte-defop lines).

For an example of someone digging into the byte-code to understand why he couldn't seem to advise the narrowing/widening functions, see limits of advice.

NOTE: as @Malabarba's comment points out, there's a great post over at Null Program on Emacs Byte-Code Internals.