What are the potential pitfalls of enabling lexical-binding for a buffer?

Question

This was inspired by the discussion of lexical-binding vs lexical-let in this question. As lexical-binding gives you the ability to have useful closures people may be used to in other languages like JavaScript why wouldn't you enabled it all the time?

Assuming backwards compatibility with older Emacsen is not a concern what pitfalls should you be looking for if enabling it in legacy code buffers?

Answer 1

A major pitfall is that the binding semantics for undefined variables—i.e. variables not defined with defvar and friends—change with lexical-binding: Without it, let binds everything dynamically, but with lexical-binding enabled undefined variables are bound lexically, and even elided completely if unused in the current lexical scope.

Old code sometimes relies on this. To avoid hard dependencies for optional features, it'd bind dynamic variables without requiring the corresponding library or declaring the variable itself:

(let ((cook-eggs-enabled t))
  (cook-my-meal))

If the cooking feature is optional, we don't want to force unnecessary dependencies onto the user, so we don't use (require 'cook) and instead rely on autoloading of the cook-my-meal function.

It's obvious for the human reader that cook-eggs-enabled is not a local variable, but still refers to some global dynamic variable from the cook library here. Without lexical-binding this code works as intended: cook-eggs-enabled is bound dynamically, whether defined or not.

With lexical-binding however, it breaks: cook-eggs-enabled is now bound lexically (and then optimized away, because it's not used), so the global dynamic variable cook-eggs-enabled is not ever touched at all and still nil by the time cook-my-meal is called, so we surprisingly won't have any eggs in our meal.

Luckily, these issues are very easy to spot: The byte compiler naturally warns about an unused lexical binding here.

The fix is simple: Either add a (require 'cook) (for features that aren't really optional anyway), or—to avoid hard dependencies—declare the variable as dynamic variable in your own code. There is a special defvar form for this:

(defvar cook-eggs-enabled)

This defines cook-eggs-enabled as dynamic variable, but doesn't affect the docstring, the load-history (and thus find-variable and friends) or anything else, except the binding nature of the variable.