Deep eager macroexpansion

Question

This works:

(require 'clojure-mode)
(when (member 'clojure-mode my-packages)
  (define-clojure-indent
    (-> 1)
    (->> 1)))

This gives the error Wrong type argument: listp, 1:

(when (member 'clojure-mode my-packages)
  (require 'clojure-mode)
  (define-clojure-indent
    (-> 1)
    (->> 1)))

Here's what I think is happening in the second version:

1. define-clojure-indent is a macro defined by clojure-mode, which is not loaded by default. So define-clojure-indent is only defined after (require 'clojure-mode) is evaluated.
2. Since both the require and the define-clojure-indent call are in the same top-level form, all macroexpansion happens before any evaluation happens. This appears to be similar to the Gilardi scenario from Clojure.
3. Both -> and ->> are macros defined by dash.el, which I also had loaded. They have the behavior that both of the forms (-> 1) and (->> 1) expand to just 1.

4. It appears that since Emacs does its macroexpansion on the define-clojure-indent form before it has loaded clojure-mode, it does not yet know that define-clojure-indent is actually a macro. Therefore, it goes ahead and macroexpands recursively, transforming

   (define-clojure-indent
     (-> 1)
     (->> 1))
   

   to

   (define-clojure-indent
     1
     1)
   

   by interpreting -> and ->> as macros instead of simply unevaluated symbols.

5. When clojure-mode is actually loaded and define-clojure-indent is invoked as a macro, it throws an error because it expects its arguments to be lists, not integers.

Whereas in the first version, since the require and define-clojure-indent are in different top-level forms, by the time Emacs gets to macroexpanding the define-clojure-indent form, it already knows that define-clojure-indent is a macro, and doesn't try to do macroexpansion on its arguments. So (-> 1) and (->> 1) are passed to define-clojure-indent as lists of symbols, as I would expect, and everything works.

So here is my question. If Emacs doesn't know whether or not something is a macro, why would it do macroexpansion on the arguments? In this situation, it clearly produces unintended behavior. As far as I know, this would never happen in the Clojure macro system. And I can't think of any situation in which the current functionality would be desired.

Surely I am missing something here. Is there an elegant way to avoid this problem? Or do I have a fundamental misunderstanding?

(P.S. I am aware that I could use eval-after-load. The same problem still occurs, but I used require in the example because it was simpler. Full code—obviously not yet working, because of this and other errors—available here.)

Answer 1

@Stefan explained why you see what you see: Your require statement is not at top level of the file.

A solution to the byte-compiler not evaluating your (non top-level) require statement is to force its evaluation, using (eval-and-compile (require 'clojure-mode)).

The fact that require is evaluated by the byte-compiler when it is at top level is pointed out in the Elisp manual, node Named Features (which I found by using i require in the manual). There you see this:

When require is used at top level in a file, it takes effect when you byte-compile that file (*note Byte Compilation::) as well as when you load it. This is in case the required package contains macros that the byte compiler must know about. It also avoids byte compiler warnings for functions and variables defined in the file loaded with require.

Answer 2

(require <foo>) is treated specially when it's at top-level (it causes the compiler to require the file also, rather than only do the require at run-time).

If the require is not performed at compilation time, then your code can't be compiled properly since the define-clojure-indent macro won't be known during compilation, leading to mis-compilation, indeed.

If the byte-compiler never macro-expanded calls in args to something that is not known to be a function, then lots of existing code would break: all the warnings like function <foo> is not known to be defined would turn into compilation errors. We could do that, but lots of existing files would then fail to compile.