10

Suppose I have a file named elisp-defvar-test.el containing:

;;; elisp-defvar-test.el ---  -*- lexical-binding: t -*- 

(defvar my-dynamic-var)

(defun f1 (x)
  "Should return X."
  (let ((my-dynamic-var x))
    (f2)))

(defun f2 ()
  "Returns the current value of `my-dynamic-var'."
  my-dynamic-var)

(provide 'elisp-dynamic-test)

;;; elisp-defvar-test.el ends here

I load this file and then go into the scratch buffer and run:

(setq lexical-binding t)
(f1 5)
(let ((my-dynamic-var 5))
  (f2))

(f1 5) returns 5 as expected, indicating that the body of f1 is treating my-dynamic-var as a dynamically scoped variable, as expected. However, the last form gives a void-variable error for my-dynamic-var, indicating that it is using lexical scoping for this variable. This seems at odds with the documentation for defvar, which says:

The defvar form also declares the variable as "special", so that it is always dynamically bound even if lexical-binding is t.

If I change the defvar form in the test file to supply an initial value, then the variable is always treated as dynamic, like the documentation says. Can anyone explain why the scoping of a variable is determined by whether or not defvar was supplied with an initial value when declaring that variable?

Here is the error backtrace, in case it matters:

Debugger entered--Lisp error: (void-variable my-dynamic-var)
  f2()
  (let ((my-dynamic-var 5)) (f2))
  (progn (let ((my-dynamic-var 5)) (f2)))
  eval((progn (let ((my-dynamic-var 5)) (f2))) t)
  elisp--eval-last-sexp(t)
  eval-last-sexp(t)
  eval-print-last-sexp(nil)
  funcall-interactively(eval-print-last-sexp nil)
  call-interactively(eval-print-last-sexp nil nil)
  command-execute(eval-print-last-sexp)
  • 4
    I think the discussion in bug#18059 is relevant. – Basil Apr 24 '18 at 10:57
  • Great question, and yes, please do see the discussion of bug #18059. – Drew Apr 24 '18 at 16:49
  • I see, so it looks like the documentation will be updated to address this in Emacs 26. – Ryan Thompson Apr 24 '18 at 17:51
8

Why the two are treated differently is mostly "because that's what we needed". More specifically, the single-argument form of defvar appeared a long time ago, but later than the other and was basically a "hack" to silence compiler warnings: at execution time it had no effect at all, so as an "accident" it meant that the silencing behavior of (defvar FOO) only applied to the current file (since the compiler had no way to know that such a defvar had been executed in some other file).

When lexical-binding was introduced in Emacs-24, we decided to re-use this (defvar FOO) form, but that implies that it now does have an effect.

Partly to preserve the previous "only affects the current file" behavior, but more importantly to allow a library to use toto as a dynamically-scoped var without preventing other libraries from using toto as a lexically scoped var (usually the package-prefix naming convention avoids those conflicts, but it's not used everywhere sadly), the new behavior of (defvar FOO) was defined to apply only to the current file, and was even refined so it only applies to the current scope (e.g. if it appears within a function, it only affects uses of that var within that function).

Fundamentally, (defvar FOO VAL) and (defvar FOO) are just two "completely different" things. They just happen to use the same keyword for historical reasons.

  • 1
    +1 for the answer. But Common Lisp's approach is clearer and better, IMHO. – Drew Apr 24 '18 at 16:48
  • @Drew: I mostly agree, but reusing the (defvar FOO) makes the new mode much more compatible with old code. Also, IIRC a problem with CommonLisp's solution is that it is pretty costly for a pure-interpreter like Elisp's (e.g. every time you evaluate a let you have to look inside its body in case there's a declare that affects some of the vars). – Stefan Apr 24 '18 at 21:20
  • Agreed on both counts. – Drew Apr 24 '18 at 23:05
4

Based on experimentation, I believe the issue is that (defvar VAR) with no init value only has an effect on the library(s) it appears in.

When I added (defvar my-dynamic-var) to the *scratch* buffer, the error no longer occurred.

I originally thought this was on account of evaluating that form, but I then noticed firstly that simply visiting the file with that form present was sufficient; and furthermore that merely adding (or removing) that form in the buffer, without evaluating it was enough to change what happened when evaluating (let ((my-dynamic-var 5)) (f2)) inside that same buffer with eval-last-sexp.

(I have no real understanding of what is happening here. I find the behaviour surprising, but I'm unacquainted with the details of how this functionality is implemented.)

I will add that this form of defvar (with no init value) prevents the byte compiler from complaining about uses of an externally-defined dynamic variable in the elisp file being compiled, but on its own it doesn't cause that variable to be boundp; so it's not strictly defining the variable. (Note that if the variable was boundp then this issue would not occur at all.)

In practice I suppose this is going to work out ok provided that you do include (defvar my-dynamic-var) in any lexical-binding library which uses your my-dynamic-var variable (which presumably would have a real definition elsewhere).


Edit:

Thanks to the pointer from @npostavs in the comments:

Both eval-last-sexp and eval-defun use eval-sexp-add-defvars in order to:

Prepend EXP with all the defvars that precede it in the buffer.

Specifically it locates all defvar, defconst, and defcustom instances. (Even when commented out, I notice.)

As this is searching the buffer at call-time, it explains how these forms can have an effect in the buffer even without being evaluated, and confirms that the form must appear in the same elisp file (and also earlier than the code being evaluated).

  • 2
    IIUC, bug#18059 confirms your exeriments. – Basil Apr 24 '18 at 10:55
  • 2
    Seems that eval-sexp-add-defvars checks for defvars in the buffer text. – npostavs Apr 24 '18 at 11:14
  • 1
    +1. Clearly this feature is not clear, or is not clearly presented to users. The doc fix for bug #18059 helps, but this is still something mysterious, if not fragile, for users. – Drew Apr 24 '18 at 16:51
0

I cannot reproduce this at all, evaluating the latter snippet works just fine here and returns 5 as expected. Are you sure that you're not evaluating my-dynamic-var on its own? That will throw an error because the variable is void, it hasn't been set to a value and it will only have one if you dynamically bound it to one.

  • 1
    Did you set lexical-binding non-nil before evaluating the forms? I get the behavior you describe with lexical-binding nil, but when I set it to non-nil, I get the void-variable error. – Ryan Thompson Apr 24 '18 at 6:52
  • Yes, I saved this to a separate file, reverted, checked that lexical-binding is set and evaluated the forms sequentially. – wasamasa Apr 24 '18 at 9:47
  • @wasamasa Reproduces for me, maybe you've accidentally given my-dynamic-var a top-level dynamic value in your current session? I think that could mark it permanently special. – npostavs Apr 24 '18 at 13:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.