9

My example is simplified:

(defvar wtf 10)

(defun f (wtf)
  (lambda ()
    (cl-incf wtf)))
(setq f (f 20))

(setq g (let ((wtf 30))
          (lambda ()
            (cl-incf wtf))))

(list (funcall f)
      (funcall g)
      wtf)
;; ==> (21 11 11)

I have 2 understandings conflicting with each other:

  • (defvar wtf 10) made wtf bound dynamically so the variable g would get a closure which captured nothing.
    However, this cannot explain what (f 20) returned.

  • (defun f (wtf) ...) created a lexical environment so (f 20) would return a closure which captured the whole lexical environment.
    However, this cannot explain what value the variable g was set.

Of course, neither of them is correct.
So what is the right understanding?


It seems that Emacs Lisp is a little different. I am confused.

enter image description here

6
  • 1
    From my experiments it looks like a symbol that is already marked as special goes only into the lexical environment of a closure if it is a function argument of an enclosing lambda that is bound to some symbol with the help of fset. The enclosing lambda can be defined and fset by a defun as in your case. But you are right, the behavior is wired.
    – Tobias
    Mar 16, 2023 at 8:46
  • BTW, you do not need the sharp quotes because lambda is self-quoting. Furthermore, for repeatability it is better to separate the marking of the symbol as special and the binding of the symbol to a value: (defvar wtf) (setq wtf 10).
    – Tobias
    Mar 16, 2023 at 8:48
  • @Tobias: I am new to Elisp and I don't know the difference it makes whether to add a #' or not. I used to write some CL code and I used #' there. It seems that Elisp doesn't refuse this notation, so I continue to do this way because it makes lambda expressions more prominent. Thanks for your suggestion; without #', the code may look cleaner.
    – shynur
    Mar 16, 2023 at 9:32
  • May I add a weird case that shows a flaw in phils answer?
    – Tobias
    Mar 16, 2023 at 13:40
  • @Tobias: Of course you're welcome. It's your right. But I won't look at it for a while, because I'm reading the manual to try to figure out the definitions of some terms (This helps me to understand your answers better).
    – shynur
    Mar 16, 2023 at 13:51

3 Answers 3

5

(defvar wtf 10) made wtf bound dynamically so the variable g would get a closure which captured nothing.

This is correct.

However, this cannot explain what (f 20) returned.

It can if, under lexical binding, function arguments are lexical regardless.

Did you try byte-compiling your code? You should see the likes of this for your function f:

Warning: Lexical argument shadows the dynamic variable wtf

Note that this means you never need to worry that your choice of function argument names might inadvertently bind some dynamic variable of the same name -- potentially causing other functions, called later in the stack, to obtain an unintended value for the variable -- which is liable to be much more confusing/broken behaviour than we get with things the way they are (with the shadowing confined to the lexical scope of the function with that argument).

5
  • 1
    This is almost my comment but you are missing the bit that the function must be fbound to a symbol. The function arguments of a not fbound lambda are also not contained in the lexical environment of a closure defined within the lambda if they have an outer dynamical binding. (Emacs 28.1) I tried to find the statements that make the behavior that wired in the C-code but was not successful.
    – Tobias
    Mar 16, 2023 at 13:08
  • 2
    Looks like the problem I described in my last comment only affects interactively executed functions: github.com/emacs-mirror/emacs/blob/…
    – Tobias
    Mar 16, 2023 at 14:15
  • 1
    Ah, true there does seem to be some weirdness happening there. I have no insight into that.
    – phils
    Mar 16, 2023 at 14:27
  • 4
    Please consider filing a bug report - at the very least for the doc - for that specific case (weirdness). This stuff needs to be communicated clearly to users. It sounds like you're well-placed to specify the problem clearly and propose a fix. ;-)
    – Drew
    Mar 16, 2023 at 17:44
  • SE seems to limit the length of a comment, so I posted an answer as my comment.
    – shynur
    Mar 17, 2023 at 12:33
4

A long comment instead of an answer. (SE limits the length of a comment.)


Hi phils, this is my comment on your answer.

potentially causing other functions, called later in the stack, to obtain an unintended value for the variable

I wrote an example. Is that what you're trying to say? :

;; -*- lexical-binding: t; -*-

(defvar wtf 0)

(defun global-val-of-wtf ()
  wtf)

(defun f (wtf)
  (global-val-of-wtf))

(f 1)
;; ==> 0

;; As contrast,
;; when under dynamical binding rule,
;; (f 1) ==> 1

If this example is indeed the case you wanted to point out, then I think I can understand the reason why:

under lexical binding, function arguments are lexical regardless.

Such a rule really makes sense. (Is it one of the things that makes Emacs Lisp different from Common Lisp?)

But I couldn't find a sentence about this feature in the manual.
All I found was a slightly related sentence in 12.10.5 Converting to Lexical Binding:

The byte-compiler will also issue a warning if you use a special variable as a function argument.

which implies that funtion arguments are lexically bound.


Added for Tobias:

May I add a weird case that shows a flaw in phils answer?

We all agree that this is a good feature. But there is an exception, as Tobias's comment mentioned:

;; -*- lexical-binding: t; -*-

(defvar wtf 0)

(setq ans1
      (funcall (funcall (lambda (wtf)
                          (lambda ()
                            wtf))
                        1))
      )
;; ==> 0

(defun f (wtf)
  (lambda ()
    wtf))

(setq ans2
      (funcall (f 1))
      )
;; ==> 1

(setq ans (list ans1 ans2))
;; ==> (0 1)

Save this lisp code to a file. With Emacs 28.1 we get the following strange behavior:

  • When we run eval-buffer on the file buffer ans is set to (0 1).
  • When we run emacs-lisp-byte-compile-and-load on the file buffer ans is set to (0 0).

The byte-compiler even issues a warning for f:

Lexical argument shadows the dynamic variable wtf

But the behavior we get for the byte compiled case is that one for dynamical binding.

4
  • "I wrote an example. Is that what you're trying to say?" -- yes, exactly so.
    – phils
    Mar 18, 2023 at 1:11
  • I can't comment much on the "different to Common Lisp" question, but the nearest relative of Emacs Lisp is MacLisp. Obviously elisp has evolved since, and the cl/cl-lib library does explicitly provide a heap of Common Lisp-isms; but those aside I don't believe there was ever any attempt to provide parity with CL specifically (RMS wasn't a fan of CL, as I understand it). They are both Lisps (and Lisp-2s) so they'll have lots in common by default, but there will be lots of differences too.
    – phils
    Mar 18, 2023 at 1:36
  • 1
    I believe you'll find dl.acm.org/doi/abs/10.1145/3386324 to be a worthwhile read.
    – phils
    Mar 18, 2023 at 1:50
  • 1
    Thanks. Yes this is the strange Emacs behavior which I was referring to. I modified the example a bit to make inspection of the results possible after running the byte-compiled stuff. The behavior is really strange and I really agree with @Drew that it is time for a bug-report. Interactive evaluation, evaluation of source code and evaluation of byte-compiled code should all give the same result. I hadn't the possibility to check native-compiled code. For all cases the result should be that one, expected for lexical binding if lexical-binding is file-locally set to t. I.e., ans = (1 1).
    – Tobias
    Mar 18, 2023 at 5:11
1

This answer is an extension of Phils answer.

Your question shows the potential danger of locally binding special-declared variables.

The convention of prefixing symbol names with library names normally protects us from accidentally using special-declared symbols for local bindings that should have lexical scope.

But there are exceptions like displayed-month and displayed-year from calendar.el.

One can ensure lexical binding with the help of uninterned symbols.

The principle is described through a simple documented example:

;; -*- lexical-binding: t -*-
(require 'cl)
;; reset:
(unintern 'wtf)
(unintern 'foo)
;; declare `wtf' as special:
(defvar wtf)
(setq wtf 0)

(defmacro f ()
  (let ((sym (make-symbol "wtf"))) ;; `make-symbol' returns an uninterned symbol with name `wtf'
    `(let ((,sym 1)) ;; We substitute here the uninterned symbol `wtf'.
       (lambda ()
     ,sym))))

(setq ans (funcall (f)))
;; => 1

It is clear that the definition of the macro f in the above example is cumbersome.

But a special lexlet macro as defined in the following can relieve us from that work.
First comes the library code defining the lexlet macro and below that the user code resembling the simplified introductory example.

The macro lexlet generates uninterned duplicates of all locally bound symbols, locally binds the uninterned symbols to their given values and replaces the locally bound interned symbols in the body with their uninterned counterparts.

The definition of the macro is kept simple. But there is a small caveat. One cannot use the function definition of the locally bound symbols.

;; -*- lexical-binding: t; -*-
;; Here starts the library code.

(defun lexlet--search (data symbol-alist)
  "Replace symbols according to SYMBOL-ALIST in DATA."
  (cond
   ((symbolp data)
    (or
     (alist-get data symbol-alist)
     data))
   ((vectorp data)
    (apply #'vector (seq-map #'lexlet--search data symbol-alist)))
   ((consp data)
    (cons (lexlet--search (car data) symbol-alist)
          (lexlet--search (cdr data) symbol-alist)))
   (t data)))
;; Test:
;; (lexlet--search '(lambda () wtf) '((wtf . var)))

(defmacro lexlet (bindings &rest body)
  "Like `let' but with lexical binding for all symbols in BINDINGS.
If a symbol has a local binding in BINDINGS its binding during
the execution of BODY is lexical even if the symbol is declared as special.

This macro requires `lexical-binding' set to t."
  (declare (indent 1)
           (debug ((&rest (symbol sexp)) body)))
  (let* ((symbol-alist (mapcar
                        (lambda (binding)
                          (let ((symbol (car binding)))
                            (cons
                             symbol
                             (make-symbol (symbol-name symbol)))))
                        bindings))
         (new-bindings (mapcar
                        (lambda (binding)
                          (let* ((old-symbol (car binding))
                                 (new-symbol (alist-get old-symbol symbol-alist)))
                            (cons new-symbol (cdr binding))))
                        bindings)))
    (append
     (list
      #'let
      new-bindings)
     (lexlet--search body symbol-alist))))

(provide 'lexlet)

With lexlet the user code of the introductory example looks friendlier:

(require 'lexlet)

(defvar wtf)
(setq wtf 0)

(defun f ()
  (lexlet ((wtf 1))
    (lambda ()
      wtf)))

(setq ans (funcall (f)))
;; => 1

It is important that the macro extends the lexical environment created by an outer let.
For an example eval with an environment given as argument lexical does not extend an outer lexical environment and is therefore only of limited use for our purpose.

But, lexlet is okay in that regard:

;; -*- lexical-binding: t -*-
(require 'lexlet)
;; reset:
(unintern 'wtf)
(unintern 'lexbound)

(defvar wtf)
(setq wft 0)

(defun f ()
  (let ((lexbound 1))
    (lexlet ((wtf 2))
      (lambda ()
    (list
     lexbound wtf)))))

(setq ans (funcall (f)))
;; => (1 2)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.