Unusual behavior when overriding `calculate-lisp-indent'

Question

I've modified calculate-lisp-indent as a better solution to the indentation questions here, here and here. By "modified" I mean I overrode the function with advice. After doing this when I restarted emacs and I tried to indent a lisp form, I'd get the error void-variable calculate-lisp-indent-last-sexp.

Here's where things get interesting. You may be inclined to think that's it's some problem with the code I added to calculate-lisp-indent. But after failing to figuring out what was wrong with the code I added (which by the way didn't even mention calculate-lisp-indent), I tried overriding calculate-lisp-indent with a function that has the exact same body as calculate-lisp-indent. What happened then? I got the same error. Wierd.

Ok, now I started looking at the code for lisp-indent-function and I could not find where the variable calculate-lisp-indent-last-sexp was set. It looked as if it was just unbound.

Check out the beginning of the lisp-indent-function. calculate-lisp-indent-last-sexp was used but not defined.

 (let ((normal-indent (current-column))
        (orig-point (point)))
    (goto-char (1+ (elt state 1)))
    (parse-partial-sexp (point) calculate-lisp-indent-last-sexp 0 t)
...

A small digression: go to say I'm not sure why parse-partial-sexp is doing here. Based on it's documentation I thought it just returned the list, but I don't see the value it's returning being stored anywhere. Does it have any (undocumented) side-effects?

So my first though was calculate-lisp-indent-last-sexp must be a global variable. But when I tried to describe it with describe-variable I couldn't find it.

Also super interesting. I noticed that after restarting emacs and getting the void variable error, if I re-evaluated the function I wrote to override calculate-lisp-indent then I don't get errors and my changes to calculate-lisp-indent take effect as I would expect.

;; If I re-evaluate this, then the indentation works and I get no errors.
(defun void~calculate-lisp-indent (&optional parse-start)
  "Has same body as `calculate-lisp-indent`."
  ...)

What is going on here?

EDIT: After starting emacs with the advice I performed a few checks.

(foundp #'void~calculate-lisp-indent) ; => t
(advice-member-p #'void~calculate-lisp-indent #'calculate-lisp-indent) ; => a non-nil value

This confirms the advice is a defined function and that it is advising calculate-lisp-indent. I don't understand why re-evaluating it makes a difference.

Answer 1

Maybe this will help with part of your question. It seems to be a multi-part question, i.e., multiple questions (it's generally better to ask one question at a time).

1. C-h f calculate-list-indent tells you the function is defined in lisp-mode.el. Clicking on lisp-mode.el in *Help* visits that library.

2. Searching for calculate-lisp-indent-last-sexp in lisp-mode.el shows that it is declared as a global/special variable, i.e., a variable that is scoped dynamically. That declaration is not a definition - there is no default value assigned, and there is no doc string.

   (defvar calculate-lisp-indent-last-sexp)
   

3. That explains why describe-variable describes it only to the extent of saying that its value is void. The help function doesn't try to say where it was declared (it could be declared in any number of places), and it doesn't show any doc for it because there is none.

4. Searching further for that variable shows that it is bound in function calculate-lisp-indent.

   (let ((indent-point (point))
         state
         ;; setting this to a number inhibits calling hook
         (desired-indent nil)
         (retry t)
         calculate-lisp-indent-last-sexp containing-sexp)
     ...)
   

5. When a dynamically scoped variable is bound with let it is bound for the duration of the calling code (the let). It is not bound lexically by the let. This means that it remains bound for the duration of any code invoked in the let body. That includes the invocation of lisp-indent-function: (funcall lisp-indent-function indent-point state).

Answer 2

I am currently discussing a potential patch for as an answer to this question.

The code that answers my original question looks like this:

(if (or
     (= (point) calculate-lisp-indent-last-sexp)

     (eq (char-after (1+ containing-sexp)) ?:)

     (eq (char-before containing-sexp) ?')

     (let ((quoted-p nil)
           (point nil)
           (positions (nreverse (butlast (elt state 9)))))
      (save-excursion
        (while (and positions (not quoted-p))
         (setq point (pop positions))
         (setq quoted-p
          (or (eq (char-before point) ?')
           (goto-char (1+ point))
           (looking-at-p "[[:space:]\n]*quote\\_>")))))
      quoted-p))
    ;; Containing sexp has nothing before this line
    ;; except the first element.  Indent under that element.
    nil
  ;; Skip the first element, find start of second (the first
  ;; argument of the function call) and indent under.
  (progn (forward-sexp 1)
   (parse-partial-sexp (point)
    calculate-lisp-indent-last-sexp
    0 t)))

This patch should be put in the body of calculate-lisp-indent (I don't show it in the body here for brevity). It should go where I denote <<CODE GOES HERE>>.

(defun calculate-lisp-indent ()
  ...
  (cond ((looking-at "\\s(")
         ;; First element of containing sexp is a list.
         ;; Indent under that list.
         )
   ((> (save-excursion (forward-line 1) (point))
     calculate-lisp-indent-last-sexp)
    ;; This is the first line to start within the containing sexp.
    ;; It's almost certainly a function call.
    <<CODE GOES HERE>>>
    (backward-prefix-chars))
   (t ...))
  ...)

Here is a demonstration of how this patch works:

;; Quoted list
'(hello hi ho
  ho hi)

;; Backquote is not touched
`(hello hi
        ho)

;; Nested quoted list
'((hello hi ho
   hello))

;; Explicitly quoted list
(quote (a b c
        d e f))

I will try to update this post in case some detail of this patch changes. You can also see this link to see my progress on this patch.