lexical-binding: How to create a let-bound function that recursively calls itself

Question

The custom function below my-jump-to works after being evaluated in a scratch buffer, but does not work when I stick it into a library with lexical-binding non-nil in the header. When byte-compiling, I get a warning: reference to free variable ‘the-fn’. When I restart Emacs after byte-compiling, and try to use the function, I get a *Backtrace*: Debugger entered--Lisp error: (void-variable the-fn). I tried using the trick of putting a #' in front of the (lambda ..., but that did not appear to have any effect. I tried using a backtick in front of the (lambda ..., and placing a comma before the-fn, but that didn't work either. Is it possible to make this work inside a library with lexical-binding non-nil in the header, and if so, then how can that be achieved?

(defun my-jump-to ()
"Jump to an imenu item using ido."
  (let* ((completion-ignore-case t)
         (collection (imenu--make-index-alist))
         (first-amended-collection nil)
         (the-fn
           (lambda (index-alist depth)
             (dolist (entry index-alist)
               (setq imenu-list--line-entries (append imenu-list--line-entries (list entry)))
               (push entry first-amended-collection)
               (when (imenu--subalist-p entry)
                 (funcall the-fn (cdr entry) (1+ depth))))))
         second-amended-collection)
    (funcall the-fn collection 0)
    (mapc (lambda (elt)
            (push (cons (car elt) (list elt)) second-amended-collection))
           first-amended-collection)
    (let ((choice (ido-completing-read "SELECT:  " second-amended-collection nil 'confirm)))
      (imenu (car (cdr (assoc choice second-amended-collection)))))))

Answer 1

Your problem is that the let-bound symbol the-fn is not yet available when you define the lambda.

Use letrec instead of let*. It first binds all variables and afterwards sets them.

Example:

(defun my-test ()
  (letrec ((the-fn (lambda (recursive)
                     (message "Calling the-fn with arg %s." recursive)
                     (when recursive
                       (funcall the-fn nil)))))
    (funcall the-fn t)))

This example is equivalent to:

(defun my-test ()
  (let* (the-fn)
    (setq the-fn (lambda (recursive)
                   (message "Calling the-fn with arg %s." recursive)
                   (when recursive
                     (funcall the-fn nil))))
    (funcall the-fn t)))

The same but with immediate byte-compilation and a call of my-test for testing:

(let ((lexical-binding t))
  ;; The actual defun but with byte-compilation:
  (defalias 'my-test
    (byte-compile
     (lambda ()
       (let* (the-fn)
         (setq the-fn (lambda (recursive)
                        (message "Calling the-fn with arg %s." recursive)
                        (when recursive
                          (funcall the-fn nil))))
         (funcall the-fn t))))))

(my-test)

The message output is:

Calling the-fn with arg t.
Calling the-fn with arg nil.