How does Emacs see the generated autoloads files?

Question

I have some of my own Elisp code in my own directory. Like I have a file /tmp/foo/foo.el with this in it:

;;; foo.el -- Foo

;;;###autoload
(defun foo-hello ()
  (message "hello, world"))

(provide 'foo)

;;; foo.el ends here

I open Emacs and run M-x make-directory-autoloads RET /tmp/foo/foo-autoloads.el RET.

Then I add this to my ~/.emacs.d/init.el:

(add-to-list 'load-path "/tmp/foo/")
(foo-hello)

Then I start Emacs but I get this error:

eval-buffer: Symbol’s function definition is void: foo-hello

What did I miss? How do I make Emacs see my generate autoloads file and use it to automatically load the autoloads when I use them? Is there an extra step involved?

If there is an extra step involved, then how does Emacs automatically see the autoloads installed with package manager. Like I never did anything special to make Emacs recognize ~/.emacs.d/elpa/magit-20230402.1342/magit-autoloads.el. How does Emacs see that file automatically but it does not see my autoloads file?

Answer 1

There is no magic with files of autoload declarations -- just like any other *.el file, Emacs knows nothing about the contents of an *-autoloads.el file unless that file is loaded; so in order to make Emacs process your file you will need to load it explicitly.

I never did anything special to make Emacs recognize ~/.emacs.d/elpa/magit-20230402.1342/magit-autoloads.el. How does Emacs see that file automatically but it does not see my autoloads file?

Simply because the package manager in Emacs explicitly loaded that file as part of its package-initialize activities.

In 28.2 that looks like this:

(defun package--autoloads-file-name (pkg-desc)
  "Return the absolute name of the autoloads file, sans extension.
PKG-DESC is a `package-desc' object."
  (expand-file-name
   (format "%s-autoloads" (package-desc-name pkg-desc))
   (package-desc-dir pkg-desc)))

(defun package--activate-autoloads-and-load-path (pkg-desc)
  "Load the autoloads file and add package dir to `load-path'.
PKG-DESC is a `package-desc' object."
  (let* ((old-lp load-path)
         (pkg-dir (package-desc-dir pkg-desc))
         (pkg-dir-dir (file-name-as-directory pkg-dir)))
    (with-demoted-errors "Error loading autoloads: %s"
      (load (package--autoloads-file-name pkg-desc) nil t))
    (when (and (eq old-lp load-path)
               (not (or (member pkg-dir load-path)
                        (member pkg-dir-dir load-path))))
      ;; Old packages don't add themselves to the `load-path', so we have to
      ;; do it ourselves.
      (push pkg-dir load-path))))

ELPA packages follow conventions, and so such things can be done automatically. Emacs can't make any guesses about your custom code though, so you need to do it yourself.

Answer 2

I am not sure how Emacs usually does this, but you should simply require the autoloads before calling (foo-hello), using (require 'foo-autoloads).

Now instead of loading the full foo 'package' directly, Emacs initially only loads the autoload definitions, and then of course loads foo only when calling (foo-hello).

Answer 3

In your "foo-autoloads.el" generated by make-directory-autoloads, you will see the line:

(autoload 'foo-hello "foo" nil nil nil)

Why not just write this one line directly to init.el? (without even generating "foo-autoloads.el"). Since your "foo.el" is ready to be auto-loaded, it may be sufficient to write a very general autoloading configuration (autoload 'foo-hello "foo"), without 3 nil parameters, in init.el unless you want to autoload everything (multiple files besides "foo.el") under your "foo" directory at once.

Answer 4

I've written tons of Emacs packages and it was a nightmare to ensure that the generated autoload:s were up to date. Also, I didn't want to install the packages since I would end up having two copies of each source file, one in the source location and one in .emacs.d/elpa.

Instead, I came up with the following code that sits in my init file. It allows me to point to the source file and let Emacs synthesise autoload:s from the ###autoload constructs.

;; ----------------------------------------
;; Silence messages.
;;

(defvar my--silence-messages nil)


(defadvice message (around my-message activate)
  (unless my--silence-messages
    ad-do-it))


(defmacro my-with-silent-messages (&rest body)
  "Like `progn' but silence messages emitted by talky functions."
  `(let ((my--silence-messages t))
     ,@body))


;; ----------------------------------------
;; On the fly autoloads.
;;
;; Apply the ";;;###autoload" construct of a file. Mainly used for
;; packages written by me to avoid having to install them using the
;; package manager, while still taking advantage of the autoload
;; annotations.

(defun my-autoload-onthefly (file &optional dir add-to-load-path)
  "Create autoloads on the fly for FILE.

If DIR is specified, FILE is expected to be relative it.

Add DIR to `load-path'. If ADD-TO-LOAD-PATH is non-nil, add the
full path of FILE to `load-path'."
  (require 'autoload)
  (with-temp-buffer
    (unless dir
      (setq dir (file-name-directory file)))
    (add-to-list 'load-path dir :append)
    (let ((path (concat dir file)))
      (when (or add-to-load-path
                ;; For some reason, `generate-file-name' does not
                ;; include any part of the path in older Emacs
                ;; versions.
                (version< emacs-version "24.0"))
        (add-to-list 'load-path (file-name-directory path)))
      (my-with-silent-messages
       (let ((buffer-file-name (concat dir "dummy.el")))
         (generate-file-autoloads path)))
      (eval-buffer))
    ;; Return content of temp buffer, to help debugging.
    (buffer-string)))

Also in my init file:

(my-autoload-onthefly "~/emacs/src/font-lock-studio/font-lock-studio.el")