org mode publish to html uses cache

Question

The issue is that although I set my build-site.el with org-publish-cache nil, it still creates cache. Plus, it’s not really working as even when I change a file, it says nothing has changed and thus doesn’t compile the files.

What am I doing wrong? Or is it a bug or something?

;; Load the publishing system
(require 'org)
(require 'ox-publish)
(require 'ox-html)

;; Necessary for code blocks
(require 'package)
(setq package-user-dir (expand-file-name "./.packages"))
(setq package-archives '(("melpa" . "https://melpa.org/packages/")
                         ("elpa" . "https://elpa.gnu.org/packages/")))

;; Initialize the package system
(package-initialize)
(unless package-archive-contents
  (package-refresh-contents))


;; Install dependencies
(package-install 'htmlize)


;; Customize the HTML output
(setq org-html-validation-link nil            ; Don't show validation link
      org-html-head-include-scripts nil       ; Use own scripts
      org-html-head-include-default-style nil ; Use own styles
      org-publish-use-timestamps-flag nil ; Don’t use timestamps
      org-publish-cache nil
      ;;      org-publish-timestamp-directory "org-timestamps/"
      org-html-head "<link rel=\"stylesheet\" href=\"https://cdn.simplecss.org/simple.min.css\">"
      org-html-doctype "html5"
      org-html-html5-fancy t
      org-export-with-smart-quotes t
      )


;; Define the publishing project
(setq org-publish-project-alist
      '(
        ("notes"
         :base-directory "./../"
         :base-extension "org"
         :publishing-directory "./public"
     :exclude "./"
         :recursive t
         :publishing-function org-html-publish-to-html
         :org-html-preamble nil
     :with-author t           ;; Don't include author name
         :with-creator t            ;; Include Emacs and Org versions in footer
         :with-toc t                ;; Include a table of contents
         :section-numbers nil       ;; Don't include section numbers
         :time-stamp-file nil
     :auto-sitemap nil)
        ("static"
         :base-directory "./../"
         :base-extension "css\\|js\\|png\\|jpg\\|gif\\|pdf\\|mp3\\|ogg\\|swf"
         :publishing-directory "./public/assets"
     :exclude "./"
         :recursive t
         :publishing-function org-publish-attachment
         :time-stamp-file nil
     :auto-sitemap nil)
        ("cours"
         :components ("notes" "static"))))

(org-publish-all t)
(message "Build complete!")

Answer 1

IIUC, your setup looks like this¹:

.
├── notes
│   ├── file.org
│   └── image.png
├── other-notes
│   ├── other-file.org
│   └── other-image.png
└── site
    └── build-site.el

4 directories, 5 files

to begin with. Then, after cd ./site, you execute emacs --batch --load build-site.el, with the resulting structure looking like this:

.
├── notes
│   ├── file.org
│   └── image.png
├── other-notes
│   ├── other-file.org
│   └── other-image.png
└── site
    ├── build-site.el
    ├── packages
    │   ├── archives
    │   │   ├── elpa
    │   │   │   ├── archive-contents
    │   │   │   └── archive-contents.signed
    │   │   └── melpa
    │   │       └── archive-contents
    │   └── htmlize-20240915.1657
    │       ├── htmlize-autoloads.el
    │       ├── htmlize.el
    │       ├── htmlize.elc
    │       └── htmlize-pkg.el
    └── public
        ├── assets
        │   ├── notes
        │   │   └── image.png
        │   └── other-notes
        │       └── other-image.png
        ├── notes
        │   └── file.html
        └── other-notes
            └── other-file.html

15 directories, 16 files

I was able to accomplish this with the following build-site.el:

;; Load the publishing system
(require 'org)
(require 'ox-publish)
(require 'ox-html)

;; Necessary for code blocks
(require 'package)
(setq package-user-dir (expand-file-name "./packages"))
(setq package-archives '(("melpa" . "https://melpa.org/packages/")
                         ("elpa" . "https://elpa.gnu.org/packages/")))

;; Initialize the package system
(package-initialize)
(unless package-archive-contents
  (package-refresh-contents))


;; Install dependencies
(package-install 'htmlize)


;; Customize the HTML output
(setq org-html-validation-link nil            ; Don't show validation link
      org-html-head-include-scripts nil       ; Use own scripts
      org-html-head-include-default-style nil ; Use own styles
      org-publish-use-timestamps-flag nil ; Don’t use timestamps
      org-html-head "<link rel=\"stylesheet\" href=\"https://cdn.simplecss.org/simple.min.css\">"
      org-html-doctype "html5"
      org-html-html5-fancy t
      org-export-with-smart-quotes t
      )


;; Define the publishing project
(setq org-publish-project-alist
      '(
        ("notes"
         :base-directory "./.."
         :base-extension "org"
         :publishing-directory "./public"
         :exclude "site/.*"
         :recursive t
         :publishing-function org-html-publish-to-html
         :org-html-preamble nil
         :with-author t           ;; Don't include author name
         :with-creator t            ;; Include Emacs and Org versions in footer
         :with-toc t                ;; Include a table of contents
         :section-numbers nil       ;; Don't include section numbers
         :time-stamp-file nil
         :auto-sitemap nil)
        ("static"
         :base-directory "./.."
         :base-extension "css\\|js\\|png\\|jpg\\|gif\\|pdf\\|mp3\\|ogg\\|swf"
         :publishing-directory "./public/assets"
         :exclude "site/.*"
         :recursive t
         :publishing-function org-publish-attachment
         :time-stamp-file nil
         :auto-sitemap nil)
        ("cours"
         :components ("notes" "static"))))

(org-publish-all t)
(message "Build complete!")

Since the site directory is under the base directory, it is important to exclude everything in the site directory: nothing in there contributes to the final site. That's basically the only difference between your build-site.el and this one.

Here's the output of the Emacs run (--batch is preferable to -Q here, since you don't need to do anything with the GUI: see the documentation):

user@host:/path/to/base/directory/site$ emacs --batch --load build-site.el 
Importing package-keyring.gpg...
Importing package-keyring.gpg...done
Contacting host: melpa.org:443
Contacting host: melpa.org:443
Package refresh done
Contacting host: elpa.gnu.org:443
Contacting host: elpa.gnu.org:443
Package refresh done
Setting ‘package-selected-packages’ temporarily since "emacs -q" would overwrite customizations
Setting ‘package-selected-packages’ temporarily since "emacs -q" would overwrite customizations
Contacting host: melpa.org:443
Contacting host: melpa.org:443
Parsing tar file... 
Parsing tar file...done
Extracting... \ 
Extracting...done
  INFO     Scraping 2 files for loaddefs... 
  INFO     Scraping 2 files for loaddefs...done
  GEN      htmlize-autoloads.el
Checking /home/nick/src/publish/site/packages/htmlize-20240915.1657...
Compiling /home/nick/src/publish/site/packages/htmlize-20240915.1657/htmlize-autoloads.el...
Compiling /home/nick/src/publish/site/packages/htmlize-20240915.1657/htmlize-pkg.el...
Compiling /home/nick/src/publish/site/packages/htmlize-20240915.1657/htmlize.el...
Done (Total of 1 file compiled, 2 skipped)
Package ‘htmlize’ installed.
Resetting org-publish-cache
Publishing file /home/nick/src/publish/notes/file.org using ‘org-html-publish-to-html’
Publishing file /home/nick/src/publish/other-notes/other-file.org using ‘org-html-publish-to-html’
Resetting org-publish-cache
Publishing file /home/nick/src/publish/notes/image.png using ‘org-publish-attachment’
Publishing file /home/nick/src/publish/other-notes/other-image.png using ‘org-publish-attachment’
Build complete!
user@host:/path/to/base/directory/site$  

Subsequent runs differ only in that the htmlize package is not installed again:

user@host:/path/to/base/directory/site$ emacs --batch --load build-site.el 
Setting ‘package-selected-packages’ temporarily since "emacs -q" would overwrite customizations
‘htmlize’ is already installed
Resetting org-publish-cache
Publishing file /home/nick/src/publish/notes/file.org using ‘org-html-publish-to-html’
Publishing file /home/nick/src/publish/other-notes/other-file.org using ‘org-html-publish-to-html’
Resetting org-publish-cache
Publishing file /home/nick/src/publish/notes/image.png using ‘org-publish-attachment’
Publishing file /home/nick/src/publish/other-notes/other-image.png using ‘org-publish-attachment’
Build complete!
user@host:/path/to/base/directory/site$ 

The directory structure is exactly the same as before: nothing gets added or subtracted by the second run.

---

Footnote:

[1] I use site and package, rather than your .site and .package, but that should make no substantive difference in anything.

Answer 2

[EDIT]

Nope, I’m wrong. What’s written below is incorrect. Refer to the comments below this answer for the correct answer.

---

Thanks to @NickD’s successful attempt to compile, I was able to identify the issue.

It lies in the :base-directory and the :exclude variables. The latter variable inherits from the former.

In details:

;; Define the publishing project
(setq org-publish-project-alist
      '(
        ("notes"
         :base-directory "./.."
         :base-extension "org"
         :publishing-directory "./public"
         :exclude "./")

I thought that the :exclude variable would take the same base value as the :base-directory one. In other words, let’s say I have .site where I execute my commands and my based directory is a folder up, I set :base-directory to ./... I also set the :exclude variable to ./ to exclude the directory I am executing the script in, here .site. However, :exclude inherits from :base-directory, therefore actually excluding my base-directory to be published….

To solve the issue, all I need to refer the exclude directory from the :base-directory starting point. So this works:

(setq org-publish-project-alist
      '(
        ("notes"
         :base-directory "./.."
         :base-extension "org"
         :publishing-directory "./public"
         :exclude ".site/")