Merge history from multiple eshells

Question

Before switching to eshell I had zsh setup so that it would:

• Write out to the history file after every command
• Append to rather than overwrite the history file so when running multiple shells they would all get merged into one big history

Both zsh options are documented here (see APPEND_HISTORY and INC_APPEND_HISTORY).

This is super useful when combined with a large history size because you can open a new shell weeks after issuing a command and find it in your history (without these options a large history is useless since it will only contain the history of the most recently closed shell). It also means you can open new shells and have them immediately know your most recent commands in other shells. Is there any way to setup eshell with this behavior? The first bullet seems easy enough but appending looks trickier...

Answer 1

Disclaimer: I don't use eshell, so take this with a grain of salt.

eshell appears to call eshell-write-history to write history, which takes an optional argument append which defaults to nil. This argument seems to be unused in eshell presently, but does appear to work (it passes the argument through to write-region, which does properly append).

There are a couple of options here.

1. (setq eshell-save-history-on-exit nil) and call eshell-write-history yourself
2. Redefine eshell-write-history to satisfy your requirement.

Personally, I'd go with 1.

As an example:

(setq eshell-save-history-on-exit nil)
(defun eshell-append-history ()
  "Call `eshell-write-history' with the `append' parameter set to `t'."
  (when eshell-history-ring
    (let ((newest-cmd-ring (make-ring 1)))
      (ring-insert newest-cmd-ring (car (ring-elements eshell-history-ring)))
      (let ((eshell-history-ring newest-cmd-ring))
        (eshell-write-history eshell-history-file-name t)))))
(add-hook eshell-pre-command-hook #'eshell-append-history)

Thanks to @joseph-garvin for the corrected, working eshell-append-history function

This doesn't handle dynamically loading the new history contents into a shell (eg run command X in shell A, and having it appear in history in shell B without reloading; like zsh's SHARE_HISTORY). I don't know how efficient eshell-read-history is, so I'd be hesitant to run it in a hook.

It is also possible that you will end up with duplicate entries with this eshell-append-history function. You may need to do some shenanigans with clearing all but the most recent entry from eshell-history-ring, then resetting it to the old value after writing history.

E.g.

(let ((old-ring (copy-list eshell-history-ring)))
  (setq eshell-history-ring (list (car eshell-history-ring)))
  ; write
  (setq eshell-history-ring old-ring))

Answer 2

My solution is to wrap the function eshell-write-history. It is called when you exit intentionally, but also in other places, e.g. when you close Emacs without quitting eshell first. You want that history as well, and if you only wrap eshell-exit-hook you risk losing it: yes that hook is called on Emacs exit, but because it, itself, calls eshell-write-history, it risks undoing your custom history work.

Long story short: patch eshell-write-history to just force always appending history.

There's another gotcha which is explained in the comments.

Code:

;; Record the size of existing history when eshell starts
(add-hook 'eshell-mode-hook
          (lambda ()
            (setq-local hly/eshell-history-on-start (ring-length eshell-history-ring))))

(defun hly/eshell-write-history-around (orig-fun &rest args)
  "Always force ‘eshell-write-history’ in append mode"
  ;; I can’t use ring-resize because eshell uses ring-insert-at-beginning on the
  ;; history ring, and that doesn’t play nicely with ring-resize when the
  ;; original ring was larger than the total history.  Iterating and copying
  ;; one-by-one is the only option I can think of.
  (let* ((full eshell-history-ring)
         (n (- (ring-length eshell-history-ring) hly/eshell-history-on-start))
         (eshell-history-ring (make-ring n)))
    (cl-loop for i downfrom (1- n) to 0
             do (ring-insert eshell-history-ring (ring-ref full i)))
    (cl-destructuring-bind (&optional fname append) args
      (funcall orig-fun fname t))))

Bonus customization which I recommend: change the default history file, lest you accidentally overwrite it one day running eshell from within emacs -Q:

(setq eshell-history-file-name "~/.emacs.d/eshell/history-my")

P.S.: Beware eshell-save-history-on-exit: that only controls the behavior on Emacs exit I described above, not regular eshell exit. From the docs (emphasis mine):

**History is always preserved after sanely exiting an Eshell buffer.**
However, when Emacs is being shut down, this variable determines
whether to prompt the user.
If set to nil, it means never save history on termination of Emacs.
If set to ask, ask if any Eshell buffers are open at exit time.
If set to t, history will always be saved, silently.