63

I use emacs in different places and I want to have a similar setup and packages installed everywhere. I guess I can use a version control repository for the setup files. Since I use Prelude, that would be ~/.emacs.d/personal/.

I don't know how to do with packages. Is there a file somewhere in .emacs.d/ with the list of installed packages that I can use to make emacs in other machines to also install those listed there?

6
  • 2
    For me the installed packages are just part of my Emacs version control repository. I byte-compile the packages with the oldest version of Emacs I want/need to use and also put the .elc files into the repository. This offers maximum control and consistency. The trade-off is the (relatively) large size of the repository. My 6 year old git repository has a size of 120MB. While I could probably get away with 1/10 of it if I wouldn't include packages, those few "wasted" megabytes really don't worry me.
    – paprika
    Sep 28, 2014 at 21:05
  • 1
    I should add that while ELPA/MELPA/... are pretty popular nowadays, still not all packages are available through them. So if you use a package that needs manual installation you might not want to replicate the effort on each new machine you use (plus for each package upgrade). Again, an easy solution is to add the package to your Emacs version control repository.
    – paprika
    Sep 28, 2014 at 21:08
  • Can't you just byte compile on site and get away with ignoring .elc files in git ?
    – Vamsi
    Sep 28, 2014 at 23:42
  • @Vamsi: I put the byte-compiled files into the repository because some (non-ELPA) packages are a bit finicky to compile due to their dependencies. For these I don't like to repeat the process if not necessary.
    – paprika
    Sep 30, 2014 at 12:57
  • @paprika Could you provide some details how to obtain your setup? Do you add everything in .emacs.d plus .emacs to the repository and that's it?
    – Beginner
    Jan 31, 2015 at 8:09

12 Answers 12

52

For emacs 25 or newer refer to this answer. For emacs < 25 there's no automatically generated manifest file that you can synchronize to achieve the desired effect.

That said, something you can do is add calls to package-install in your emacs configuration itself.

(package-install 'auctex)

The idea is that package-install is idempotent, so if the package is already present, nothing will actually happen. Assuming you have such a call for every package you use (or at least the leaves in the dependency graph), that would effectively synchronize your packages across machines.


For multiple packages you can use the following:

(setq my-package-list '(package1 package2 packageN))
(mapc #'package-install my-package-list)
5
  • 2
    I am surprised how much simpler this approach is compared to the solution suggested here. Is there any difference? The snippet also uses package-install.
    – Stenskjaer
    Sep 30, 2014 at 10:02
  • 1
    There have been changes to package.el since that linked answer. It's possible that at the time package-install performed operations on existing packages, not just uninstalled ones. Sep 30, 2014 at 12:30
  • 4
    This technique is unfortunately problematic -- even if the package-archives repository is uniform between machines and specified in SCM. It does not assure that package versions are identical between machines. The issue is that package versions are not specified; these individual packages may diverge over time and their dependencies may become incompatible. This can happen quite easily on active package archives such as melpa.
    – ctpenrose
    Apr 7, 2016 at 0:49
  • @ctpenrose: Do you have a suggestion to avoid this problem?
    – student
    May 13, 2016 at 12:29
  • @student I have minimized the problem by using melpa-stable and updating packages less often.
    – ctpenrose
    Apr 21, 2017 at 5:13
36

I keep my .emacs.d directory in version control. Then, in my init.el and subsequent files I use use-package to define package setup. Not only does use-package lazily load your packages, it will download them on demand if they don't exist from whatever package repos you've set up.

For example, I use go-mode, but not on every machine. In my init.el I have the following:

(use-package go-mode
  :ensure t
  :config
  (progn
    (defun my-go-mode-hook ()
      (linum-mode t)
      (setq tab-width 4)
      (add-hook 'before-save-hook 'gofmt-before-save))
    (add-hook 'go-mode-hook 'my-go-mode-hook)))

This adds a mode hook, but more importantly, by specifying :ensure t it will download the package on demand.

To keep a machine in sync, you can just checkout or pull from the repo and start up Emacs. Any new packages will be downloaded and installed.

4
  • This is the solution I now use too, rather than Cask, primarily because (as T. Verron noted) Cask does not work (well) on Windows, and is yet another dependency.
    – Andy
    May 7, 2015 at 4:40
  • 2
    This is the way I use also, but instead of doing :ensure go-mode that is repeating the package name, you can just specify :ensure t
    – Pedro Luz
    Nov 10, 2016 at 20:37
  • Good point! This is an old answer. I'll update it.
    – elarson
    Nov 11, 2016 at 17:30
  • 1
    You can also use the :hook keyword to simplify your code. Aug 7, 2018 at 13:12
18

In Emacs-25, there is the variable package-selected-packages, so you can customize this variable and use package-install-selected-packages to make sure they're installed.

2
  • Note that I see it, the name of that command is a little confusing. Can I change it to package-install-selected-packages?
    – Malabarba
    Apr 27, 2015 at 17:41
  • Assuming you mean "Now" instead of "Note", yes.
    – Stefan
    Apr 27, 2015 at 18:02
9

What you want to use is Cask, which lets you create a Cask file specifying which packages to install on cask install. It can be used to manage dependencies of a package, and "dependencies" of your Emacs configuration with ease. Put your Cask file in under version control, and install/update packages on a per-machine basis.

2
  • 5
    It is worth noting that (as of today) this solution doesn't work for windows machines.
    – T. Verron
    Sep 30, 2014 at 8:37
  • 1
    I no longer use this solution, for your very reason (and that Cask is yet another dependency). Great for making packages; horrible for configuration management.
    – Andy
    May 7, 2015 at 4:41
7

An alternative approach would be the following: since I do not only want to synchronise my Emacs packages, but also other files (e.g. .emacs, .bashrc, but also other directories) between my server and my laptop, I started to use unison, to sync the files and directories. So when working on my laptop I simple run unison laptop before anything else. My ~/.unison/laptop.prf file has the following section for Emacs related files:

path = .emacs
path = .emacs.d
ignore = Path {.emacs.d/semanticdb}

Since my Emacs packages (and also my Emacs backups and bookmarks) are stored in ~/.emacs.d this makes sure I've got everything on all my machines.

An alternative approach would be put the .emacs.d directory in a directory that is synced with OwnCloud, DropBox or any other file syncing service and then create symlinks from ~/.emacs.d to that shared directory.

5

While package.el is the standard way to install packages, you might also want to try el-get which is very useful for installing packages which are not (or cannot be) on elpa. This answer deals with synchronizing such packages.

The way you ensure that given packages are installed when using el-get is to add something like the following to you init file

(el-get 'sync '(packages))

where packages are the list of packages you want to be installed. This function is similar to package-install it installs the packages only if they are not already installed, otherwise it simply initializes the packages.

5

I use a little trick "stolen" from emacs-starter-kit (I think):

(defun maybe-install-and-require (p)
  (when (not (package-installed-p p))
   (package-install p))
  (require p))

So when I require a package, I simply use:

(maybe-install-and-require 'magit)

On emacs startups, evaluating my config, package.el will provide to install magit if it's not installed.

You can find my configuration here:

https://github.com/mdallastella/emacs-config/

2
  • 1
    Following same philosophy, you could use 'paradox-require' from paradox
    – csantosb
    Nov 4, 2014 at 16:57
  • Github link needs to be updated. Its giving a 404 error.
    – prash
    Nov 20, 2020 at 6:13
4

I have this setup-packages.el in my emacs setup which is a hybrid of code from Prelude and Tomorokoshi's blog on Package Management.

setup-packages.el does the following:

  • Create a directory for elpa packages if one doesn't exists and add it and its sub-directories to the load-path.
  • Update package-archives list with Melpa.
  • Check if you have all the packages listed in my-packages list installed. If a package is not installed, install it.

How to implement

  • Save the setup-packages.el below to your ~/.emacs.d/ directory.
  • Set user-emacs-directory, setup-packages-file and my-packages variables in your init.el and do (load setup-packages-file).

When you start emacs for the first time on a machine that does not have these packages installed, all the pacakges listed in my-packages will get auto-installed.

setup-packages.el

;; setup-packages.el - Package management

(require 'cl)
(require 'package)

;; Set the directory where you want to install the packages
(setq package-user-dir (concat user-emacs-directory "elpa/"))

;; Add melpa package source when using package list
(add-to-list 'package-archives '("melpa" . "http://melpa.org/packages/") t)

;; Load emacs packages and activate them
;; This must come before configurations of installed packages.
;; Don't delete this line.
(package-initialize)
;; `package-initialize' call is required before any of the below
;; can happen

;; Auto install the required packages
;; Method to check if all packages are installed
(defun packages-installed-p ()
  (loop for p in my-packages
        when (not (package-installed-p p)) do (return nil)
        finally (return t)))

;; if not all packages are installed, check one by one and install the missing ones.
(unless (packages-installed-p)
  ;; check for new packages (package versions)
  (message "%s" "Emacs is now refreshing its package database...")
  (package-refresh-contents)
  (message "%s" " done.")
  ;; install the missing packages
  (dolist (p my-packages)
    (when (not (package-installed-p p))
      (package-install p))))

(provide 'setup-packages)

init.el

You would need the following in your init.el:

(setq user-home-directory  (getenv "HOME"))
(setq user-emacs-directory (concat user-home-directory ".emacs.d/"))
(setq setup-packages-file  (expand-file-name "setup-packages.el" user-emacs-directory))

;; A list of packages to ensure are installed at launch
(setq my-packages
      '(
        ;; package1
        ;; package2
       ))

(load setup-packages-file nil :nomessage) ; Load the packages
3

I have ~/emacs directory which is mercurial-version controlled and contains everything my emacs setup consists of (~/emacs/site-lisp for manually downloaded libraries, ~/emacs/elpa for elpa-installed libraries, ~/emacs/etc/ for splitted .emacs, ~/emacs/dot-emacs.el which I symlink as ~/.emacs). It required some tweaking of some packages to have all important files inside this tree, but it works well. Those few bits which are machine-specific I have implemented by conditionals on system name.

So after I install/reconfigure/change anything, I simply commit pull/push all the changes between all machines I use.

Extra benefit is that I have complete history of my config and can go back/bisect/revert in case anything goes wrong.

PS mercurial seems particularly suitable as it has natural two-side pull/push, but similar setup should not be hard to achieve with git or any other dvcs.

2

In order to mirror my config I decided to go for a different approach, using Syncthing; every change in any of my config files propagates to any other of my pcs without having to care about it, so when I upgrade packages I only have to do it in one of the pcs.

2

RSYNC:  Synchronize select folders/files using rsync, either over a home network or via ssh to a remote server.

rsync is a one-way synchronization utility that is able to delete files on the target, so please be sure to back-up your data on both the source and target locations, and thoroughly test using the --dry-run option, before doing the real thing.

To read about how to properly configure the .authinfo file, see https://www.gnu.org/software/emacs/manual/auth.html An example .authinfo file contents (which may contain multiple different entries) is as follows:

machine mymachine login myloginname password mypassword port myport

Configure and use the function rsync-remote to synchronize over ssh to a remote server. Or, use the function rsync-local to synchronize on the same computer or over a trusted home network.

(require 'auth-source)

;;; EXAMPLE:
;;;   (get-auth-info "12.34.567.89" "username")
;;;   (get-auth-info "localhost" "root")
(defun get-auth-info (host user &optional port)
  (let ((info (nth 0 (auth-source-search
                      :host host
                      :user user
                      :port port
                      :require '(:user :secret)
                      :create t))))
    (if info
      (let* ((port (plist-get info :port))
             (secret-maybe (plist-get info :secret))
             (secret
               (if (functionp secret-maybe)
                 (funcall secret-maybe)
                 secret-maybe)))
          (list port secret))
    nil)))

(defun rsync-filter (proc string)
  (cond
    ((string-match
       "^\\([a-zA-Z0-9_\\-\\.]+\\)@\\([a-zA-Z0-9_\\-\\.]+\\)'s password: "
       string)
      (let* ((user (substring string (match-beginning 1) (match-end 1)))
             (host (substring string (match-beginning 2) (match-end 2)))
             (password (car (cdr (get-auth-info host user)))))
        (process-send-string proc (concat password "\n"))))
    ((not (or (string-match "files\\.\\.\\.\r" string)
              (string-match "files to consider\n" string)))
      (with-current-buffer (messages-buffer)
        (let ((inhibit-read-only t))
          (goto-char (point-max))
          (when (not (bolp))
            (insert "\n"))
          (insert string)
          (when (not (bolp))
            (insert "\n")))))))

(defun rsync-remote ()
"Use rsync to a remote server via ssh.  Back-up your data first!!!"
(interactive)
  (let* (
      (host "localhost")
      (username "root")
      (port (or (car (get-auth-info host username))
                (number-to-string (read-number "Port:  "))))
      (source
        (let ((dir (expand-file-name (locate-user-emacs-file "elpa/"))))
          (if (file-directory-p dir)
            dir
            (let ((debug-on-quit nil)
                  (msg (format "`%s` is not a valid directory." dir)))
              (signal 'quit `(,msg))))))
      (target "/private/var/mobile/elpa/")
      (ssh "/usr/bin/ssh")
      (rsync "/usr/bin/rsync")
      (rsync-include-file "/path/to/include-file.txt")
      (rsync-exclude-file "/path/to/exclude-file.txt")
      (rsh (concat "--rsh=ssh -p " port " -l " username))
      (host+target (concat host ":" target)))
    (start-process
        "rsync-process"
        nil
        rsync
        "-avr" ;; must specify the `-r` argument when using `--files-from`
        "--delete"
        ;; The paths inside the exclusion file must be relative, NOT absolute.
        ;;; (concat "--files-from=" rsync-include-file)
        ;;; (concat "--exclude-from=" rsync-exclude-file)
        rsh
        source
        host+target)
    (set-process-filter (get-process "rsync-process") 'rsync-filter)
    (set-process-sentinel
      (get-process "rsync-process")
      (lambda (p e) (when (= 0 (process-exit-status p))
        (message "rsync-remote:  synchronizing ... done."))))))

(defun rsync-local ()
"Use rsync locally -- e.g., over a trusted home network.
 Back-up your data first!!!"
  (interactive)
  (let (
      (rsync-program "/usr/bin/rsync")
      (source
        (let ((dir (expand-file-name
                     (file-name-as-directory
                       (read-directory-name "Source Directory: " nil nil nil nil)))))
          (if (file-directory-p dir)
            dir
            (let ((debug-on-quit nil)
                  (msg (format "`%s` is not a valid directory." dir)))
              (signal 'quit `(,msg))))))
      (target (expand-file-name
                (file-name-as-directory
                  (read-directory-name "Target Directory: " nil nil nil nil)))))
    (unless (y-or-n-p (format "SOURCE:  %s | TARGET:  %s" source target))
      (let ((debug-on-quit nil))
        (signal 'quit `("You have exited the function."))))
    (start-process "rsync-process"
      nil
      rsync-program
      "--delete"
      "-arzhv"
      source
      target)
    (set-process-filter (get-process "rsync-process") #'rsync-process-filter)
    (set-process-sentinel
      (get-process "rsync-process")
      (lambda (p e)
        (when (= 0 (process-exit-status p))
        (message "Done!"))))))
2

https://github.com/redguardtoo/elpa-mirror creates a local repository of all installed packages.

Usage is simple, just run M-x elpamr-create-mirror-for-installed.

On other machines, insert (setq package-archives '(("myelpa" . "~/myelpa/"))) into your .emacs and restart Emacs.

Now on all the machines, you get exactly same version of packages.

1
  • Thank you, this is a wonderful answer for my use-case, elpa-mirror should be more widely known.
    – gsl
    Nov 8, 2020 at 7:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.