How to implement remove-key to completely undo the effect of define-key

Question

A long time ago I implemented remove-key to completely undo the effect of define-key, but when I just tried that implementation, it did not work reliably. How would you implement remove-key so that for any given key this is true:

(let ((map (make-sparse-keymap)))
  (define-key map KEY 'bound)
  (remove-key map KEY)
  map)
=> (keymap)

It should also work for non-sparse keymaps. And when a sub-keymap is involved which becomes empty when the key is removed, then the sub-keymap should also be removed.

Here's the documentation - now I only need an implementation :-)

(defun remove-key (keymap key)
  "In KEYMAP, remove key sequence KEY.
Make the event KEY truely undefined in KEYMAP by removing the
respective element of KEYMAP (or a sub-keymap) as opposed to
merely setting it's binding to nil.

There are several ways in which a key can be \"undefined\":

   (keymap (65 . undefined) ; A
           (66))            ; B

As far as key lookup is concerned A isn't undefined at all, it is
bound to the command `undefined' (which doesn't do anything but
make some noise).  This can be used to override lower-precedence
keymaps.

B's binding is nil which doesn't constitute a definition but does
take precedence over a default binding or a binding in the parent
keymap.  On the other hand, a binding of nil does _not_ override
lower-precedence keymaps; thus, if the local map gives a binding
of nil, Emacs uses the binding from the global map.

All other events are truly undefined in KEYMAP.

Note that in a full keymap all characters without modifiers are
always bound to something, the closest these events can get to
being undefined is being bound to nil like B above."
  ...)

Why do I want that? It has already been mentioned in the above doc-string, but here it is again, quoting (elisp)Format of Keymaps (emphasis mine).

When the binding is nil, it doesn't constitute a definition but it does take precedence over a default binding or a binding in the parent keymap. On the other hand, a binding of nil does not override lower-precedence keymaps; thus, if the local map gives a binding of nil, Emacs uses the binding from the global map.

(define-key map KEY nil) does not remove the entry for KEY, it just sets the binding to nil, which "doesn't constitute a definition", but which never-the-less can have an effect.

Answer 1

Eventually I came up with a solution that satisfies me and added it to my keymap-utils package.

(defun kmu-remove-key (keymap key)
  "In KEYMAP, remove key sequence KEY.
Make the event KEY truely undefined in KEYMAP by removing the
respective element of KEYMAP (or a sub-keymap) as opposed to
merely setting its binding to nil.

There are several ways in which a key can be \"undefined\":

   (keymap (65 . undefined) ; A
           (66))            ; B

As far as key lookup is concerned A isn't undefined at all, it is
bound to the command `undefined' (which doesn't do anything but
make some noise).  This can be used to override lower-precedence
keymaps.

B's binding is nil which doesn't constitute a definition but does
take precedence over a default binding or a binding in the parent
keymap.  On the other hand, a binding of nil does _not_ override
lower-precedence keymaps; thus, if the local map gives a binding
of nil, Emacs uses the binding from the global map.

All other events are truly undefined in KEYMAP.

Note that in a full keymap all characters without modifiers are
always bound to something, the closest these events can get to
being undefined is being bound to nil like B above."
  (when (stringp key)
    (setq key (kmu-parse-key-description key t)))
  (define-key keymap key nil)
  (setq key (cl-mapcan (lambda (k)
                         (if (and (integerp k)
                                  (/= (logand k ?\M-\0) 0))
                             (list ?\e (- k ?\M-\0))
                           (list k)))
                       key))
  (if (= (length key) 1)
      (delete key keymap)
    (let* ((prefix (vconcat (butlast key)))
           (submap (lookup-key keymap prefix)))
      (when (and (not (eq submap 'ESC-prefix))
                 (= (length submap) 1))
        (delete (last key) submap)
        (kmu-remove-key keymap prefix)))))

---

The old answer because it discusses some details:

After looking at it again I realized that my existing implementation actually works quite well, but is very picky about the format of the KEY. It has to be a vector in the "internal format", so [?\C-i] would work but [(control ?i)] would not. All elements have to be either integers or symbols.

(defun remove-key (keymap key)
  (define-key keymap key nil)
  (setq key (cl-mapcan (lambda (k)
                         (if (and (integerp k)
                                  (/= (logand k ?\M-\^@) 0))
                             (list ?\e (- k ?\M-\^@))
                           (list k)))
                       key))
  (if (= (length key) 1)
      (delete key keymap)
    (let* ((prefix (vconcat (butlast key)))
           (submap (lookup-key keymap prefix)))
      (delete (last key) submap)
      (when (= (length submap) 1)
        (remove-key keymap prefix)))))

?\M-\^@ is Emacs' value for the meta key.

To make this function less picky about the format of the KEY, this might work:

(setq key (if (stringp key)
              (edmacro-parse-keys key t)
            (edmacro-parse-keys (key-description key) t)))

The first line (define-key keymap key nil) should take care of the case where KEY is part of the full keymap's char-table, assuming this is correct:

Note that in a full keymap all characters without modifiers are always bound to something, the closest these events can get to being undefined is being bound to nil like B above.

That first line should also prevent the parent keymap from being modified, in the non-char-table case. I do not avoid doing that for a full keymap, because there is no full-keymap-p predicate. Usually the first element of a full keymap is the char-table, but last I checked that was not actually enforced, a keymap also functions as a full keymap if a later element is a char-table. It just seemed wiser to do some potentially unnecessary work instead of risking getting it wrong.

Answer 2

You could start with

(defun remove-key (keymap key)
  (let ((binding (assq key keymap))
        (parentbinding (assq key (keymap-parent keymap))))
    (and binding
         (not (eq binding parentbinding)) ;Don't change the parent!
         (delq binding keymap))))

This will only work for a sparse keymap and expects key to be a single key, rather than a key sequence. [ And it's guaranteed 100% untested, so even for that limited case it might not work. ]

Answer 3

This is a simplified version of information extracted from the accepted answer posted by tarsius which contains more advanced information and requires a package for kmu-parse-key-description. This small stand alone function undoes the effect of define-key by removing the declaration from the keymap. Removing the declaration is more effective than setting its value to nil.

(defun remove-key (keymap key)
  (define-key keymap key nil)
  (setq key (cl-mapcan (lambda (k)
                         (if (and (integerp k)
                                  (/= (logand k ?\M-\^@) 0))
                             (list ?\e (- k ?\M-\^@))
                           (list k)))
                       key))
  (if (= (length key) 1)
      (delete key keymap)
    (let* ((prefix (vconcat (butlast key)))
           (submap (lookup-key keymap prefix)))
      (delete (last key) submap)
      (when (= (length submap) 1)
        (remove-key keymap prefix)))))

Thanks to tarsius, the above code works.

I used remove-key to remove troublesome cua keys which, when set to nil using define-key caused C-x to behave unpredictably, if any reaction at all.

(remove-key cua--prefix-repeat-keymap (kbd "C-x"))
(remove-key cua-global-keymap (kbd "<C-return>"))

The C-x declaration is successfully removed. Unfortunately, cua is somehow still affecting C-return is still a problem, but remove-key did remove the declaration as promised. Some interlinks can get complex. Using remove-key get rid of the declaration actually simplifies the situation.