"Backspace" key sometimes also deletes forward in python-mode

Question

Start Edit 1

To answer questions.

Emacs Versions:

• GNU Emacs 25.2.2 (x86_64-pc-linux-gnu, GTK+ Version 3.22.21) of 2017-09-22, modified by Debian
• GNU Emacs 26.3 (build 1, x86_64-w64-mingw32) of 2019-08-29

python-mode version (from C-c C-v) on both platforms:

• Using python-mode version 6.2.3

However, M-x describe-package for python-mode on the Windows shows:

python-mode is an installed package.

     Status: Installed in  python-mode-6.2.3/  (unsigned). Delete
    Version: 6.2.3
    Summary: Python major mode
Other versions: 6.2.3 (melpa).
[...]

Whereas on Linux, M-x describe-package for python-mode says I can install it (even though I'm using 6.2.3 as per C-c C-v above:

python-mode is an available package.

     Status: Available from melpa -- [Install]
    Archive: melpa
[...]

Changing backspace behavior

Changing normal-erase-is-backspace so that backspace deletes forward results in expected behavior (it deletes forward and not also backward), as far as I can tell, though the macro weirdness persists.

I am updating to the Melpa version 20200224.1148 on Linux and trying it out now to see if it changes anything.

Output of describe-key

On Linux

In python-mode on Linux, C-h deletes backward. I ran M-x describe-key on Backspace and got:

DEL runs the command delete-backward-char (found in global-map), which is an
interactive compiled Lisp function in  simple.el .

It is bound to DEL.

(delete-backward-char N &optional KILLFLAG)

This function is for interactive use only;
in Lisp code use  delete-char  instead.

Delete the previous N characters (following if N is negative).
If Transient Mark mode is enabled, the mark is active, and N is 1,
delete the text in the region and deactivate the mark instead.
To disable this, set option  delete-active-region  to nil.

Optional second arg KILLFLAG, if non-nil, means to kill (save in
kill ring) instead of delete.  Interactively, N is the prefix
arg, and KILLFLAG is set if N is explicitly specified.

When killing, the killed text is filtered by
 filter-buffer-substring  before it is saved in the kill ring, so
the actual saved text might be different from what was killed.

In Overwrite mode, single character backward deletion may replace
tabs with spaces so as to back over columns, unless point is at
the end of the line.

NB, M-x describe-key on C-h in python-mode results in nothing but prints You can run the command describe-key with C-h k (in *Messages* as well as the mini-buffer)

Whereas running M-x describe-key-briefly on C-h AND Backspace prints DEL runs the command backward-delete-char-untabify to the mini-buffer.

I am thoroughly confused.

On Windows

C-h k Backspace works! Go figure. I use the exact same .emacs and install packages using the Emacs package manager from Melpa on both OSes (they're in a Git repo). Output:

<backspace> runs the command py-electric-backspace (found in
python-mode-map), which is an interactive compiled Lisp function in
 python-mode.el .

It is bound to <backspace>, <menu-bar> <Python> <Other> <Electric>
<Electric backspace>.

(py-electric-backspace &optional ARG)

Delete preceding character or level of indentation.

When  delete-active-region  and (use-region-p), delete region.

Unless at indentation:
  With  py-electric-kill-backward-p  delete whitespace before point.
  With  py-electric-kill-backward-p  at end of a list, empty that list.

Returns column reached. 

[back]

M-x describe-key on Backspace does the same while M-x describe-key on Backspace prints <backspace> runs the command py-electric-backspace.

End Edit 1

Start Edit 2

A few more updates (so far ony trouble shooting on Linux):

• Looking at the output of M-x describe-bindings while in python-mode clearly shows <backspace> bound to py-electric-backspace
• Running M-x py-electric-backspace always does the right thing
• If I bind py-electric-backspace from within the buffer to some other key (e.g. by running M-x eval-expression then (define-key python-mode-map "\C-n" 'py-electric-backspace)), it works as expected.
• Removing all key-bindings in my ~/.emacs does not solve this
• M-x describe-bindings also shows <backspace> mapped to DEL under the Function key map translations: section.
• Manually setting the normal-erase-is-backspace via M-x set-variable (0 = Off, 1 = Maybe, 2 = On or even t or nil) does nothing to solve the problem but does toggle Backspace between Backspace and DEL.
• Manually running M-x normal-erase-is-backspace turns Backspace into C-h

End Edit 2

EDIT 3 {

I don't believe the root of the problem has really been found. This problem manifests independently of any terminals. It also occurs in graphical Emacs. I decided to try to trace what Emacs was doing using a pre-command-hook:

(defvar *event-log* nil)
(defun log-commands nil
  (let ((buf (current-buffer)))
    (cl-push `(,buf ,this-command ,(this-command-keys))
          *event-log*)))
(add-hook 'pre-command-hook 'log-commands)

This causes Emacs to log every single keypress to an in-memory list.

From there, I opened a Python source file and put the cursor where the block appears:

def foobar(baz,qux):
    try:
        quux.xyzzy()▋
    except:
        pass

From there, I added .foobar and then tried to delete it by pressing Backspace twice (enough to reproduce the behavior and also revert back to the correct behavior). The resulting event log, which is in reverse chronological order:

 (#<buffer foobar.py> python-indent-dedent-line-backspace "^?") 
 (#<buffer foobar.py> delete-char "^D") 
 (#<buffer foobar.py> self-insert-command "r")
 (#<buffer foobar.py> self-insert-command "a")
 (#<buffer foobar.py> self-insert-command "b")
 (#<buffer foobar.py> self-insert-command "o")
 (#<buffer foobar.py> self-insert-command "o")
 (#<buffer foobar.py> self-insert-command "f")
 (#<buffer foobar.py> self-insert-command ".")

It clearly shows that the same key is being received as two distinct keys, and therefore it's executing two different functions. Why this happens remains a mystery, but because it can happen in a GUI session, we can rule out the possibility of terminal settings being to blame. Also note that it's impossible for things like terminal settings or X11 keymap bindings to suddenly change as a result of Emacs keypresses, and even if it was, the behavior would not be consistent between terminals, local X11 sessions, and X11-forwarded sessions.

}

As the title says. Sometimes when I press Backspace, it deletes the character after the cursor (to the right) usually also deleting the character before it (to the left) in python-mode. This seems to happen only on some lines and after at least a 0.25 pause (does not happen when typing quickly).

I have only observed this in python-mode. At first, I thought it might be the python-mode and elpy-mode combo, but it persists with elpy-mode disabled and unloaded.

This seems to happen more frequently when within an f-string (python), at the end of a line, or at the end of a bracket or parentheses pair. I suspect that it might have something to do with hitting the backspace key when python-mode is "thinking" (e.g. trying to autocomplete).

Incidentally, python-mode also seems to make keyboard macros useless, causing all sorts of errors. I often find myself setting the buffer to text-mode, running a macro, and then setting it back to python-mode.

I've tried to set various py-* variables that have to do with delaying or reducing autocomplete, but they don't seem to do anything.

I was not having all these problems with my old and now deprecated python-mode, but it is no more.

Apparently, there is no python-mode tag.

Answer 1

Short Answer

Don't try to create one .emacs file for all your environments and conditions under which you might access them unless you have time on your hands.

Longer answer for those who might be in a similar circumstance...

How to troubleshoot

Thanks to the input from commenters and others. I'm putting things that helped be get to some of the issues.

• Check which keypresses Emacs is getting.
  • In a Linux terminal, you can use showkey
  • In Emacs, command-line-mode from Melpa may help.
  • On Windows, you may also want to check out KeyPose
  • Note xev may be useful if you're running Emacs using X11 remotely.
• Check your key-bindings in Emacs:
  • Run M-x describe-bindings to list all the key bindings in the current buffer.
  • Run C-hk<key> or M-x describte-key then press <key> where <key> is the key you want to check.
  • Try running M-x normal-erase-is-backspace
  • Try running M-x set-variable on normal-erase-is-backspace and setting it:
    • 0 = Off
    • 1 = Maybe
    • 2 = On
  • Check your .emacs for any global-set-key, define-key or similar statements

Know that how (as a GUI or in a terminal with '-nw' etc.), in which environment (Linux, Mac, Windows, WSL) and from where you are accessing it (directly on the computer, through command-line ssh, through PuTTy/MobaXTerm, through RDP, TeamVewier, etc.) and more can affect which keys Emacs ultimately "sees".

The Root of the Problem

Emacs was receiving different key-presses / characters depending on the system and my attempt to have one universal .emacs file to rule them all caused conflicting handling under different circumstances and modes.

For example, I noticed that in one instance (Emacs on Linux using MobiXTerm on Windows) the terminal was sending ^H (aka C-h) instead of the keyboard Backspace, while in others (Emacs on Linux under XTerm) it was sending Backspace while in others still (Emacs under Ubuntu 18.0.4 under WSL thought remote connection, but not when on the same machine) Emacs was thinking it was getting Delete.

Note: I noticed the -back-and-forward delete issue also happening in lisp-mode during one of my configuration permutation attempts, so it seems at least likely that there would have been other major modes that would have had issues.

Although I am not 100% sure, I believe it was happening when the mode (e.g., python-mode), executed some time-delayed hooks for handling Backspace, Delete, or C-h, the disparate configurations caused some oddities. It always required a bit of a time-delay and appeared to happen in python-mode at the end-of-line or other mode-specific trigger cases like deleting characters at or around a character "closing" a mode syntax grouping (e.g., ),], }, ", ', or backtick)

My Solution

I was able to "fix" this by having different settings in different environments by making sure that Emacs was getting the keys it thought it was getting and expanding my .emacs a bit to do some more checking.

I was able to create a config that detects all scenarios I tested except WSL (Windows Subsystem for Linux), but for now, that case is rare enough that I did not bother to test it after testing the keystrokes.

Thanks to everyone for the troubleshooting help.