I would like to bind C-h to do exactly the same thing as backspace, no matter the mode I'm in. So far I bound C-h to delete-backward-char in global-map, but the results are inconsistent across the modes (it doesn't work well with electric-pair for example, python mode is broken as well).
(define-key key-translation-map [?\C-h] [?\C-?]), this also works for when backspaces as parts of key sequences, e.g.
C-x DEL (translated from C-x C-h) runs the command backward-kill-sentence
I found the answer, for future reference:
(global-set-key (kbd "C-h") (kbd "<backspace>"))