overlay makes text unreadable -- where does it come from? Overlay face is "undefined face"

Question

There is an overlay making parts of stack traces in my inferior python buffers very difficult to read:

I would like to change the coloring of the yellow part of line 42. If I run M-x describe-char there it reveals that there is an overlay applied:

There is an overlay here:
 From 2370 to 2371
  evaporate            t
  face                 (:background "#F0DFAF")
  insert-behind-hooks  (ansi-color-freeze-overlay)
  modification-hooks   (ansi-color-freeze-overlay)

It's the combination of the foreground color and the background #F0DFAF that's problematic. I'm not sure how to figure out where that background is coming from, or change it. Enter over (:background "#F0DFAF") gives me:

Face: :background undefined face.

The same sequence (M-x describe char, enter over an overlay face) elsewhere gives me an opportunity to customize the face.

I'm using Doom emacs and the zenburn theme, in case that's helpful.

-------EDIT--------

After reading the comment from @dalanicolai I'm getting somewhere. If I run ansi-color-for-comint-mode-off, the stack trace shows the color escape sequences. The one one causing the problem applies [43], which puts ansi-yellow in the background and leaves the foreground alone. And for a dark-background theme that results in some light-colored text on a yellow background. If I set the value for ansi-yellow to black:

(setq ansi-color-normal-colors-vector [ansi-color-black ansi-color-red ansi-color-green ansi-color-black ansi-color-blue ansi-color-magenta ansi-color-cyan ansi-color-white])

the yellow background becomes black and the stacktrace is readable. Great! But specifying a foreground color would be a better solution than telling it that yellow is now black. So... The question now becomes how to tell comint to use a different color for that situation.

I've dug around some in ansi-color-process-output and the functions it calls without much luck in connecting the treatment of comint output to actual colors.

What ended up working was customizing the face ansi-color-yellow locally for python-mode buffers. I'm using doom emacs so this involved the (after! python) and custom-theme-set-faces doom macros. Here's the relevant section of my config.el:

(after! python
  (setq python-shell-interpreter "jupyter"
        python-shell-interpreter-args "console --simple-prompt"
        python-shell-completion-native-disabled-interpreters '("jupyter")
        lsp-file-watch-threshold 9999)
  (custom-theme-set-faces! '(doom-zenburn)
   `(ansi-color-yellow :background, "#F18C96")))

It's a hack, but I haven't seen it to affect anything other than error messages in inferior python buffer stack traces. And for now I've spent about all the time I can spare on it.

Answer 1

It's IPython, not Emacs, that does the highlighting here. IPython sets the background color to ansiyellow and Emacs adopts it.

There's an open issue for IPython: https://github.com/ipython/ipython/issues/13446

Although colors in IPython are customizable by using themes, it seems that themes are only applied to program code, but not to the exception handler that uses hardcoded colors.

To fix this issue search for bg:ansiyellow in IPython's file core/ultratb.py and change it to whatever you like best, e.g. bg:#3f4a99.

See also: https://stackoverflow.com/a/70837141/7864179

FWIW here are some links for customizing IPython themes. But so far themes are not applied to the exception handler.

• Configuration
• highlighting_style