Does this do what you want?
It provides a
display-buffer-reuse-major-mode-window function for use with
display-buffer-alist, and configures an entry which targets
You can then adapt the
display-buffer-alist entry to target additional modes if you want this behaviour for other kinds of buffer as well.
;; Always open compilation buffers in the same window.
(cons (lambda (buffer alist)
(eq major-mode 'compilation-mode)))
'((inhibit-same-window . nil)
(reusable-frames . visible)
(inhibit-switch-frame . nil)))))
(defun display-buffer-reuse-major-mode-window (buffer alist)
"Return a window displaying a buffer in BUFFER's major mode.
Return nil if no usable window is found.
If ALIST has a non-nil `inhibit-same-window' entry, the selected
window is not eligible for reuse.
If ALIST contains a `reusable-frames' entry, its value determines
which frames to search for a reusable window:
nil -- the selected frame (actually the last non-minibuffer frame)
A frame -- just that frame
`visible' -- all visible frames
0 -- all frames on the current terminal
t -- all frames.
If ALIST contains no `reusable-frames' entry, search just the
selected frame if `display-buffer-reuse-frames' and
`pop-up-frames' are both nil; search all frames on the current
terminal if either of those variables is non-nil.
If ALIST has a non-nil `inhibit-switch-frame' entry, then in the
event that a window on another frame is chosen, avoid raising
(let* ((alist-entry (assq 'reusable-frames alist))
(frames (cond (alist-entry (cdr alist-entry))
((if (eq pop-up-frames 'graphic-only)
(window (let ((mode (with-current-buffer buffer major-mode)))
(if (and (eq mode (with-current-buffer (window-buffer)
(not (cdr (assq 'inhibit-same-window alist))))
(and (window-live-p w)
(eq mode (with-current-buffer (window-buffer w)
(not (eq w (selected-window)))
(throw 'window w)))
(when (window-live-p window)
(prog1 (window--display-buffer buffer window 'reuse alist)
(unless (cdr (assq 'inhibit-switch-frame alist))
(window--maybe-raise-frame (window-frame window)))))))