Can a mode setup function fail?

Question

If I define a new mode using define-derived-mode or define-minor-mode, can the mode setup somehow signal failure so that the mode is not activated?

All of the following modes are activated despite the error.

(define-derived-mode foo-major-mode fundamental-mode "Foo"
  "Docstring"
  (error "Unable to set up Foo major mode"))

(define-minor-mode foo-local-minor-mode
  "Docstring"
  (when foo-local-minor-mode
    (error "Unable to set up Foo local minor mode")))

(define-minor-mode foo-global-minor-mode
  "Docstring"
  :global t
  (when foo-global-minor-mode
    (error "Unable to set up Foo global minor mode")))

Answer 1

No, the mode has already done a whole bunch of stuff before your error happens.

Use M-x pp-macroexpand-last-sexp to see what your definitions expand to, and where your mode body code actually runs.

If you want to prevent a mode from being enabled, the logic needs to be in the place which tries to enable the mode.

Alternatively, you can add some before advice to your mode function -- that will run before anything else happens. (I wouldn't particularly recommend this approach without knowing that there was a good reason for doing it, though.)

Answer 2

For a minor mode, you can do:

...
(setq foo-local-minor-mode nil)
(error "Unable to set up Foo local minor mode")

For a major mode it's a bit different: there's basically no such thing as "removing" a major mode, the only thing you can hope to do is to go back to the previous major mode (but that requires saving it beforehand) or switch to some "bare" major mode.

To "go back", the only standard option is major-mode-restore, which requires you've done a major-mode-suspend bedforehand. This is used for example in image-mode to switch between the "view image" and the "view underlying data" mode (where the underlying data is often binary junk but can be text in cases like Postscript, SVG, XPM, ...).

To set a "bare" major mode you could do:

(fundamental-mode)

which will revert to (surprise, surprise) fundamental-mode.