How can I build a regexp insensitive to case?

Question

I have a bunch of keywords:

• set
• write
• quit

which each can be shortened down to one letter:

• s se set
• w wr wri writ write
• q qu qui quit

Further, these keywords are case-insensitive, so the following are added to the mix:

• s S se sE Se SE ...
• w W wr wR Wr WR ...
• q Q qu qU Qu QU ...

Given a list of the basic keywords ("set" "write" "quit"), how can I create a regular expression that will match them as I describe?

---

I've approached it by providing a list to regexp-opt, but the case-insensitivity list is proving hard to write. Here is what I have so far:

(defun mumps--command-variants (command &optional stack)
  "Returns the possible abbreviations of COMMAND.
For example, input of \"set\" would yield the list
  \(\"s\" \"se\" \"set\")"
  (let ((case-insensitive-command-list
         (mumps--case-insensitize command))))
  (if (= 1 (length command))
      (cons command stack)
    (mumps--command-variants
     (substring command 0 (1- (length command)))
     (cons command stack))))

Note on use-case

This is going to be used to create font-lock keywords for a major mode for editing MUMPS code. MUMPS is an... interesting language: command names are case-insensitive (and can be abbreviated like this), but everything else in the language is case-sensitive (and cannot be abbreviated). Furthermore, there is no syntactical difference between a expression which uses a command and that which uses a function (a subroutine of commands), but one is case-insensitive/abbreviated and the other is not.

Answer 1

After reading the comments in a previous answer, it appears that you would like to write a font-lock rule which would match one regexp without case, whereas the others would still be matched with a case.

The easiest way to do this is to break out the matcher of the font-lock rule to a function, for example:

Before:

 ...
 (MY-REGEXP
  (0 font-lock-variable-name-face)
  (1 another-face)
  ...)
 ...

After:

 (defun my-match-the-regexp (limit)
   (let ((case-fold-search t))
     (re-search-forward MY-REGEXP limit t)))

 (my-match-the-regexp
  (0 font-lock-variable-name-face)
  (1 another-face)
  ...)

Answer 2

The docstring for re-search-forward says:

Search case-sensitivity is determined by the value of the variable case-fold-search, which see.

Hence, you can set case-fold-search to t:

Documentation: Non-nil if searches and matches should ignore case.

EDIT For font locking, you can set font-lock-keywords-case-fold-search to t:

Documentation: Non-nil means the patterns in font-lock-keywords are case-insensitive. This is set via the function font-lock-set-defaults, based on the CASE-FOLD argument of font-lock-defaults.

Answer 3

I have exactly the same issue in a different language (but similarly odd, from the sound of it).

Let's start with a single keyword to match: "write"

You can build up a solution starting with the regular expression provided in this answer (or actually one of its comments) about matching a word and all initial substrings:

w(?:r(?:i(?:t(?:e)?)?)?)?

From this basic pattern, you can generate regular expressions for whatever keyword you need to match. To introduce case insensitivity as described, a quick modification including both lower- and upper-case for each letter is available:

[wW](?:[rR](?:[iI](?:[tT](?:[eE])?)?)?)?

From here the addition of word boundaries \b and some hideous escaping, we have a regular expression that's ready for deployment as a matcher in an element in font-lock-keywords:

(setq mumps-write-regex "\\b[wW]\\(?:[rR]\\(?:[iI]\\(?:[tT]\\(?:[eE]\\)?\\)?\\)?\\)?\\b")
(setq mumps-font-lock-keywords
  `(
    (,mumps-write-regex . 'font-lock-keyword-face)
    ;; ...other matchers and faces here...
    ))

This should match all of the scenarios described in the question, with case insensitivity, but requires manual generation of the regular expression.

---

As is explored in the question parts of this are also doable through regexp-opt, which should generate optimized regular expressions to match sequences of words, if combined with a bit of lisp:

;; function to generate a list of initial substring
(defun initial-substrings (word)
  (setq word (downcase word))
  (cond
   ((> (length word) 0) (cons word (initial-substrings (substring word 0 -1) start)))
   (t '())
   ))

;; generate a regular expression that matches that list
(setq mumps-write-regex (regexp-opt (initial-substrings "write") 'words))
     ⇒ "\\<\\(w\\(?:r\\(?:i\\(?:te?\\)?\\)?\\)?\\)\\>"

The generated regular expression closely resembles our initial case-sensitive regular expression, though it avoids the innermost (?:e)? and uses probably more proper boundary matching with < and >.

To generalize this regexp-opt approach to case-insensitive matches, it can get a little out of control because one needs to generate all case-permuted versions of all of the strings and substrings.

Instead, a better approach is to use this generated case-sensitive regular expression inside a matcher function -- as Lindydancer shows in another answer -- that sets case-fold-search to t just for that font-lock-keywords element:

;; matcher function leveraging the generated regex with case insensitivity
(defun mumps-write-matcher (limit)
  (let ((case-fold-search t))
    (re-search-forward mumps-write-regex
                       limit 'no-error)))

Putting it all together, a regexp-opt based approach that doesn't involve writing regular expressions manually is:

;; function to generate a list of initial substring
(defun initial-substrings (word)
  (setq word (downcase word))
  (cond
   ((> (length word) 0) (cons word (initial-substrings (substring word 0 -1) start)))
   (t '())
   ))

;; generate a regular expression that matches that list
(setq mumps-write-regex (regexp-opt (initial-substrings "write") 'words))

;; matcher function leveraging the generated regex with case insensitivity
(defun mumps-write-matcher (limit)
  (let ((case-fold-search t))
    (re-search-forward mumps-write-regex
                       limit 'no-error)))

;; set the local font-lock-keywords variable
(setq mumps-font-lock-keywords
      `(
        (,'mumps-write-matcher . 'font-lock-keyword-face)
        ;; ...other matchers and faces here...
        ))

---

Neither of the approaches above cover the question's original requirements about accepting a list of different keywords, all of which should be matched together in a case-insensitive fashion.

At a minimum, you could craft a regex for each keyword and separate them with \| in the regular expression in either approach. In the regexp-opt aproach, using append to join the lists returned by initial-substrings prior to invoking regexp-opt should work.