Different backslash behaviour between interactive regexps and code regexps

Question

We usually use \| to depict OR in Regex. Using a\|b as input to Regexp I-search and Occur both gave the desired result. But re-search-forward seems unable to recognize this notation.

abcdcba

(re-search-forward "a\|b")

;; Search failed: "a|b"

Answer 1

Elisp regexps are represented as strings, which means backslashes are interesting, as they are not only special to regexps, but also when writing strings.

Emacs requires a literal \ character to be escaped in the double-quoted read syntax for strings and so, when the code is processed by the lisp reader, "\\" becomes a string object containing a single \ character; and hence that single backslash is what the regexp engine sees when it uses that string object.

So in your instance, the regexp a\|b is represented by "a\\|b" in the double-quoted read syntax for strings.

Conversely "a\|b" is the regexp a|b (because \| is not a special construct in the read syntax for strings, so all we have done there is needlessly escaped a | character; hence "a\|b" is no different to "a|b"), and a|b contains no regexp-special constructs, so it matches the three-character sequence a|b literally.

The elisp manual explains further:

`\' has two functions: it quotes the special characters (including
`\'), and it introduces additional special constructs.

Because `\' quotes special characters, `\$' is a regular
expression that matches only `$', and `\[' is a regular expression
that matches only `[', and so on.

Note that `\' also has special meaning in the read syntax of Lisp
strings (*note String Type::), and must be quoted with `\'.  For
example, the regular expression that matches the `\' character is
`\\'.  To write a Lisp string that contains the characters `\\',
Lisp syntax requires you to quote each `\' with another `\'.
Therefore, the read syntax for a regular expression matching `\'
is `"\\\\"'.

-- C-hig (elisp)Regexp Special

It is also worth noting that \ is not special within a character alternative (this is also true for most other regexp-special characters), and therefore [\] (aka "[\\]") matches a backslash.

As a `\' is not special inside a character alternative, it can never
remove the special meaning of `-' or `]'.  So you should not quote
these characters when they have no special meaning either.  This would
not clarify anything, since backslashes can legitimately precede these
characters where they _have_ special meaning, as in `[^\]' (`"[^\\]"'
for Lisp string syntax), which matches any single character except a
backslash.