I have these strings:

  • First Type:

  • Second type:


Belong to the second type even strings with no "-" in their names, such as math/123456789 or CoRR/123456789.

I have to do some "search and replace" on all these strings, so I would like to define a variable ARXIV that matches all of these strings. So, I have defined:

;; arXiv:1234.56789vK
(setq ARXIV-A "arXiv:\\([0-9]+\\)\\.\\([0-9]+\\)\\(v[0-9]+\\)?")

;; hep-th/1234567vK
;; math.XX/1234567vK
(setq ARXIV-NA "\\(astro-ph\\|cond-mat\\|gr-qc\\|hep-ex\\|hep-lat\\|hep-ph\\|hep-th\\|math-ph\\|nlin\\|nucl-ex\\|nucl-th\\|physics\\|quant-ph\\|math\\|CoRR\\|q-bio\\|q-fin\\|stat\\|eess\\|econ\\)\\(\\.[A-Z]+\\)?/\\([0-9]+\\)\\(v[0-9]+\\)?")

(setq ARXIV (concat "\\("


---------- Buffer: foo ----------
---------- Buffer: foo ----------

(perform-replace (concat "\\(\\[\\)?"   
                 "\\1 BEG \\2 END \\3" t t nil 1 nil (point-min) (point-max))

---------- Buffer: foo ----------
[ BEG arXiv:1234.56789 END 1234
[ BEG arXiv:1234.56789 END 1234
[ BEG arXiv:1234.56789v1 END 1234
[ BEG hep-th/123456 END 
[ BEG hep-th.QA/123456 END 
[ BEG quant-ph.QA/123456 END 
[ BEG hep-th/123456v12 END 
---------- Buffer: foo ----------

The problem is that \\3 does not correspond to \\(\\]\\)? seeing as how ARXIV-A and ARXIV-NA do not have (always) the same length, I think.

I have tried different configurations of my variables, but the problem is always the same: in replacement code \digit does not work as I was expecting after ARXIV. Indeed,

(perform-replace (concat "\\(\\[\\)"    
                 "\\2" t t nil 1 nil (point-min) (point-max))

works as I was expecting (groups are useless here, but this is only an example to clear better my question).

Is there a way to do this "search and replacement" with only one variable?

P.S. Any better idea to entitle my question is welcome!

1 Answer 1


The problem is that \\3 does not correspond to \\(\\]\\)?

(concat "\\(\\[\\)?"

Gives you the following regexp:


Which has 10 groups. \\(\\]\\) is group 10, not group 3.

C-hig (elisp)Regexp Backslash confirms that only groups 1-9 can be referenced by the \DIGIT syntax, so you would need to modify the regexp to reduce the number of groups being captured, which you would typically do by using non-capturing (or "shy") groups in your regexp for the groups which you do not need to reference independently.

You specify a non-capturing group by using \\(?: to open the group, instead of just \\(

You can alternatively set/clobber an explicit group number by opening the group with \\(?NUM:

In your case:

(concat "\\(\\[\\)?"

Would mean \\3 would reference \\(?3:\\]\\) (and the original group 3 would not be accessible).

Finally, to incorporate your own useful comment into this answer:

While the \DIGIT syntax is limited to groups 1-9, higher group numbers are accessible in elisp with, E.g., (match-string 10)

Quoting C-hig (elisp)Regexp Backslash

\(?: ... \)
is the “shy group” construct. A shy group serves the first two purposes of an ordinary group (controlling the nesting of other operators), but it does not get a number, so you cannot refer back to its value with ‘\DIGIT’. Shy groups are particularly useful for mechanically-constructed regular expressions, because they can be added automatically without altering the numbering of ordinary, non-shy groups.

Shy groups are also called “non-capturing” or “unnumbered groups”.

\(?NUM: ... \)
is the “explicitly numbered group” construct. Normal groups get their number implicitly, based on their position, which can be inconvenient. This construct allows you to force a particular group number. There is no particular restriction on the numbering, e.g., you can have several groups with the same number in which case the last one to match (i.e., the rightmost match) will win. Implicitly numbered groups always get the smallest integer larger than the one of any previous group.

  • 1
    phils: I know about shy groups, thanks, but here I need to match all kind of archives and I was wondering if it could be possible with only one variable, instead of 3 or more. However, (concat "\\1 BEG \\2 END "(match-string 10)) in replacement part does the trick, thanks. Commented Mar 29, 2020 at 10:56
  • I don't actually understand the "only one variable, instead of 3 or more" aspect. The number of variables you define (if any), and their value(s), is entirely up to you, so I'm not sure what you mean by that.
    – phils
    Commented Mar 29, 2020 at 11:22
  • You might like the C-h f rx regexp compiler, for writing large regexps in a more structured and readable form. If you do some searching, you should find examples of how to use it. (Or if you're already using Emacs 27 then you'll find it documented in the elisp manual at C-h i g (elisp)Rx Notation.)
    – phils
    Commented Mar 29, 2020 at 11:26
  • I believe the answer is no; but also, there would be no difference between what you've described and using shy groups. If "everything in the variable" is considered a single group, then there's no reason for the variable to contain capturing groups, because you wouldn't/couldn't reference them in your scenario. I can only assume you have some other use for the variable in which those capturing groups are actually needed?
    – phils
    Commented Mar 29, 2020 at 12:02
  • ... If so, and if you are generating the regexp such that you don't know in advance how many groups there are, then I suggest that you generate two versions -- one with shy groups, and one without.
    – phils
    Commented Mar 29, 2020 at 12:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.