3

I frequently need a regular expression, that approximately matches identifiers in a given environment. Ignoring the "must not start with a number" requirement, this would mean e.g.

[[:alnum:]_]+      e.g. Python, C, ...

When used in complicated expressions, this is rather verbose. So I was wondering if there is any shorter way to express "character that may be part of an identifier or keyword".

Things that didn't work:

\w or \sw  . . . . Matches only word characters.
\s_  . . . . . . . Matches symbol constituents,
                   EXCEPT for word characters.
\(?:\w\|\s_\)  . . Does the job, but very verbosely.
                   Delegates knowledge of the languages rules
                   to the syntax table.
\_<.*?\_>  . . . . Works for simple cases, but breaks down
                   when trying to be specific; e.g. \_<.*?z\_>
                   will match the whole line
                      foo bar baz
                   rather than just "baz".
5
  • AFAIK your "very verbose" word-or-symbol-constituent match is pretty standard.
    – phils
    Oct 18, 2019 at 12:51
  • You could also use \_<.*?\_> for a non-greedy match between symbol-start and symbol-end positions.
    – phils
    Oct 20, 2019 at 13:00
  • 1
    @phils This breaks down for non-trivial examples. E.g. if I want to match all identifiers ending with _foo, I'd try \_<.*?_foo\_>. but this would match the whole line baz_baz bar_bar foo_foo.
    – kdb
    Oct 21, 2019 at 14:42
  • Yep, that's a very different criteria. Sorry, I was thinking that you wanted to match every symbol.
    – phils
    Oct 21, 2019 at 20:33
  • @phils I updated the question to be more clear on this.
    – kdb
    Oct 21, 2019 at 20:38

1 Answer 1

3

I do not know a shortcut for identifier regexps. Maybe, that is due to the complicated rules that exist for identifiers of some languages.

Nevertheless you can define your own format strings and convert them by format-spec to regexps with identifiers.

The following lisp code defines a function my-re for transforming extended regular expressions with %i as regexp for identifiers to normal regular expressions.

Furthermore, it exemplarily defines my-re-search-forward for searching with extended regexps.

format-spec implies some constraints:

  • if you do not want the identifier regexp at some place where %i occurs in the search string you have to replace it with %%i
  • the regexp may not end with a single % if that is a really problem you can replace it with something like %\{1\}
(defconst my-identifier-re "\\(?:\\w\\|\\s_\\)\\(?:\\w\\|\\s_\\|[0-9]\\)+"
  "Regular expression for identifiers.")

(defconst my-re-format-with-identifier
  `((?i . ,my-identifier-re)
    (?% . "%") ;; escape
    ;; self-insert all others:
    ,@(cl-loop for c from 0 upto 255
           collect (cons c (string ?% c))))
  "Format for format string with ")

(defun my-re (re)
  "Transform extended REGEXP into regexp."
  (format-spec re my-re-format-with-identifier))

(defun read-interactive (fun)
  "Read argument list like `interactive' does."
  (call-interactively `(lambda (&rest args)
             ,(interactive-form fun)
             args)))

(defun my-re-search-forward (re &rest args)
  "Imitate `re-search-forward' with RE and ARGS but add %i as identifier re."
  (interactive (read-interactive 're-search-forward))
  (apply #'re-search-forward
     (my-re re)
     args))

(defun my-occur (re &rest args)
  "Run `occur' with `re-search-forward' replaced by `my-re-search-froward'."
  (interactive (read-interactive 'occur))
  (apply #'occur (my-re re) args))

Example: my-occur input for a search for C-function declarations/headers. It is not complete but covers quite many situations and there are only a few false positives.

^ \{0,3\}\(%i[&*]?[[:space:]
]+\)+%i[[:space:]
]*( *\(%i[&*]? *\)*\(, *\(%i[&*]? *\)+\)*)[[:space:]
]*\({\|;\)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.