4

How do I search for "foo bar" ignoring comments and newlines in emacs? For example, given the tex file fragment

blah blah foo % comment 1
% comment 2
bar blah blah

I'd like to be able to type "foo bar" at the emacs search prompt and have my cursor land at the end of "bar" in the third line.

Qualification: I'm actually using Aquamacs in OS X, but hopefully a good answer for emacs would carry over.

2

I can fake this via C-hv search-whitespace-regexp

It wouldn't be a full solution unless your comment marker character could never occur in any other context (e.g. the string "100%").

For testing select a window of the buffer with the test text, call M-: and paste the following lisp snippet into the minibuffer.

(progn
  (setq-local search-whitespace-regexp "\\([[:space:]\n]\\|%.*\n\\)+")
  (setq-local isearch-lax-whitespace t))

The following figure shows the matches in an extended test text for an isearch starting at the beginning of the buffer.

isearch matches skipping comments and newlines

| improve this answer | |
  • Even if this solution is not as general and stringent as mine the return to investment ratio is very high here. – Tobias Dec 15 '17 at 15:55
  • Thanks for taking the time to add that demonstration, @Tobias. – phils Dec 24 '17 at 7:53
  • Thanks. This is excellent, really simple. Is there a way to tweak this so that it won't also find occurrences of "foo bar" in comments? I.e., if you replace "comment 1" or "comment 2" in my original example by "comment 1 foo bar" and "comment 2 foo bar", then the search will still yield a unique hit (the same one as before), skipping the two "foo bar" occurrences in comments. – Dominic Jan 20 '18 at 5:41
  • @Dominic, by default this does what you've just asked? Appending "foo bar" to the two comments does not alter the behaviour -- the search still extends from the initial foo to the un-commented bar. If you're seeing something different, try again starting from emacs -Q. If it still doesn't work, which version of Emacs are you using? I've just tested successfully with 24.5, 25.3, and a 26 pre-release. – phils Jan 20 '18 at 10:44
  • @phils: Thanks for responding. I'm still using Aquamacs in OS X, which doesn't skip "foo bar"s within comments when I used your lisp snippet. I'm now curious about why there would be such a stark difference in behaviour between Aquamacs and emacs. – Dominic Jan 23 '18 at 21:56
1

If you use library hide-comnt.el then you can hide or ignore comments, including for searching (Isearch or Lisp searching).

If the region is active then the effect is limited to the region.

  • Command hide/show-comments-toggle toggles hiding and showing comments.

  • Command hide/show-comments hides comments or, with a prefix arg, shows them.

  • Macro with-comments-hidden temporarily hides comments while it executes a body of code. You do not see the commented text disappear and reappear – it is effectively just ignored while processing the body. The macro uses hide/show-comments to hide the comments.

| improve this answer | |
  • Evaluating (setq isearch-regexp-lax-whitespace t search-space-regexp "[\n\t ]") M-x hide/show-comments-toggle and isearch for foo bar in the test text fails. – Tobias Dec 14 '17 at 20:52
  • @Tobias: Yes, see my comments about Isearch. – Drew Dec 16 '17 at 0:32
1

If you use library Icicles then you can use Icicles search to do the following:

  • Search inside or outside of comments. A comment is a THING. Use M-s M-s t and tell it that the kind of THING you want to search is comment.

  • Search inside other kinds of THING, optionally ignoring such THINGs when inside comments. Options icicle-ignore-comments-flag and icicle-hide-whitespace-before-comment-flag control this ignoring. You can toggle this ignoring using C-M-; at any time. When comments are ignored, the candidate things (e.g. sexps) to be searched are only those outside of comments.

  • You can use icicle-search-text-property (C-c ") to search the zones of text that do not have a text-property value that you specify (e.g., property face with faces font-lock-comment-face and font-lock-comment-delimiter-face - which means search outside comments.

    This is an example of searching the complement of a set of zones. You can turn this context-complementing feature on and off using C-M-~ (icicle-toggle-search-complementing-domain) during Icicles-search completion.

| improve this answer | |
  • (let ((icicle-ignore-comments-flag t) (icicle-hide-whitespace-before-comment-flag t)) (icicle-search (point-min) (point-max) "foo[[:space:]\n]+bar" nil)) (with and withoug icicle-hide-whitespace-before-comment-flag) does not indicate any matches for the test case of the OP. So comments are not really ignored? – Tobias Dec 15 '17 at 17:30
  • icicle-ignore-comments-flag pertains only to icicle-search-thing, i.e., searching within THING contexts.Context that are, in effect, contiguous (e.g. because of invisible text) are not coalesced to a single context. The search pattern is looked for in each context separately. – Drew Dec 15 '17 at 19:04
  • Similarly, C-c " searches zones of text properties. You can search outside such zones (after C-M-~) - e.g., search outside comment-font-locked zones, or you can search inside them. But you cannot search across them with a single search pattern. So I guess this does not really help with the use case cited. – Drew Dec 15 '17 at 19:10
  • I suspected so. For that reason this answer does not meet the specified test in the question. Phil comes very close with very simple means. That is the reason I upvoted his answer. My approach is to search a clone buffer instead of the original one. Before the search I delete all invisible regions in the clone. For user-interaction (i.e., isearch-commands) the clone positions are transformed into positions of the original and the match data is set accordingly. – Tobias Dec 15 '17 at 19:13
  • Agreed............... – Drew Dec 15 '17 at 21:48
1

The following lisp code provides the command isearch-visible. It creates a clone of the current buffer and deletes all invisible text there. All isearch operations in the original buffer are then delegated to the clone.

Therefore, you can use Drew's hide/show-comment command from the package hide-comnt.el to hide the comments in your TeX buffer.

Afterwards you can search the visible text as the invisible text never existed with isearch-visible.

If you set search-whitespace-regexp to [[:space:]\n] and use M-x isearch-toggle-lax-whitespace you can use foo bar as search string with isearch-visible in your test TeX buffer to get what you want.

If you want to try the code just paste it into your init file and restart emacs.

CHANGE-LOG:

2017-12-15, 20:30:

  1. bugfix: error wrong argument type nil; expected number or marker

  2. bugfix: if the match end was at the beginning of a hidden region the calculation of the end position in the original buffer was wrong. This lead to match intervals that were too long.

(require 'subr-x)

(defvar-local clone-visible-clone nil
  "Visible clone of current buffer if there exists one.")

(defvar-local clone-visible-original nil
  "Original buffer of clone.")

(defun clone-visible-buffer ()
  "Function used as result of `isearch-search-fun-function'.
It searches within `visible-isearch-clone-buffer' instead of the original buffer.
Sets `clone-visible-clone' in the original buffer and `clone-visible-original' in the clone."
  (let* ((original-buf (current-buffer))
         buffer-file-name)
    (setq clone-visible-clone
      (with-current-buffer (clone-buffer)
        (setq clone-visible-original original-buf)
        ;; First scan: put original position into 'clone-visible text property
        (cl-loop for int being the intervals property 'invisible
             if (null (invisible-p (car int)))
             do (put-text-property (car int) (cdr int) 'clone-visible-int int))
        ;; Second scan: Remove invisible regions
        (cl-loop for int being the intervals property 'invisible
             if (invisible-p (car int))
             do (delete-region (car int) (cdr int)))
        (current-buffer)))))

(defsubst clone-visible-assert ()
  "Check whether we have a working clone."
  (cl-assert (buffer-live-p clone-visible-clone)
         nil
         "Visible clone %s defective." clone-visible-clone))


(defun clone-visible-original-position (&optional clone-position)
  "Returns position in original buffer corresponding to CLONE-POSITION in visible clone.
CLONE-POSITION defaults to `point'.
We assume that the clone is current."
  (unless clone-position
    (setq clone-position (point)))
  (when-let ((original-int (get-text-property clone-position 'clone-visible-int))
         (clone-int-start (or (previous-single-property-change clone-position 'clone-visible-int) (point-min))))
    (+ clone-position (- (car original-int) clone-int-start))))

(defun clone-visible-clone-position (original-position)
  "Find position in clone corresponding to ORIGINAL-POSITION in current buffer."
  (clone-visible-assert)
  (and original-position
       (with-current-buffer clone-visible-clone
     (cl-loop
      with original-int
      for int being the intervals property 'clone-visible-int
      if (and (setq original-int (get-text-property (car int) 'clone-visible-int))
          (>= original-position (car original-int))
          (< original-position (cdr original-int)))
      return (+ original-position (- (car int) (car original-int)))
      ))))

(defun clone-visible-search-regexp (regexp &optional bound noerror)
  "Searches the visible clone in place of the original buffer.
Uses `isearch-search-fun-default' for the search."
  (let* ((original-buffer (current-buffer))
         (clone-pt (clone-visible-clone-position (point)))
         (clone-bd (clone-visible-clone-position bound))
         (search-fun (isearch-search-fun-default))
         (original-pt (with-current-buffer clone-visible-clone
                        (goto-char clone-pt)
                        (setq clone-pt (funcall search-fun regexp clone-bd noerror))
                        (when clone-pt
                          (let ((match-data (butlast (match-data t))))
                            (cl-loop for int on match-data by #'cddr do
                                     (setcar int (clone-visible-original-position (car int)))
                                     (setcar (cdr int) (1+ (clone-visible-original-position (1- (cadr int))))))
                            (set-match-data (append match-data (list original-buffer)))
                            (match-end 0))))))
    (goto-char (or original-pt bound (point-max)))
    original-pt))

(defun isearch-visible ()
  "Like `isearch-forward' but only searches the visible part of current buffer."
  (interactive)
  (let ((isearch-search-fun-function
     (lambda ()
       #'clone-visible-search-regexp))
    (clone (clone-visible-buffer)))
    (when (buffer-live-p clone)
      (unwind-protect
      (progn
        (isearch-mode t t nil t)))
      (kill-buffer clone))))
| improve this answer | |
  • Looks good. I think something like this is probably the way to go, here. Either this way or some other way is needed to work around the problem that the text to search is really a disjoint set of zones, and what is requested is to search across the "joints", pretending that the zones to search are in fact contiguous in some places (and so can be coalesced). In the case of Isearch this is essentially bug #27896. – Drew Dec 16 '17 at 0:30
1

Library Isearch+ lets you do this in several ways.

In particular:

  • You can dynamically add a search filter that limits Isearch to zones outside comments. Function isearchp-not-in-comment-or-delim-p does this. You can add a filter while searching by hitting C-z & (command isearchp-add-filter-predicate). When prompted for the filter function, you can enter that function name or you can use completion and choose ~[;+].

    (Completion candidates have abbreviated names, and the full filter-predicate name accompanies them as annotations in *Completions*. For this abbreviation:

    • ~ means complement: NOT.
    • [...] means within.
    • ; means comments.
    • + means include the comment delimiters.

    (You can also just set isearch-filter-predicate to isearchp-not-in-comment-or-delim-p, but dynamic Isearch filtering gives you a quick way to filter on the fly.)

  • You can search the text of THINGS of various kind (sexps, lists, defuns, lines, pages, sentences, filenames, strings, comments, xml/html elements, symbols,…), using command isearchp-thing. This is equivalent to using command isearchp-thing-define-contexts, which marks such zones with a text property, and then using isearchp-property-forward.

  • When searching THINGS, you can toggle ignoring comments, using C-M-; (command isearchp-toggle-ignoring-comments). You can also hide or show code comments during Isearch, using M-; (command isearchp-toggle-hiding-comments).

  • You can search within character-property zones. Example: search within zones having a face text property with a value of font-lock-comment-face or font-lock-string-face. Search overlays or text properties.

    From within Isearch: C-t (or C-M-t for regexp search). First time, or with a prefix argument, you are prompted for the property and its values. See the doc string of command isearchp-property-forward.

  • For searching within THINGS or searching within character-property zones, you can do the opposite: search outside the given type of THING or outside the given type of property. You can toggle this complementing (inside/outside) anytime during Isearch using C-M-~.

All of these suffer from what is currently a fundamental limitation of Isearch filtering: search hits have to be entirely within the text that remains after filtering, but search itself matches all of the text. See Emacs bug #27896 for a description of the problem. It would be good for someone to provide a solution.

| improve this answer | |
  • (setq isearch-regexp-lax-whitespace t search-space-regexp "[\n\t ]") M-x isearchp-toggle-ignoring-comment and subsequent isearch for foo bar in the test text fails. – Tobias Dec 14 '17 at 21:09
  • @Tobias: Yes, see my updates here. – Drew Dec 16 '17 at 0:32
0

Call me old fashioned, but I would just use standard emacs regexp search

foo\s-*\(//[^
]
\s-*\)*bar

That’s off the top of my head. I will test it in emacs tomorrow - I don’t have emacs on my iPhone :-(.

I use isearch-forward-re to the exclusion of all else. Standard key binding C-M-s. You might prefer re-search-forward.

Commentary

  • \s- → character class whitespace
  • \s-* → 0 to infinite whitespace
  • \(…\) → group

Within the group:

  • // → your comment
  • [^␤] (with an actual line break instead of — type C-q C-j) → any character that is not a newline

And thus

//[^
]*
\s-* 

→ comment to end of line, followed by a newline and whitespace at beginning of line.

All of that in a \(…\) group, *’ed to repeat 0 to infinite times.

The funkiest part is the newlines in the regexp. I get them by typing C-q (quote-char) followed by C-j, since C-j is newline.

If all of the above is in an elisp string, then the backslashes will have to be doubled, and you can use \n for a newline.

In the really old days I might have typed [ blank tab ]* - which is illegible. Now I usually remember to type \s-*. But note that syntax class tables can vary.

Modern emacs uses character classes like [:blank:] for tab and space, or [:space:] for any whitespace including tab and space and newline.

So my original regexp might be more readable as

foo[[:space:]]*\(//\([[:blank:]]*[^[:space:]]*\)*\)*[[:space:]]*bar

The trick here is that I don’t know of any easily readable version of “any character except newline”, so instead I do a group of ( any blanks followed by any non-spaces ), which is equivalent when starred,

Again, untested.

foo\s-*\(//[^
]
\s-*\)*bar
| improve this answer | |
  • 1
    . (dot) is the regexp for "any character except newline". – phils Dec 24 '17 at 7:55
  • @phils yep, perl confusion, where . may or may not inclusion newline. //.* for comment to end if line, followed by your favorite way of saying newline. – Krazy Glew Dec 24 '17 at 8:10
  • 1
    The OP wants the search engine to ignore the comments. He does not want to be forced to think about interleaving comments and to construct more complicated regexps considering potential occurrences of comments. – Tobias Dec 24 '17 at 10:49
  • @Tobias fair enough. – Krazy Glew Dec 24 '17 at 16:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.