OK, this has been driving me nuts every time I've run into it for over a year now and I finally sat down and figured out what is going on... Unfortunately I don't have a fix---only the actual explanation for the behavior the original poster asked about (I'm on Emacs 29.1.50, MacOS, but that's not super-relevant to the main point here)
% g -> dired-mark-files-containing-regexp
When I run
find-name-dired, I don't toggle mark on all files and hit
Q. I like to mark the files using
% g to test that my regexp working as I expect, then do
Q on the marked files.
(The fact that
% g finds the expected files with new lines in the query, but
Q reports no results found is what was driving me bonkers)
When in the list of files produced by
% g runs the command
In the definition, we see the given regexp is being passed directly to Emacs'
(defun dired-mark-files-containing-regexp (regexp &optional marker-char)
(let ((prebuf (get-file-buffer fn)))
(message "Checking %s" fn)
;; For now we do it inside emacs
;; Grep might be better if there are a lot of files
(if (and prebuf (not dired-always-read-filesystem))
(re-search-forward regexp nil t)))
(re-search-forward regexp nil t)))))))
Matching files are marked.
Q -> dired-do-find-regexp-and-replace
When you have marked files and do
Q, this runs
dired-do-find-regexp-and-replace. The help for this function states:
REGEXP should use constructs supported by your local `grep' command.
That was my tip-off to what the issue is here, but following further into the details of what
dired-do-find-regexp-and-replace does (indenting under a function call describes what the called function does):
- Assigns "from" (the find pattern) and "to" (the replacement).
- Calls (dired-do-find-regexp from) (returns xrefs of matches)
- Does a bunch of stuff to list files recursively and handle ignored files to produce
- Internally relabels the received
- Calls (xref-matches-in-files regexp files)
- This function's documentation says: "See
xref-search-program-alist for how to control which program to use when looking for matches."
xref-search-program defaults to "grep" and specifies the variable value must reference a corresponding entry in
xref-search-program is why this doesn't work.
From Gnu Grep's documentation:
How can I match across lines?
Standard grep cannot do this, as it is fundamentally line-based. Therefore, merely using the [:space:] character class does not match newlines in the way you might expect.
With the GNU grep option -z (–null-data), each input and output “line” is null-terminated; see Other Options. Thus, you can match newlines in the input, but typically if there is a match the entire input is output, so this usage is often combined with output-suppressing options like -q
MacOS's default grep doesn't have the -R or -z options. It's easy to install Gnu Grep from homebrew (
brew install grep, which must be run as ggrep), but the output is messy and I suspect not the output that Emacs expects to deal with in the context of creating a buffer listing xrefs to matching regions.
I do not want to get into customizing weird variables for parsing xref lines or whatever in order to make this work.
Why it works in a single buffer
query-replace-regexp (C-M-%) doesn't bring grep into it. It just uses Emacs regexp searching, and then calls
perform-replace on regions matching the query.
If the initial searching step of
dired-do-find-regexp-and-replace (Q on marked files) finds results, it doesn't call
query-replace-regexp; it calls
xref-query-replace-in-results, which calls
xref--query-replace-1, which finally calls
perform-replace (but it has been wrapped in a bajillion layers of special behavior and assumptions at this point).
ugrep: A possible solution? (nope)
In looking at
xref-search-program-alist, I noticed it contains an entry for ugrep, which I had not used before. Its README file states:
Matches multiple lines with \n or \R in regex patterns, no special options are required to do so!
WARNING: It looks like it would be A PAIN to install ugrep on Windows if you are not setup in subsystem for Linux.
It was easy to install on my Mac with
brew install ugrep.
The ugrep README has a section "Using ugrep within Emacs", but following those instructions did not work for me (Emacs version 29.1.50).
Manuel Uberti writes:
Once set, you can just hit C-x p g (project-find-regexp) in your project and let ugrep do his magic.
In my terminal the following works to find the regions I want to fix:
ugrep -rn '^\s+# rubocop:todo Layout\/LineLength\n(\s+#.*?\n)\s+# rubocop:enable Layout\/LineLength\n' .
C-x p g with either of the following input values gets no results:
'^\s+# rubocop:todo Layout\/LineLength\n(\s+#.*?\n)\s+# rubocop:enable Layout\/LineLength\n'
^\s+# rubocop:todo Layout\/LineLength\n(\s+#.*?\n)\s+# rubocop:enable Layout\/LineLength\n
However, if I follow the
find-name-dired > t > Q workflow with the same regexps, it still doesn't work:
Note: ‘\n’ here doesn’t match a newline; to do that, type C-q C-j instead
No matches for: ^\s+# rubocop:todo Layout\/LineLength\n(\s+#.*?\n)\s+# rubocop:enable Layout\/LineLength\n
Since one of the features of ugrep is matching newlines, I'd hoped you wouldn't have to use the C-q C-j syntax. Well, I also hoped the search would work. Unfortunately it also didn't work with line breaks entered via C-q C-j:
No matches for: ^\s+# rubocop:todo Layout\/LineLength
)\s+# rubocop:enable Layout\/LineLength
SO, I'm at writing a quick script to deal with this for now, but there's the explanation why
% g multiline works in the
find-name-dired buffer, but
Q with the same query does not.