How to recursively search / grep file contents in a directory / subdirectories in Emacs?

Question

I'm using Emacs for quite some time and I still somehow fail to understand what is the best way to (recursively) grep for a string within a (project) directory and get the results presented either as a dired buffer or in some even more useful way. Please enlighten me.

Answer 1

Well, since the original question doesn't mention rgrep, I'll go ahead and point it out here. This is the simplest option already built into Emacs, and I've found it to be more than sufficient for most things.

From the documentation,

Recursively grep for REGEXP in FILES in directory tree rooted at DIR.

The search is limited to file names matching shell pattern FILES.

So, for instance, to search all python files beneath my home directory for uses ofsome_function, I would call it with some_function, *.py, ~/ as the arguments. Results are displayed in a new buffer.

Answer 2

I happily use M-x find-grep-dired for years. It returns the results as a dired buffer, which you could use.

Answer 3

Solution 1 (best solution):

Install counsel (https://github.com/abo-abo/swiper/blob/master/counsel.el)

Then M-x counsel-git-grep.

No setup needed (git knows the project root and files to exclude). Both git grep and counsel is efficient.

The project need be managed by git.

counsel requires ivy-mode.

Solution 2:

This solution uses grep and works on any project. It's inferior to the Solution 1 because it's slower and need manual setup. It's also based on ivy-mode.

(defvar simple-project-root "~/.emacs.d")
(defun simple-grep ()
  (interactive)
  (unless (featurep 'ivy)
    (require 'ivy))
  (let* ((default-directory (file-name-as-directory simple-project-root))
         (keyword (read-string "Keyword:")))
    (ivy-read (format "Grep at %s:" simple-project-root)
              (split-string (shell-command-to-string (format "grep -rsnI %s ." keyword)) "\n" t)
              :action (lambda (val)
                        (let* ((lst (split-string val ":"))
                               (linenum (string-to-number (cadr lst))))
                          ;; open file
                          (find-file (car lst))
                          ;; goto line if line number exists
                          (when (and linenum (> linenum 0))
                            (goto-char (point-min))
                            (forward-line (1- linenum))))))))

You need create .dir-locals.el to setup simple-project-root, see https://www.gnu.org/software/emacs/manual/html_node/emacs/Directory-Variables.html for technical details

The code in solution 2 is just a prototype. My real implementation is much more complex. See counsel-etags-grep in https://github.com/redguardtoo/counsel-etags/blob/master/counsel-etags.el

Summary:

Those are the best two solutions I know.

If any other better solutions exist, they need at least solve below problems to be production-ready,

• how to get the keyword to grep (for example, get keyword from selected region)

• escape the keyword

• if more efficient grep program exist, we should use it (ripgrep, the_silver_searcher/ag, ... etc), or else fallback the default grep

• candidate window should use full width of screen and users can filter candidates interactively (that's why people use ivy or helm)

• we should show the relative path in candidate window

• able to re-use previous grepped result. So previous result should be saved. You can use ivy-resume from ivy or helm-resume from helm

• When you save previous grepped result, the context of previous result should also be saved. For example, in the code of of Solution 2. default-directory is context. See https://github.com/abo-abo/swiper/issues/591 for more details.

• Extended regular expression should be used because it's simpler and is already used by counsel-git-grep and the the_silver_searcher/ag.

Answer 4

Here is an example of a custom grep function that recursively greps file contents (using a regexp for the search term) in a directory and its subdirectories, and displays the results with lines of context before and after. The built-in compile.el and grep.el libraries provide for several methods to jump to the location in the file containing the search results. There is no need to install anything else for this example to work -- all that is needed is a full version of Emacs and a computer that has the grep utility installed. The variable grep-program can be adjusted to point to a particular location of the grep utility if the default is not sufficient.

(defun recursive-grep ()
"Recursively grep file contents.  `i` case insensitive; `n` print line number;
`I` ignore binary files; `E` extended regular expressions; `r` recursive"
(interactive)
  (let* ((search-term (read-string "regex:  "))
         (search-path
           (directory-file-name (expand-file-name (read-directory-name "directory:  "))))
         (default-directory (file-name-as-directory search-path))
         (grep-command
           (concat
             grep-program
             " "
             "-inIEr --color=always -C2"
             " "
             search-term
             " "
             search-path)))
    (compilation-start grep-command 'grep-mode (lambda (mode) "*grep*") nil)))

Answer 5

I'd like to add another solution to @chen bin's solution which also uses counsel (but you don't need to have a project maintained by git for this).

You need to install ag (the silver searcher) and counsel provides the command counsel-ag which searches the current directory for the entered string.

If you want to search within a project(not just the current working directory), add this to your .emacs :-

  (defun rag/counsel-ag-project-at-point ()
    "use counsel ag to search for the word at point in the project"
    (interactive)
    (counsel-ag (thing-at-point 'symbol) (projectile-project-root)))

This function will use ag to recursively search the project root directory.

Edit: the above function defaults to the symbol at point for searching. If you do not like that behaviour, replace (thing-at-point 'symbol) with nil. And if you want the search results in it's own buffer press C-c C-o

Edit2: if you've ripgrep installed, you can replace counsel-ag with counsel-rg in the above function and it'll just as fine :)