Grep pipe support and how to filter grep results

Question

I try to filter the output of M-x grep with grep --color=always -nH -e "text_to_find" ~/path/to/files/*.org | grep -v "text_to_ignore" but it doesn't seem to be working. The 'to be ignored' part is still not filtered away.

Is it true that grep in Emacs do not support piping? If so, how do we filter grep results?

Answer 1

Note that the first grep grep --color=always -nH -e "text_to_find" ~/path/to/files/*.org puts escape sequences around matches for text_to_find to colorize those matches.

If the second search string text_to_ignore contains text_to_find. Matches of text_to_ignore in the original files are not found in the output of the first grep call because of those escape sequences.

Leave the --color=always option out to avoid that effect. The matches of text_to_find are no longer highlighted with yellow color but the second grep works if text_to_find is part of text_to_ignore.

If you really need to colorize the matches for text_to_find you can colorize them as a last step:

grep -nH -e "text_to_find" ~/path/to/files/*.org | grep -v "text_to_ignore | grep --color=always -e "text_to_find""

Answer 2

What you were attempting with a pipe works fine for me (tested in 26.1 and 25.3).

• Which version are you using?
• Does your command work outside of Emacs?

Edit: C-hv grep-use-null-device might mess up your command, by preventing the second grep from reading stdin. This would be apparent in the command line displayed in the grep buffer. If this is the issue, then hopefully you can simply disable that option via the customize interface.

---

Failing that, you can always filter the results after the fact with the flush-lines and keep-lines commands, after toggling the read-only state with C-xC-q.

Personally I like/use the winnow library for streamlining that:

Use x to remove lines matching a pattern, or m to keep only the lines matching a pattern.

Answer 3

Another alternative:

The elgrep library allows searching files without external programs and with user-defined search function.

The command M-x elgrep-menu -- also available as menu item Tools -> Search Files (elgrep)... -- provides an input field Search Function.

The following search function searches for the given regular expression text but ignores lines that match text_to_ignore.

(lambda (re &rest _) (and (re-search-forward re nil t) (null (string-match-p "text_to_ignore" (thing-at-point 'line)))))

Remarks:

• elgrep only checks whether the return value of the search function is non-nil
• it evaluates the match data after a successful return of the search function
• string-match-p preserves the match data
• (thing-at-point 'line) uses forward-line and therefore does also not change the match data

The following image shows the full dialog of elgrep-menu.

Answer 4

The elgrep library has evolved. So there is now a better alternative way to search for lines matching text_to_find but not matching text_to_ignore. One does no longer need to explicitly input Elisp functions.

One can define records of a file, and specify a regular expression that must not match in a record to be considered in the search for another regular expression. Full lines are an example for records.

Settings for your usecase in the elgrep-menu (Tools → Search Files (Elgrep)...):

• Set Expression to List of regexps
• Input text_to_find as first regexp
• Input !text_to_ignore as second regexp Note, that the leading ! negates the expression. The negated expression must not match within the record.
• Input the directory and the file name regexp. Note that you can use completion for the input of the directory. Furthermore resetting the elgrep menu sets the directory to the directory of the last visited file.
• Set Beginning of Record to Regexp and use ^ as regexp standing for beginning-of-line.
• Set End of Record to Regexp and use $ as regexp standing for end-of-line.

Note that this method can be inefficient for very large files with many lines (order of 100MB with 100 Mio lines) with only few matches for text_to_find because each line is searched for the regexps.