How to use Helm to grep across a list of xz compressed files?

Question

I'm trying to implement a simple helm extension for work which among other things provides an action to "grep across selected log file candidates". I've got this all working for actions like this:

    (defun visit-logs-action (log-directory log-name)
      "Visit the selected log files"
      (let ((full-paths (mapcar (lambda (fn) (concat log-directory "/" fname)) 
                                (helm-marked-candidates))))
      (mapc #'find-file full-paths)))

(in the above I pass log-directory myself from a lambda and helm gives me log-name but I ignore it to instead consider all selected candidates (based on reading handling multiple selections)

I'm having trouble with a "grep across selected log files" primarily because they are xz compressed. I tried looking at the occur family of functions or even helm-swoop but those go off buffers. The closest I've come so far is with helm-ag by doing something like this:

    (defun grep-logs-action (log-directory log-name)
      "Any swoop/helm-ag/occur over the files would be great here."
      (let ((helm-ag-command-option "-z"))
        (helm-do-ag log-directory (helm-marked-candidates))))

but the output of that looks corrupt. Haven't found a good hook either for this in helm-grep

I also tried to find a way to use helm-rg but so far that does not seem to accept a list of paths and instead only a directory (and that works poorly because the log directory is huge with too many large logs). Am wondering if there are other simple alternatives that I could try here to hop into a swoop/grep/occur really anything to fulfill that extra functionality

Answer 1

First Things First

helm-find-files (Helm FF) already includes an action to run zgrep on the marked candidates, you can run it with M-g z (C-u to recurse).

But maybe your zgrep doesn’t support the .xz format, so...

Let’s Use ripgrep (rg) Instead

Unfortunately, the_silver_searcher (ag) has a bug, so we’ll have to stick to rg.

(defun helm-ff-rg-z (_candidate)
  "Grep with `rg -z' from Helm FF."
  (let* ((candidates (helm-marked-candidates :with-wildcard t))
         (relative-candidates (mapcar (lambda (candidate)
                                        (file-relative-name candidate helm-ff-default-directory))
                                      candidates))
         (helm-grep-ag-command (concat "rg -z --color=always --smart-case --no-heading --line-number --with-filename %s -- %s"
                                       " "
                                       (mapconcat (lambda (relative-candidate)
                                                    ;; make sure everything is quoted properly
                                                    (string-replace "%" "%%" (shell-quote-argument relative-candidate)))
                                                  relative-candidates
                                                  " "))))
    (helm-grep-ag helm-ff-default-directory helm-current-prefix-arg)))

(defun helm-ff-run-rg-z ()
  "Run `helm-ff-rg-z' from Helm FF with a key binding."
  (interactive)
  (with-helm-alive-p
    (helm-exit-and-execute-action 'helm-ff-rg-z)))
(put 'helm-ff-run-rg-z 'helm-only t)

(push '("Grep with `rg -z' (`C-u' to select file types)" . helm-ff-rg-z) (cdr (last helm-find-files-actions)))
(define-key helm-find-files-map (kbd "M-g R") 'helm-ff-run-rg-z)

You can now mark some candidates (including whole directories) and grep them with rg -z by pressing M-g R from Helm FF.

Result

Marked candidates

Grep For foo

Both files contain foo:

Grep For baz

Only the second file contains baz:

Answer 2

M-x zrgrep would be the normal way.

I've just noticed there's no zlgrep though -- that would be a good feature request to M-x report-emacs-bug for. A manual workaround might be M-! zgrep -nH <pattern> <files> RET and then M-x grep-mode in the output buffer (although the command line options might vary between systems).