I am running a command that produces a few thousand lines of output, which I would like to store in a buffer. I want to run it in the background and be alerted when it is finished. The problem is that it typically takes of order 10 seconds to report back, when the command itself takes a second or less to run.

Here is a trivial example. I set up my command to produce 2000 lines of output

(setq bjm-cmd "find /Users/bjm/ -ls | head -2000")

then use following to save the output to a string and write to a buffer

  (message "shell-command-to-string")
  (message (format-time-string "%c" (current-time)))
  (with-temp-buffer (insert-string
                     (shell-command-to-string bjm-cmd)))
  (message (format-time-string "%c" (current-time))))

This works effectively instantly.

Now if I instead run the command asynchronously:

(defun bjm-test-sentinel (process event)
  (if (string= event "finished\n")
      (message "finished")
    (message "failed"))
  (message (format-time-string "%c" (current-time))))

  (message "start-process-shell-command")
  (message (format-time-string "%c" (current-time)))
  (setq bjm-proc (start-process-shell-command "dummy" "dummy" bjm-cmd))
  (set-process-sentinel bjm-proc #'bjm-test-sentinel))

it takes about 15 seconds to complete.

I presume this is because the sentinel is parsing the output in some way - right?

My work around is to redirect the output of my command to a file and then read the file into the buffer. i.e.

(setq bjm-cmd "find /Users/bjm/ -ls | head -2000 > /tmp/junk")

This runs quickly, but feels like a bit of a hack - is there a better way?

The context for this is that I am working on a package to call OS X spotlight using the ivy and counsel libraries, but I think the question is a generic one.

  • For me, running a command that produces 10,000 lines of output takes 9.4 seconds to finish synchronously, and 9.4 seconds to finish asynchronously. What type of system are you on, and what emacs version? – Jordon Biondo Sep 24 '15 at 15:25
  • It's not about the sentinel — the sentinel is run when the process exits. You might be thinking about a filter function, which would receive the output as it arrives from the process. – legoscia Sep 24 '15 at 15:51
  • @JordonBiondo that is interesting. By default I am running emacs 24.4 on OS X 10.10.5. I am using the emacsformacosx build. I also tested with the macports build of emacs 24.5 with the same result (also starting emacs with -Q gives the same result). – Ben Maughan Sep 24 '15 at 21:14
  • @legoscia I see what you mean. I guess it is whatever part of emacs that is watching for the process to change state that is bothered by the fact that the process is generating many lines of output. Redirecting the output of the shell command to a file or /dev/null makes it run just as fast as with shell-command-to-string; I am puzzled! – Ben Maughan Sep 24 '15 at 21:55
  • @JordonBiondo I have now tested on a linux machine running emacs 24.4 and the problem goes away so this appears to be a bug associated with emacs on the Mac. Thanks for the help. – Ben Maughan Sep 25 '15 at 9:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.