0

I have this code:

(with-output-to-temp-buffer OUTPUT_BUFFER
  (shell-command
   (format "unzip -p %s %s | cat" ZIP_FILE_NAME LOG_FILE_NAME)
   OUTPUT_BUFFER OUTPUT_BUFFER))

I read a .log file from a .zip file and insert its content in a temp-buffer.

I can't understand why, if the log file is "empty" (0 byte) the temp-buffer size is 1. I.e. it seems that shell-command inserts a newline in the temp buffer.

Can I prevent this?

5
  • What makes you think it does that? AFAICS it does not: ` (shell-command "cat /tmp/does-not-exist 1>/dev/null 2>&1" "OUTPUT_BUFFER" "OUTPUT_BUFFER")` writes nothing to OUTPUT_BUFFER, as you can check by saving it and doing an ls -l on it. Maybe you are fooled by the exit status it reports in the echo area?
    – NickD
    Mar 20, 2023 at 21:44
  • 1
    I tested with buffer size and moving cursor in the temp buffer. Maybe because "Shell commands usually terminate their output with a newline" emacs.stackexchange.com/a/21906/15606?
    – Gabriele
    Mar 20, 2023 at 21:51
  • 2
    Just evaluating (with-output-to-temp-buffer "test") already produces the size of 1. You can try to find out what happens by expanding the macro using pp-macroexpand-last-sexp, however, that did not make me any wiser. But I am not sure why you are using with-output-to-temp-buffer, you could just use (shell-command (format "unzip -p %s %s | cat" ZIP_FILE_NAME LOG_FILE_NAME) OUTPUT_BUFFER OUTPUT_BUFFER) directly, for the exact same effect... except of the mysterious 1 buffer size Mar 20, 2023 at 22:25
  • @dalanicolai I used with-output-to-temp-buffer because i want "always" the output buffer to be displayed. It doesn't happen "if the output is short enough to display in the echo area". If my.log file is empty shell-command doesn't displays the output buffer.
    – Gabriele
    Mar 21, 2023 at 10:35
  • 1
    Ah okay, I was not aware of this shell-command behavior indeed. In that case I would probably just, in the 'shell-command', pass (one of) the output buffers wrapped in a pop-to-buffer, e.g. as follows (shell-command "ls" (pop-to-buffer "out") "out")... Mar 21, 2023 at 11:22

2 Answers 2

2

The newline that the OP is worried about has nothing to do with shell-command or the command it executes (whether that command produces output or not - which in the case the OP describes, it does not).

The OP's case confuses the issue because he uses the same buffer in three different places: as the buffer used by with-output-to-temp-buffer and also as the buffer where the stdout and stderr of the command will end up.

Let us dispense with the command first. If you create a zip archive foo.zip that has (among other things) an empty file foo in it and you try to extract it, its stdout and stderr are both empty:

$ unzip -p foo.zip foo | wc
0 0 0
$ unzip -p foo.zip foo 2>&1 | wc
0 0 0

(piping into cat is superfluous in any case). So the command produces NO output.

Next, (shell-command "unzip -p foo.zip foo" "some-buffer" "some-buffer") leaves the buffer empty: no newline added. So "Shell commands usually terminate their output with a newline" is not the answer in this case.

@dalanicolai in his comment identified the culprit: with-output-to-temp-buffer does the following according to its doc string:

Bind ‘standard-output’ to buffer BUFNAME, eval BODY, then show that buffer.

Note that the standard-output stream has nothing to do with the stdout of the command that shell-command executed. It is usually used for *Help* buffers where

... it binds ‘standard-output’ to that buffer, so that output generated with ‘prin1’ and similar functions in BODY goes into the buffer.

So using it in this case is probably a misunderstanding of what it really does on the part of the OP. But it is nevertheless what adds that final newline into the buffer. Since we have dispensed with both the command itself and shell-command as possible culprits, let's take a simpler example as suggested by the doc string above:

(princ "foo")

That outputs three characters (no newline) to the standard-output stream. Now try:

(with-output-to-temp-buffer "FOO" (princ "foo"))

The buffer contains four characters: the macro has added a newline.

To dig further, let's expand the macro call:

(macroexpand '(with-output-to-temp-buffer "FOO" (prin1 "foo")))

---> (let*
    ((old-dir default-directory)
     (buf
      (with-current-buffer
          (get-buffer-create "FOO")
        (prog1
            (current-buffer)
          (kill-all-local-variables)
          (setq default-directory old-dir)
          (setq buffer-read-only nil)
          (setq buffer-file-name nil)
          (setq buffer-undo-list t)
          (let
              ((inhibit-read-only t)
               (inhibit-modification-hooks t))
            (erase-buffer)
            (run-hooks 'temp-buffer-setup-hook)))))
     (standard-output buf))
  (prog1
      (progn
        (prin1 "foo"))
    (internal-temp-output-buffer-show buf)))

So the macro calls the function internal-temp-output-buffer-show on the buffer. You can see that it is the culprit:

(with-current-buffer "FOO"
   (erase-buffer)
   (princ "foo" (current-buffer)))

The buffer size of FOO is 3. But with:

(with-current-buffer "FOO"
   (erase-buffer)
   (princ "foo" (current-buffer))
   (internal-temp-output-buffer-show (get-buffer "FOO")))

the buffer size is now 4 because of the added newline. Digging into internal-temp-output-buffer-show, there is a hook that is run at the end: (run-hooks 'temp-buffer-show-hooks) and the hook includes the function help-mode-finish. Finally, help-mode-finish calls help-make-xrefs and this outputs a newline:

          ...
          ;; Delete extraneous newlines at the end of the docstring
          (goto-char (point-max))
          (while (and (not (bobp)) (bolp))
            (delete-char -1))
          (insert "\n")
          ...

That's how the newline gets added.

The moral of the story is that with-output-to-temp-buffer is the wrong macro to use: it's a special-purpose macro, meant to populate buffers that are frequently rewritten (like the *Help* buffer), so using it in other cases is fraught with dangers. @dalanicolai's comment:

But I am not sure why you are using with-output-to-temp-buffer, you could just use (shell-command (format "unzip -p %s %s | cat" ZIP_FILE_NAME LOG_FILE_NAME) OUTPUT_BUFFER OUTPUT_BUFFER) directly, for the exact same effect... except for the mysterious 1 buffer size.

is spot-on.

EDIT: in response to the OP's comment about displaying the output buffer of the shell command always, one can do something like this:

(let ((foo (get-buffer-create "FOO")))
  (display-buffer foo)
  (with-current-buffer foo (erase-buffer))
  (shell-command "echo baz" foo foo))

@dalanicolai's suggestion of pop-to-buffer doesn't quite work, because it makes the popped buffer be the current buffer, which you don't want to do in this case. Depending on your needs, you can e.g. make this into a function where you pass in the command as a string, mark the buffer read-only as with-output-to-temp-buffer does, erase it (as I do) or leave the previous contents and move to the end before running the next shell command, etc.

4
  • 1
    Excellent analysis.
    – phils
    Mar 21, 2023 at 6:46
  • I used with-output-to-temp-buffer because i want "always" the output buffer to be displayed. It doesn't happen "if the output is short enough to display in the echo area". If my.log file is empty shell-command doesn't displays the output buffer.
    – Gabriele
    Mar 21, 2023 at 9:44
  • See the edit in the answer.
    – NickD
    Mar 21, 2023 at 16:48
  • I used save-selected-window combined with pot-to-buffer to restore the current buffer.
    – Gabriele
    Mar 23, 2023 at 9:29
0

"Shell commands usually terminate their output with a newline" may well be the answer -- although on my GNU/Linux system I cannot reproduce the issue for your specific example.

Speaking generally though, the shell-command* functions naturally give you all of the output from the shell, and that might include a trailing newline. If you want to bypass the shell, there are more direct ways of running processes. See C-hig (elisp)Processes and the "synchronous processes" and "asynchronous processes" nodes in particular. I've just updated my answer in the related Q&A you'd found to cover one of those options: https://emacs.stackexchange.com/a/21912/454

Alternatively (and also covered by that same linked answer), you can run a shell command which will suppress the newline. I use printf for that:

(shell-command-to-string "echo foo")
"foo
"

(shell-command-to-string "printf %s foo")
"foo"

(shell-command-to-string "printf %s \"$(echo foo)\"")
"foo"

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.