3 edited tags
| link
2 added 31 characters in body
source | link

I started off doing my theory in a let*. I'm using shell just as an example, the real target is another interpreter, so async-shell-command won't solve my real problem.

and it didn't work. I am evaling everything in the *scratch* buffer with C-u C-x C-eC-uC-xC-e, using *scratch* like a traditional lisp REPL.

I started off doing my theory in a let*

and it didn't work. I am evaling everything in the *scratch* buffer with C-u C-x C-e, using *scratch* like a traditional lisp REPL.

I started off doing my theory in a let*. I'm using shell just as an example, the real target is another interpreter, so async-shell-command won't solve my real problem.

and it didn't work. I am evaling everything in the *scratch* buffer with C-uC-xC-e, using *scratch* like a traditional lisp REPL.

1
source | link

Asynchrony when collecting output in a process buffer?

I want to capture output from a process buffer. This is not a duplicate of How do I reliably get output from a process? because that questioner is presupposing comint and an architecture for a function that returns the output. My question is at a lower level.

My theory was to get the value of point before sending a command string to the process buffer, call that value p1; send the command; get the new value of point, call it p2, then call (buffer-substring p1 p2) to get the output.

My theory works when sequentially executed in the REPL in the *scratch* buffer. The commands don't work when sequentially executed in a progn or in a let*.

I started off doing my theory in a let*

(with-current-buffer "*shell*"
  (let* (( p1   (point))
         ( _    (process-send-string "*shell*" "date\n"))
         ( p2   (point))
         ( text (buffer-substring-no-properties p1 p2)))
    `((p1 ,p1) (p2 ,p2) (text ,text))
    ))

((p1 326) (p2 326) (text ""))

and it didn't work. I am evaling everything in the *scratch* buffer with C-u C-x C-e, using *scratch* like a traditional lisp REPL.

I unrolled everything into separate calls setting global variables to investigate why it didn't work, and discovered that it DOES work if I run the commands sequentially in the REPL:

Here, I execute each line independently and show the output:

(setq p1 (with-current-buffer "*shell*" (point)))

275

(process-send-string "*shell*" "date\n")

nil

(setq p2 (with-current-buffer "*shell*" (point)))

326

(with-current-buffer "*shell*"
  (buffer-substring-no-properties p1 p2))

"date Sat Sep 15 10:47:38 PDT 2018

~ ⌚ 10:47:38 $ "

That's great. My theory is not completely crazy. Now, package that all back up in a let*, just in baby steps. First, just put it all in a progn, so that instead of four commands, I will just have one:

(progn (setq p1 (with-current-buffer "*shell*" (point)))
       (process-send-string "*shell*" "date\n")
       (setq p2 (with-current-buffer "*shell*" (point)))
       (setq text (with-current-buffer "*shell*"
                    (buffer-substring-no-properties p1 p2)))
       `((p1 ,p1) (p2 ,p2) (text ,text))
       )

((p1 473) (p2 473) (text ""))

Now it doesn't work.

To summarize: the commands work when sequentially executed in the REPL in the *scratch* buffer. The commands don't work when sequentially executed in a progn or in a let*.

There is some asynchrony going on that I don't understand and don't know how to manage. The first call to point, the call that sets the value of p1, somehow happens after the call to process-send-string, even though I explicitly call point before calling process-send-string. p1 has the same value before and after the call of process-send-string, and that value is at the end of the buffer. There is no way that the first call of point can clairvoyantly know what the value of point will be after processing the command, so the call of point must happen after the call of process-send-string, or else some magical updating is going on, or some kind of lazy evaluation.

My questions are:

  1. (this is what I really want) What is the right way to collect output from a command buffer?

  2. Out of curiosity, what about this hidden asynchrony? How and where is it documented? How could I work my way around it?