Multi-processing via TRAMP with asynchronous shell commands

Question

I found the following code in TRAMP:

(defun tramp-adb-handle-shell-command
  (command &optional output-buffer error-buffer)
  "Like `shell-command' for Tramp files."
  (let* ((asynchronous (string-match "[ \t]*&[ \t]*\\'" command))
     ;; We cannot use `shell-file-name' and `shell-command-switch',
     ;; they are variables of the local host.
     (args (list "sh" "-c" (substring command 0 asynchronous)))
     current-buffer-p
     (output-buffer
      (cond
       ((bufferp output-buffer) output-buffer)
       ((stringp output-buffer) (get-buffer-create output-buffer))
       (output-buffer
        (setq current-buffer-p t)
        (current-buffer))
       (t (get-buffer-create
           (if asynchronous
           "*Async Shell Command*"
         "*Shell Command Output*")))))
     (error-buffer
      (cond
       ((bufferp error-buffer) error-buffer)
       ((stringp error-buffer) (get-buffer-create error-buffer))))
     (buffer
      (if (and (not asynchronous) error-buffer)
          (with-parsed-tramp-file-name default-directory nil
        (list output-buffer (tramp-make-tramp-temp-file v)))
        output-buffer))
     (p (get-buffer-process output-buffer)))

    ;; Check whether there is another process running.  Tramp does not
    ;; support 2 (asynchronous) processes in parallel.
    (when p
      (if (yes-or-no-p "A command is running.  Kill it? ")
      (ignore-errors (kill-process p))
    (tramp-compat-user-error p "Shell command in progress")))

    (if current-buffer-p
    (progn
      (barf-if-buffer-read-only)
      (push-mark nil t))
      (with-current-buffer output-buffer
    (setq buffer-read-only nil)
    (erase-buffer)))

    (if (and (not current-buffer-p) (integerp asynchronous))
    (prog1
        ;; Run the process.
        (apply 'start-file-process "*Async Shell*" buffer args)
      ;; Display output.
      (pop-to-buffer output-buffer)
      (setq mode-line-process '(":%s"))
      (shell-mode))

      (prog1
      ;; Run the process.
      (apply 'process-file (car args) nil buffer nil (cdr args))
    ;; Insert error messages if they were separated.
    (when (listp buffer)
      (with-current-buffer error-buffer
        (insert-file-contents (cadr buffer)))
      (delete-file (cadr buffer)))
    (if current-buffer-p
        ;; This is like exchange-point-and-mark, but doesn't
        ;; activate the mark.  It is cleaner to avoid activation,
        ;; even though the command loop would deactivate the mark
        ;; because we inserted text.
        (goto-char (prog1 (mark t)
             (set-marker (mark-marker) (point)
                     (current-buffer))))
      ;; There's some output, display it.
      (when (with-current-buffer output-buffer (> (point-max) (point-min)))
        (display-message-or-buffer output-buffer)))))))

In particular,

    ;; Check whether there is another process running.  Tramp does not
    ;; support 2 (asynchronous) processes in parallel.
    (when p
      (if (yes-or-no-p "A command is running.  Kill it? ")
      (ignore-errors (kill-process p))
    (tramp-compat-user-error p "Shell command in progress")))

means that if one runs async-shell-command once in a TRAMP buffer, then one cannot run another one in parallel.

In fact, if one runs async-shell-command in one TRAMP buffer and tries to run another async-shell-command in another unrelated TRAMP buffer (representing different connection, e.g. local sudo), this will also fail with the prompt to kill the first one.

That limitation appears weird to me, especially the second case. To further support my point, if one runs shell in a TRAMP buffer and runs it one or many more times, one will get as many concurrent shells via that TRAMP connection as requested. So why async-shell-command is the underdog here? I'm not a fan of running fully-blown shells when it's not necessary.

Answer 1

The quoted limitation does not exist any longer. Moreover, the function tramp-adb-handle-shell-command does not exist any longer. All different implementations of shell-command in Tramp have been unified in tramp-handle-shell-command, which is able to run several asynchronous processes in parallel.

This is brand-new, it will appear with Tramp 2.4.2 (planned to be released end of June). For the time being, you could use either Emacs or Tramp git repository to test this.

Answer 2

Here is my current workaround by the way:

    (if (file-remote-p default-directory)
        (let* ((buffer         (get-buffer-create "*Async Shell Command*"))
               (buffer-name    (buffer-name buffer))
               (buffer-process (get-buffer-process buffer))
               (filter         nil))
          (when buffer-process
            (setq buffer (generate-new-buffer buffer-name)))
          (with-current-buffer buffer
            (shell-command--save-pos-or-erase))
          (make-comint-in-buffer command buffer "/bin/sh" nil "-c" command)
          (setq buffer-process (get-buffer-process buffer))
          (setq filter         (process-filter buffer-process))
          (cl-assert (eq filter #'comint-output-filter))
          (with-current-buffer buffer
            (require 'shell)
            (shell-mode)
            (set-process-sentinel buffer-process #'shell-command-sentinel)
            (set-process-filter   buffer-process #'comint-output-filter)
            (display-buffer buffer '(nil (allow-no-window . t)))))
      (let ((async-shell-command-buffer 'new-buffer))
        (async-shell-command command)))

It's a synthesis of what shell-command does in case of a async-shell-command together with what make-comint-in-buffer already does so far to establish the asynchronous process via start-file-process (see comint-exec-1). Just wondering if this is somewhat better or worse than the stock async-shell-command (even for non-remote cases)? In any case, it works fine for multi-processing via TRAMP. Why (maybe because of start-file-process)?