I'm using comint to communicate with a REPL.

From time to time the REPL sends large amounts of data. This causes a significant lag (about one second), because my filter function gets called on a bunch of small strings, instead of fewer, larger strings. (even with a fast filter function, parsing 1MB of output still ends up taking 0.001 ms per filter call, times 1000 = 1s).

Is there a way to get emacs to package the output into larger chunks? process-adaptive-read-buffering is already set to t.

The manual says:

The output to the filter may come in chunks of any size. A program that produces the same output twice in a row may send it as one batch of 200 characters one time, and five batches of 40 characters the next. If the filter looks for certain text strings in the subprocess output, make sure to handle the case where one of these strings is split across two or more batches of output; one way to do this is to insert the received text into a temporary buffer, which can then be searched.

  • Reading the doc, it looks like enabling process-adaptive-read-buffering does the exact opposite of what you want. – politza Apr 8 '15 at 18:39
  • Is that so? I am misunderstanding this sentence? "This behavior can be remedied to some extent by setting the variable process-adaptive-read-buffering to a non-nil value (the default), as it will automatically delay reading from such processes, thus allowing them to produce more output before Emacs tries to read it. " – Clément Apr 8 '15 at 18:49
  • Never mind, I misunderstood your question. – politza Apr 8 '15 at 18:50
  • Like the manual says, it could help if you simply insert the text into a buffer and then only process the buffer's text once it gets long enough. – Jordon Biondo Apr 8 '15 at 19:10
  • @Jordon I'm already adding the text in a buffer, actually – Clément Apr 9 '15 at 3:23

two options:

  1. Advise comint functions which use (accept-process-output proc), so that accept-process-output is redefined to send nil instead of proc. Without proc, it will use adaptive read buffering and a longer timeout. This was broken in recent emacs, (after emacs-24.3.91) but fixed and improved by me after ~25.05.50.

  2. Change comint's accept-process-output to a function which does accept-process-output in a loop, with a custom timeout.

  • I can't really do that on the computers of users of my package though :/ But I look forward to your patch! – Clément Jun 16 '15 at 2:59
  • You could try #1 programatically in a few ways: pre-advice which redifines accept-process-output with a wrapper on the relevant comint-send or whatever it's called, then removes the wrapper post-advice. Or having a mode-specific wrapper around the comint-send, if your package has a mode. – Ian Kelling Jun 16 '15 at 3:36
  • Or #2. Or just wait for the patch to come in. – Ian Kelling Jun 16 '15 at 3:38
  • Thanks for the pointers. I'll try this. The patch approach sounds promising, too; even without using comint I'm seeing this problem. – Clément Jun 16 '15 at 3:40
  • So my patch went in, and it enables #1, for before emacs-24.3.91, or maybe even better after ~25.05.50 (very latest sources as of now). You can also temporarily modify process-output-filter, and restore it to it's previous value, like is done in readline-complete. The relevant patch. lists.gnu.org/archive/html/bug-gnu-emacs/2015-07/msg00123.html – Ian Kelling Jul 6 '15 at 5:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.