14

I use Emacs with Geiser to hack on some Scheme code. As I'm playing around in the REPL I sometimes evaluate expressions that result in a lot of output, often all on one line.

For example, I just played with SRFI-41 (streams) and created a character stream from a big file; then I forced the stream and Geiser barfed the whole contents of the file as a character stream into my buffer. Almost immediately, Emacs ground to a halt as more and more characters where appended to the output line, and no matter how long I kept pressing C-g or C-c C-c I could not make Emacs (or Geiser) stop.

This broke my whole Emacs session as Emacs now completely ignores my input, thinking that it needs to give priority to printing this massive character stream all on one line into an unresponsive Geiser REPL buffer.

Is there something I can do to protect my Emacs session from my destructive curiosity? (Why does Emacs get so slow when displaying very long lines anyway?) Can I set a limit for long lines and tell Emacs that it's okay to simply not attempt to display very long lines?

  • possible duplicate of How do I handle files with extremely long lines? – nanny Dec 18 '14 at 21:32
  • 2
    Well, my question is not about displaying long lines per se; I want to know how I can avoid this kind of thing in the first place (Emacs reads the line from an inferiour process, it's not read from a file I could fix); and it's about how I can prevent losing my Emacs session to Emacs's dedication to a single dynamic buffer. – rekado Dec 18 '14 at 21:39
  • "Well, my question is not about displaying long lines" Then maybe you should change your title. Perhaps you want to filter the inferior process output and add new line after a certain amount of characters? – nanny Dec 18 '14 at 21:41
  • Really though, this has little to do with long lines. yes in an ansi-term for instance has a similar (but not that horrible) effect. Really it's just the volume of text that gives emacs pause. – PythonNut Dec 19 '14 at 0:07
  • Text insertion in a buffer is pretty fast, it's the redisplay operations that make it appear slower than it actually is. To be honest, running yes in a VTE terminal emulator maxes out all of my CPU cores, so I'd not use it as example. – wasamasa Dec 19 '14 at 10:01
12

As already answered in the comments, Emacs becoming very slow in its redisplay for long lines is a well-known issue. Fixing it would be very nice, but needs lots of thought to be pulled off correctly. I have an idea of how it could be accomplished based on section 6.3 of this document (basically, store visual line information in the current buffer and update it on insertion of whitespace, display properties, window changes, etc., then use that information in the redisplay code to avoid scanning for it all the time), but am not familiar enough with the C internals to pull it off.

There are workarounds though. The most obvious ones would be tuning display-related parameters (like, enabling visual line truncation in the graphical Emacs instance, using a non-graphical Emacs to have that done automatically, disabling Bidi features, etc.) and preprocessing the file contents you're reading in. A less obvious one is automatically post-processing the files, be it by actually truncating their lines or adding text properties that make the lines appear shorter than they actually are. To turn this into a more interesting answer, I'll present a pretty ugly hack of the former option that will only work for comint-derived modes:

(defun my-comint-shorten-long-lines (text)
  (let* ((regexp "^\\(.\\{80\\}\\).*?$")
         (shortened-text (replace-regexp-in-string regexp "\\1" text)))
    (if (string= shortened-text text)
        text
      (propertize shortened-text 'help-echo text))))

(add-hook 'comint-preoutput-filter-functions 'my-comint-shorten-long-lines)

This defines my-comint-shorten-long-lines, a function that takes a string possibly consisting of many lines and uses the power of regular expressions to replace any line in it with a length of 80 characters or more with a shortened version that displays the original text when hovering over it. When used as hook in comint-preoutput-filter-functions it will filter all comint output before it's displayed.

However, this rendition of the hack has a pretty serious weakness. In modes that have basic fontification going on (like, M-x ielm), it will happily cut off lines that are part of a string and will that way fontify everything until the next quote as string! That's not what we want and can be fixed with a bit more regex mastery (but will presumably break inside a REPL for a language like Python). While we're at it, let's highlight shortened output, too:

(defun my-comint-shorten-long-lines (text)
  (let* ((regexp "^\\(.\\{80\\}\\).*?\\(\"?\\)$")
         (shortened-text (replace-regexp-in-string regexp "\\1\\2" text)))
    (if (string= shortened-text text)
        text
      (propertize shortened-text 'font-lock-face 'shadow 'help-echo text))))

(add-hook 'comint-preoutput-filter-functions 'my-comint-shorten-long-lines)

That's a bit better, but still ugly. Hovering on the output of something like find / in M-x shell is not appealing (we'd ideally want only to display the unshortened line, not all output), string detection is rudimentary at best and truncation could be indicated better with ellipses instead of fontifying everything. On top of that, it's not even guaranteed that the text coming in isn't turned into batches. All of this screams for doing the processing step in a temporary buffer, but will be left for the reader as exercise (or the author as potential blog post).

4

As this happened with Python likewise, solution at python-mode.el, https://launchpad.net/python-mode, is to connect to process directly, not via comint-mode.

Relies on start-process and process-send-string

For example see functions py--start-fast-process and py--fast-send-string-intern

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.