org-mode 9 too slow with long code blocks

Question

Emacs gets extremely slow when one code block in an org-mode file reaches some size — say > 2000 lines (Emacs 26.1 & org-mode 9.1.9).

Pointed out in this thread of 2015, the problem comes from syntax highlighting, activated by font-lock-mode.

Here's the profiling report

M-x profiler-start   
Type a character in the code block (inside, before or after)
M-x profiler-report

- redisplay_internal (C function)                               39467  84%
 - jit-lock-function                                            39454  84%
  - jit-lock-fontify-now                                        39454  84%
   - jit-lock--run-functions                                    39418  84%
    - run-hook-wrapped                                          39418  84%
     - #<compiled 0x11fd819>                                    39418  84%
      - font-lock-fontify-region                                39418  84%
       - font-lock-default-fontify-region                       39418  84%
        - font-lock-fontify-keywords-region                     39334  84%
         - org-fontify-meta-lines-and-blocks                    38967  83%
          - org-fontify-meta-lines-and-blocks-1                 38963  83%
           - org-src-font-lock-fontify-block                    38911  83%
            - org-font-lock-ensure                              37567  80%
             - #<compiled 0x52bfbd>                             37567  80%
              - font-lock-default-fontify-buffer                37567  80%
               - font-lock-fontify-region                       37559  80%
                - c-font-lock-fontify-region                    37559  80%
                 - font-lock-default-fontify-region              37559  80%
                  - font-lock-fontify-keywords-region              37435  80%
                   - c-font-lock-declarations                   35256  75%
                    + c-find-decl-spots                         34937  75%

From the same thread, quoting Adam M. :

According to the Emacs documentation:
'In Emacs 21 and later, turning on font-lock-mode automatically activates
the new Just-In-Time fontification provided by jit-lock-mode.'

It would appear, therefore, that just-in-time fontification is not taking
place in the code block with the whole code block being fontified at once -
and hence the delay.

Any help to confirm this hypothesis and how to enable just-in-time
fontification in the code block would be appreciated.

Does anybody know how to fix this ?

Answer 1

The function which is in charge of fontifying src code blocks is org-src-font-lock-fontify-block. This function creates a buffer with the right major mode and inserts the entire block contents there so it gets colorized. AFAIK this is what every major mode supporting some kind of nested syntax highlighting does in emacs (for example, python.el does the same to colorize input lines in python inferior mode). The drawback with this approach is that search based font locking cannot apply its usual partial JIT rehighlighting strategies based on invalidated regions of the buffer, since the entire ancillary buffer must be recolorized each time. A possible optimization could be to keep the font lock buffer alive and only update the parts that changed in the org src block (python.el does something like that, indeed) but that's not what the current implementation does and it doesn't seem to be trivial.

That said, I think keeping 2000 lines of source code inside an org src block is neither a standard use case nor a reasonable idea. You can use the #+INCLUDE ... src ... directive or split the block into a number of more reasonably sized cells. If you insist on keeping the entire block, then disable native highlighting of src code blocks.