Org babel caching for R when using session

Question

I have an file which is a research paper (here); it contains numerous R blocks which produce tables and figures. Since these share common data and preliminary calculations, they are run as a org-babel session.

However, some of these tables and figures are reasonably time-consuming (a few seconds each), such that export becomes a somewhat lengthy proposition. A lot of the time at the end I am not changing the figures at all, just making minor amendments and corrections to the text, yet each re-render still takes a long time.

Caching is what I would seem to want, but the org mode manual address this specifically:

Note that the :cache header argument will not attempt to cache results when the :session header argument is used, because the results of the code block execution may be stored in the session outside of the Org mode buffer.

My question is - what does the second half of this sentence mean? Is this "may be stored" as in "might be stored" (so babel shouldn't) or as in "can be stored" (so babel needn't). Does anyone know if there's a technical org-internal reason that caching won't work with session?

And above all - any solutions to caching long-ish R operations when using org-babel? And ideally, without writing a lot of unnatural code in the R blocks. I already do some thigns like if ( ! exists(foo) ) foo <- read.delim(...) to avoid repeating the longest data-loading operations but this can be a bit error prone and muddies the R code.

Answer 1

It means that side-effects may exist from a previous code block(s) evaluated that alter the results of the next evaluation. Consider this code in Python (because I'm not familiar with R):

#+name: one
#+BEGIN_SRC python :session
  list = some_other_list_class
#+END_SRC

#+name: two
#+BEGIN_SRC python :session
  list(foo)
#+END_SRC

The result of two will depend on whether or not one was run in the same session; therefore simple caching doesn't give the whole story.

There are two work-arounds I can think of:

Pass every variable explicitly

Anytime you'd use a variable, create a named source block that just has that value, and use those as a :var reference

#+name: one
#+BEGIN_SRC python
  range(3)
#+END_SRC

#+RESULTS: one
| 0 | 1 | 2 |

#+name: two
#+BEGIN_SRC python :var l=one
  map(lambda x: x*2, l)
#+END_SRC

#+RESULTS: two
| 0 | 2 | 4 |

#+name: three
#+BEGIN_SRC python :var a=one b=two
  import itertools
  list(itertools.product(a,b))
#+END_SRC

#+RESULTS: three
| 0 | 0 |
| 0 | 2 |
| 0 | 4 |
| 1 | 0 |
| 1 | 2 |
| 1 | 4 |
| 2 | 0 |
| 2 | 2 |
| 2 | 4 |

Evaluate manually on change, not on export

Put :eval no-export in your header arguments, and the processing won't happen on export, only when you manually call it with C-c C-c or M-x org-babel-execute-buffer