5

Context

I'm using org-babel-tangle to generate my configuration from an Org file config.org. Recently, I did some clean-up by

  1. dispatching config.org to multiple Org files config/*.org;
  2. replace the content of config.org with multiple #+INCLUDE: config/*.org directives.

To illustrate a bit, the directory tree is as follows:

config/
|______ A.org
|______ B.org
|______ C.org
|
config.org

where

## config.org
#+INCLUDE: config/A.org
#+INCLUDE: config/B.org
#+INCLUDE: config/C.org
...

and

## [ABC].org
#+BEGIN_SRC emacs-lisp :tangle target1.el
;; elisp code
#+END_SRC

#+BEGIN_SRC emacs-lisp :tangle target2.el
;; elisp code
#+END_SRC
...

Issue & attempt

However, I ran into the issue that org-babel-tangle doesn't handle #+INCLUDE directives. So, I follow the idea of this post which consists of

  1. exporting config.org to an Org file config-export.org with org-export-to-file;
  2. org-babel-tangle the exported Org file config-export.org to tangle targets.

This approach does replace the multiple #+INCLUDE directives of config.org by the content of config/*.org, but all tangle options are erased. Namely, the exported config-export.org looks like this (all emacs-lisp tangle target filename are lost):

## config-export.org
#+BEGIN_SRC emacs-lisp
;; elisp code for target1.el
#+END_SRC

#+BEGIN_SRC emacs-lisp
;; elisp code for target2.el
#+END_SRC
...

Question

How to org-babel-tangle an Org file with #+INCLUDE directives as it is intended?

(i.e. (1) include all Org files (preserving :tangle target.el), then (2) org-babel-tangle the whole Org file.)

Related question

A similar question has been raised in the Org mode mailing-list.

  • Just to clarify, are you trying to tangle to different/unique file names inside each of the included org files? – Melioratus Oct 19 at 18:41
  • 1
    @Melioratus I would want to include Org files at first, then tangle each code block to potentially different files. – Firmin Martin Oct 19 at 19:12
  • 1
    Thank you for the clarification! Very fun question! There are several ways to do what you’re asking using existing functionality without writing a lot of custom code so I’ll post a multi method answer. – Melioratus Oct 20 at 0:32
2

I don't know how to do this without changing the code (it might be possible with org-babel-pre-tangle-hook but it did not seem at all simple, so I didn't try that). But as soon as you accept code changes, it does not seem too bad.

The simplest possible way I could think of was to emulate in org-babel-tangle what org-export-as is doing WRT included files. A simple test (similar to what you describe above) succeeded easily (but see caveats at the end), so if you want to try replicating the change, I include the patch below.

The patch itself would look big, but that's only because I'm taking all of the code of org-babel-tangle and enclosing it in another layer, which changes the indentation and that's most of the change. Ignoring whitespace changes, the "real" change is much smaller: I got the patch below with git diff -b in order to ignore whitespace. You can then see that it is very small: we just enclose almost the whole body of org-babel-tangle in (org-export-with-buffer-copy ...), so that we can do transformations to the text in a temp buffer and not disturb the original and then add one more transformation to process the #+INCLUDE: directives, before the resulting buffer is then tangled:

diff --git a/lisp/ob-tangle.el b/lisp/ob-tangle.el
index b74b3fa0c..5e6eebf63 100644
--- a/lisp/ob-tangle.el
+++ b/lisp/ob-tangle.el
@@ -206,6 +206,9 @@ export file for all source blocks.  Optional argument LANG-RE can
 be used to limit the exported source code blocks by languages
 matching a regular expression."
   (interactive "P")
+  (org-export-with-buffer-copy
+   (org-export-expand-include-keyword)
+
    (run-hooks 'org-babel-pre-tangle-hook)
    ;; Possibly Restrict the buffer to the current code block
    (save-restriction
@@ -306,7 +309,7 @@ matching a regular expression."
     (mapc (lambda (pair)
         (when (cdr pair) (set-file-modes (car pair) (cdr pair))))
           path-collector)
-   (mapcar #'car path-collector)))))
+    (mapcar #'car path-collector))))))
 
 (defun org-babel-tangle-clean ()
   "Remove comments inserted by `org-babel-tangle'.

That produces the expected results WRT the tangled files I think, but you should check to make sure that it does the right thing in your case: my tests are toy tests so they might miss tricky situations that might arise in your "real world" tests.

There is one problem which I don't understand yet: the config.org buffer itself is changed in my tests, despite the org-export-with-copy-buffer, so at the end of the tangle I have to undo the change with C-x u and save it. That's probably because I don't understand what org-export-with-copy-buffer is really doing, but I have not checked that code yet.

OTOH, if that is resolved and the tests all pass, we can propose this as an enhancement to the Org mode mailing list and put that problem (from 2010!) behind us.

EDIT: I don't see any problem with org-babel-tangle-file - it calls org-babel-tangle underneath, so in theory it should do the (almost) right thing and testing it with a simple example works fine for me. Maybe you can post a (small) example that gives you problems by editing your question?

I'm also testing with the following workaround to get around the issue of config.org getting changed:

#+begin_src emacs-lisp
(copy-file "./config.org.orig" "./config.org")
(org-babel-tangle-file "./config.org")
(delete-file "./config.org")
#+end_src

Ugly, but it seems to work (except that I have to answer the "changed on disk" question).

| improve this answer | |
  • Thanks again for your answer! It's almost perfect, here are my observations: (1) We need to add (require 'ox) somewhere. I added it under the interactive call. (If it is put at the very beginning, it would cause circular load). (2) It works interactively on config.org modulo the issue of expanding #+INCLUDE on the original buffer. Programmatically, I call org-babel-tangle-file "config.org" and #+INCLUDE directives are not expanded. Maybe it's a buffer issue ? I'm not sure. – Firmin Martin Oct 19 at 20:16
  • It's quite odd that this 2010 question has not been followed up as it is a very natural thing one would want to do. Maybe at the time, a solution would require more than two lines. – Firmin Martin Oct 19 at 20:24
  • Thanks! I'll see what I can do about org-babel-tangle-file as well as about the unwanted expansion in the buffer. If this does make it into Org mode, the require will be at the top level of ob-tangle.el, rather than inside the function. But it may be that org-export-with-copy-buffer is not quite right and something else will need to take its place. – NickD Oct 19 at 20:25
  • Yes, it's a bit odd but both of the people responsible for babel and tangle moved on to other projects and the export mechanism has since been rewritten from scratch: neither of the org-export-... functions above existed in the original implementation. – NickD Oct 19 at 20:27
2

Method 1

Manual method using default org-mode keybindings

Setup

  1. Add Org to Org Babel Load Languages configuration and then apply and save new configuration.

  2. Add #+EXPORT_FILE_NAME: keyword near top of config.org.

    e.g.

    #+EXPORT_FILE_NAME: temp.org
    
  3. Add #+PROPERTY: header-args:org :results drawer replace keyword near top of config.org.

    e.g.

    #+PROPERTY: header-args:org :results drawer replace
    
  4. Update #+INCLUDE: to include the src org syntax.

    e.g.

    #+INCLUDE: config/A.org src org
    #+INCLUDE: config/B.org src org
    #+INCLUDE: config/C.org src org
    
  5. Add :eval never header to each of SRC blocks that will be tangled inside the included config/*.org files.

    e.g.

    #+BEGIN_SRC elisp :tangle tangle1.el :eval never 
      ;; Code goes here
    #+END_SRC
    

Manual Execution Steps

  1. Open config.org in emacs.

    The config.org file should look similar to the example below:

    #+EXPORT_FILE_NAME: temp.org
    #+PROPERTY: header-args:org :results drawer replace 
    
    #+INCLUDE: config/A.org src org
    #+INCLUDE: config/B.org src org
    #+INCLUDE: config/C.org src org
    
  2. Export config.org using C-c C-e O v key chord. This will create a new temp.org file and open in emacs.

    The temp.org file should look similar to the example below:

    #+export_file_name: temp.org
    #+property: header-args:org :results drawer replace
    
    #+begin_src org
      ,#+BEGIN_SRC elisp :tangle tangle1.el :eval never 
        ;; Code goes here
      ,#+END_SRC
    #+end_src
    #+begin_src org
      ,#+BEGIN_SRC elisp :tangle tangle2.el :eval never 
        ;; Code goes here
      ,#+END_SRC
    #+end_src
    #+begin_src org
      ,#+BEGIN_SRC elisp :tangle tangle3.el :eval never 
        ;; Code goes here
      ,#+END_SRC
    #+end_src
    
  3. Execute all the blocks in the temp.org using C-c C-v C-b key chord.

    The temp.org file should look similar to the example below:

    #+export_file_name: temp.org
    #+property: header-args:org :results drawer replace
    
    #+begin_src org
      ,#+BEGIN_SRC elisp :tangle tangle1.el :eval never 
        ;; Code goes here
      ,#+END_SRC
    #+end_src
    
    #+RESULTS:
    :results:
    #+BEGIN_SRC elisp :tangle tangle1.el :eval never 
      ;; Code goes here
    #+END_SRC
    :end:
    
    #+begin_src org
      ,#+BEGIN_SRC elisp :tangle tangle2.el :eval never 
        ;; Code goes here
      ,#+END_SRC
    #+end_src
    
    #+RESULTS:
    :results:
    #+BEGIN_SRC elisp :tangle tangle2.el :eval never 
      ;; Code goes here
    #+END_SRC
    :end:
    
    #+begin_src org
      ,#+BEGIN_SRC elisp :tangle tangle3.el :eval never 
        ;; Code goes here
      ,#+END_SRC
    #+end_src
    
    #+RESULTS:
    :results:
    #+BEGIN_SRC elisp :tangle tangle3.el :eval never 
      ;; Code goes here
    #+END_SRC
    :end:
    
  4. Execute the tangle in the temp.org using C-c C-v C-t key chord. This will tangle the imported SRC blocks from the config/*.org files.

    This will create the tangled files similar to the files listing below:

    config.org
    tangle1.el
    tangle2.el
    tangle3.el
    temp.org
    
    ./config:
    A.org
    B.org
    C.org
    

This answer was tested using:
emacs version: GNU Emacs 27.1
org-mode version: 9.3.7

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.