1

I'm starting using org-mode and org-babel to support my day by day scientific work and produce reproductibles papers.

Lot of people use rMarkdown/jupyter to do that, but i'm interested by investing some time to do more or less the same things using org-babel and org-mode.

Running chunk of code using org-babel is as simple as running chunk of code with RMarkdown/KnitR, and it seems org-babel is much powerfull on this side, so no problem with that actually

For me, problem arrives when i try to manage path, and start using :session property, especially with shell (sh/bash).

I define the workspace :dir for source code like this :

:PROPERTIES:
:export_file_name: reproductiblepaper
:header-args: :dir ~/home/xxx/reproductible_paper tangle:yes session: reproductible
:END:

It's common to use git or any cvs to download and run code producing result, so naively i wrote in my org code :

#+begin_src sh
git clone git@gitlab.com:xxx/yyy/mygitproject.git
#+end_src

Folder appear in ~/home/xxx/reproductible_paper/mygitproject/ as expected.

After that, i don't want to rewrite my :dir path for each #+BEGIN_SRC ... #+END_SRC block, so i use the :session capacity, and i wrote

#+begin_src sh
cd mygitproject
#+end_src

BUT, using this strategy cannot work.

When you write and export org file multiple time in a writing/publishing workflow (write -> execute/publish -> write), each command rerun using the same shell session !, bypassing the :header-args: dir: defined in property.

The result was weird, and multiple run create an infinity of nested folder due to git clone and cd command.

Following your advice what is the best way to manage path in this use case ? Killing session each run ? Use environment variable without :session to define directory path ?

UPDATE 1

a) Using absolute path, env substitution, and session :

:PROPERTIES:
:export_file_name: reproductiblepaper
:header-args: :dir ~/home/xxx/reproductible_paper tangle:yes session: reproductible
:END:
#+NAME: absfolder
#+BEGIN_SRC sh
ABS="/home/xxx/reproductible_paper"
#+END_SRC
#+begin_src sh
git clone git@gitlab.com:xxx/yyy/mygitproject.git
#+end_src
#+begin_src sh :noweb yes
<<absfolder>>
cd $ABS/mygitproject
pwd
#+end_src

If you use this solution, and killing each time the session, it works, but you cannot use interactive evaluation without creating an unstable state in your shell session. Why not.

But to do this in a clean way i need :

  • a way/command to kill/reset session at each re-export.
  • a starting script which clean previous state files/folders manipulation

Any advice do that in org ?

b) I suppose there is probably another way to do the same thing without using a session, using :var or :dir and using link dependency between each code block ?

4
  • You say, "you cannot use interactive evaluation without creating an unstable state in your shell session" but it isn't clear to me why this is the case. What is unstable?
    – mankoff
    Jan 9 '20 at 16:52
  • By unstable state, i mean, if you rerun interactivly previously runned command (git clone myproject or cd myproject/ for example), weird things happen.
    – reyman64
    Jan 9 '20 at 18:52
  • We already address cd mproject/: use absolute paths. For the git command, you could: [[ -d myproject ]] || git clone.... But your issues now aren't about Org Mode or Org Babel anymore. These are just generic coding issues about how to write code that behaves properly if it is run multiple times. You'll have these same issues regardless of Org.
    – mankoff
    Jan 9 '20 at 20:49
  • 1
    Thanks for asking your question! As @mankoff mentioned recoding those blocks will be very helpful. In addition, most org-mode SRC headers & properties will accept elisp code to dynamically set values at run time to change paths, turn on or off features, auto kill sessions, prompt for user input, etc. There are many excellent answers on emacs sx about these issues & your question provides an opportunity to pull these separate answers into a comprehensive answer for others.
    – Melioratus
    Jun 7 '20 at 13:57
1

Your use case is a bit unusual, but I think you can get what you want with a few tweaks.

As you've discovered, when you use the session argument, you will re-use the same active bash/R session each time you re-run your code. That's an issue if you use cd. You could protect against this by including a block resetting your directory at the start of your file:

#+begin_src sh
cd ~/home/xxx/reproductible_paper
#+end_src

This will allow you to continue using relative paths for the rest of your script. If you don't want this block to appear in your final document, you can hide it from the output with the header argument :exports none

Another thing that might help is to selectively exclude some code blocks from evaluation at export. You probably don't need to clone the same git repository every time you change some code further down in your document. You can control this with the eval header:

#+begin_src sh :eval noexport
git clone git@gitlab.com:xxx/yyy/mygitproject.git
#+end_src

The noexport option will allow you to interactively run the code the first time you need it, or anytime you want to repeat the code (e.g., when the repository has been updated). But it won't get run automatically whenever you export your document.

4
  • Thanks for you answer. Another path i fallow actually, waiting for future release of org, i also see that headers accept now eval() function. That could help us a lot by reinjecting correct path foreach different block without using session and noweb << cd .. path>> trick at each block.
    – reyman64
    Oct 6 '20 at 12:41
  • You should only need a single cd block at the top of your file to reset the directory, not before every block.
    – Tyler
    Oct 6 '20 at 13:29
  • Yes you're right @Tyler, i validate your answer, thanks :)
    – reyman64
    Oct 6 '20 at 14:06
  • In another scenario, I need to create another question i suppose because this dont work if i want to use tramp + cd to folder whitout using session ...
    – reyman64
    Oct 14 '20 at 9:56
0

I'm not sure how this is specific to Org. If you write a script that changes state dependent on current state, and don't unchange/reset the state elsewhere in the script, and repeatedly run the script, the state will be changed each time you run it.

What about cd ${project_root}/mygitproject?, where ${project_root} is absolute? If you use the absolute path, not the relative path, your problem may be solved.

1
  • Yes, you're right, i precise my question in update
    – reyman64
    Jan 9 '20 at 8:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.