org-mode batch export hangs when using a python session

Question

I have developed a workflow for batch-export of org-mode files, and have come across an issue in which execution halts when using a python session. I'm using emacs 28.1 on MacOS 11.6.5 with Python 3.9.5.

Here's a minimal org-mode file (python-session.org)

#+PROPERTY: header-args:python :exports both :eval yes :session

#+begin_src python :results output
import sys
print(sys.executable)
print(sys.version)
var = "from the first block"
#+end_src

#+begin_src python :results output
print(var)
#+end_src

..and some minimal code to export on batch mode

(let* ((infile (car (last command-line-args)))
       (outfile (concat (file-name-sans-extension infile) ".html")))

  (find-file infile)
  (org-mode)
  (org-babel-do-load-languages
   'org-babel-load-languages
   '((emacs-lisp . t)
     (shell . t)
     (python . t)))

  (setq org-confirm-babel-evaluate nil)
  (setq org-babel-python-command "python3")
  (setq python-indent-offset 4)
  (setq python-shell-completion-native-enable nil)
  (setq python-indent-guess-indent-offset t)
  (setq python-indent-guess-indent-offset-verbose nil)

  (org-html-export-as-html)
  (write-file outfile))

Here's the command to perform the export (and the resulting message):

% emacs --batch --load minimal-export.el --kill -- python-session.org
executing Python code block...

At this point the process hangs and must be interrupted.

The above command results in an exported html file when the :session option is removed from the first line of the file:

% emacs --batch --load minimal-export.el --kill -- python-session.org
executing Python code block...
Code block evaluation complete.
executing Python code block...
Babel evaluation exited with code 1
Code block produced no output.
Cannot fontify source block (htmlize.el >= 1.34 required)
Cannot fontify source block (htmlize.el >= 1.34 required)

Any ideas on how to troubleshoot further?

Answer 1

Ok, after learning a great deal about how to (clumisly) trace elisp code, I identified this as a bug in prompt detection in python.el: apparently the startup message for the python interpreter is not being recognized. Launching the interpreter with python -q suppresses the prompt, but the variable python-shell-interpreter-args does not appear to be respected. So the brute force solution is to advise the function that sets up inferior-python-mode to add -q:

(defun advised-python-shell-make-comint (orig-fun &rest args)
  (setq args (append '("python3 -q") (cdr args)))
  (apply orig-fun args))

That's it. Not a great long term solution, and hopefully this will get fixed in python-mode.

Answer 2

I was running into the same problem.

macOS 10.15.7, GNU Emacs 28.1, Doom Core 3.0.0-dev, Doom Modules 22.07.0-dev

And using pyenv with the +pyenv Doom Emacs setting.

My process was to run org-babel-execute-buffer(C-c C-v b) which let me C-g out of the 'executing python codeblock...' status. And then all the following codeblock executions with RET and C-c C-c ran correctly.

I have #+PROPERTY: header-args-python :session in my header, which I always ran separately from the blocks. So in short, (C-c C-v b) fixed it for me.

Hope this helps.

EDIT: Forget about my original answer, I think it was a coincidence. This really fixes it for me: rename the session to the current python process' buffer. So if the buffer is named "Python", set the session name like :session Python