Org Mode Babel: Is there a way to assert the expected results of evaluation?

Question

Is there a way to assert that the results of evaluating a code block with Babel are equal to an expected text? Ideally, I would use this to stop an export to PDF if one of the assertions does not hold.

This is an example on how to use the Unix command =touch=.

#+begin_src sh :exports code
touch main.c
touch lib.c
#+end_src

#+RESULTS:

After you execute those commands, you should see in your working directory the
following files:

#+begin_src sh :results raw :exports results
tree .
#+end_src

#+RESULTS:
.
|-- example.org
|-- lib.c
`-- main.c

0 directories, 3 files

#+EXPECTED:
.
├── lib.c
└── main.c

The first code block does not have any results nor any expected results; only its side effects are useful. (Incidentally, it has been evaluated, as you can see because there is an empty RESULTS section underneath).

The second code block is what I would like to have: when it is evaluated, it returns a result that can be compared with the EXPECTED block. As you can see, the results do not match because I forgot to include the org file itself and the summary line returned by tree. An export to PDF should be stopped with a message that points out the difference. If RESULTS and EXPECTED did match, then the EXPECTED block is not exported, but the rest of the file is.

Is there a way to do this with the currently available features of Org and Babel?

Answer 1

Here are a few approaches.

* testing conditional execution

This is an example on how to use the Unix command =touch=.

#+begin_src sh :exports code
touch main.c
touch lib.c
#+end_src

#+RESULTS:

After you execute those commands, you should see in your working directory the
following files:

#+name: tree-results
#+begin_src sh :results output :exports results
tree .
#+end_src

#+RESULTS: tree-results
: .
: ├── example.org
: ├── lib.c
: └── main.c
:
: 0 directories, 3 files



** run block with assertion block

Here we blocks that return a string yes or no, and we use that to do conditional execution in this block.


#+BEGIN_SRC emacs-lisp :var execute=assert-tree-results-nil()
(if (string= execute "no")
    (message "Not executed: %S" execute)
  (message "%S so executing" execute)
  ;;(org-open-file (org-latex-export-to-pdf)
  )
#+END_SRC

#+RESULTS:
: Not executed: "no"

#+BEGIN_SRC emacs-lisp :var execute=assert-tree-results-t()
(if (string= execute "no")
    (message "Not executed: %S" execute)
  (message "%S so executing" execute)
  ;;(org-open-file (org-latex-export-to-pdf))
  )
#+END_SRC

#+RESULTS:
: "\"yes\"" so executing

This will raise an error if the block assert fails.

#+BEGIN_SRC emacs-lisp :var execute=assert-assert-tree-results
(message "Doing it")
  ;;(org-open-file (org-latex-export-to-pdf))
#+END_SRC

#+RESULTS:
: Doing it


** using eval to do this.

Here we use elisp to execute a named block and get the results. Here if the block returns "no" the block will not be able to execute.

#+BEGIN_SRC emacs-lisp :eval (org-babel-execute-src-block nil (org-babel-lob--src-info "assert-tree-results-t"))
;(org-open-file (org-latex-export-to-pdf))
(message "doing it")
#+END_SRC

#+RESULTS:
: doing it


** Assert blocks

#+name: assert-tree-results-nil
#+BEGIN_SRC emacs-lisp :var result=tree-results :results silent
(if (string= result ".
├── lib.c
└── main.c")
    "yes"
  "no")
#+END_SRC



#+name: assert-tree-results-t
#+BEGIN_SRC emacs-lisp :var result=tree-results :results code
(if  (string= result ".
├── example.org
├── lib.c
└── main.c

0 directories, 3 files
")
    "yes"
  "no")
#+END_SRC


This raises an error if the results are not right.

#+name: assert-assert-tree-results
#+BEGIN_SRC emacs-lisp :var result=tree-results :results code
(assert  (string= result ".
├── example.org
├── lib.c
└── main.c

0 directories, 3 files
")))
#+END_SRC